Skip to Content
Former Member
Apr 24, 2005 at 07:27 PM

Processing huge file loads through XI


In File To SAP integration scenario, XI File Adapter loads and processes whole file in memory. For huge files volumes such as 1GB, it doesn't seem worth doing it through XI.

Could XI File Adapter split the incoming file into multiple chunks or messages?

If not, is there any better way in XI for processing huge loads, or should I go to ETL tool for this?

In typical EAI middleware tools, we use Java IO to read the file, compose them into chunks, and put them in the JMS Queue. The next set of components can pick the data from Queue and process in parallel. This is how we solve the integration problem of processing huge volume.

Please provide your suggestions on this.