java.io.IOException: Unable to eat more bytes.

[Former Member]

All.

I am attempting to read a 45000MB file into a PI mapping through Content Conversion. The goal it to take this file and map it into SQL Server commands to clear the target SQL table and insert the input rows into a database (approx. 300,000 rows). There is a very large target mapping (35 elements) that produces a very large XML message producing an out of memory message.

the last successful file posted was 32000MB.

We changed the format of the mapping to just insert the rows and tried to split the processing of the input file to create many smaller messages to pass through using the RecordSets per Message on Content Conversion. We tried low 1000 to high 20000.

We are now receiving the following error in the MDT. SXMB_MONI didn't register at this point.

Channel CC_XXXX_XXXX_Sender: Send binary file "
XXX\file.CSV", size 322124473 with QoS EO.

Transmitting the message to endpoint http://xxxx:8000/sap/xi/engine?type=entry using connection File_http://sap.com/xi/XI/System failed, due to: com.sap.aii.af.ra.ms.api.RecoverableException: java.io.IOException: Unable to eat more bytes.

If there is anyone that has come across this message and has idea of the best way to get this data passed, I would appreciate it.

We prefer not to touch the input file by breaking it into smaller pieces.

thanks

[Former Member]

Hi Bob,

45GB, that's a huge file ! How big is your heap memory for the J2EE ? Is there any way to split it into smaller subsets of records ? Do you plan to do this load only once or on a regular basis ?

If this happens once (or once in a while), why not using the db manufacterer load tool (like SQLLoader for ORACLE etc) ?

Rgds

Chris

[Former Member]

I apologize. The size of the file is about 45000(KB) or 45 MB not GB .

Further Info. We are running NetWeaver: NW04S_18_REL Service pack: 18

java.runtime.version: 1.4.2_11-b06

com.sap.aii.ib.client.jnlp.j2se.maxheapsize: 512m

Our BASIS staff worked with SAP to tune the system, as this was happening with smaller files 25000KB. It was working and peaked at 32000KB before the out of memory failure. It is failing after the content conversion into XML when it is trying to send it through.

Error: Transmitting the message to endpoint http://xxxxx:8000/sap/xi/engine?type=entry using connection File_http://sap.com/xi/XI/System failed, due to: com.sap.aii.af.ra.ms.api.RecoverableException: java.io.IOException: Unable to eat more bytes.

It has a very large footprint with 35 elements and about 300,000 rows.

We increased the MAX File size on the Sender channel: from 804857600 to 1000000000 and recordsets per message to 35000 to no avail.

Also note: The file will be loaded monthly. It grows each month, as it is a historical file.

The backend design is to drop the table and reload, it as opposed to appended or updating or providing a delta.

I have no access to XI Admin functions. If I need to check settings, please inform best way to access this.

thanks

Edited by: Bob Drozd on Dec 30, 2009 7:44 PM

Edited by: Bob Drozd on Dec 30, 2009 9:19 PM

[Former Member]

SAP PI has a Quick Sizer tool that helps you estimate the required memory and CPU based on message size and throughput. The webinar on guidelines to use it can be found at https://www.sdn.sap.com/irj/sdn/webinar?rid=/library/uuid/70e8b684-9f23-2b10-74bc-fdbc84ead706. Hopefully that should provide useful insights for your situation