Skip to Content
author's profile photo Former Member
Former Member

Out of Memory Exception when processing SAP messages in xMII

Hi All,

I am using xmii 12. I have configured a message listener to listen for OPERA3 messages from SAP system and have configured a transaction to get executed when such a message comes.

The transaction will write the received XML into a file in local system and will execute a custom action which will parse the file and insert its contents to DB.

But twice when an initial download for OPERA3 has been triggered, the j2ee server has restarted with an exit code of 666(out of memory exception).

The file size of the OPERA3 xml message is 57MB and it has 60000 records in it.

I have created a custom action since standard XML Loader action will slow down handling files with larger size

1) Is the problem due to large amount of records and file size?

2) Should i have to increase the jvm parameters?

3) How should i approach messages with large number of records?

Thanks

Add a comment
10|10000 characters needed characters exceeded

Related questions

2 Answers

  • Best Answer
    author's profile photo Former Member
    Former Member
    Posted on Oct 08, 2008 at 09:57 AM

    Hi,

    mass data and XML is a known issue. To process such data the complete DOM is built in memory by MII. If you want to fight this issue with

    resources the only chance you have is that you use x64 Windows Server für MII with several GB for the JVM. Typically you could try to give

    the JVM a heap size of 3GB/4GB (keep increasing till it works :-). Though this is architectural not a nice solution, so you should send, for

    example, the message zipped and base64 encoded. This message could then be received by MII and saved to a file. Decoding and dezipping

    this file should be done on file system such that a decoded and dezipped file is created. Then you can process this file with a custom action.

    Remember that you will face the same resource problems in your custom action. The solution is to use an event driven parser like SAX to parse

    the file and do the inserts in JDBC batch mode. With this procedure you can guarantee that a fixed amount of memory is used when parsing

    a big XML file.

    Good luck,

    Matthias

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Oct 07, 2008 at 11:34 AM

    You should start with your logs and looking at the jvm settings. I would also do a search on this forum for "Heap" or "Heap size". You are not the first person to run into jvm limitations while processing huge files. I would also suggest looking on the SAP ERP side of things to see if you A) really need 57M of data and B) 60000 records. If you are doing something along the lines of downloading all the material masters, then I would suggest looking into Change Pointers in the ERP system and only download those which need updating.

    Please give a bit more information on your custom action block and also what OPERA3 messages are.

    Thanks,

    Mike

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.