Handling xml message of size more than 100mb in SAP PI 7.1

[Former Member]

Dear Experts,

Is it possible for PI to pick-up and process a XML message of size more than 100 MB in PI 7.1 EHP-1?

If yes, can you please let me know how to handle it?

Thank you.

[sajith_p]

Please refer the SAP link:

https://help.sap.com/doc/saphelp_nw75/7.5.5/en-US/64/243dc80d194d9c94dff40e1f4b059e/frameset.htm

[RaghuVamseedhar]

Hi Saravana,

I have previously searched SAP documentation for "What is maximum size of file, PI can handle?". I have not found any SAP Help documentation, white paper, Article etc. When a message was raised to SAP by my team (I do not remember the message number), SAP responded that "technically there is no limit".

I would suggest you to raise a OSS message to SAP. Then share SAP response with us.

Regards,

Raghu_Vamsee

[former_member181962]

"Best performing message sizes: It has been observed that the best choice is an average

message size in the range of 1 MB to 5 MB, even slightly lower message sizes still lead to an

acceptable performance."

Source:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/2016a0b1-1780-2b10-97bd-be3ac6221...

See section 3.1

Best Regards,

Ravikanth Talagana

[RaghuVamseedhar]

Hi Saravana,

Yes, technically it is possible for PI to handle 100 or 200 MB file. If you ask SAP, reply will be "technically SAP PI can handle any size of file, based on PI system recourses (RAM and processor)". In my current project, our production system is handling 100MB XML file (even I am surprised).

Regards,

Raghu_Vamsee

[Former Member]

Thanks Raghu.

By default the configurations allow us to use only upto !00 MB message. Even quick-sizer does not allow us to size message more than 100 MB. So, I am looking for any document/pointers to process huge files ( > 200 MB ) by changing configuration or hardware.

Sanjeev, no, I am checking whether we can process huge files end-to-end in PI.

[naveen_chichili]

Hi Saravana,

As per my knowledge SAP PI can be able to process the file with 40 MB and if you do the tuning/extra parameter thange it would be able to process till 200mb to the max.

You can configure the pameters given by me which would be able to process the file with 200 mb

Thannks and Regards,

Naveen

Edited by: chichilin on Aug 26, 2011 10:36 PM

[naveen_chichili]

Hi Saravana,

it is not a best practice to more than 100mb..

you can increase below parameters and so that you would be able to process for the best..

u2022 UME Parameters : May be we need to look into the pool size and poolmax wait parameters - UME recommended parameters (like: poolmaxsize=50, poolmaxwait=60000)

u2022 Tuning Parameters: May be we need to look/define the Message Size Limit u201Clike: EO_MSG_SIZE_LIMIT = 0000100u201D under tuning category

u2022 ICM Parameters: May be we need to consider ICM parameters (ex: icm/conn_timeout = 900000. icm/HTTP/max_request_size_KB = 2097152)

Thanks and Regards,

Naveen

[Former Member]

Thanks Naveen.

What I am looking for is broader answer. I mean, whether there is no possibility in PI to process very big message [ > 200 Mb] or we can achieve it by some combination of configuration and server sizing.

[former_member194786]

Hi Saravana,

In case you do not have any mapping requirement, you could use chunk mode for Binary File Transfer within PI 7.1 EHP1. Use the below link for more details on this:

/people/niki.scaglione2/blog/2009/10/31/chunkmode-for-binary-file-transfer-within-pi-71-ehp1

Regards,

Sanjeev.

[Former Member]

We have seen 150 MB message process without any problem.

You obviously need the memory settings to be increased (java heap, abap memory parameters), ensure that garbage collection is not running frequently, adjust parameters like AFW_MAXREQUESTCONTENTLENGTH & EO_MSG_SIZE_LIMIT in SXMB_ADM and increase MaxRequestContentLength in http provider.

Ultimately all the payload information will be stored in CLOB/BOB columns which has its own limitation.

regards

[sapnwnewbie.blogspot.com|http://sapnwnewbie.blogspot.com/]

[Former Member]

Thanks Raghu_Vamsee and sapnwnewbie.blogspot.com.

I raised an OSS note with SAP. They replied that SAP support cannot answer the question as it is a consulting question and asked me to contact SAP Consulting group ( which is a paid service ).

So, as a summary of this discussion so far, following are the parameters we need to change: Can you please add/modify if anything else?

1. SXMB_ADM - AFW_MAXREQUESTCONTENTLENGTH & EO_MSG_SIZE_LIMIT

2. J2EE Visual Admin - increase MaxRequestContentLength in http provider

3. SMICM - icm/conn_timeout = 900000. icm/HTTP/max_request_size_KB = 2097152

4. J2EE Visual Admin - UME Parameters : May be we need to look into the pool size and poolmax wait parameters - UME recommended parameters (like: poolmaxsize=50, poolmaxwait=60000)