Skip to Content
avatar image
Former Member

Handling a large message(>100MB) in a Proxy to File scenario in PI 7.31

Hi,

We have a proxy to SFTP interface where PI is receiving an XML proxy message with message size between 100 MB - 150 MB.

The message has reached the maximum size limit and is unable to get out of the ECC system; it is throwing the below error:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>

- <SAP:Error SOAP:mustUnderstand="1" xmlns:SAP="http://sap.com/xi/XI/Message/30" xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">

<SAP:Category>XIServer</SAP:Category>

<SAP:Code area="INTERNAL">CLIENT_SEND_FAILED</SAP:Code>

<SAP:P1>400</SAP:P1>

<SAP:P2>Bad Request</SAP:P2>

<SAP:P3>(See attachment HTMLError for details)</SAP:P3>

<SAP:P4 />

<SAP:AdditionalText />

<SAP:Stack>Error while sending by HTTP (error code: 400 , error text: Bad Request) (See attachment HTMLError for details)</SAP:Stack>

<SAP:Retry>A</SAP:Retry>

</SAP:Error>

To add a little more context to the issue, we are on PI 7.31 (single stack). We do not want to dump the PI system with such huge size of data, rather thinking to split it into smaller chunks in ECC and stitching them back in PI into one single file. This is because, the target legacy system can not process multiple files; it can process one full load file per day.

I have come across the below blog where splitting and stitching methods are explained:

http://scn.sap.com/people/smasanta/blog/2012/12/21/how-to-handle-large-volume-of-data-for-proxy-to-file-scenario

But, the blog involves OS level commands and writing the file to a temporary folder and I assume this blog was written keeping PI 7.1 in mind.

Is there any other optimum solution using standard methods to handle the large messages in PI 7.31, bearing the below points in mind:

1. We do not want to change the system level parameters(this might impact the overall performance of the system)

2. BPM is not a preference(can go for it if there is no other option). This interface runs just once in a day, hence just thinking if developing a BPM interface is really worth it.

Awaiting for a solution to this problem.

Thank you.

Regards,

Subbalaxmi Musunuri

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

6 Answers

  • Best Answer
    Oct 28, 2015 at 05:52 AM

    Hi Subbalaxmi

    I think there are very limited standard methods to deal with your scenario. If you really need to split it on the sender side due to size limitation, you need a "staging" area to hold the intermediate messages. Temporary files approach is a common method to stage these intermediate messages. However, looking at the design mentioned in the blog, it does seem quite elaborate.

    I'd personally go for BPM (especially if you have NW BPM and not ccBPM) because it's not too difficult to design an aggregator pattern (I just so happen to complete one recently!) You can check out the following blog that list a series of post related to aggregator patterns.

    Boost your Correlation Scenarios with Conditional Start

    Rgds

    Eng Swee

    Add comment
    10|10000 characters needed characters exceeded

    • Harish, you are definitely right and I totally missed it. My bad! 😳 Thanks for pointing it out. If it can't handle the initial full 150MB load, it wouldn't be able to handle it when it eventually combine them back.

  • Oct 28, 2015 at 07:20 AM

    Hi Subbalaxmi,

    The issue doesn't  seem to be a memory problem, that it is the usual in these cases, you can change the sxmb_adm ECC parameter EO_MSG_SIZE_LIMIT. Also, i know you concern about to overload the ECC system, in this possibility you can develop a own module, or anotther scenario which responsibility was to split the files in n parts and a second scenario to take only this n parts.

    Regards.

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 29, 2015 at 01:36 AM

    Subbalaxi,

    If target file is text file (not XML), try this: -

    First interface: - Small messages from SAP ECC -> SAP PI EOIO -> Temp folder (append mode).

    Second interface: - Pick above file -> SAP PI -> SFTP. If header and trailer are required add in mapping.

    If target file is XML: - Schedule the interface (>150 MB XML) when there is less load on PI system. There is no official size limit, it depends on hardware, available java heap memory and complexity of mapping. Try to reduce complexity of mapping, PI will be able to handle huge file.

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 29, 2015 at 03:07 AM

    Hello Subbalaxmi,

    If your file is text file you can split the data coming from ECC in chunks and process that in single file using APPEND in file communication. If you want to process the records in sequence use EOIO.

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 29, 2015 at 07:49 AM

    Hi Subbalaxmi,

    • Instead of sending the data via proxy from ECC you can create the final text file using ABAP program and create the single file in ECC application server.
    • You can send this file using chunk mode option in File adapter, you can refer below blog.

    File/FTP Adapter - Large File Transfer (Chunk Mode)

    Regards,

    Praveen.


    chunk mode.jpg (19.6 kB)
    Add comment
    10|10000 characters needed characters exceeded

  • avatar image
    Former Member
    Nov 09, 2015 at 05:40 AM

    Hello All,

    Thank you for your valuable responses.

    For the problem I mentioned above, as an immediate solution, we have reduced the length of the XML tag names to a maximum of 2 - 3 characters for each field(We have more than 100 fields in the structure). Since, the XML tag names occupy more message size than the actual data, this helped in reducing the XML message size to 1/3 rd of the actual message size. This change to the message structure resolved the issue for now and the messages are getting processed successfully in both ECC and PI systems.

    Further to this, we are looking at implementing BPM(splitting and stitching approach) as a long term/ permanent solution to these kind of issues.

    Thanks once again for your inputs.

    Regards,

    Subbalaxmi Musunuri

    Add comment
    10|10000 characters needed characters exceeded