Ways to handle large volume data (file size = 60MB) in PI 7.0 file to file

[madanmohan_agrawal]

Hi,

In a file to file scenario (flat file to xml file), the flat file is getting picked up by FCC and then send to XI. In xi its performing message mapping and then XSL transformation in a sequence.

The scenario is working fine for small files (size upto 5MB) but when the input flat file size is more then 60 MB, then XI is showing lots of problem like (1) JCo call error or (2) some times even XI is stoped and we have to strat it manually again to function properly.

Please suggest some way to handle large volume (file size upto 60MB) in PI 7.0 file to file scenario.

Best Regards,

Madan Agrawal.

[Former Member · Accepted solution]

Hi Madan,

If every record of your source file was processed in a target system, maybe you could split your source file into several messages by setting up this in Recordset Per Messages parameter.

However, you just want to convert you .txt file into a .xml file. So, try firstly to setting up

EO_MSG_SIZE_LIMIT parameter in SXMB_ADM.

However this could solve the problem in Inegration Engine, but the problem will persit in Adapter Engine, I mean, JCo call error ...

Take into account that file is first proccessed in Adapter Engine, File Content Conversion and so on...

and then it is sent to the pipeline in Integration Engine.

Carlos

[madanmohan_agrawal · Accepted solution]

Hmmm.....I think I am now getting the picture clearly. Could you please confirm the below.

To put less stress in adapter engine, we are configuring the EO_MSG_SIZE_LIMIT in SXMB_ADM and

to put less stress in integration engine, we have to use recordset per message in sender FCC.

Thanks for your information!

Br,

Madan Agrawal.

[Former Member · Accepted solution]

Hi,

No, its exactly the opposite.

SXMB_ADM --> Integration Engine.

FCC --> Adapter Engine.

Carlos

[madanmohan_agrawal · Accepted solution]

Thanks Carlos !