Performance Issue - XML vs CSV

[silentbull]

Hi

We have an new interface where we will expect 80k records coming from XML or CSV and process it as idoc.

We tested the idoc part and performance is fine. We are concerned about the two options.

1. If we receive XML from the third party it will be 20 MB per day. Will this cause big performance issue in PI 7.4? Please remember that this will be coming via ICO config.

2. If we agree to receive it via CSV it will be only 5 MB, buit then when it gets converted in the pipeline steps it will convert as 20 MB with XML. Is this somethign again which do we need to worry?

Please note that we have already explored the option of splitting the files and due to complexity involved in logic we prefer one file one idoc per day.

Please let me know your views.

REgards

Sam

[NishantChavan]

I would choose XML as sender.

[silentbull]

Hi

Thanks for the replies. Does it mean it is ok to choose XML as sender or CSV?

[NishantChavan]

I have recently worked on one scenario, Tab delimited file to SOAP receiver. I am processing around 95K records (4MB Tab delimited file). The sender channel is converting that into XML. All records are getting processed within 3-4 minutes. We have 7.31 SP16 with 2 nodes.

In your scenario, PI pipeline processing time will remain same. the sender channel converts the file into XML faster than PI pipeline processing.

Thanks,

Nishant