Skip to Content
0

Performance Issue - XML vs CSV

Mar 30, 2017 at 03:14 PM

83

avatar image

Hi

We have an new interface where we will expect 80k records coming from XML or CSV and process it as idoc.

We tested the idoc part and performance is fine. We are concerned about the two options.

1. If we receive XML from the third party it will be 20 MB per day. Will this cause big performance issue in PI 7.4? Please remember that this will be coming via ICO config.

2. If we agree to receive it via CSV it will be only 5 MB, buit then when it gets converted in the pipeline steps it will convert as 20 MB with XML. Is this somethign again which do we need to worry?

Please note that we have already explored the option of splitting the files and due to complexity involved in logic we prefer one file one idoc per day.

Please let me know your views.

REgards

Sam

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

3 Answers

Nishant Chavan Mar 30, 2017 at 06:23 PM
0

I have recently worked on one scenario, Tab delimited file to SOAP receiver. I am processing around 95K records (4MB Tab delimited file). The sender channel is converting that into XML. All records are getting processed within 3-4 minutes. We have 7.31 SP16 with 2 nodes.

In your scenario, PI pipeline processing time will remain same. the sender channel converts the file into XML faster than PI pipeline processing.

Thanks,

Nishant

Share
10 |10000 characters needed characters left characters exceeded
Anandhakrishnan Balaradjou Apr 03, 2017 at 09:13 AM
0

Hi

Thanks for the replies. Does it mean it is ok to choose XML as sender or CSV?

Share
10 |10000 characters needed characters left characters exceeded
Nishant Chavan Apr 03, 2017 at 01:19 PM
0

I would choose XML as sender.

Share
10 |10000 characters needed characters left characters exceeded