Skip to Content
A B

Performance Issue - XML vs CSV

Hi

We have an new interface where we will expect 80k records coming from XML or CSV and process it as idoc.

We tested the idoc part and performance is fine. We are concerned about the two options.

1. If we receive XML from the third party it will be 20 MB per day. Will this cause big performance issue in PI 7.4? Please remember that this will be coming via ICO config.

2. If we agree to receive it via CSV it will be only 5 MB, buit then when it gets converted in the pipeline steps it will convert as 20 MB with XML. Is this somethign again which do we need to worry?

Please note that we have already explored the option of splitting the files and due to complexity involved in logic we prefer one file one idoc per day.

Please let me know your views.

REgards

Sam

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

3 Answers

  • Mar 30, 2017 at 06:23 PM

    I have recently worked on one scenario, Tab delimited file to SOAP receiver. I am processing around 95K records (4MB Tab delimited file). The sender channel is converting that into XML. All records are getting processed within 3-4 minutes. We have 7.31 SP16 with 2 nodes.

    In your scenario, PI pipeline processing time will remain same. the sender channel converts the file into XML faster than PI pipeline processing.

    Thanks,

    Nishant

    Add comment
    10|10000 characters needed characters exceeded

  • Apr 03, 2017 at 09:13 AM

    Hi

    Thanks for the replies. Does it mean it is ok to choose XML as sender or CSV?

    Add comment
    10|10000 characters needed characters exceeded

  • Apr 03, 2017 at 01:19 PM

    I would choose XML as sender.

    Add comment
    10|10000 characters needed characters exceeded