Skip to Content
avatar image
Former Member

How to split a huge file into small chunks?

The requirement here is to read a .csv file with around 100 to 200MB file size having around 50k to 100k records. As this is too huge for processing, the file must be split into chunks of 10,000 or 5000. File structure is given below. Please let me know how to configure recordsets per message option in FCC to achieve this. I already tried, but it failed.

Also, the chunks must be processed in sequence. If the sequence is missed, it will cause difficulties in further processing of data in target system.

File Structure :

Continent

Country 1

Country 2

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • Nov 07, 2017 at 12:33 AM

    Hi Poornima,

    Please refer the below blog

    https://blogs.sap.com/2011/12/26/fileftp-adapter-large-file-transfer-chunk-mode/

    regards,

    Harish

    Add comment
    10|10000 characters needed characters exceeded

    • Former Member

      Hi Harish,

      Thank you very much for sharing this information. Will refer to it when changing configuration in file channel.

      Regards,

      Poornima S