Skip to Content

Can I process a file with multiple sets of data?

I have a CSV file from a vendor that I need to process that has multiple sets of data--not multiple schemas, but entirely different sets. Here is an example:

** Message
** Timestamp
Data Set 1
Value 1, Value 2, Value 3, Value 4, Value 5
Data, Data,,Data,
Data, Data, Data,,Data

** Message
** Timestamp
Data Set 2
Value 1, Value 2, Value 3
Data,,
Data,Data,Data

** Message
** Timestamp
Data Set 3
Value 1, Value 2, Value 3, Value 4
Data, Data, Data, Data
Data,,Data,

I need to retrieve the data from the second set. I don't know how to have Data Services skip the first set and skip the top three rows of file information before processing the file. The record count for each set varies, of course, so I can't set a row to start reading the file. Can this be done?

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • Best Answer
    Apr 16 at 04:09 PM

    I solve this type of problem by dumping everything into a 2-column table first, the 1st column is populated with a sequence number, the 2nd contains the input data. Then using DS logic (transforms and functions) for further processing.

    Have a look at

    https://wiki.scn.sap.com/wiki/display/EIM/How+to+transform+a+flat+file+into+a+master-detail+structure+using+lookup_seq

    for a slightly comparable use case.

    Add comment
    10|10000 characters needed characters exceeded