Skip to Content
avatar image
Former Member

Columns are not loading properly from Flat File (Pipe Delimiter)

Hi All,

I have a Flat File (Pipe Delimiter) contains 28 Columns.

Input File contains 200000 Records. All records loaded properly except one record.

For that record, first 6 columns are not loaded. 7th column from Flat file started populating into 1st Column in Template Table (Target Table) and So On.

Note:

    1) For Previous record, Last 6 columns are NULLs.

    2) If I load this one record Individually, its being loaded correctly

Could you please let me know how to load this row properly along with huge file.

Regards.

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

4 Answers

  • avatar image
    Former Member
    Oct 27, 2015 at 10:06 PM

    I think this is regarding your earlier post.

    When you are loading from flat files, these are bound to happen unless the file is formatted well. The error you have is a delimiter issue because of NULL values in the 6th column or there is no 7th column for that record. When you run DS job it will throw an error with the row number. To fix this do the following -

    1) Open flat file in excel or Notepad++

    2) Go to row number which is displayed in DS.

    3) Keep your cursor on the 6th column and see whats going on.

    4) Either fix yourself manually or inform the guys who are generating flat file for you.

    Add comment
    10|10000 characters needed characters exceeded

    • Keep all output fields as varchar and set Adaptable Schema to Yes. Your job won't run into error and you'll be able to examine the contents of the culprit record in output. That might shed a light on your issue.

  • avatar image
    Former Member
    Oct 28, 2015 at 04:37 AM

    Hi Arun,

    I tried to load same data through Excel. All data loaded with out any Issues. I am still breaking my head whats wrong with Flat File.

    Regards.

    Add comment
    10|10000 characters needed characters exceeded

    • Former Member

      Hi,

      Can you attach the records that are throwing the error? I need some records before row number <15386> and some after.

      Arun

  • avatar image
    Former Member
    Oct 28, 2015 at 05:32 AM

    Hi Arun,

    My Client machine do not allow other websites. So I can not attach the screen shot.

    I am sorry for that.

    Regards.

    Add comment
    10|10000 characters needed characters exceeded

  • avatar image
    Former Member
    Oct 29, 2015 at 01:19 AM

    Can we say it is Data Issue.

    But the record which is prior to failed record is loaded successfully and these two records are similar in format.

    I have asked my client to generate Excel format. They said, they can not generate Excel format.

    Please do let me know if you have any other alternatives.

    Regards.

    Add comment
    10|10000 characters needed characters exceeded

    • Former Member

      Do you have access to DB? If so can you try loading directly to DB instead of DS? Check if thats throwing an error?