We are attempting to use the standard Import package to import a very large data file.
The data file was generated with Time dim values as columns.
Account, Category, Datasrc, Entity, Time1, Time2, ......, Time12
act1, cat1, data1, ent1,500,200,0,0,0,0,0,0,140,0,0,0
Which using *MVAL in the Transformation file will actually result in 12 rows to be imported, one for each Time.
act1,cat1,data1,ent1,time1,500
act1,cat1,data1,ent1,time2,200
act1,cat1,data1,ent1,time3,0
:
act1,cat1,data1,ent1,time12,0
Since we are already starting with an extremely large file, this is taking far too long and resulted in the actual connection timing out not long after the conversion process.
Is there anyway to suppress rows with 0 as the amount to be imported?
We would like to be able to do this using the standard Import process (no custom packages), so we are hoping there's an solution that can be implemented within the Transformation or Conversion files.
Thanks