Skip to Content
avatar image
Former Member

SDI not loading all rows from Virtual table into Physical table.

Hi,

We are using DP Agent on a server,File Share for config ,data files . We created a Remote Source in our SAP HANA server registered to the DP Agent.

I am successfully able to create a virtual table which points to an CSV file using file adapter and i am successfully able to load data from Virtual table to Physical table using

INSERT INTO PhysicalTable (select columns from VirtualTable)

file size 4 GB..total 23 million rows took more than 4 hours..

when i changes the Fetch Size in DP Agent from default 1000 rows to 100000 rows. and tried to load the data in to PhysicalTable using the same syntax as above , it loaded nearly 1 million rows in 6 minutes and the showed the INSERT INTO statement finished successfully.

Statement 'INSERT INTO "PROJECT041"."PSP_Data" (select "UPC" , "Desc1" , "Desc2" , "Size" , "Measure" , ...' successfully executed in 6:17.709 minutes (server processing time: 6:17.711 minutes) - Rows Affected: 1279931

But i have 23 million rows in the file , why is it not loading all the rows after i changed the fetch size.?? as i mentioned above i was able to load all rows with default fetch size of 1000 but took a lot of time..

Regards,

Sai

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • avatar image
    Former Member
    Jun 30, 2017 at 07:47 PM

    any updates from experts.??

    Add comment
    10|10000 characters needed characters exceeded