Skip to Content
0

SDI not loading all rows from Virtual table into Physical table.

Jun 28, 2017 at 06:28 PM

52

avatar image

Hi,

We are using DP Agent on a server,File Share for config ,data files . We created a Remote Source in our SAP HANA server registered to the DP Agent.

I am successfully able to create a virtual table which points to an CSV file using file adapter and i am successfully able to load data from Virtual table to Physical table using

INSERT INTO PhysicalTable (select columns from VirtualTable)

file size 4 GB..total 23 million rows took more than 4 hours..

when i changes the Fetch Size in DP Agent from default 1000 rows to 100000 rows. and tried to load the data in to PhysicalTable using the same syntax as above , it loaded nearly 1 million rows in 6 minutes and the showed the INSERT INTO statement finished successfully.

Statement 'INSERT INTO "PROJECT041"."PSP_Data" (select "UPC" , "Desc1" , "Desc2" , "Size" , "Measure" , ...' successfully executed in 6:17.709 minutes (server processing time: 6:17.711 minutes) - Rows Affected: 1279931

But i have 23 million rows in the file , why is it not loading all the rows after i changed the fetch size.?? as i mentioned above i was able to load all rows with default fetch size of 1000 but took a lot of time..

Regards,

Sai

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

1 Answer

Saikrishna Tarapareddy Jun 30, 2017 at 07:47 PM
0

any updates from experts.??

Share
10 |10000 characters needed characters left characters exceeded