Skip to Content
avatar image
Former Member

SAP HANA SDI Flowgraph vs Insert INTO SELECT

Hi,

I have DP Agent running on my local laptop connecting to a remote HANA server. I have a huge csv file (1gb) on my laptop which i want to upload as a HANA table. i am able to create a remote source and create a virtual table of the file.If i want to create a physical table from the file what is the better option.

i am able to create a physical file using INSERT INTO phisical_table SELECT columns from Virtual table.

It took almost 1h 30mins to load the physical table with 6 million records.

is there a better way? should i try with Flowgraph , will there be a difference.?

Regards,

Sai

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • Feb 13, 2017 at 11:34 PM

    Hello Sai,

    when using Flowgraphs you can specify partitions that will basically allow you to read in parallel from the file.

    When you add a virtual table as a Data Source double click on the Data Source Node and go to the Partition Tab.

    There you can specify the partitions which will be loaded in parallel.

    This should increase your load performance. However the limiting factor at the end is the network. So one more thing to tweak is the fetchsize in the dpagentconfig.ini file located on the DPAgent machine. Increasing this number will lower the network overhead created when sending the rows over to hana.

    Kind Regards,

    Timo

    Add comment
    10|10000 characters needed characters exceeded

    • Hello Sai,

      They are at the end modifying the same parameter. The DPAgent Tool however is retriggering the config to be reloaded. If you change it through the ini file directly you may need to restart the agent in order to get the settings picked up.

      Did you also try the partitions in the flowgraph? This should give you some additional boost :)

      Kind Regards,
      Timo