Skip to Content
0

performance issue in loading the data

Jul 15, 2017 at 10:01 AM

42

avatar image

hi all,

I am trying to load the data from hana to ODL in json format.
JSon formance has so many nested scemas.
I have to load 40 million records.

so i have designed 5 data flows which run parallelly and can load any number of records each.
when each data flow process 500 records, total 2500 records will be processed in 5 mnts by all the 5 dfs.

but i cant take this approach to load 40 million records.

i have tested this with query transform(better performance) and xml_Map transform(not good compared to query transform).

Is there a way to improve performance of the job ?
As per my understanding, nested schemas inside the json format are taking longer time.
Appreciate if you can suggest the best and optimal way to handle json format ?

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

0 Answers