Skip to Content
avatar image
Former Member

Improving performace of the job while loading the data into Json format

Hi All,

I am trying to load data into json format and from json to ODL(operation data layer which is cloud).

The json format I am loading has nested schemas.

I have to load 40 millions of data which is transactional data.

so to improve the performance of the job, I have designed 5 parallel dfs which process 500 transactions each at a time. 500 transactions are processed in 5 mnts by single df. so in 5 mnts, I can process max 2500 transactions. if I process more than 500 records( lets say for 1000-40 mnts and 1500-1 hr 20 mnts), it takes more time.

I have tried this with query transforms(better performance),xml_map transform(not better than query). Is there a way to improve the performance of the job ?

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

0 Answers