Skip to Content
0

Table Comparison taking too long to compare

Jan 13 at 08:35 PM

64

avatar image

Hello Experts,

I have a dataflow which is having Table A (4900000 records) LEFT joined with 3 other tables (max records 800000) followed by a table comparison ,map operation and target table. All the tables are created in Hana Database.

Now every time I execute my job,this dataflow takes almost 35 minutes to execute and load the data into target. Tried lot of tuning techniques but doesn't seem to make much difference.

Thanks in advance.

Ankit

10 |10000 characters needed characters left characters exceeded

Hello Ankit,

Have to try with Lookup functions.

It may decrease your execution time.

Thanks,

Chandrasekhar. K

0
Chandra Sekhar K

Hi Chandrasekhar,

Thanks for your reply.

Let me try with Lookup and get back.

Regards,

Ankit

0
* Please Login or Register to Answer, Follow or Comment.

1 Answer

Best Answer
Arun Sasi Jan 16 at 09:03 AM
0

Which is the Comparison method used in the TC transform.

Are your Source tables from a same Datastore or different one?

Try using a Data Transfer transform after the Query transform with Transfer Type as Table.

Below is the data flow:

Source(LOJ)>Query_Transform>Data_Transfer>TC>MO>Target Table

You can also go with the Bulk Loader options on the HANA Target table which is mentioned in the below article. As you have TC and MO in the data flow Bulk Loader should improve the performance.

https://wiki.scn.sap.com/wiki/display/EIM/Bulk+Loading

Regards

Arun Sasi

Show 2 Share
10 |10000 characters needed characters left characters exceeded

Hi Arun,

Thanks for the reply.

Initially I had used Cache Comparison that I later changed to Sorted-Input(Execution time decreased by around 10 mins).

The Sources are from 2 datastores within HANA.

Used Data Transfer with Table option but that also was not helpful(execution time went up again).

As part of our requirement, we're already using the Bulk loader mode as 'Append' and I'm facing this issue after all these settings.

Regards,

Ankit

0

Ankit, I assume that there are Primary Keys(s) on the HANA target table. Check if you can rebuild the indexes on the Target table. If there are more than 1 indexes then it is possible that the indexes are fragmented. This might help from the performance perspective. Check with your DBA if you dont have rights to rebuild the indexes

Regards

Arun Sasi

0