Skip to Content
Former Member
Jul 21, 2017 at 10:44 AM

Table comparison performance after altering key columns

53 Views Last edit Jul 21, 2017 at 10:49 AM 2 rev


I am using DS

Database : SQL Server 2008 R2.

Input data size ~100000 rows

Target comparison table size ~ 80 million records

We are witnessing a huge issue in performance we after we moved few columns from the key columns to comparison list.


Generated key column is set

Comparison method : Row by row

Before : Input pk columns were 5 and the dataflow took around 10-15 minutes to complete

After : Input pk columns left are 3 and the df now doesnt complete even after running for 12 hours and has to be aborted.

I tried using sorted input as well but doesnt help at all. Tried running as a separate process but no change.

The reason why the few key columns were moved to compare list is because those values are expected to change and we need to update the existing records instead of creating new ones.