I am using DS 126.96.36.199
Database : SQL Server 2008 R2.
Input data size ~100000 rows
Target comparison table size ~ 80 million records
We are witnessing a huge issue in performance we after we moved few columns from the key columns to comparison list.
Generated key column is set
Comparison method : Row by row
Before : Input pk columns were 5 and the dataflow took around 10-15 minutes to complete
After : Input pk columns left are 3 and the df now doesnt complete even after running for 12 hours and has to be aborted.
I tried using sorted input as well but doesnt help at all. Tried running as a separate process but no change.
The reason why the few key columns were moved to compare list is because those values are expected to change and we need to update the existing records instead of creating new ones.