Skip to Content
avatar image
Former Member

Table comparison performance after altering key columns

Hi,

I am using DS 14.2.1.224

Database : SQL Server 2008 R2.

Input data size ~100000 rows

Target comparison table size ~ 80 million records

We are witnessing a huge issue in performance we after we moved few columns from the key columns to comparison list.

Settings:

Generated key column is set

Comparison method : Row by row

Before : Input pk columns were 5 and the dataflow took around 10-15 minutes to complete

After : Input pk columns left are 3 and the df now doesnt complete even after running for 12 hours and has to be aborted.

I tried using sorted input as well but doesnt help at all. Tried running as a separate process but no change.

The reason why the few key columns were moved to compare list is because those values are expected to change and we need to update the existing records instead of creating new ones.

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

0 Answers