Usage many transforms in the same data flow

[Former Member]

I have a case, where I need to remove duplicate record, and retain most recent date record. So, I used 3 Query transform,

1) Ordering,

2) Generate ROW-NUMBER for each group and

3) Filter record based on row number

Apart from that, I used 3 more query transforms for some other functionality. Finally I end up having 6 query transforms in the same data flow.

Is there is any restrictions in using more transforms in the same data flow? If yes, will it affect the performance?

Kindly show me some light on it.

Thanks

[former_member187605 · Accepted solution]

There's no theoretical restriction on the complexity of a dataflow. As a rule of thumb, for practcial reasons, I try to make my flows fit the size of a my screen.

Performance is affected by many parameters. Check for a technique to improve the performance when removing duplicate records.