Skip to Content
avatar image
Former Member

Usage many transforms in the same data flow

I have a case, where I need to remove duplicate record, and retain most recent date record. So, I used 3 Query transform,

1) Ordering,

2) Generate ROW-NUMBER for each group and

3) Filter record based on row number

Apart from that, I used 3 more query transforms for some other functionality. Finally I end up having 6 query transforms in the same data flow.

Is there is any restrictions in using more transforms in the same data flow? If yes, will it affect the performance?

Kindly show me some light on it.

Thanks

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • Best Answer
    Oct 26, 2015 at 09:26 AM

    There's no theoretical restriction on the complexity of a dataflow. As a rule of thumb, for practcial reasons, I try to make my flows fit the size of a my screen.

    Performance is affected by many parameters. Check Efficient extraction of most recent data from a history table for a technique to improve the performance when removing duplicate records.

    Add comment
    10|10000 characters needed characters exceeded