Skip to Content
avatar image
Former Member

How to run just n number of rows

We are in the process of converting from one ETL tool to Data Services 3.0. I have gone through the tutorial and searched the forum, but could not find a way to run the first n rows of source. For example, my source has 3000000 rows, but instead of running all the rows through the job, I just want to run the first 1000 rows to make sure I am getting my desired result before running all 3000000 rows.

Thanks,

Dan

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

3 Answers

  • Best Answer
    avatar image
    Former Member
    Oct 14, 2008 at 11:11 AM

    You could include a where clause on the query that is pulling the data to just restrict it to a subset of rows.

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 18, 2008 at 07:26 PM

    Dan,

    Use a SQL transform and configure it to load data from your source. Create a Select * From <Table_Name> Where rownum < (# of rows you want) in the text area.

    Connect that to a Query transform or not and ->> table.

    -Gera Mats

    Consultant, Global Field Services

    gera.mats at sap.com

    Edited by: Gera Mats on Oct 18, 2008 9:27 PM

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 27, 2008 at 09:32 AM

    This is what the debug feature is about. You would open the dataflow and on the line from the source table you add a filter saying "less than 1000 rows". When you execute that, this filter gets ignored. But when you "Start Debugging" instead of "Execute", this filter will be merged into the reader and hence just 1000 rows are executed. Further more, once the job is completed, you get asked if the debugging session should be shut down. If you say "No", you then can open the dataflow again and see the rows with their data that have been moved through all the steps.

    (Keep in mind, to edit anything, you have to end the debug session - you cannot edit a dataflow while it is still debugged)

    Add comment
    10|10000 characters needed characters exceeded