Skip to Content
author's profile photo Former Member
Former Member

d filter on selected fields during Extraction in Data Service

Hi

We would like to add filter on the data source to extract only some specific range of values. For a huge dataset (around 5M), we only need probably 200 records. Instead of extract the whole 5M records, we need to find a way to apply a filter during or before the extraction. ( The 'where' condition in Query is only do the selection after extraction, which is not what we want) Pl give me some help on that! Thanks!

CC

Add a comment
10|10000 characters needed characters exceeded

Assigned Tags

Related questions

5 Answers

  • Posted on Apr 29, 2016 at 09:13 AM

    Hi,

    You can use SQL transform to extract data.Select TOP 200 * from source table.


    sql.JPG (9.3 kB)
    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Apr 29, 2016 at 11:35 AM

    Whats your Source Database

    Regards

    Arun Sasi

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Apr 29, 2016 at 01:41 PM

    Why do you not want to use WHERE?.Explain.

    Use JOIN : Load the key values in another table and join with your source table.

    Example : You have a table with 5m address records but you are interested in countries US ,AU and SG. Place these country codes in a separate table and join with the main to table to filter out the records.

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Apr 29, 2016 at 02:44 PM

    Source is the ODP object import from ERP via RFC

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Apr 29, 2016 at 03:52 PM

    Use JOIN : Load the key values in another table and join with your source table.

    Example : You have a table with 5m address records but you are interested in countries US ,AU and SG. Place these country codes in a separate table and join with the main to table to filter out the records.

    Add a comment
    10|10000 characters needed characters exceeded

    • Former Member

      Hi Magesh,

      Thanks for your reply. My resource is ODP object from ERP, instead of tables. 'Load the key values in another table' , is it going happen in Data Service? Our case is, in ERP, RSA6, I checked a few columns as 'selection'. And then , we would like somewhere in data service I could set specific values for those columns and then just read data set based on those values. Should I use global variable?

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.