Skip to Content

Generic Extractor based upon calday - delta logic - lower limits

Hi Experts,

We have created Generic Extractor with generic delta based upon 'CALDAY' and upper limit of 1. Now as part of our requirement we have to run this multiple times in a day so we are thinking about maintaining lower limits as below so that we can extract it multiple times in day.

As per my understanding it should behave in the following way after maintaining lower limit

If lower limit of 1 is defined

RSA7-1 < Delta <= D

[D = Day on which the BW load is run, (RSA7 -1) = means Day on which last successful delta completed in BW minus 1 ]

If we are running the load on 09/19/2016 then

09/17/2016 <Delta <=09/19/2016 – On 19th we are getting all records of 18th and partial records of 19th (Note - More records can be created on 19th )

Now if we run the load on 09/20/2016 then

09/18/2015 <Delta <=20 – On 20th we are getting all records of 19th and partial records of 20th (Note - more records can be created for 20th ).

Are there any other specific issues that we will have after maintaining lower limits.

Thanks

Add a comment
10|10000 characters needed characters exceeded

Related questions

6 Answers

  • author's profile photo Former Member
    Former Member
    Posted on Oct 18, 2016 at 03:16 PM

    I think there is some confusion about the lower limits and upper limits.

    If you have the upper limit then the delta upper limit will be subtracted by your set upper limit. For example, if you have upper limit at 1 and you are extracting data on 10/19/2016 and your last successful delta is on 10/18/2016 then system will set the upper limits by subtracting 1 from the day of extraction. So system will set the limits as 10/18/2016 - 10/18/2016. So it will miss the data of the day of extraction. So do not have upper limit when you want to extract multiple times a day.

    Follow what Phani said - No upper Limit and Lower limit as 1. You should be fine then for extracting multiple times a day.

    Also you will have repeated data in the datasource so better use DSO.

    Thanks

    Amit

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Oct 18, 2016 at 07:51 AM

    Calday : always contains the day of the last change.

    When you wanted multiple data loads per day, better practice with ‘Timestamp’ through safety intervals. Thank you.

    -

    mgande

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Oct 18, 2016 at 11:10 AM

    There would be no issues. Even if there are some data inclusive of what was loaded last time but the values would be different as delta operates based on the timestamp. So, unless there is a change in the same data, it wont be captured in the delta.

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Oct 18, 2016 at 01:32 PM

    You should not have any issue as your delta will run through the change log entries.

    Only any subsequent changes of delta records will be updated to target when the delta is repeated (upper/lower limits). All new records at the time of the run will be picked up anyways.

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Oct 18, 2016 at 03:02 PM

    Hi,

    you want run the multiple times run for a day can use the timestamp option.

    in cal day you mention lower limit 1 it will pick the records from last date.

    Thanks,

    Phani.

    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Oct 20, 2016 at 03:23 AM

    Hi,

    Please note TIMESTAMP is not available in base tables and we shall be using DSO as some of the records would be repeating.

    Kindly provide specific answers to below.

    If lower limit of 1 is defined

    RSA7-1 < Delta <= D

    [D = Day on which the BW load is run, (RSA7 -1) = means Day on which last successful delta completed in BW minus 1 ]

    If we are running the load on 09/19/2016 then

    09/17/2016 <Delta <=09/19/2016 – On 19th we are getting all records of 18th and partial records of 19th (Note - More records can be created on 19th )

    Now if we run the load on 09/20/2016 then

    09/18/2015 <Delta <=20 – On 20th we are getting all records of 19th and partial records of 20th (Note - more records can be created for 20th ).

    Are there any other specific issues that we will have after maintaining lower limits as above.

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.