Skip to Content
0

Generic Extractor based upon calday - delta logic - lower limits

Oct 18, 2016 at 03:36 AM

560

avatar image
Former Member

Hi Experts,

We have created Generic Extractor with generic delta based upon 'CALDAY' and upper limit of 1. Now as part of our requirement we have to run this multiple times in a day so we are thinking about maintaining lower limits as below so that we can extract it multiple times in day.

As per my understanding it should behave in the following way after maintaining lower limit

If lower limit of 1 is defined

RSA7-1 < Delta <= D

[D = Day on which the BW load is run, (RSA7 -1) = means Day on which last successful delta completed in BW minus 1 ]

If we are running the load on 09/19/2016 then

09/17/2016 <Delta <=09/19/2016 – On 19th we are getting all records of 18th and partial records of 19th (Note - More records can be created on 19th )

Now if we run the load on 09/20/2016 then

09/18/2015 <Delta <=20 – On 20th we are getting all records of 19th and partial records of 20th (Note - more records can be created for 20th ).

Are there any other specific issues that we will have after maintaining lower limits.

Thanks

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

6 Answers

avatar image
Former Member Oct 18, 2016 at 03:16 PM
1

I think there is some confusion about the lower limits and upper limits.

If you have the upper limit then the delta upper limit will be subtracted by your set upper limit. For example, if you have upper limit at 1 and you are extracting data on 10/19/2016 and your last successful delta is on 10/18/2016 then system will set the upper limits by subtracting 1 from the day of extraction. So system will set the limits as 10/18/2016 - 10/18/2016. So it will miss the data of the day of extraction. So do not have upper limit when you want to extract multiple times a day.

Follow what Phani said - No upper Limit and Lower limit as 1. You should be fine then for extracting multiple times a day.

Also you will have repeated data in the datasource so better use DSO.

Thanks

Amit

Share
10 |10000 characters needed characters left characters exceeded
avatar image
Former Member Oct 18, 2016 at 07:51 AM
0

Calday : always contains the day of the last change.

When you wanted multiple data loads per day, better practice with ‘Timestamp’ through safety intervals. Thank you.

-

mgande

Share
10 |10000 characters needed characters left characters exceeded
avatar image
Former Member Oct 18, 2016 at 11:10 AM
0

There would be no issues. Even if there are some data inclusive of what was loaded last time but the values would be different as delta operates based on the timestamp. So, unless there is a change in the same data, it wont be captured in the delta.

Share
10 |10000 characters needed characters left characters exceeded
Francis Yesudas Oct 18, 2016 at 01:32 PM
0

You should not have any issue as your delta will run through the change log entries.

Only any subsequent changes of delta records will be updated to target when the delta is repeated (upper/lower limits). All new records at the time of the run will be picked up anyways.

Share
10 |10000 characters needed characters left characters exceeded
avatar image
Former Member Oct 18, 2016 at 03:02 PM
0

Hi,

you want run the multiple times run for a day can use the timestamp option.

in cal day you mention lower limit 1 it will pick the records from last date.

Thanks,

Phani.

Share
10 |10000 characters needed characters left characters exceeded
avatar image
Former Member Oct 20, 2016 at 03:23 AM
0

Hi,

Please note TIMESTAMP is not available in base tables and we shall be using DSO as some of the records would be repeating.

Kindly provide specific answers to below.

If lower limit of 1 is defined

RSA7-1 < Delta <= D

[D = Day on which the BW load is run, (RSA7 -1) = means Day on which last successful delta completed in BW minus 1 ]

If we are running the load on 09/19/2016 then

09/17/2016 <Delta <=09/19/2016 – On 19th we are getting all records of 18th and partial records of 19th (Note - More records can be created on 19th )

Now if we run the load on 09/20/2016 then

09/18/2015 <Delta <=20 – On 20th we are getting all records of 19th and partial records of 20th (Note - more records can be created for 20th ).

Are there any other specific issues that we will have after maintaining lower limits as above.

Show 1 Share
10 |10000 characters needed characters left characters exceeded
Former Member

There will be no issues (or missed data) if you maintain lower limit as 1 and NO Upper Limit.

Thanks

Amit

0