on 03-23-2018 12:17 PM
Hi guys,
I specified to keep rejected records in the standard /CPMB/LOAD_DELTA_IP 10.1 Data Manager package.
Delta's load fine, however when there is an error, and the keep rejected records flag has been set as '1', the rejected records are not picked up (after correcting the error that caused the error in the 1st place).
There is a SAP Note 2449327 - Rejected data that are stored the last time are
missing, that was issued in March 2017 that describes my problem exactly, but the correction i.e. to specify VALIDATERECORDS = NO in the transformation file has no effect.
I see someone else asked this question around a year ago but there was no positive response.
Please let me know, if others are using the /CPMB/LOAD_DELETA_IP and where there are rejected records, that these rejected records are then picked upon a subsequent BPC Delta load.
Thanks for any help.
Kind regards
Mark Dean
Thank you Peter, for the great summary of what happens and when, regarding BPC Rejected Delta records operation.
I have now got it working. Thanks Vadim & Peter
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Mark,
I've worked with this feature and tested it in internal test environments and for me it worked.
The feature 'IF Keep Rejected Data' is an enhancement of the loading of transaction data.
As it was initially designed, rejected data meant that the data was invalid and shouldn't be kept any longer. We just output the rejected data to end user in this case.
However, customers asked us to provide a feature to keep rejected data at write back step:
a). Process chain /CPMB/LOAD_DELTA_IP is a special chain, the current package execution is based on result of the last package execution. b). There is a requirement that if data is rejected due to a locked work status for one data region, then it should be possible to unlock this data region and reload the rejected data.
Because the package execution is based on previous execution, we provide the enhancement to keep the data which are rejected at WRITE BACK step for the next package running. (Work Status checking is at write back phase)
Execution: If you check package log, you will notice there are 3 tasks: Task1: SOURCE DELTA INFOPROVIDER ---> This task is used to load the stored data (including data which are rejected the last time). If load data finishes with success, it will read data from request which are loaded the last time; Otherwise, It will finish with error and will not do the next steps.
Please note that the step of loading the stored data is achieved by calling WRITE BACK. So if the data still gets rejected, it will be stored for the next package execution.
Task2: CONVERT --> If task 1 has new data on top of the data which is stored the last time, this data has to be converted as per your transformation file and conversion file.
Task3: APPEND LOAD DELTA INFOPROVIDER --> This task calls the write back again to try and write the new data which has been converted.
If there are any rejected data at this step, they will be stored for the next package execution and will be processed at the first task.
NOTE: If 'VALIDATERECORDS=YES' in the transformation file, this means you would like to validate data at Task2 (CONVERT Task). In this case, data will be rejected at CONVERT step. We don't keep this kind of rejected data. If you change keyword to 'VALIDATERECORDS=NO', the data will not be validated at CONVERT task. In this case, data will be put to TASK3 (APPEND LOAD DELTA INFOPROVIDER TASK). This TASK will reject this kind of data by calling Write back and they will be stored.
So if any data is rejected in the CONVERT task of your Delta Load log this data will not be loaded again on subsequent executions.
Please confirm in your case that the rejected data is correctly rejected in the APPEND task. If so, once you correct the reason for the rejection the subsequent execution should load this data.
-Peter
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
"specify VALIDATERECORDS = NO in the transformation file has no effect" - please describe your test in details!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
15 | |
3 | |
2 | |
1 | |
1 | |
1 | |
1 | |
1 | |
1 | |
1 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.