challenge in the data load

[Former Member]

Hi BI Gurus,

There is a cube(with 6 years of data) and delta load is happening over it. Now, the client wants to add a new key figure to it. He needs data for the newly added keyfigure. Data must come via the new loads and also data must be updated for the earlier 6 years. The delta must not be disturbed at any cost. Here is my way of doing it, let me know if i am right or wrong.

Existing flow : Datasource ----> Standard DSO -----> Cube

This is planned to be done as a weekend activity. Stops all of the process chain running for the Infocube.

Create a duplicate cube with all the characteristics and keyfigures of the original cube. Do a cube copy via full load DTP.

Go to t-code RSMRT and remodel the cube by adding the keyfigure and setting its value to constant 0.

Add the same keyfigure to the DSO ( There is no remodeling required for the DSO hence we can add it to the "data field" directly).

Go to the ECC, lock the system to avoid any transactions by the clients. Then clear the queue ( records from LBWQ & SM13 )

Clear the delta queue by running the Infopackages multiple times until you get zero records.

Enhance the datasource and Replicate the datasource to the BI side.

In the ECC, Go to tcode LBWG and delete the setup tables then fill the "set up" table using the OLI7BW.

Create a "repair full request" infopackage to pull all the records from the "setup" tables to the PSA ( no selction is specified and hence all of the records for the 6 years comes to the PSA. I don't know how long this is going to run and how successful its going to be !!!! )

Change the transformation of the DSO and link the newly added field in the datasource/PSA to the newly added keyfigure in the DSO.

Load the data to the DSO via DTP,the data that was created via the "repair full request" comes to the DSO. After the DTP load has become successful, because of the overwrite property of the DSO, only the latest records exist in the DSO and now the new keyfigure contains data for earlier and current loads.

Now change the transformation between the DSO and the Cube, set all the old keyfigures to "no transformation" and just link the newly added keyfigure in the cube with the same keyfigure of the DSO.

Load the data to the cube via DTP and immediately compress the data. Now the newly added keyfigure will contain all of the data for the previous loads too.

Now change the transformation between the DSO and cube,remove all "no transformation" which were set earlier, for all of the old keyfigures and revert them to their original setting.

Remove the "repair full request"check box of the Infopackage.

Remove the locks set in the ECC side.

Enable "process chains" again.

Is this fine or am i missing something?

[ravi_chandra3 · Accepted solution]

Hi sapbi ,

Do one thing , don't disturb that flow because delta records might be corrupted . Better to create the another IO or IC and add the primary characteristic and the newly added keyfigure only and now u have 2 infoproviders 1 infocube for the old data and the other with the newly added keyfigure and now create the mulitprovider on the top of these 2 infoprovider. So ur delta won't get corrupted.

Hope u got it,

Regards,

RaviChandra

[Former Member · Accepted solution]

Hi Ravi,

Thanks for the update. I wish to create a separate infoproviders with the newly added KF alone and then use multiprovider. But please let me know, whether the steps that i have mentioned by me in the first post is right?

[former_member188080]

Hi,

Delta will happen only for current period..

you are having data say for 5-6 years..

better to create new data flow or copy the old cube and keep it..

next is delete the original cube data..

load first the latest or current fiscal year period data provided you should have all the options.for time period,yera selection etc..

once this is done you can load remaining fiscal year period full repair..

then you go back 1 year then another and so on all full repair..

Your idea of remodelling u can try but I am not sure how successful it will be ..it might work or might not...so keep alternative plan ready may be..

Thanks and regards

Kiran

[Former Member]

Thanks, still i wanted to know whether the steps given by me in the first post is right or wrong.

is there anything to be modified in those steps.

I am not going to implement it.

But I want to know if its right or wrong because out of curiosity 🙂 🙂

[ravi_chandra3]

Hi

All those steps are perfect , but in case if u lost delta records then the large amount of data will be lost. So as per my convienience i would suggest this method.Because in that step u are not modifying any flow .

Hope u got it,

Regards,

RaviChandra

[Former Member]

Thanks to all. Closing the thread now.

[Former Member]

If you are so unsure of the loads, try using an alternate data flow to load the data populating the key fields & a this 'new' field, then join these both using a MP/Infoset? This solves major burden worrying reload of 6 years data isn't it? I think max effort for this would be a CMOD extractor code to be written in R/3 or a if using generic ext, then creation of d/source with particular fields. A majority of the future field additions can be isolated this manner, since efforts to redo the entire the 6 years loads can possibly take >2* days. Hope that helps. Thanks, Arun Bala

(*depends on your architecture, hoping it has a reduced TCO).

[sowdip]

I agree with Arivind, if you are confident that you can load all the 6 years of data during weekend period.

Go ahead and flush out all the data in data targets , reload full and initialize delta w/o data.

Rgds,

Prashanth

[Former Member]

Hi,

Correct me if i am wrong.

Since ur enhancing the data-targets with new key-figure every where including datasource.

Then why  to go with repair-full request.

Go for a full load by deleting all the data from different stages.

Regards,

Aravind.

[Former Member]

Hi Aravind,

The Infopackage is delta enabled. If change it to "full" from "delta", then definitely these is a possibility of "delta"getting corrupted. Then we need to do a "re init", which is not desired by client. He wants the existing "delta" not to be disturbed.