Hello,
Welcome to my new blog.
In my previous blog I had explained how you can use ‘Migrate your Data – Migration Cockpit’ & SAP BODS as ETL Platform for SAP Data Migration.
Now will try to explain, how can we increase the performance of Migration Cockpit (Migrate Data Using Staging Tables/File) or How can we process high volume of data efficiently while migration.
Let’s Discuss !!
As we know, Migration Cockpit has two migration approaches:
Migration approach by 'File' is incorporated within the Staging table approach.
i.e. Template files are provided for every migration object and you can use these templates to fill the data & same will be inserted in staging tables when you will upload the file in Migration Cockpit.
So, once data is there in staging table and prepared to be loaded, we can consider below points to increase job performance.
Migration Cockpits offers the possibility to parallelize the activities, simulation and migration by using several data transfer jobs.
Data transfer jobs are responsible for transferring the data from staging tables to target SAP system. The default number of background jobs for a project (Maximum number of Jobs for project) is 15 in SAP S/4 HANA, though it can differ from environment to environment based on configuration. For single migration objects, the default number of data transfer jobs is 1.
You should consider increasing the number of data transfer jobs in the following situations:
Avoid situation where many people are working in parallel on separate migration projects, or where one user triggers multiple activities for several projects in the same data migration context. Running multiple objects in parallel can flood the job queue.
It delivers better results to migrate one object with a full number of jobs, complete the data migration and then start with the migration of the next object.
In "Migrate Your Data – Migration Cockpit" you have a option "Maximum number of Jobs for project" which denotes the number of background jobs available and can be assigned to your project.
This value is editable but you can not exceed this more than a certain number which comes from parameter rdisp/wp_no_btc. This is nothing but number of batch work processes (background processing) in your system.
If it is required to increase this value in order to get more performance, you can check with BASIS team for further assistance.
If you are migrating data to SAP S/4HANA, the default size limit for each uploaded XML file is 100MB, but it depends on parameter icm/HTTP/max_request_size_KB, which controls the size of the http request. You can increase the size limit for each uploaded XML file to 160MB by changing this system parameter, which has default value 102400 kb (100MB).
If required, you can zip several files together, but note that the combined size of all the XML files you want to add to the zip file must also not exceed 160MB.There is no such limit in case you are migrating using "Staging tables".
For further information you can check 2719524 - SAP S/4HANA Migration Cockpit: XML template size limits - SAP for Me
During the production load/Mock load, there should be no other active background processes (jobs) in the system - besides the processes for the migration itself. Check thoroughly which other processes are necessary. Postpone all processes which are not completely necessary. You can check with BASIS for assistance.
Test cycles are executed in Sandbox/Quality/Val environment which may not have have performance as good as Production. This makes it difficult to get correct estimation of load time, parameter configuration and other aspects. Therefore, it is recommended that at least one mock load should be executed in a production like environment.
That brings me to the end of this blog !!
If you have anything to add on top of it, kindly share feedback or thoughts in comments. Your contribution will be highly appreciated.
For further knowledge on this you can check :
Welcome to my new blog.
In my previous blog I had explained how you can use ‘Migrate your Data – Migration Cockpit’ & SAP BODS as ETL Platform for SAP Data Migration.
Now will try to explain, how can we increase the performance of Migration Cockpit (Migrate Data Using Staging Tables/File) or How can we process high volume of data efficiently while migration.
Let’s Discuss !!
As we know, Migration Cockpit has two migration approaches:
- Migrate Data Using Staging Tables
- Migrate Data Directly from SAP System
Migration approach by 'File' is incorporated within the Staging table approach.
i.e. Template files are provided for every migration object and you can use these templates to fill the data & same will be inserted in staging tables when you will upload the file in Migration Cockpit.
So, once data is there in staging table and prepared to be loaded, we can consider below points to increase job performance.
→ A. Job Management
Migration Cockpits offers the possibility to parallelize the activities, simulation and migration by using several data transfer jobs.
- Open "Migrate Your Data – Migration Cockpit" App in Fiori.
- Open the project where you would like to modify the number of background jobs for the data transfer.
- Select “Job Management” to adjust job settings.
- Click "Edit" and in the field "Number Jobs" enter number of jobs higher than 1 to perform job parallelization of the data transfer and it will improve the performance of the data migration.
- Note that if an activity has already started for a migration object, then increasing or decreasing the number of jobs for the migration object will have no effect on that activity.
Data transfer jobs are responsible for transferring the data from staging tables to target SAP system. The default number of background jobs for a project (Maximum number of Jobs for project) is 15 in SAP S/4 HANA, though it can differ from environment to environment based on configuration. For single migration objects, the default number of data transfer jobs is 1.
You should consider increasing the number of data transfer jobs in the following situations:
- You have a lot of data to transfer.
- The migration object is complex with multiple staging tables.
- Using one job results in an unacceptable data transfer time.
♦ Precaution
Avoid situation where many people are working in parallel on separate migration projects, or where one user triggers multiple activities for several projects in the same data migration context. Running multiple objects in parallel can flood the job queue.
It delivers better results to migrate one object with a full number of jobs, complete the data migration and then start with the migration of the next object.
In "Migrate Your Data – Migration Cockpit" you have a option "Maximum number of Jobs for project" which denotes the number of background jobs available and can be assigned to your project.
This value is editable but you can not exceed this more than a certain number which comes from parameter rdisp/wp_no_btc. This is nothing but number of batch work processes (background processing) in your system.
If it is required to increase this value in order to get more performance, you can check with BASIS team for further assistance.
→ B. Template size limit for 'File' approach
If you are migrating data to SAP S/4HANA, the default size limit for each uploaded XML file is 100MB, but it depends on parameter icm/HTTP/max_request_size_KB, which controls the size of the http request. You can increase the size limit for each uploaded XML file to 160MB by changing this system parameter, which has default value 102400 kb (100MB).
If required, you can zip several files together, but note that the combined size of all the XML files you want to add to the zip file must also not exceed 160MB.There is no such limit in case you are migrating using "Staging tables".
For further information you can check 2719524 - SAP S/4HANA Migration Cockpit: XML template size limits - SAP for Me
→ C. Cutover/Mock cycle planning
During the production load/Mock load, there should be no other active background processes (jobs) in the system - besides the processes for the migration itself. Check thoroughly which other processes are necessary. Postpone all processes which are not completely necessary. You can check with BASIS for assistance.
→ D. Production like environment
Test cycles are executed in Sandbox/Quality/Val environment which may not have have performance as good as Production. This makes it difficult to get correct estimation of load time, parameter configuration and other aspects. Therefore, it is recommended that at least one mock load should be executed in a production like environment.
That brings me to the end of this blog !!
If you have anything to add on top of it, kindly share feedback or thoughts in comments. Your contribution will be highly appreciated.
For further knowledge on this you can check :
- SAP Managed Tags:
