How increase performance of SAP DataServices?

[former_member335073]

Hi,

About the System:

The system having 30 countries.

Each country processing 50k records.

Data flows are using Data Clean and GAC transforms.

Current setup is:

BI Platform 4.2 SP03

Data Service 4.2SP 08

We have one job server and all country repo added to that. all the country job running at the same time.

Any suggestion what should the setup to increase performance? as the above setup taking too much time to process the data.

Regards,

Palash

[DayaJha]

Hi Palash,

I totally agree with Dirk comments let me add few more things in that.

If you want to validate the CPU & Memory usage either go for stats from operating system team or else you can see the same status usage via Data Services Management Console -> Job Execution Stats-> Each Process Wise Usage of CPU/Memory and data load

Also refer below link that will help you to investigate in detail

Pageable Cache and DSConfig

https://wiki.scn.sap.com/wiki/display/EIM/Pageable+Cache+and+DSConfig

Row Per Commit & Array Fetch Size in BODS

https://blogs.sap.com/2015/07/13/row-per-commit-array-fetch-size-in-bods/

Degree of Parallelism

https://wiki.scn.sap.com/wiki/display/EIM/Degree+of+Parallelism

Monitor Sample Rate

https://wiki.scn.sap.com/wiki/display/EIM/Monitor+Sample+Rate

I hope this will help!!!!

Regards,

Daya Jha

[former_member187605]

Are you aware of the complexity of the DQ transforms? By nature, the throughput of DQ jobs is much lower than the one of DI jobs. DQ jobs are often CPU-bound. You can speed up the processing by adding more CPU power to your DS box.

[former_member335073]

Hi Dirk,

Thanks for the reply..

We have 9CPUs with 80GB RAM in the AIX 7.1 server.

Regards,

Palash

[former_member187605]

You cannot possibly run 30 parallel jobs on such a configuration. I don't think RAM will be an issue. But the max number of simultaneous DQ processes will be 8 only.

[former_member335073]

if we split the countries to different job server, will it improve the performance?

Currently 1 Jobserver having all countries repo attached, if we create jobserver per 8 or 10 countries, will it help?

As per the requirement all country job should start at the same time. hence we can't change the timing of the jobs.

Any suggestion?

Regards,

Palash

[former_member187605]

No, that won't help. The jobserver performs housekeeping activities only, it typically doesn't consume much CPU time. That's spent in the al_engine processes that are spawned when you execute a job. And with your hardware, you cannot actively run more than 8 of them at the time.