Skip to Content
avatar image
Former Member

How increase performance of SAP DataServices?

Hi,

About the System:

The system having 30 countries.

Each country processing 50k records.

Data flows are using Data Clean and GAC transforms.

Current setup is:

BI Platform 4.2 SP03

Data Service 4.2SP 08

We have one job server and all country repo added to that. all the country job running at the same time.

Any suggestion what should the setup to increase performance? as the above setup taking too much time to process the data.

Regards,

Palash

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

2 Answers

  • Oct 16, 2017 at 12:15 PM

    Are you aware of the complexity of the DQ transforms? By nature, the throughput of DQ jobs is much lower than the one of DI jobs. DQ jobs are often CPU-bound. You can speed up the processing by adding more CPU power to your DS box.

    Add comment
    10|10000 characters needed characters exceeded

    • No, that won't help. The jobserver performs housekeeping activities only, it typically doesn't consume much CPU time. That's spent in the al_engine processes that are spawned when you execute a job. And with your hardware, you cannot actively run more than 8 of them at the time.

  • Oct 22, 2017 at 07:12 AM

    Hi Palash,

    I totally agree with Dirk comments let me add few more things in that.

    If you want to validate the CPU & Memory usage either go for stats from operating system team or else you can see the same status usage via Data Services Management Console -> Job Execution Stats-> Each Process Wise Usage of CPU/Memory and data load

    Also refer below link that will help you to investigate in detail

    Pageable Cache and DSConfig

    https://wiki.scn.sap.com/wiki/display/EIM/Pageable+Cache+and+DSConfig

    Row Per Commit & Array Fetch Size in BODS

    https://blogs.sap.com/2015/07/13/row-per-commit-array-fetch-size-in-bods/

    Degree of Parallelism

    https://wiki.scn.sap.com/wiki/display/EIM/Degree+of+Parallelism

    Monitor Sample Rate

    https://wiki.scn.sap.com/wiki/display/EIM/Monitor+Sample+Rate

    I hope this will help!!!!

    Regards,

    Daya Jha

    Add comment
    10|10000 characters needed characters exceeded