Data Services 4.2.9 performance on Windows Server 2016

[richard_west]

We are testing an upgrade to DataServices 4.2.9 on a new Windows Server 2016 installation.

We are seeing some really disappointing performance numbers which seem to be tied to the Windows Server 2016 OS.

For comparison, we set up two identical servers in our VMware environment.

-- 12 cores
-- 48 GB RAM

One has Windows Server 2012 R2 installed, while the other has Windows Server 2016 installed.

A local install of IPS 4.2.4 is installed on each system, with Data Services 4.2.9 installed after that.

The same repository was upgraded into each installation. So this is a shared nothing install -- but everything is the same except the OS.

I then ran a job that creates a local Persistent Cache copy of a database - which contains ~4.5 million rows.

I then run a series of jobs that use this Persistent Cache database as a candidate lookup selector in a Match transform. I'm running the same input data files against both servers, and I'm running a test of 48 jobs against each server.

The results are dramatically different. The Windows 2012 R2 system completes all jobs in 25 minutes and 43 seconds. With an average of 32.2 seconds per job.

The Windows 2016 system takes a total of 1 hour 58 minutes 36 seconds to complete the same set of jobs. With an average of 2 minutes 28.3 seconds per job.

This is almost a 4X performance decrease!

If I turn off the Persistent Cache and instead query the SQL Database directly for the candidate selection process in the Match transform, the performance numbers are very close. Still a little slower than the locally cached copy running on Windows 2012, but still similar to each other.

Has anyone else noticed this huge performance difference with any of transforms or data sources? Are there any settings in Windows or Data Services that I should check, or modify?

I'm disappointed that I might need to stand up a new server that's running an OS already 5 years old, but the performance numbers are making that decision for me at this point.

[richard_west · Accepted solution]

Hi Arun,

Thanks for the reply.

The Persistent Cache data was recreated in Data Services 4.2.9.

The issue appears to be that in the default installation of Windows Server 2016 the Windows Defender Features are installed. This is killing the performance of the local persistent cache data file.

I have confirmed that removing the Windows Defender Features from the Windows Server 2016 instance resolves the performance issue that I was experiencing.

[former_member213714 · Accepted solution]

Hi Richard,

This is good information. I will create a KBA to capture this in case others run into this sort of performance issue. Curious, do you see this same behavior with older DS versions on Windows 2016 or was SP9 the first version you tested with Windows 2016? Thanks.

Doug

[former_member198401]

Richard,

Your deployment looks OK and Windows 2016 Server is also supported on DS 4.2 SP09

Could you please check the following details on the Windows 2016 Server box

1. Required Service Packs and Patches for Windows Server 2016

Microsoft publishes updates periodically which enhance the stability and performance of their Windows Server products. Therefore it is mandatory to run Windows Update before starting any installation activities.

2. Which is the version of JVM on the Windows 2016 Server

3. Which is the Repository Database(with version) and Reporting Database installed on the Data Services Job Server. Check the PAM Guide for SP09

PAM

4. Did you do a thorough regression testing of the Jobs after upgrade to 4.2 SP09?

5. What was the previous version of Data Services before you upgraded to 4.2 SP09

Excerpt from DS Upgrade Guide:

Persistent cache data used in earlier versions of Data Services cannot be used in Data Services 4.2.

Refer Upgrade Guide

DS Upgrade Guide

Regards

Arun Sasi

[mirunalini_seshadri]

I have a general question regarding Persistent cache. Since you are using it, i would like to know if this is an issue for you too. My environment is Data Service 4.2 sp 13 on windows 2016 server. I am having issues mapping any date, datetime field to a persistent cache table. As soon as I save the job, it crashes the designer. I am doing an upgrade from 4.1 to 4.2. In 4.1 everything works fine. The jobs that use persistent cache with dates are not working in 4.2

Thank you for your time.

Regards

Miru

[former_member598107]

mirunalini.seshadri

Please refer SAP Note. Please check if Windows Defender on the server where DS 4.2 Job server is installed.

https://launchpad.support.sap.com/#/notes/2497752

Regards

Arun Sasi

[mirunalini_seshadri]

FYI - this is a Data Services software bug. It will be fixed in Data Service 4.2 SP 13 patch 4 and SP14 Patch4.

Thanks

Miru