Skip to Content

Data Services 4.2.9 performance on Windows Server 2016

We are testing an upgrade to DataServices 4.2.9 on a new Windows Server 2016 installation.

We are seeing some really disappointing performance numbers which seem to be tied to the Windows Server 2016 OS.

For comparison, we set up two identical servers in our VMware environment.

-- 12 cores
-- 48 GB RAM

One has Windows Server 2012 R2 installed, while the other has Windows Server 2016 installed.

A local install of IPS 4.2.4 is installed on each system, with Data Services 4.2.9 installed after that.

The same repository was upgraded into each installation. So this is a shared nothing install -- but everything is the same except the OS.

I then ran a job that creates a local Persistent Cache copy of a database - which contains ~4.5 million rows.

I then run a series of jobs that use this Persistent Cache database as a candidate lookup selector in a Match transform. I'm running the same input data files against both servers, and I'm running a test of 48 jobs against each server.

The results are dramatically different. The Windows 2012 R2 system completes all jobs in 25 minutes and 43 seconds. With an average of 32.2 seconds per job.

The Windows 2016 system takes a total of 1 hour 58 minutes 36 seconds to complete the same set of jobs. With an average of 2 minutes 28.3 seconds per job.

This is almost a 4X performance decrease!

If I turn off the Persistent Cache and instead query the SQL Database directly for the candidate selection process in the Match transform, the performance numbers are very close. Still a little slower than the locally cached copy running on Windows 2012, but still similar to each other.

Has anyone else noticed this huge performance difference with any of transforms or data sources? Are there any settings in Windows or Data Services that I should check, or modify?

I'm disappointed that I might need to stand up a new server that's running an OS already 5 years old, but the performance numbers are making that decision for me at this point.

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

2 Answers

  • Best Answer
    Jun 30, 2017 at 03:10 PM

    Hi Arun,

    Thanks for the reply.

    The Persistent Cache data was recreated in Data Services 4.2.9.

    The issue appears to be that in the default installation of Windows Server 2016 the Windows Defender Features are installed. This is killing the performance of the local persistent cache data file.

    I have confirmed that removing the Windows Defender Features from the Windows Server 2016 instance resolves the performance issue that I was experiencing.

    Add comment
    10|10000 characters needed characters exceeded

    • Hi Richard,

      This is good information. I will create a KBA to capture this in case others run into this sort of performance issue. Curious, do you see this same behavior with older DS versions on Windows 2016 or was SP9 the first version you tested with Windows 2016? Thanks.

      Doug

  • Jun 16, 2017 at 06:37 AM
    -1

    Richard,

    Your deployment looks OK and Windows 2016 Server is also supported on DS 4.2 SP09

    Could you please check the following details on the Windows 2016 Server box

    1. Required Service Packs and Patches for Windows Server 2016

    Microsoft publishes updates periodically which enhance the stability and performance of their Windows Server products. Therefore it is mandatory to run Windows Update before starting any installation activities.

    2. Which is the version of JVM on the Windows 2016 Server

    3. Which is the Repository Database(with version) and Reporting Database installed on the Data Services Job Server. Check the PAM Guide for SP09

    PAM

    4. Did you do a thorough regression testing of the Jobs after upgrade to 4.2 SP09?

    5. What was the previous version of Data Services before you upgraded to 4.2 SP09

    Excerpt from DS Upgrade Guide:

    Persistent cache data used in earlier versions of Data Services cannot be used in Data Services 4.2.

    Refer Upgrade Guide

    DS Upgrade Guide

    Regards

    Arun Sasi

    Add comment
    10|10000 characters needed characters exceeded