Skip to Content
0

Data Services 4.2.9 performance on Windows Server 2016

Jun 15, 2017 at 08:12 PM

198

avatar image

We are testing an upgrade to DataServices 4.2.9 on a new Windows Server 2016 installation.

We are seeing some really disappointing performance numbers which seem to be tied to the Windows Server 2016 OS.

For comparison, we set up two identical servers in our VMware environment.

-- 12 cores
-- 48 GB RAM

One has Windows Server 2012 R2 installed, while the other has Windows Server 2016 installed.

A local install of IPS 4.2.4 is installed on each system, with Data Services 4.2.9 installed after that.

The same repository was upgraded into each installation. So this is a shared nothing install -- but everything is the same except the OS.

I then ran a job that creates a local Persistent Cache copy of a database - which contains ~4.5 million rows.

I then run a series of jobs that use this Persistent Cache database as a candidate lookup selector in a Match transform. I'm running the same input data files against both servers, and I'm running a test of 48 jobs against each server.

The results are dramatically different. The Windows 2012 R2 system completes all jobs in 25 minutes and 43 seconds. With an average of 32.2 seconds per job.

The Windows 2016 system takes a total of 1 hour 58 minutes 36 seconds to complete the same set of jobs. With an average of 2 minutes 28.3 seconds per job.

This is almost a 4X performance decrease!

If I turn off the Persistent Cache and instead query the SQL Database directly for the candidate selection process in the Match transform, the performance numbers are very close. Still a little slower than the locally cached copy running on Windows 2012, but still similar to each other.

Has anyone else noticed this huge performance difference with any of transforms or data sources? Are there any settings in Windows or Data Services that I should check, or modify?

I'm disappointed that I might need to stand up a new server that's running an OS already 5 years old, but the performance numbers are making that decision for me at this point.

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

2 Answers

Best Answer
Richard West Jun 30, 2017 at 03:10 PM
0

Hi Arun,

Thanks for the reply.

The Persistent Cache data was recreated in Data Services 4.2.9.

The issue appears to be that in the default installation of Windows Server 2016 the Windows Defender Features are installed. This is killing the performance of the local persistent cache data file.

I have confirmed that removing the Windows Defender Features from the Windows Server 2016 instance resolves the performance issue that I was experiencing.

Show 1 Share
10 |10000 characters needed characters left characters exceeded

Hi Richard,

This is good information. I will create a KBA to capture this in case others run into this sort of performance issue. Curious, do you see this same behavior with older DS versions on Windows 2016 or was SP9 the first version you tested with Windows 2016? Thanks.

Doug

0
Arun Sasi Jun 16, 2017 at 06:37 AM
-1

Richard,

Your deployment looks OK and Windows 2016 Server is also supported on DS 4.2 SP09

Could you please check the following details on the Windows 2016 Server box

1. Required Service Packs and Patches for Windows Server 2016

Microsoft publishes updates periodically which enhance the stability and performance of their Windows Server products. Therefore it is mandatory to run Windows Update before starting any installation activities.

2. Which is the version of JVM on the Windows 2016 Server

3. Which is the Repository Database(with version) and Reporting Database installed on the Data Services Job Server. Check the PAM Guide for SP09

PAM

4. Did you do a thorough regression testing of the Jobs after upgrade to 4.2 SP09?

5. What was the previous version of Data Services before you upgraded to 4.2 SP09

Excerpt from DS Upgrade Guide:

Persistent cache data used in earlier versions of Data Services cannot be used in Data Services 4.2.

Refer Upgrade Guide

DS Upgrade Guide

Regards

Arun Sasi

Share
10 |10000 characters needed characters left characters exceeded