SAP on Azure : Highly Available File Share Cluster...

former_member642947 · ‎08-04-2020

Introduction

This blog provides step by step guide for Highly Available (HA) File Share cluster on Windows Operating System using SIOS Datakeeper which will provide ‘sapmnt’ and ‘trans’ individual/separate fileshare for multiple SAP environments in the landscape. This will cover all the aspects of single of failure (SPOF) for SAP Application layer. Blog will skip the setup of Database layer of SAP environment.

Purpose

To build HA environment for deploying SAP landscape in Azure we need to have SMB/NFS HA Fileshare environment which will host the common filesystems(like ‘sapmnt’ and ‘trans’) for all the environments. This will help to achieve the availability SLA and optimization of the landscape design. We can either use availability set or availability zone to deploy the Fileshare cluster based on the availability requirements. This blog covers one of the multiple approaches for deploying the fileshare for SAP systems in Windows environment.

System Design

Below is the example scenario design of the Production SAP landscape for an organization. Windows fileshare cluster is the central location for hosting the common filesystems for all/multiple SIDs and they are hosted (and can be maintained) independent of each other.

In this blog, as an example lets us consider that we have four SAP SID in our landscape which have HA setup for ASCS and database. There is one common SMB File share cluster which will hosting four separate SMB fileshare, each of which have separate virtual hostname and separate storage associated with it. Only specific mountpoints can be mounted to the relevant ASCS and application servers.

File share names:

\\sapglobalPR1\sapmnt

\\sapglobalPR2\sapmnt

\\sapglobalC01\sapmnt

\\sapglobalC02\sapmnt

Each fileshare will have separate role in WFSC cluster and separate drive for the share. Drives are replicated(sync) between the cluster nodes using SIOS Datakeeper. Internal Load balancers are used to create virtual IPs of the fileshares.

Solution meets the requirements of SAP Fileshare as described in SAP Note 2492395 (example 2) in which it states that we can’t change the name ‘sapmnt’ for SAP Fileshare.

Following the details of the reference setup which are described in the blog:

Hostname	Description	OS Version	Drive Letters
azwinfileshare1	SMB Fileshare Node1	Windows 2019	I: (PR1), J: (PR2), M: (C01), N: (C02)
azwinfileshare2	SMB Fileshare Node2	Windows 2019	I: (PR1), J: (PR2), M: (C01), N: (C02)
sapglobalPR1	Virtual hostname for PR1 fileshare	-	-
sapglobalPR2	Virtual hostname for PR2 fileshare	-	-
sapglobalC01	Virtual hostname for C01 fileshare	-	-
sapglobalC02	Virtual hostname for C02 fileshare	-	-

The setup has used Azure Zone features which provides 99.99% of VM availability. Same setup can be used for Availability Set based deployment which provides 99.95% of availability for the cluster VMs.

Solution can be extended to other Azure region for Disaster Recovery(DR) setup by setting up Azure Site Recovery (ASR) of the Fileshare VMs.

Overview of the Installation Steps

Overall steps for the deployment of the Fileshare cluster for SAP Application layer HA setup are :

Preform the preparatory steps.

Define the Internal Load balancer.

Deploy the WFSC Failover Cluster.

Setup and configure the SIOS DataKeeper Replication.

Setup the Fileshare as Failover Cluster Roles.

Setup SAP ASCS/ERS Cluster using Fileshare.

Utilize the Fileshare for SAP PAS Installations.

Optimization steps for SAP Environment using Fileshare.

Test the Cluster Setup

Preparations

Read the required Installation Guide, SAP Notes, SAP on Azure docs and download the installation media.

Deploy the VM in Availability Zones as per the system architecture and Choose Operating System as Windows Server for Application Layer.

Add equal number & size of disks to SMB File share VMs.

Join the VMs to the Domain.

Define Page File in Temp Disk (D Drive).

Check that necessary Ports are open in Windows firewall.

Disable the Continuous Availability Feature in Windows using the instructions in this link.

Define the Internal Load Balancer

Create a standard Internal load balancer and define the setup as below:

Front-end IP	Backend Pool	Health probe port	Load balancing rule
10.20.10.15 (virtual IP for sapglobalPR1)	azwinfileshare1 and azwinfileshare2	62200	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)
10.20.10.16 (virtual IP for sapglobalPR2)		62205	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)
10.20.10.17 (virtual IP for sapglobalC01)		62210	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)
10.20.10.18 (virtual IP for sapglobalC02)		62215	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)

In case of using a Basic Load balancer in Availability Set based deployment, load balancing rules needs to created for port 445 and 5985.

Setup Windows Cluster

Login to the fileshare VMs as administrator

In the Add Role and Features Wizard, Install the .NET Framework 3.5 and Failover Clustering for both the nodes.

Set up the failover cluster by using Failover Cluster Manager. In Failover Cluster Manager, select Create Cluster, and then add only the name of the first cluster (node A). Do not add the second node yet; you add the second node in a later step.

Enter the network name (virtual host name) of the cluster.

Continue with next steps to create the cluster.

Note : If we are using Windows Server 2016, once cluster is created then we need to update the IP of the cluster (which also needs to be defined as A-record in AD/DNS) and then start the cluster.

Add Note2 to the cluster

Uncheck ‘Add all eligible storage to the cluster’

Continue with the steps to add the Node 2 to the cluster. Once Node2 is successfully added. Check that both the Nodes are showing status as ‘Up’.

Configure the cloud share witness by selecting the option to add quorum to cluster.

Enter the storage account details for the cloud witness.

Check the status of the cluster and cloud Witness as ‘Online’.

Change below cluster parameters thresholds so they adapt failover detection to conditions in Azure.

SameSubNetDelay = 2000

SameSubNetThreshold = 15

RouteHistoryLength = 30

We can see further details about these setting in the link.

Setup SIOS Datakeeper Replication

Create Domain User ‘DataKeeperSvc’ and add it in local admin group of both cluster nodes.

Use the dism.exe command-line tool on both nodes. For this type of installation, you need to access the SxS directory on the Windows installation media. At an elevated command prompt, enter this command:

Dism /online /enable-feature /featurename:NetFx3 /All /Source:installation_media_drive:\sources\sxs /LimitAccess

Start the Installation of the latest version of SIOS Datakeeper in both cluster nodes.

Note the information about Ports to be opened for SIOS Datakeeper.

Maintain the Service Account details
- Enter license details of SIOS DataKeeperRestart the fileshare VMs before starting the SIOS Datakeeper configuration.

Configure SIOS DataKeeper
- Open SIOS Datakeeper in Node 1, click on ‘connect’.
- Enter the IP Address of Fileshare Cluster Node 1
- Repeat the step to connect to Node 2.
- Check the server overview which shows four drives on both nodes.
- Create the Replication Job
- Select the source server and volume for replication
- Select the target server and volume
- Choose the mode of replication as ‘Synchornus’
- Add the shared volume to Windows cluster.
- Repeat the above steps to Create jobs for all the volumes.
- Check the status of replication.
- Check the shared volume status in the Failover Cluster

Configure Fileshares

Create New A Record in AD/DNS for all the virtual hostnames of Fileshare. IP address will be same as its defined as front-end IP of ILB.

Create Role in Failover cluster.

Select the Role Type as ‘File Server’.

Choose ‘File Server for General Use’

Define the name same as virtual hostname.

Choose the storage volume to be used for this fileshare. ‘sapmnt’ folder needs to be created as folder in this drive.

Role creation successfully completed.

Update the IP Address and Start the Role.

Update the probe port for the Cluster Role
- Check the current probe port (using powershell command) which will be ‘0’ as its yet to be defined.

Get-ClusterResource "IP Address 10.20.10.15" | Get-ClusterParameter

- Run the powershell commands to set the probe port

$ClusterNetworkName = “Cluster Network 1"



$IPResourceName = "IP Address 10.20.10.15"



$ILBIP = "10.20.10.15"



Import-Module FailoverClusters



Get-ClusterResource $IPResourceName | Set-ClusterParameter -Multiple @{Address=$ILBIP;ProbePort=62200;SubnetMask="255.255.255.255";Network=$ClusterNetworkName;EnableDhcp=0}

- Restart the Role to activate configuration
- Check the ProbePort configuration

Get-ClusterResource "IP Address 10.20.10.15" | Get-ClusterParameter

Repeat the above steps to create the Failover cluster roles for other Fileshares.

\\sapglobalPR2\sapmnt



\\sapglobalC01\sapmnt



\\sapglobalC02\sapmnt

Check the status of the Roles.

Create the ‘sapmnt’ share for each path.

SAP ASCS/ERS HA Setup with Fileshare

Below are the high level steps for setting up SAP ASCS/ERS HA setup on Windows Server using Fileshare. Please refer to reference section for links with more details.

Perform all preparations steps as described in the “prepare” section.

Create the loadbalancer for ASCS and ERS (e.g. for S/4 HANA).

Create front-end IP, backend-pool, health probe, and loadbalancing rule for ASCS/ERS cluster.

As an example, following will be ILB details for PR1 (which is S/4 HANA environment)

Front-end IP	Backend Pool	Health probe port	Load balancing rule
10.20.10.25 (ASCS Virtual IP)	azwinpr1ascs1 and azwinpr1ascs2	62300	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)
10.20.10.26 (ERS Virtual IP)	azwinpr1ascs1 and azwinpr1ascs2	62302	Enable HA Port, Enable Floating IP, Idle Timeout (30 Minutes)

In case of Basic loadbalancer, we need to define load balancing rule for all the allowed ports.

For ASCS : 32xx, 36xx, 39xx, 81xx, 5xx13, 5xx14, 5xx16

For ERS : 33xx, 5xx13, 5xx14, 5xx16

Windows-specific ports for every SAP system: 5985

For SAP Java environment, please refer to port requirement in SAP documentation.

Add A-record in AD/DNS for virtual hostname of ASCS and ERS (e.g. for S/4 HANA) virtual hostnames and assign the front-end IP defined in the ILB.

Login to ASCS/ERS VMs as administrator user.

Add registry entries on both cluster nodes of the SAP ASCS/SCS instance
- Entry 1

Path : HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters



Variable name : KeepAliveTime



Variable type : REG_DWORD (Decimal)



Value : 120000

- Entry 2

Path : HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters



Variable name : KeepAliveInterval



Variable type : REG_DWORD (Decimal)



Value : 120000

In the Add Role and Features Wizard, add “failover clustering” to both cluster nodes.

Create the Failover cluster by adding Node1. If OS is Windows 2016, update the IP of the cluster and then start the cluster. IP needs to be defined against the cluster name in the AD/DNS.

Add Node2 to the cluster.

Add the cloud witness to the cluster.

Change cluster thresholds so they adapt failover detection to conditions in Azure.

SameSubNetDelay = 2000

SameSubNetThreshold = 15

RouteHistoryLength = 30

We can see further details about these setting in the link.

Start the SWPM to install the ASCS First Cluster Node.

While Installing ASCS First Cluster Node, Select the Cluster share configuration as ‘File Share Cluster’

Enter the hostname of file share cluster, in this case we use ‘sapglobalPR1’ for PR1 setup.

Add following parameters in Instance profile

enque/encni/set_so_keepalive = true



service/ha_check_node = 1

Configure the probe port for the Cluster role using the powershell.

Get-ClusterResource "SAP PR1 IP" | Get-ClusterParameter



 



$ClusterNetworkName = “Cluster Network 1”



$IPResourceName = "SAP PR1 IP"



$ILBIP = "<Virtual IP>"



Import-Module FailoverClusters



Get-ClusterResource $IPResourceName | Set-ClusterParameter  -Multiple @{Address=$ILBIP;ProbePort=<probePort>;SubnetMask="255.255.255.255";Network=$ClusterNetworkName;EnableDhcp=0}

For ERS Role (e.g. in S/4 Hana)

Get-ClusterResource "SAP PR1 ERS IP" | Get-ClusterParameter



 



$ClusterNetworkName = “Cluster Network 1”



$IPResourceName = "SAP PR1 ERS IP"



$ILBIP = "<Virtual IP>"



Import-Module FailoverClusters



Get-ClusterResource $IPResourceName | Set-ClusterParameter  -Multiple @{Address=$ILBIP;ProbePort=<probePort>;SubnetMask="255.255.255.255";Network=$ClusterNetworkName;EnableDhcp=0}

Restart the Cluster Roles to activate the changes.

Run SWPM on Node2, to install the ‘Additional Cluster Node’.

Using Fileshare in SAP PAS Setup

Following are the some relevant entries for Fileshare location while installing SAP PAS instance using SWPM.

Select the Profile directory which is in the File share cluster

Select the Fileshare virtual hostname for transport Directory. Transport directory hostname can be same as ‘sapmnt’ virtual host. Otherwise we can create a separate virtual host, ILB front-end IP, Cluster Role, storage volume for transport host in the Fileshare Cluster.

Optimization Steps for SAP Setup using Fileshare

There are several recommendations and SAP Notes available to tune the SAP deployment using fileshare.

SAP Note 1823833 - Accessing shares via SMB3.x will result in long waiting times

Make sure Continuous availability is disabled in all the SAP and fileshare VMs as described in the link.

SAP Note 2802770 - AL11 running slow on Windows

Specially Check the setting of your drives NTFS configuration with "Fsutil 8Dot3Name query" as described in SAP Note 662452

SAP Note 1911507 - Background Jobs canceled during failover of (A)SCS instance in windows failover cluster

SMB 3.0 must be active in all the VMs.

File Server Resource Manager – We can use quota management to set limits on the fileshare and define rules & alerts.

Failover Cluster Testing

Failover testing needs to be performed for Fileshare cluster, ASCS Cluster and Database cluster of the SAP environment.

There are mainly 2 kinds of HA testing needs to be performed:

Planned – In case of planned unavailability of the one the node of the cluster pair. This can be tested by stopping the services on the node and verifying that SAP services are available to users.

Unplanned – This can be tested by crashing the VMs and removing the storage from VMs. In windows environment, VM crashing can be achieved by the tool ‘notmyfault’.