How to configure X.509 client certificate authentication for SAP host agent in LVM

Hi LVM community,

on our SAP LVM 2.1 SP7_1 system I try to configure X.509 client certificate authentication for accessing our host agents. I already went successfully through the following steps:

1. I created a SSL server PSE for the host agent (in directory /usr/sap/hostctrl/exe/sec as described in SSL für den SAP-Host-Agenten auf UNIX konfigurieren - SAP-Host-Agent - SAP Library)
2. I created a certificate signing request and ordered a signed certificat from my CA.
3. I imported the signed certificate into the SSL server PSE of the host agent.

Afterwards I was able to activate HTTPS for the host agent communication. I verified the connection with the "Test Connection" button. Everything is working fine.

Now I want to get to the next level and activate X.509 client certificate authentication for the host agent connection. Unfortunately I didn't find any documentation which describes how to configure this correctly. So I just followed the hints LVM provided:

After switching "Authentication Type" to "X.509 Client Certificate" LVM pointed me to the Key Storage in NWA. According to the LVM mouse-over hint of the field "Private Key Name" the private key needs to be configured in the "LVMView" of the Key Storage.

So I created another SSL private key for my LVM system in the LVMView of the NWA key storage. I also got it signed from my CA and imported the signed certificate into the LVMView:

Afterwards I was able to select my SSL private key name in the Host Agent Configuration:

Though the configuration seems to be correct, the authentication does not work. If I press the "Test Connection" button, I get a "Invalid credentials" error message now:

I already turned on tracing for the host agent. In sapstartsrv.log I see the following messages every time I do the connection test:

[Thr 140437377509120] Tue Jul 5 09:39:11 2016

[Thr 140437377509120] NiIPeekListen: peek successful for hdl 1

[Thr 140437377509120] NiIPeekListen: peek successful for hdl 1

[Thr 140437377509120] NiIAccept: hdl 1 accepted connection

[Thr 140437377509120] NiICreateHandle: hdl 18 state NI_INITIAL_CON

[Thr 140437377509120] NiIInitSocket: set default settings for hdl 18/sock 22 (I4; ST)

[Thr 140437377509120] NiIBlockMode: set blockmode for hdl 18 FALSE

[Thr 140437377509120] NiIAccept: state of hdl 18 NI_ACCEPTED

[Thr 140437377509120] NiIAccept: hdl 1 accepted hdl 18 from 149.216.2.50:56382

[Thr 140437377509120] NiIAccept: hdl 18 took local address 149.216.2.50:1129

[Thr 140437377509120] NiIBlockMode: set blockmode for hdl 18 TRUE

[Thr 140437310097152] ->> SapSSLSessionInit(&sssl_hdl=0x7fba1bfb3e08, role=2 (SERVER), auth_type=0 (NO_CLIENT_CERT))

[Thr 140437310097152] <<- SapSSLSessionInit()==SAP_O_K

[Thr 140437310097152] in: args = "role=2 (SERVER), auth_type=0 (NO_CLIENT_CERT)"

[Thr 140437310097152] out: sssl_hdl = 0x2725de0

[Thr 140437310097152] ->> SapSSLSetNiHdl(sssl_hdl=0x2725de0, ni_hdl=18)

[Thr 140437310097152] NiIBlockMode: leave blockmode for hdl 18 TRUE

[Thr 140437310097152] SSL NI-sock: local=149.216.2.50:1129 peer=149.216.2.50:56382

[Thr 140437310097152] <<- SapSSLSetNiHdl(sssl_hdl=0x2725de0, ni_hdl=18)==SAP_O_K

[Thr 140437310097152] ->> SapSSLSessionStart(sssl_hdl=0x2725de0)

[Thr 140437310097152] Server-configured Ciphersuites: "TLS_RSA_WITH_AES128_GCM_SHA256:TLS_RSA_WITH_AES256_GCM_SHA384:TLS_RSA_WITH_AES128_CBC_SHA:TLS_RSA_WITH_AES256_CBC_SHA:TLS_RSA_WITH_RC4_128_SHA:TLS_RSA_WITH_RC4_128_MD5:TLS_RSA_WITH_3DES_EDE_CBC_SHA:TLS_RSA_WITH_DES_CBC_SHA:TLS_RSA_EXPORT_WITH_DES40_CBC_SHA:TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5:TLS_RSA_EXPORT_WITH_RC4_40_MD5"

[Thr 140437310097152] Client-offered Ciphersuites: "TLS_RSA_WITH_AES128_GCM_SHA256:TLS_RSA_WITH_AES128_CBC_SHA:TLS_RSA_WITH_3DES_EDE_CBC_SHA:TLS_RSA_WITH_RC4_128_SHA:TLS_RFC5746_INFO_SCSV"

[Thr 140437310097152] No Client Certificate

[Thr 140437310097152] Cached session resumed (TLSv1.2)

[Thr 140437310097152] HexDump of native SSL session ID { &buf= 0x271ab34, buf_len= 32 }

[Thr 140437310097152] 00000: d1 c1 bc 7b 2a 6f a2 ef 56 a4 0f 8e 2e 96 d2 03 ...{*o.. V.......

[Thr 140437310097152] 00010: fa f1 9d 1e a8 46 37 3b af b9 cb 9c 02 bd 97 7b .....F7; .......{

[Thr 140437310097152] <<- SapSSLSessionStart(sssl_hdl=0x2725de0)==SAP_O_K

[Thr 140437310097152] status = "resumed SSL session, client cert NOT requested"

[Thr 140437310097152] ->> SapSSLRead(sssl_hdl=0x2725de0, buf=0x25c7a50, maxlen=32768, timeout=-1, &readlen=0x7fba1bfb3d3c)

[Thr 140437310097152] <<- SapSSLRead(sssl_hdl=0x2725de0)==SAP_O_K

[Thr 140437310097152] ... = "buf= 0x25c7a50, max=32768, received=1153"

[Thr 140437310097152] HTTP Parse - Start

[Thr 140437310097152] PutHeader: Host , 7

[Thr 140437310097152] PutHeader: mHeaderSet 128

[Thr 140437310097152] PutHeader: Content-Type , 4

[Thr 140437310097152] PutHeader: mHeaderSet 144

[Thr 140437310097152] PutHeader: CallingType , 29

[Thr 140437310097152] PutHeader: Content-Length , 3

[Thr 140437310097152] PutHeader: mHeaderSet 152

[Thr 140437310097152] PutHeader: SAP-PASSPORT , 29

[Thr 140437310097152] PutHeader: SOAPAction , 29

[Thr 140437310097152] - Parsing buffer 'POST HTTP/1.1'

[Thr 140437310097152] HTTPMessage::AddBodyContent: Allocate in 0x0x271f560 8192 bytes (left=8192)

[Thr 140437310097152] HTTPMessage::AddBodyContent: Copy in 0x0x271f560 547 bytes (Size = 8192, Left=7645)

[Thr 140437310097152] Trying to lock HTTPHandlerManager::GetInstance

[Thr 140437310097152] Successfully locked HTTPHandlerManager::GetInstance

[Thr 140437310097152] Successfully unlocked HTTPHandlerManager::GetInstance

[Thr 140437310097152] Start executing Webmethod ACOSPrepare

[Thr 140437310097152] Operation ACOSPrepare; Socket type Network SSL Socket; Remote IP 149.216.2.50; Remote port 56382; Username Not Available

[Thr 140437310097152] No username set for DefaultOperationCredentialAuthenticator

[Thr 140437310097152] ->> SapSSLGetPeerInfo(sssl_hdl=0x2725de0, &cert=(nil), &cert_len=(nil),

[Thr 140437310097152] &subject_dn=0x7fba1bfb3250, &issuer_dn=(nil), &cipher=(nil))

[Thr 140437310097152] Current Cipher: TLS_RSA_WITH_AES128_GCM_SHA256

[Thr 140437310097152] <<- SapSSLGetPeerInfo(sssl_hdl=0x2725de0)==SAP_O_K

[Thr 140437310097152] out: cert_len = <no cert>

[Thr 140437310097152] out: cipher = "TLS_RSA_WITH_AES128_GCM_SHA256"

[Thr 140437310097152] Unauthorized (user authentication required)

[Thr 140437310097152] *** ERROR => Webmethod ACOSPrepare failed: Unauthorized: User authentication required [saphostcontr 1297]

[Thr 140437310097152] NiIPeek: peek for hdl 18 timed out (r; 0ms)

[Thr 140437310097152] NiIPeek: peek successful for hdl 18 (w)

[Thr 140437310097152] HostControl_SendHeader: HTTP/1.1 401 Unauthorized : null

[Thr 140437310097152] HostControl_SendHeader: WWW-Authenticate : Basic realm="gSOAP Web Service"

[Thr 140437310097152] HostControl_SendHeader: Server : gSOAP/2.7

[Thr 140437310097152] HostControl_SendHeader: Send Additional Header -> No GSOAPHTTPRequest found

[Thr 140437310097152] HostControl_SendHeader: Send Additional Header -> No GSOAPHTTPRequest found

[Thr 140437310097152] HostControl_SendHeader: Connection : close

[Thr 140437310097152] HostControl_SendHeader: Send Additional Header -> No GSOAPHTTPRequest found

[Thr 140437310097152] ->> SapSSLWrite(sssl_hdl=0x2725de0, buf=0x25c7a50, len=803, timeout=-1, &writelen=0x7fba1bfb3d54)

[Thr 140437310097152] <<- SapSSLWrite(sssl_hdl=0x2725de0)==SAP_O_K

[Thr 140437310097152] ... = "buf= 0x25c7a50, written= 803 of 803 (all)"

[Thr 140437310097152] NiIShutdownHandle: shutdown -w of hdl 18

[Thr 140437310097152] ->> SapSSLSessionDone(&sssl_hdl=0x271a918)

[Thr 140437310097152] <<- SapSSLSessionDone()==SAP_O_K

[Thr 140437310097152] in: sssl_hdl = 0x2725de0

[Thr 140437310097152] ... ni_hdl = 18

[Thr 140437310097152] NiICloseHandle: shutdown and close hdl 18/sock 22

From my point of view the SSL handshake seems to fail. I'm wondering why the authentication is tried without client certificate, because that's what I wanted to activate.

At this point I'm out of ideas. Has anybody successfully configured this in LVM or can point me to a documentation were this configuration is explained?

Your help is greatly appreciated.

Kind regards

Benny