Skip to Content

ADS Error 6326 Enabling TLS communication encryption on an AES256-encrypted database

Recently we migrated our .dbf tables into .adt via .add data dictionary v 11.1.0.24.

After some efforts we have it working fine so far.

Now we require to encrypt communications as well using TLS and we have done the following:

a) Created 2 certificates, one for the server (with password) the other for the client.

b) Placed the server certificate (servercert.pem) in the same folder where the Advantage Database Server is located and added the following entries to the registry:

"TLS_KEY_FILE"="servercert.pem"
"TLS_KEY_PASSWORD"="xxxxx"
"RECEIVE_TLS_PORT"=dword:00001877

restarted ads_cfg.exe and via netstats -a confirm the server is listening to ports 6262 and 6263:

TCP 0.0.0.0:6262 Server7:0 LISTENING
TCP 0.0.0.0:6263 Server7:0 LISTENING

c) Copied the client certifcate (clientcert.pem) to the folder where our executable resides.

Note that we have already created the .add using AES256 encryption and enabled the ADS_DD_ENCRYPT_COMMUNICATION flag. The corresponding SSL dlls (libeay32.dll and ssleay32.dll) are located in Window\SysWOW64 (32 bit version) and the 64 bit version are also located where ads_cfg.exe, ads.exe live.

d) Launch our app which calls adsConnect101() passing the following connection string (pwd and url have been replaced with bogus text):

Data Source="C:\develop\v23\sample\abacus.add"; ServerType=2; User ID="ABASYS"; Password="abc"; Shared=TRUE; DDPassword="xyz"; CommType=TLS; TLSCertificate="C:\develop\v23\Programs\clientcert.pem"; TLSCommonName=www.abc.com;

And here is where we're stopped on our tracks with this error:

Error 6326: An error occurred in the OpenSSL library. OpenSSL Error: 5, Socket Error: 0

If I edit ads.ini (which is located where our exe resided, c:\develop\v23\programs) and set LAN_PORT=6263 (instead of the default 6262) we instead get error 6097 (I've added rules to the Windows Firewall to allow TCP, UDP for ports 6262 and 6263 and even temporarily disabled the firewall, but either error will persists.

We don't have any issues if the .add is not encrypted or encrypted with aes256; only when we try to have TLS communication encryption for the latter.

At this stage I don't know what else is missing:

+ additional SSL libraries?

+ additional configuration either on the server or client side or both?

Any guidance is greatly appreciated.

Add comment
10|10000 characters needed characters exceeded

  • Hello Luis,

    Like described in this KBA this issue looks to be related to a communication issue.

    The best will be to start a network sniff and check, what exactly happens with the communication packets. May this will bring you to the right direction to enclose the problem.

    Best regards,
    Hakan

  • Get RSS Feed

2 Answers

  • Oct 14, 2016 at 04:20 PM

    Hakan:

    I did come across that KBA researching the problem. I was hoping TLS_PORT was implemented, but I guess it never was.

    I changed LAN_PORT to 6263, but then instead of error 6326, we got error 6097.

    Eventually, the solution was to locate all ads.ini files in the search path (to be on the safe side), find the [servername] and set
    LAN_PORT TO 0. With this, we were able to open the .add using TLS without any more errors.

    In the past we has our apps create ads.ini with the [servername] section and the LAN_IP and LAN_PORT keywords to resolve discovery issues. This now complicates things for those clients that will want to upgrade to an encrypted database + communication encryption.

    Is there a better way to resolve the issue other than have to hunt down ads.ini and fix them up? Too bad the TLS_PORT solution wasn't implemented.

    Regards,

    Luis.

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 26, 2016 at 10:04 PM

    Hakan:

    Is there a way via ads.ini or elswhere where we can specify both, the lan_port and the tls_port?

    Though setting LAN_PORT=0 took care of the issue, we have some clients that still get error 6420 so this solution only seems to work mostly, but not universally.

    Thanks.

    Add comment
    10|10000 characters needed characters exceeded