Skip to Content
avatar image
Former Member

SAP HANA Dynamic Tiering: FAIL: process hdbesserver HDB Extended Storage Server not running

Hello gurus,

We are facing one problem with SAP HANA Dynamic Tiering. The esserver service crashed, and I am not able to start this service.

During the HDB start I am getting this error FAIL: process hdbesserver HDB Extended Storage Server not running.

I tried to go throw trace files and I can’t see any related error in this trace files.

In esserver*.trc file I can see only this error (SAP IQ):

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.283292 i esserver UNKNOWN(0) : 140538166503168 Database Option for asynchronous I/O is enabled.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.283875 i esserver UNKNOWN(0) : 140538166503168 1 DBFile '/hana/es_data/HD2/mnt01024/es/system/HD2ESDB.es' is enabled for asynchronous I/O.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.284360 i esserver UNKNOWN(0) : 140538166503168 2 DBFile '/hana/es_data/HD2/mnt01024/es/user/HD2ESDB_usr.es' is enabled for asynchronous I/O.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.284821 i esserver UNKNOWN(0) : 140538166503168 3 DBFile '/hana/es_log/HD2/mnt01024/es/delta/HD2ESDB_rlv.rlv' is enabled for asynchronous I/O.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.287157 i esserver UNKNOWN(0) : 140538166503168 Database BIO is asynchronous I/O enabled for RLV.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.287636 i esserver UNKNOWN(0) : 140538166503168 Log buffer manager using asynchronous I/O for RLV.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.288084 i esserver UNKNOWN(0) : 140538166503168 RLV Flusher Enabled.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.288530 i esserver UNKNOWN(0) : 140538166503168 RLV Logging Enabled.

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.289561 i esserver UNKNOWN(0) : 140538166503168 Last Commit Txn ID from database: 43299314

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.290077 i esserver UNKNOWN(0) : 140538166503168 Starting RV Store Recovery for 0 Tables

[01218]{0000000003}[-1/-1] 2017-04-21 08:30:37.290535 i esserver UNKNOWN(0) : 140538166503168 Creating forward scan of log stream 1, table 0

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.821355 i esserver UNKNOWN(0) :

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.821973 i esserver UNKNOWN(0) :

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.822458 i esserver UNKNOWN(0) : **************************************************

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.822928 i esserver UNKNOWN(0) : ***SAP IQ Abort:

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.823549 i esserver UNKNOWN(0) : ***From:stcxtlib/st_server.cxx:2004

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.824008 i esserver UNKNOWN(0) : ***PID: 10265

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.824466 i esserver UNKNOWN(0) : ***Message: caught signal 11, program abort

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.824956 i esserver UNKNOWN(0) : ***Thread: 140538166503168(TID: 1218)

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.825416 i esserver UNKNOWN(0) : **************************************************

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.825862 i esserver UNKNOWN(0) :

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.826921 i esserver UNKNOWN(0) :**Error from IQ connection:

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.827484 i esserver UNKNOWN(0) :**Time of error:2017-04-21 08:31:01.826

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.828471 i esserver UNKNOWN(0) :**IQ Version:HANA DT/1.0.122/10594/P/Rev122.06

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.829653 i esserver UNKNOWN(0) :**OS info:IQ built on: Enterprise Linux64 - x86_64 - 2.6.18-194.el5,Executed on: Linux/smbhd2dt/2.6.32-573.el6.x86_64/#1 SMP Wed Jul 1 18:23:37 EDT 2

015/x86_64

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.830580 i esserver UNKNOWN(0) :**Command status when error occured:NO COMMAND OR CURSOR ACTIVE

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.831492 i esserver UNKNOWN(0) : Dump all thread stacks at stcxtlib/st_server.cxx:2004 for PID: 10265

[01218]{0000000003}[-1/-1] 2017-04-21 08:31:01.863066 i esserver UNKNOWN(0) :

***************** This is the STACKTRACE ***************

In esserver.out file is the same error:

Server built for X86_64 processor architecture

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.097336 i esserver UNKNOWN(0) : 31924K of memory used for caching

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.097362 i esserver UNKNOWN(0) : Minimum cache size: 31924K, maximum cache size: 262144K

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.097373 i esserver UNKNOWN(0) : Using a maximum page size of 4096 bytes

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.101128 i esserver UNKNOWN(0) : Multiprogramming level: 150

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.101150 i esserver UNKNOWN(0) : Automatic tuning of multiprogramming level is disabled

=============================================================

IQ server starting with:

100 connections(-gm )

32 cmd resources( -iqgovern )

1060 threads(-iqmt )

512 Kb thread stack size(-iqtss)

542720 Kb thread memory size ( -iqmt * -iqtss )

16 IQ number of cpus( -iqnumbercpus )

10000000 MB maximum size of IQMSG file ( -iqmsgsz )

10 copies of IQMSG file archives ( -iqmsgnum )

=============================================================

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.140896 i esserver UNKNOWN(0) : Starting database "HD2ESDB" (/hana/es_data/HD2/mnt01024/es/db/HD2ESDB.db) at Fri Apr 21 2017 08:30

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.141982 i esserver UNKNOWN(0) : Database recovery in progress

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.142006 i esserver UNKNOWN(0) :Last checkpoint at Fri Apr 21 2017 08:24

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.142024 i esserver UNKNOWN(0) :Database "HD2ESDB" transaction log offset after last checkpoint is 196434257

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:32.142036 i esserver UNKNOWN(0) :Checkpoint log...

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:36.920829 i esserver UNKNOWN(0) :Transaction log: HD2ESDB.log...

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:36.920859 i esserver UNKNOWN(0) :Database "HD2ESDB" transaction log starts at offset 196423049

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:36.928271 i esserver UNKNOWN(0) : Database "HD2ESDB" mirroring: local status for this server: role=unknown, state=synchronizing, sequence=1, yielding=N

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:36.928394 i esserver UNKNOWN(0) : Database "HD2ESDB" mirroring:becoming primary server

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:36.928414 i esserver UNKNOWN(0) : Database "HD2ESDB" transaction log current offset is 196434477

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:36.928878 i esserver UNKNOWN(0) :Rollback log...

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:37.013220 i esserver UNKNOWN(0) :Checkpointing...

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:37.013295 i esserver UNKNOWN(0) : Starting checkpoint of "HD2ESDB" (HD2ESDB.db) at Fri Apr 21 2017 08:30

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:37.029068 i esserver UNKNOWN(0) : Finished checkpoint of "HD2ESDB" (HD2ESDB.db) at Fri Apr 21 2017 08:30

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:37.029115 i esserver UNKNOWN(0) : Recovery complete

Using licenses from: /usr/local/flexlm/licenses/license.dat

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:37.201968 i esserver UNKNOWN(0) : Update previous log GUID to: b7afd90c-19f6-11e7-8000-846468576517

Update current log GUID to :8f1eba1a-267d-11e7-8000-f9490b5987c1

[00000]{0000000000}[-1/-1] 2017-04-21 08:30:37.206821 i esserver UNKNOWN(0) : Database "HD2ESDB" (HD2ESDB.db) started at Fri Apr 21 2017 08:30

[00000]{0000000000}[-1/-1] 2017-03-21 08:30:37.223168 i esserver UNKNOWN(0) : IQ Server esHD220.

**************************************************

***SAP IQ Abort:

***From:stcxtlib/st_server.cxx:2004

***PID: 10265

***Message: caught signal 11, program abort

***Thread: 140538166503168(TID: 1218)

**************************************************

sh: /usr/bin/pstack: No such file or directory

Can someone help me?

Thank you so much,

Jakub Skarke

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • Apr 26, 2017 at 02:19 PM

    For a system down issue I would recommend opening a Support case and specifying the HAN-DYT component.

    I don't see anything obvious in the trace file to explain why the DT esserver process isn't starting.

    Add comment
    10|10000 characters needed characters exceeded