Instance XXX/XXX reached state UNKNOWN after having state STARTING S4HANA 1809 Installation

[former_member265769]

hdbcmdout.txtinstdir-java.txtinstdir-sapcontrol-s4h-d00.txtinstdir-sapinst-loginquirer.txtinstdir-sapinst1.txtsapinst1.txthdbstdout.txterror-1-min.jpg

error-20.jpg

Hello Gurus,

We are currently in process of installing SAP S4HANA 1809 On Premise on RED HAT Enterprise Linux 7.6.

We already installed SAP HANA DATABASE (Platform 2.0) Version-SPS04 and its status is running (all green) in HANA Studio, please view attachments.

At the Step "Start Instance" system triggered an error stating as follows:

Instance XXX/XXX reached state UNKNOWN after having state STARTING-Giving Up....

All Database related processes were running successfully as shown in the image attached. Also our swap size is 128 gb, so i guess that shall not be an issue either.

I have NOT YET assigned any static ip to my Ethernet adapters (we are 2 ethernet adapters) in our motherboard, do we need to assign static ip to each of the ethernet adapter or just assigning to 1 ethernet adapter and assigning the same ip & hostname in /etc/hosts file is enough to get rid of this error message.

Hence request you to please guide me to resolve this issue.

Also hostname command at terminal, returns only the HOSTNAME only and not full hostname + domain name, so I guess that is correct configuration only.

Also some of the error logs I have attached, please view all attachments.

Need guidance here ASAP to resolve this issue and also I apologize for any inconvenience caused thereby.

Regards

[dennispadia · Accepted solution]

Hello Pavan,

As per the log you have shared, you are getting below error which means that there is some problem in semaphore (as you can see semget in the error)

*** ERROR =>e=28 semget(20095,1,2016)(28:Nospacelefton device)[semux.c500]

You can refer below SAP Note for more help on this error

1. 1584228 - Internal error in secure store/Semaphore run out on a system.

2. Also there is a wiki which describe this error in more details and what needs to be done.

3. Other similar thread which has encountered the same issue of semget and they have made changes to kernel.sem.

Kindly revisit your kernel parameters that are required for SAP.

Regards,

Dennis.

[former_member265769 · Accepted solution]

Hi Dennis,

Thanks for your time. I will follow the steps you mentioned and will update you the status in few days.

Regards,

Pavan

[former_member265769]

Hi Dennis,

Thanks for your reply. This issue has been resolved and I have completed the S4HANA 1809 installation successfully yesterday.

How it was resolved ? As per SAP Note 1048303 I have updated the Linux Kernel Parameters as listed below in file /etc/sysctl.conf and after saving this file, entered this command to activate the changes -- sysctl -p

Also as per Mr. Shabeer suggestions, earlier added the Static IP and assigned the same to my host name in /etc/hosts file.

kernel.msgmni=1024
kernel.sem=1250 256000 100 1024
vm.max_map_count=2000000 

I really appreciate everyone's suggestions and very thankful to everyone.
Regards,
Pavan

[shabeer]

*** ERROR => e=28 semget(20095,1,2016) (28: No space left on device) [semux.c      500]

Dev_w0 log says your file system is full. Did you look in to that?

[former_member265769]

Hi Shabeer,

Yes, I saw this in dev_w0 but not sure why this is being triggered. We have ample space allocated for /hana/data-380gb, /shared -200gb, /log-85gb, /usr/sap-59gb and even for swap we have allocated 128 gb. Also during hana database installation through hdblcmgui, system defaulted complete ram capacity which I have not changed it, so is this could be an issue ???

SAP Note: 2535340, describes this below error and it points to linux kernel parameters but I have not altered any linux kernel related parameters, its not mentioned anywhere in sap application server installation guide.

** ERROR =>e=28 semget. 

Thanks

Pavan

[shabeer]

Run "df -h" and provide the output.

[former_member265769]

Sure will post the screenshot in a day.

[former_member265769]

df-h-screenshot.jpg

Hi Shabeer,

Please find attached df-h command and results screenshot.

Thanks

Pavan

[former_member265769]

Hi Shabeer,

Thanks for your reply. This issue has been resolved and I have completed the S4HANA 1809 installation successfully yesterday.

How it was resolved ? As per SAP Note 1048303 I have updated the Linux Kernel Parameters as listed below in file /etc/sysctl.conf and after saving this file, entered this command to activate the changes -- sysctl -p

Also as per your suggestions, earlier added the Static IP and assigned the same to my host name in /etc/hosts file.

kernel.msgmni=1024
kernel.sem=1250 256000 100 1024
vm.max_map_count=2000000 

Regards,
Pavan

[shabeer]

Since you have changed the IP address in middle of the installation, here I would suggest to

stop the installation, Uninstall the application server and Install it again. This should no take much time.

[dennispadia]

Hello Pavan,

Kindly share dev_disp, dev_ms and dev_w0 files. You can find that under /usr/sap/S4H/D00/work. You are getting below error as per the log, so if you check this error in google you will get many different resolution. So in order to dig further, you need to check dev_dsp and dev_w0 log files.

Regards,

Dennis.

[former_member265769]

@ Mr. Dennis Padia,

Hi Dennis,

Thanks so much for your time. As per your advice, I have attached the dev_w0 & dev_disp files. Also dev_ms file is not there in work directory.

Also just to let you know, I am performing installation on RHEL 7.6 directly, I mean its not through a Virtual Machine. Its a Direct Installation in RHEL.

Please review the attachments.

Also thanks for taking your time and I apologize for the inconvenience.

Thanks

Pavandev-disp.txtdev-w0.txtdev-rd.txt

[former_member265769]

Hi Dennis,

Thanks so much for your time. Due to some error in this blog, I replied to you with attachments below. For some reason, return comments to your message with attachments was not going through, hence posted a separate posting with attachments, please review it.

thanks

Pavan

[kaus19d]

Hi Pavan Kumar Srigundugurthi,

I have gone through you attached files & below would be my suggestion as per the log messages,

A)

26.08.2019 00:40:40 GetProcessList OK name, description, dispstatus, textstatus, starttime, elapsedtime, pid sapcpe, Executable Copy, YELLOW, Started, 2019 08 26 00:40:39, 0:00:01, 132196

--> Can try with Latest Kernel copy

B)

Open your browser and paste the following URL address to access the GUI https://server1:4237/sapinst/docs/index.html

Logon users: [root]

--> generally if we run the sapinst from root also, in browser, we use <SIDadm> not the root user, although this is a very minor thing

C)

'/usr/sap/sapservices' --> Check the ASCS02 & D00 service entry details are there or not 3602 ms server port

--> please cross verify deatils against /etc/services & DEFAULT.PFL INFO

D)

2019-08-25 21:25:50.629 (root/sapinst) Problem with the DNS configuration: could not determine the DNS domain of host server1

+

26.08.2019 00:37:01 GetProcessList OK name, description, dispstatus, textstatus, starttime, elapsedtime, pid disp+work, Dispatcher, YELLOW, Server startup procedure running, 2019 08 26 00:36:51, 0:00:10, 129970 igswd_mt, IGS Watchdog, GREEN, Running, 2019 08 26 00:36:51, 0:00:10, 129971 gwrd, Gateway, YELLOW, Scheduled, , , -1 icman, ICM, YELLOW, Scheduled, , , -1

--> Dispatcher Yellow, so please check in dev_w0, dev_disp in work directory, most important is to resolve your hostname against your ip-address. So what you can do is alternatively, connect your vm to a Public Network or setup a static IP, below is a procedure example, https://www.networkworld.com/article/3353039/how-to-configure-a-static-ip-address-on-linux.html

E)

Minor additional suggestion/precaution, Check the env values in SIDadm "dbms_type= hdb" & /etc/services for sapmsSID.

Hope that during SWPM selection you have taken the Without FQDN option, I mean "Local Installation" option

Thanks,

Kaushik

[former_member265769]

Hi Kaushik,

Thanks for taking time and for providing your valuable inputs. Due to some error in this blog as its not allowing me to submit return comments to your answer, hence I posted a separate reply addressing you below, please go through my answers and also review the attachments in that posting.

Thanks

Pavan

[former_member265769]

@ Mr. Kaushik dev-disp.txtdev-w0.txtdev-rd.txt

Hi Kaushik,

Thanks for taking your time and for providing valuable inputs. Also just to let you know, I am performing installation on RHEL 7.6 directly, I mean its not through a Virtual Machine. Its a Direct Installation in RHEL.dev-disp.txtdev-w0.txtdev-rd.txt

Point 1, which you mentioned, regarding Kernel, Yes, I am using the latest kernel SAPEXE28.SAR & SAPEXEDB28.SAR and latest IGS.exe and IGSHELPER.EXE.

Point 2, which you mentioned, I ran the ./sapinst using root user, as it was mentioned in some SAP note.

Point 3, which you mentioned, Entries are there in /usr/sap/services

Point 4, I have assigned the Static IP & entered the same and hostname in /etc/hosts file

Point 5, Yes, I have opted for Without FQDN option only.

but still problem persists. Also as per your suggestion, I have attached dev_w0 and dev_disp files with another post below, as there is a problem here, for some reason its not accepting attachments for return comments.

Also thanks for taking your time and I apologize for the inconvenience.

Thanks

Pavan

[kaus19d]

Hi Pavan Kumar Srigundugurthi,

Could you try stop & start the instances with the below commands,

sapcontrol -nr 00 -function Stop

sapcontrol -nr 02 -function Stop

sapcontrol -nr 02 -function Start

sapcontrol -nr 00 -function Start

Then we can cross verify the results, also the free space issue is in the shared memory. Try the command "free -h"

Thanks,

Kaushik

[former_member265769]

Hi Kaushik,

Thanks for your time. This issue is resolved and I have installed s4hana 1809 installation successfully. Issue was with Kernel Parameters, I have posted the solution clearly below.

Thanks

Pavan

[shabeer]

First, you need a static IP, and adjust the same IP in host file.

[former_member265769]

Hi Shabeer,

Thanks for taking your time and providing valuable inputs.

I just created a static ip with subnet mask for Ethernet adapter 1 and entered the same IP and host name in /etc/hosts.Thereafter I have started the HANA Database using Hana Studio and all processes in Hana studio appears GREEN.

Then I re-started ./sapinst and choose option --> Continue with Existing Run and again at 81% same error triggered.

Instance XXX/XXX reached state UNKNOWN after having state STARTING-Giving Up

Also just to inform you that i have NOT created any domain name, our hostname is - server1. Do we have to create a domain name as well along with hostname ?

Also i read in some other forums that before restarting the SWPM again, I need to start the INSTANCE first, do i have to do this, if so from where I have to start the instance manually ?

Not sure, what was wrong, database is running, Static IP is created and assigned the same + Hostname in /etc/hosts file. Still issue persists. No SAP note provides clear and correct solution for this problem.

Please review the attached screenshots of Static IP which i have assigned in Network Settings & a screenshot of /etc/hosts file, and a Screenshot of command: "ifconfig" static-ip-1.jpgstatic-ip-2.jpgetc-hosts-file.jpgifconfig.jpg

may be there is something wrong in these, which I cannot figure it out. Hence request you to provide your valuable inputs to resolve this issue. Again, I apologize for the inconvenience.

Regards,

Sristatic-ip-1.jpgstatic-ip-2.jpgetc-hosts-file.jpgifconfig.jpg