Install instance of basics of ASCS01 Stuck

[former_member252955]

Hi Expert

During Unicode system build for SAP ERP 6.0 EHP6 from non unicode in phase of Install instance of basics of ASCS01  since last 24 Hour it's showing that starting ASCS instance only.

Please suggest to diagnose this issue.

[Reagan]

Did you check the log file sapcontrol_DEV_ASCS01.log?

If possible, try to start the stop all running SAP processes and start the instance manually from the OS level using the devadm user.

[former_member252955]

Hi

I tried to start sap instance manually but when I issue the command startsap r3 then it neither prompt error nor success just remain on that line even after 10 hour.

It stucks on start command there is no response.

Regards

Prakash

[former_member252955]

04.03.2016 17:55:11

GetInstanceProperties

FAIL: NIECONN_REFUSED (Connection refused), NiRawConnect failed in plugin_fopen()

StartService

OK

04.03.2016 17:55:14

GetInstanceProperties

FAIL: NIECONN_REFUSED (Connection refused), NiRawConnect failed in plugin_fopen()

04.03.2016 17:55:24

GetInstanceProperties

OK

property, propertytype, value

Process List, NodeWebmethod, GetProcessList

Access Points, NodeWebmethod, GetAccessPointList

Syslog, NodeWebmethod, ABAPReadSyslog

Enque Locks, NodeWebmethod, EnqGetLockTable

Enque Statistic, NodeWebmethod, EnqGetStatistic

Protected Webmethods, Attribute, ABAPAcknowledgeAlerts,ABAPGetSystemWPTable,ABAPGetWPTable,ABAPReadRawSyslog,ABAPReadSyslog,AnalyseLogFiles,Bootstrap,ConfigureLogFileList,CreateSnapshot,DeleteSnapshots,EnqGetLockTable,EnqGetStatistic,EnqRemoveLocks,GetAccessPointList,GetAlerts,GetAlertTree,GetCallstack,GetEnvironment,GetLogFileList,GetProcessParameter,GetQueueStatistic,GetStartProfile,GetTraceFile,GetVersionInfo,HACheckConfig,HACheckFailoverConfig,HAFailoverToNode,HAGetFailoverConfig,ICMGetCacheEntries,ICMGetConnectionList,ICMGetProxyConnectionList,ICMGetThreadList,InstanceStart,InstanceStop,J2EEControlCluster,J2EEControlComponents,J2EEControlProcess,J2EEDisableDbgSession,J2EEEnableDbgSession,J2EEGetApplicationAliasList,J2EEGetCacheStatistic,J2EEGetCacheStatistic2,J2EEGetClusterMsgList,J2EEGetComponentList,J2EEGetEJBSessionList,J2EEGetProcessList,J2EEGetProcessList2,J2EEGetRemoteObjectList,J2EEGetSessionList,J2EEGetSharedTableInfo,J2EEGetThreadCallStack,J2EEGetThreadList,J2EEGetThreadList2,J2EEGetThreadTaskStack,J2EEGetVMGCHistory,J2EEGetVMGCHistory2,J2EEGetVMHeapInfo,J2EEGetWebSessionList,J2EEGetWebSessionList2,ListDeveloperTraces,ListLogFiles,ListSnapshots,OSExecute,ParameterValue,ReadDeveloperTrace,ReadLogFile,ReadSnapshot,RestartInstance,RestartService,RestartSystem,SendSignal,SetProcessParameter,ShmDetach,Shutdown,Start,StartBypassHA,StartSystem,Stop,StopBypassHA,StopService,StopSystem,WebDispGetServerList,MtChangeStatus,MtCustomizeWrite,MtDbsetToWpsetByTid,MtDestroyMarkNTry,MtReset,PerfCustomizeWrite,ReadDirectory,ReadFile,ReadProfileParameters,Register,SnglmgsCustomizeWrite,SystemObjectSetValue,ToolSet,ToolSetRuntimeStatus,TriggerDataCollection,Unregister,UtilAlChangeStatus

CentralServices, Attribute, YES

StartPriority, Attribute, 1

SAPSYSTEM, Attribute, 01

SAPSYSTEMNAME, Attribute, DEV

SAPLOCALHOST, Attribute, unidev

INSTANCE_NAME, Attribute, ASCS01

Webmethods, Attribute, Start,InstanceStart,StartBypassHA,Bootstrap,Stop,StopBypassHA,InstanceStop,Shutdown,ParameterValue,GetProcessList,GetStartProfile,GetTraceFile,GetAlertTree,GetAlerts,RestartService,StopService,GetEnvironment,ListDeveloperTraces,ReadDeveloperTrace,RestartInstance,SendSignal,GetVersionInfo,GetQueueStatistic,GetInstanceProperties,OSExecute,ReadLogFile,AnalyseLogFiles,ListLogFiles,GetAccessPointList,GetSystemInstanceList,StartSystem,StopSystem,RestartSystem,AccessCheck,GetProcessParameter,SetProcessParameter,ShmDetach,GetNetworkId,GetSecNetworkId,RequestLogonFile,CreateSnapshot,ReadSnapshot,ListSnapshots,DeleteSnapshots,GetCallstack,ABAPReadSyslog,ABAPReadRawSyslog,ABAPGetWPTable,ABAPAcknowledgeAlerts,ABAPGetSystemWPTable,J2EEGetProcessList,J2EEGetProcessList2,J2EEControlProcess,J2EEGetThreadList,J2EEGetThreadList2,J2EEGetThreadCallStack,J2EEGetThreadTaskStack,J2EEGetSessionList,J2EEGetWebSessionList,J2EEGetWebSessionList2,J2EEGetCacheStatistic,J2EEGetCacheStatistic2,J2EEGetApplicationAliasList,J2EEGetVMGCHistory,J2EEGetVMGCHistory2,J2EEGetVMHeapInfo,J2EEGetEJBSessionList,J2EEGetRemoteObjectList,J2EEGetClusterMsgList,J2EEGetSharedTableInfo,J2EEGetComponentList,J2EEControlComponents,ICMGetThreadList,ICMGetConnectionList,ICMGetCacheEntries,ICMGetProxyConnectionList,WebDispGetServerList,EnqGetLockTable,EnqRemoveLocks,EnqGetStatistic,HACheckConfig,HACheckFailoverConfig,HAGetFailoverConfig,HAFailoverToNode

04.03.2016 17:55:25

GetInstanceProperties

OK

property, propertytype, value

Process List, NodeWebmethod, GetProcessList

Access Points, NodeWebmethod, GetAccessPointList

Syslog, NodeWebmethod, ABAPReadSyslog

Enque Locks, NodeWebmethod, EnqGetLockTable

Enque Statistic, NodeWebmethod, EnqGetStatistic

Protected Webmethods, Attribute, ABAPAcknowledgeAlerts,ABAPGetSystemWPTable,ABAPGetWPTable,ABAPReadRawSyslog,ABAPReadSyslog,AnalyseLogFiles,Bootstrap,ConfigureLogFileList,CreateSnapshot,DeleteSnapshots,EnqGetLockTable,EnqGetStatistic,EnqRemoveLocks,GetAccessPointList,GetAlerts,GetAlertTree,GetCallstack,GetEnvironment,GetLogFileList,GetProcessParameter,GetQueueStatistic,GetStartProfile,GetTraceFile,GetVersionInfo,HACheckConfig,HACheckFailoverConfig,HAFailoverToNode,HAGetFailoverConfig,ICMGetCacheEntries,ICMGetConnectionList,ICMGetProxyConnectionList,ICMGetThreadList,InstanceStart,InstanceStop,J2EEControlCluster,J2EEControlComponents,J2EEControlProcess,J2EEDisableDbgSession,J2EEEnableDbgSession,J2EEGetApplicationAliasList,J2EEGetCacheStatistic,J2EEGetCacheStatistic2,J2EEGetClusterMsgList,J2EEGetComponentList,J2EEGetEJBSessionList,J2EEGetProcessList,J2EEGetProcessList2,J2EEGetRemoteObjectList,J2EEGetSessionList,J2EEGetSharedTableInfo,J2EEGetThreadCallStack,J2EEGetThreadList,J2EEGetThreadList2,J2EEGetThreadTaskStack,J2EEGetVMGCHistory,J2EEGetVMGCHistory2,J2EEGetVMHeapInfo,J2EEGetWebSessionList,J2EEGetWebSessionList2,ListDeveloperTraces,ListLogFiles,ListSnapshots,OSExecute,ParameterValue,ReadDeveloperTrace,ReadLogFile,ReadSnapshot,RestartInstance,RestartService,RestartSystem,SendSignal,SetProcessParameter,ShmDetach,Shutdown,Start,StartBypassHA,StartSystem,Stop,StopBypassHA,StopService,StopSystem,WebDispGetServerList,MtChangeStatus,MtCustomizeWrite,MtDbsetToWpsetByTid,MtDestroyMarkNTry,MtReset,PerfCustomizeWrite,ReadDirectory,ReadFile,ReadProfileParameters,Register,SnglmgsCustomizeWrite,SystemObjectSetValue,ToolSet,ToolSetRuntimeStatus,TriggerDataCollection,Unregister,UtilAlChangeStatus

CentralServices, Attribute, YES

StartPriority, Attribute, 1

SAPSYSTEM, Attribute, 01

SAPSYSTEMNAME, Attribute, DEV

SAPLOCALHOST, Attribute, unidev

INSTANCE_NAME, Attribute, ASCS01

Webmethods, Attribute, Start,InstanceStart,StartBypassHA,Bootstrap,Stop,StopBypassHA,InstanceStop,Shutdown,ParameterValue,GetProcessList,GetStartProfile,GetTraceFile,GetAlertTree,GetAlerts,RestartService,StopService,GetEnvironment,ListDeveloperTraces,ReadDeveloperTrace,RestartInstance,SendSignal,GetVersionInfo,GetQueueStatistic,GetInstanceProperties,OSExecute,ReadLogFile,AnalyseLogFiles,ListLogFiles,GetAccessPointList,GetSystemInstanceList,StartSystem,StopSystem,RestartSystem,AccessCheck,GetProcessParameter,SetProcessParameter,ShmDetach,GetNetworkId,GetSecNetworkId,RequestLogonFile,CreateSnapshot,ReadSnapshot,ListSnapshots,DeleteSnapshots,GetCallstack,ABAPReadSyslog,ABAPReadRawSyslog,ABAPGetWPTable,ABAPAcknowledgeAlerts,ABAPGetSystemWPTable,J2EEGetProcessList,J2EEGetProcessList2,J2EEControlProcess,J2EEGetThreadList,J2EEGetThreadList2,J2EEGetThreadCallStack,J2EEGetThreadTaskStack,J2EEGetSessionList,J2EEGetWebSessionList,J2EEGetWebSessionList2,J2EEGetCacheStatistic,J2EEGetCacheStatistic2,J2EEGetApplicationAliasList,J2EEGetVMGCHistory,J2EEGetVMGCHistory2,J2EEGetVMHeapInfo,J2EEGetEJBSessionList,J2EEGetRemoteObjectList,J2EEGetClusterMsgList,J2EEGetSharedTableInfo,J2EEGetComponentList,J2EEControlComponents,ICMGetThreadList,ICMGetConnectionList,ICMGetCacheEntries,ICMGetProxyConnectionList,WebDispGetServerList,EnqGetLockTable,EnqRemoveLocks,EnqGetStatistic,HACheckConfig,HACheckFailoverConfig,HAGetFailoverConfig,HAFailoverToNode

04.03.2016 17:55:26

Stop

OK

04.03.2016 17:55:27

GetProcessList

OK

name, description, dispstatus, textstatus, starttime, elapsedtime, pid

04.03.2016 17:55:27

GetProcessList

OK

name, description, dispstatus, textstatus, starttime, elapsedtime, pid

[Reagan]

Before you start the system, stop all the processes, clean the shared memory segments for ASCS and PAS using cleanipc and delete all sapstream files in the /tmp file system. Check if the DB2 database is up and running. Switch to <SID>adm and run R3trans -d and check the output. Start SAP. Open a new session and check the logs in the home directory of <SID>adm user and also in the work directories of ASCS and PAS. Without logs there is nothing other can do for you.

Additionally, check and make sure you have all the APAR's installed on AIX. There is a bug reported and I suspect you must be hitting that bug.

Check this KBA 2213761 - Software Provisioning Manager hangs during ASCS instance installation in phase start/stop ...

[former_member252955]

Hi Benjamin

As per your suggestion I did but since last 10 Minutes I am waiting to get it up.

[Reagan]

No idea why you are looking into the nmon details. Have you checked the log file /home/devadm/startsap_ASCS01.log file? Also check the SAP KBA I have mentioned in my previous reply. Your problem seems to be with the missing patches at the AIX level.

[former_member252955]

Below is the log from home directory.

Trace of system startup/check of SAP System DEV on Fri Mar  4 23:24:20 EETDT 2016

Called command: /usr/sap/DEV/SYS/exe/run/startsap start

-------------------------------------------

23:24:21

Starting Startup Agent sapstartsrv

-------------------------------------------

23:24:32

Instance Service on host unidev started

-------------------------------------------

23:24:32

starting SAP Instance ASCS01

-------------------------------------------

23:24:32

Startup-Log is written to /home/devadm/startsap_ASCS01.log

-------------------------------------------

23:24:32

/usr/sap/DEV/ASCS01/exe/sapcontrol -prot NI_HTTP -nr 01 -function Start

Regards

Prakash

[Reagan]

Run the below command as devadm and check the processes running at the OS level.

sapcontrol -nr 01 -function GetProcessList

If not then you will need to review the AIX patches mentioned in the KBA

[former_member252955]

Thanks

I requested AIX Admin team to look in to it. I will update once it done.

Regards

Prakash

[former_member185239]

Hi Prakash,

Can you please paste the latest log from the work directory of ASCS01.

With Regards

Ashutosh Chaturvedi

[former_member252955]

Below are the log from work directory of ASCS01

---------------------------------------------------

trc file: "sapstartsrv.log", trc level: 0, release: "721"

---------------------------------------------------

pid        8519930

[Thr 01] Fri Mar  4 17:55:13 2016

No halib defined => HA support disabled

CCMS agent initialization for instance type SCS: return code 0.

CCMS agent start: return code 0.

Initializing SAPControl Webservice

Starting AutoRestart thread

[Thr 515] Fri Mar  4 17:55:14 2016

AutoRestart thread started

SapSSLInit failed => https support disabled

Starting WebService thread

Webservice thread started, listening on port 50113

Trusted http connect via Unix domain socket '/tmp/.sapstream50113' enabled.

trusted unix domain socket user is stopping SAP System at 2016/03/04 17:55:26

trusted unix domain socket user is starting SAP System at 2016/03/04 17:55:29

[Sriram2009]

Hi Prakash.

Just do the full system restart and then check the same phase. if it is same issue kindly update the SAPinst log as attachment.

Regards

SS

[former_member252955]

Hi Sriram

There was service in the name of NIMSH which was using 3901 and 3902 port so was getting Instance inconsistent error but since I stopped this service and started again from that time it stucks on starting ascs01.

I have stopped and started again SAPINST.

------------------------------------------------------------------------------------------------------------------------------------------

INFO 2016-03-04 16:16:48.181 (root/sapinst)

Creating file /upgrade/unitmp/dev_sap_kernel_test_04_Mar_2016_16_14_09.

INFO 2016-03-04 16:16:48.182 (root/sapinst)

Removed file /upgrade/unitmp/dev_sap_kernel_test_04_Mar_2016_16_14_09.

INFO 2016-03-04 16:16:48.183 (root/sapinst)

Real group ID set to 209.

INFO 2016-03-04 16:16:48.184 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_exe.6881336.1457068429.

INFO 2016-03-04 16:16:48.184 (root/sapinst)

Working directory changed to /upgrade/dev.

INFO 2016-03-04 16:16:48.184 (root/sapinst)

Creating file /upgrade/unitmp/dev_sap_kernel_test_04_Mar_2016_16_14_11.

INFO 2016-03-04 16:16:48.185 (root/sapinst)

Removed file /upgrade/unitmp/dev_sap_kernel_test_04_Mar_2016_16_14_11.

WARNING 2016-03-04 16:16:48.185 (root/sapinst)

The environment variable DISPLAY is not set; therefore the GUI cannot be started.

INFO 2016-03-04 16:16:48.185 (root/sapinst)

Creating file /upgrade/unitmp/sapinst_instdir/x.

INFO 2016-03-04 16:16:48.186 (root/sapinst)

Removed file /upgrade/unitmp/sapinst_instdir/x.

INFO 2016-03-04 16:16:48.186 (root/sapinst)

Authorizations set for /upgrade/unitmp/sapinst_instdir.

INFO 2016-03-04 16:16:48.188 (root/sapinst)

Creating file /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP/x.

INFO 2016-03-04 16:16:48.188 (root/sapinst)

Removed file /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP/x.

INFO 2016-03-04 16:16:48.190 (root/sapinst)

Creating file /upgrade/unitmp/sapinst_instdir/x.

INFO 2016-03-04 16:16:48.190 (root/sapinst)

Removed file /upgrade/unitmp/sapinst_instdir/x.

INFO 2016-03-04 16:16:48.191 (root/sapinst)

Authorizations set for /upgrade/unitmp/sapinst_instdir.

INFO 2016-03-04 16:16:48.191 (root/sapinst)

Copied file '/upgrade/unitmp/dev_sap_kernel' to '/upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP'.

INFO 2016-03-04 16:16:48.192 (root/sapinst)

Removed file /upgrade/unitmp/dev_sap_kernel.

INFO 2016-03-04 16:16:48.192 (root/sapinst)

Copied file '/upgrade/dev/dev_sap_kernel' to '/upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP'.

INFO 2016-03-04 16:16:48.192 (root/sapinst)

Removed file /upgrade/dev/dev_sap_kernel.

INFO 2016-03-04 16:16:48.643 (root/sapinst)

Copied file '/upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP/keydb.xml' to '/upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP/keydb.3.xml'.

INFO 2016-03-04 16:16:50.013 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_exe.6881336.1457068429.

INFO 2016-03-04 16:16:50.020 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP.

INFO 2016-03-04 16:16:50.026 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_exe.6881336.1457068429.

INFO 2016-03-04 16:16:50.033 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP.

INFO 2016-03-04 16:16:50.159 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_exe.6881336.1457068429.

INFO 2016-03-04 16:16:50.166 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP.

INFO 2016-03-04 16:16:50.473 (root/sapinst)

Copied file '/upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP/inifile.xml' to '/upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP/inifile.2.xml'.

INFO 2016-03-04 16:19:20.467 (root/sapinst)

Execute step startASCSInstance of component |NW_ABAP_OneHost|ind|ind|ind|ind|0|0|NW_Onehost_System|ind|ind|ind|ind|onehost|0|NW_CI_Instance|ind|ind|ind|ind|ci|0

INFO 2016-03-04 16:20:02.542 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_exe.6881336.1457068429.

INFO 2016-03-04 16:20:02.558 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP.

INFO 2016-03-04 16:20:02.686 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_exe.6881336.1457068429.

INFO 2016-03-04 16:20:02.693 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP.

INFO 2016-03-04 16:20:02.693 (root/sapinst)

Starting instance DEV/ASCS01...

INFO 2016-03-04 16:20:03.137 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_exe.6881336.1457068429.

INFO 2016-03-04 16:20:03.191 (root/sapinst)

Working directory changed to /upgrade/unitmp/sapinst_instdir/BS2011/ERP606/LM/COPY/DB6/SYSTEM/CENTRAL/AS-ABAP.

INFO 2016-03-04 16:20:03.340 (root/sapinst)

Node sapcontrol_DEV_ASCS01.log already exists.

INFO 2016-03-04 16:20:03.550 (root/sapinst)

Output of /usr/sap/DEV/ASCS01/exe/sapcontrol -prot NI_HTTP -nr 01 -function GetInstanceProperties is written to the logfile sapcontrol_DEV_ASCS01.log.

-----------------------------------------------------------------------------------------------------------------------------------------------

[Sriram2009]

Hi Prakash.

In this case you have to change the instance number from 01 to 03 and above.you have to start the fresh installation with different instance number.

Regards

SS

[former_member252955]

HI Sriram

I have stopped NIMSH service else the error was ASCS instance is inconsistent state.

I have done full restart as well but no success.

Once instance build over I will change instance port. At this moment no service running on 3901 and 3902.

Regards

Prakash

[Sriram2009]

Hi Prakash.

When your are starting the new installation you have to manually select the SAP instance number to 05.

Regards

SS

[former_member252955]

Hi Sriram

At this moment no service is running on 3901 and 3902 so no issue with instance no.

We do not want to change the instance no.

Regards

Prakash

[Sriram2009]

Hi Prakash.

You can check with OS admin team how to change the Nimsh instance number and then start the SAP installation

Regards

SS

[former_member252955]

Hi Sriram

Do I need to uninstall system and then install, is this mandatory?

Regards

Prakash

[Sriram2009]

Hi Prakash.

It will be better if your are doing reinstall the OS with different Nimsh and then start the SAP with 01 instance number. An year back we faced same issue in AIX

Regards

SS

[former_member252955]

Hi Sriram

My concern from re installation is only for SAP system no for AIX.

Is there any other workaround for this as I am at end of SAP System build phase.

Regards

Prakash

[Sriram2009]

Hi Prakash.

I am not that much sound in AIX, Kindly refer the blog

nimsh and SAP (Chris&amp;#39;s AIX Blog)

Regards

SS