SAP Cluster service issue

Here is the description of the PRD cluster scenario. ( windows 2008 + oracle)

We have 2 nodes .

1. host-erpn01 ( Have ASCS , Database instance, Enqueue and Dialog

Instance installed)

2. host-erp02 ( Have Central Instance, Dialog Instance and Enqueue installed)

When we move "SAP SID" service using "failover cluster management tool" from one node to another its fails and we have to manually select the "SAP SID cluster service" and "SAP SID cluster instance" to online.

These both service and instance were coming online after manual selection, however after some time in the mmc console of node 2 the sap instances hosted on node1 are in red cross and are giving " cannot connect to sap service dcom interface error 800706BA"

We replaced the sapstartsrv.exe from working directory of ASCS instance to CI executable directory.

Now the disp+work is stopped for CI instance. Also in the CI instance executable directory we can see five files with name of sapstartsrv i.e

sapstartsrv.exe.new , sapstartsrv.exe.tmp, sapstartsrv.new, sapstartsrv.pdb and actual sapstartsrv.exe file.

Here is the log of sapstartsrv.log CI work directory from node2.

"----

trc file: "sapstartsrv.log", trc level: 0, release: "701"

pid 1968

Mon Oct 11 15:55:33 2010

SAP HA Trace: Build in SAP Microsoft Cluster library '701, patch 32, changelist 1046543' initialized

Initializing SAPControl Webservice

SapSSLInit failed => https support disabled

Starting WebService Named Pipe thread

Starting WebService thread

Webservice named pipe thread started, listening on port
.\pipe\sapcontrol_01

Webservice thread started, listening on port 50113

GCCIA\csrvadmin is starting SAP System at 2010/10/11 16:09:07

SAP HA Trace: FindClusterResource: SAP resource not found [sapwinha.cpp, line 334]

SAP HA Trace: SAP_HA_FindSAPInstance returns: SAP_HA_NOT_CLUSTERED [sapwinha.cpp, line 907]"

or you can view other logs from the work directory dump at

http://s000.tinyupload.com/index.php?file_id=45384422007535688902

Now when we try to start the SAPSID_00 service manually its giving error "The SAPSID_00 service failed to start due to the following error: The system cannot find the path specified.

Please advice.

Regards

Edited by: Tech GCCIA on Oct 11, 2010 3:27 PM

Edited by: Tech GCCIA on Oct 11, 2010 3:28 PM

SAP Managed Tags:
Windows Server

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Accepted Solutions (0)

Answers (3)

The issue got resolved by adding "cluster disk" resource to the dependency of SAP SID 00 service in failover cluster management tool.

Now the switchover is working fine.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Great, happy to see that your issue resolved.

I got same problem and finally solved it. My analysis is that sapstartsrv.exe unable to response timely to SAP cluster resouces monitor and thus it's status is marked as failed. My solution is to increase value of "Delay Time Before Polling" from 10 to 60. If you still got same problem, try increase a bit, e.g. 70, 80, 90, ... and you should be able to fix the problem.

For custer issue depends how you configured the cluster groups. Make sure you are able to ping from bother server to each other with host names.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

for sapstartsrv.exe follow Note 1345206 - Handling and preventing sapstartsrv.exe corruptions

The internal update mechanism only takes place, if sapstartsrv.exe.new (which is a copy of sapstartsrv.exe and should have the same time stamp) is newer than sapstartsrv.exe.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

The stamp of sapstartsrv.exe and sapstartsrv.exe.new is as follows:

sapstartsrv.exe 2/25/2009 2:40 A.M

sapstartsrv.exe.new 2/25/2009 2:41 A.M

I have already replaced the sapstartsrv.exe.new to sapstartsrv.exe , however the issue persists .

SAP and DB cluster group are reachable from from nodes and have valid entry in DNS server.

Please advice.

Regards

Is your sapstartsrv issue resolved or not? You have to make sure that the timestamp on sapstartsrv.exe and sapstartsrv.exe.new are same.

if yes, then can you paste the new error logs?

Hi Sunil ,

The issue is there as mentioned in the first post . We tried to replace the sapstartsrv from ASCS executable directory however , the timestamp of both files are not same and the disp+work process is stopped.

Regards

Your SAP applications won't comeup until you make sure that the timestamp on both files are same. follow the SAP mentioned earlier.

In the DIR_CT_RUN directory, copy sapstartsrv.exe.new to sapstartsrv.exe.

Copy sapstartsrv.exe from the DIR_CT_RUN directory to the DIR_EXECUTABLE directory.

I replaced the sapstartsrv.exe.new to sapstartsrv.exe in un/ntadm64 folder of shared disk and copied the new sapstartsrv.exe to /exe folder of ASCS instance shared disk.

The time stamp of sapstartsrv.exe and sapstartsrv.exe.new is same at both ASCS executable directory and also at CI ( local disk) executable directory , however the disp+work again stopped after coming green for a while.

Please advice.

Regards

Ok, can you please paste the logs from dev_disp and dev_w0

Please find the logs form Central instance work directory.

dev_disp logs

trc file: "dev_disp", trc level: 1, release: "701"

sysno 01

sid GCP

systemid 562 (PC with Windows NT)

relno 7010

patchlevel 0

patchno 32

intno 20020600

make: multithreaded, Unicode, 64 bit, optimized

pid 4136

Mon Oct 11 18:05:42 2010

kernel runs with dp version 241000(ext=110000) (@(#) DPLIB-INT-VERSION-241000-UC)

length of sys_adm_ext is 576 bytes

SWITCH TRC-HIDE on ***

***LOG Q00=> DpSapEnvInit, DPStart (01 4136) [dpxxdisp.c 1286]

shared lib "dw_xml.dll" version 32 successfully loaded

shared lib "dw_xtc.dll" version 32 successfully loaded

shared lib "dw_stl.dll" version 32 successfully loaded

shared lib "dw_gui.dll" version 32 successfully loaded

shared lib "dw_mdm.dll" version 32 successfully loaded

rdisp/softcancel_sequence : -> 0,5,-1

use internal message server connection to port 3900

Mon Oct 11 18:05:47 2010

WARNING => DpNetCheck: NiAddrToHost(1.0.0.0) took 5 seconds

***LOG GZZ=> 1 possible network problems detected - check tracefile and adjust the DNS settings [dpxxtool2.c 5529]

MtxInit: 30000 0 0

DpSysAdmExtInit: ABAP is active

DpSysAdmExtInit: VMC (JAVA VM in WP) is not active

DpIPCInit2: start server >gccia-erpn02_GCP_01 <

DpShMCreate: sizeof(wp_adm) 28032 (1752)

DpShMCreate: sizeof(tm_adm) 5912704 (29416)

DpShMCreate: sizeof(wp_ca_adm) 24064 (80)

DpShMCreate: sizeof(appc_ca_adm) 8000 (80)

DpCommTableSize: max/headSize/ftSize/tableSize=500/16/552064/552080

DpShMCreate: sizeof(comm_adm) 552080 (1088)

DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0

DpShMCreate: sizeof(slock_adm) 0 (104)

DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0

DpShMCreate: sizeof(file_adm) 0 (72)

DpShMCreate: sizeof(vmc_adm) 0 (1864)

DpShMCreate: sizeof(wall_adm) (41664/36752/64/192)

DpShMCreate: sizeof(gw_adm) 48

DpShMCreate: SHM_DP_ADM_KEY (addr: 0000000008600050, size: 6612384)

DpShMCreate: allocated sys_adm at 0000000008600050

DpShMCreate: allocated wp_adm at 0000000008602270

DpShMCreate: allocated tm_adm_list at 0000000008608FF0

DpShMCreate: allocated tm_adm at 0000000008609050

DpShMCreate: allocated wp_ca_adm at 0000000008BAC8D0

DpShMCreate: allocated appc_ca_adm at 0000000008BB26D0

DpShMCreate: allocated comm_adm at 0000000008BB4610

DpShMCreate: system runs without slock table

DpShMCreate: system runs without file table

DpShMCreate: allocated vmc_adm_list at 0000000008C3B2A0

DpShMCreate: allocated gw_adm at 0000000008C3B320

DpShMCreate: system runs without vmc_adm

DpShMCreate: allocated ca_info at 0000000008C3B350

DpShMCreate: allocated wall_adm at 0000000008C3B360

MBUF state OFF

DpCommInitTable: init table for 500 entries

rdisp/queue_size_check_value : -> off

ThTaskStatus: rdisp/reset_online_during_debug 0

EmInit: MmSetImplementation( 2 ).

MM global diagnostic options set: 0

<ES> client 0 initializing ....

<ES> InitFreeList

<ES> block size is 4096 kByte.

Using implementation view

<EsNT> Using memory model view.

<EsNT> Memory Reset disabled as NT default

<ES> 4092 blocks reserved for free list.

ES initialized.

rdisp/http_min_wait_dia_wp : 1 -> 1

***LOG CPS=> DpLoopInit, ICU ( 3.0 3.0 4.0.1) [dpxxdisp.c 1681]

***LOG Q0K=> DpMsAttach, mscon ( SAPGCP) [dpxxdisp.c 12483]

DpStartStopMsg: send start message (myname is >gccia-erpn02_GCP_01 <)

DpStartStopMsg: start msg sent to message server o.k.

CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.

CCMS: Initalizing shared memory of size 60000000 for monitoring segment.

CCMS: Checking Downtime Configuration of Monitoring Segment.

CCMS: start to initalize 3.X shared alert area (first segment).

DpMsgAdmin: Set release to 7010, patchlevel 0

MBUF state PREPARED

MBUF component UP

DpMBufHwIdSet: set Hardware-ID

***LOG Q1C=> DpMBufHwIdSet [dpxxmbuf.c 1048]

DpMsgAdmin: Set patchno for this platform to 32

Release check o.K.

Mon Oct 11 18:06:47 2010

my types changed after wp death/restart 0xbb --> 0xb9

my types changed after wp death/restart 0xb9 --> 0xa9

my types changed after wp death/restart 0xa9 --> 0x89

Mon Oct 11 18:07:07 2010

ERROR => DpWPCheck: W0 (pid 3744) died (severity=0, status=0) [dpxxdisp.c 15662]
ERROR => DpWPCheck: W2 (pid 972) died (severity=0, status=0) [dpxxdisp.c 15662]
ERROR => DpWPCheck: W3 (pid 4588) died (severity=0, status=0) [dpxxdisp.c 15662]
ERROR => DpWPCheck: W5 (pid 4120) died (severity=0, status=0) [dpxxdisp.c 15662]

my types changed after wp death/restart 0x89 --> 0x88

ERROR => DpWPCheck: W9 (pid 1696) died (severity=0, status=0) [dpxxdisp.c 15662]
ERROR => DpWPCheck: W10 (pid 4472) died (severity=0, status=0) [dpxxdisp.c 15662]

my types changed after wp death/restart 0x88 --> 0x80

ERROR => DpWPCheck: W13 (pid 4432) died (severity=0, status=0) [dpxxdisp.c 15662]
ERROR => DpWPCheck: W14 (pid 3428) died (severity=0, status=0) [dpxxdisp.c 15662]
ERROR => DpWPCheck: W15 (pid 3700) died (severity=0, status=0) [dpxxdisp.c 15662]
DP_FATAL_ERROR => DpWPCheck: no more work processes
DISPATCHER EMERGENCY SHUTDOWN ***

increase tracelevel of WPs

NiWait: sleep (10000ms) ...

NiISelect: timeout 10000ms

NiISelect: maximum fd=873

NiISelect: read-mask is NULL

NiISelect: write-mask is NULL

Mon Oct 11 18:07:17 2010

NiISelect: TIMEOUT occured (10000ms)

dump system status

Workprocess Table (long) Mon Oct 11 15:07:17 2010

========================

No Ty. Pid Status Cause Start Err Sem CPU Time Program Cl User Action Table

0 DIA 3744 Ended no 2 0 0

1 DIA 3828 Ended no 1 0 0

2 DIA 972 Ended no 2 0 0

3 DIA 4588 Ended no 2 0 0

4 DIA 2576 Ended no 1 0 0

5 DIA 4120 Ended no 2 0 0

6 DIA 4888 Ended no 1 0 0

7 DIA 4992 Ended no 1 0 0

8 DIA 3280 Ended no 1 0 0

9 DIA 1696 Ended no 2 0 0

10 UPD 4472 Ended no 2 0 0

11 BTC 3616 Ended no 1 0 0

12 BTC 3712 Ended no 1 0 0

13 BTC 4432 Ended no 2 0 0

14 SPO 3428 Ended no 2 0 0

15 UP2 3700 Ended no 2 0 0

Dispatcher Queue Statistics Mon Oct 11 15:07:17 2010

===========================

--------

+
+
+
+--

Typ

now

high

max

writes

reads

--------

+
+
+
+--

NOWP

2000

--------

+
+
+
+--

DIA

2000

--------

+
+
+
+--

UPD

2000

--------

+
+
+
+--

ENQ

2000

--------

+
+
+
+--

BTC

2000

--------

+
+
+
+--

SPO

2000

--------

+
+
+
+--

UP2

2000

--------

+
+
+
+--

max_rq_id 13

wake_evt_udp_now 0

wake events total 9, udp 7 ( 77%), shm 2 ( 22%)

since last update total 9, udp 7 ( 77%), shm 2 ( 22%)

Dump of tm_adm structure: Mon Oct 11 15:07:17 2010

=========================

Term uid man user term lastop mod wp ta a/i (modes)

Workprocess Comm. Area Blocks Mon Oct 11 15:07:17 2010

=============================

Slots: 300, Used: 1, Max: 0

--------

+
+--

owner

pid

eyecatcher

--------

+
+--

DISPATCHER

-1

WPCAAD000

NiWait: sleep (5000ms) ...

NiISelect: timeout 5000ms

NiISelect: maximum fd=873

NiISelect: read-mask is NULL

NiISelect: write-mask is NULL

Mon Oct 11 18:07:22 2010

NiISelect: TIMEOUT occured (5000ms)

DpHalt: shutdown server >gccia-erpn02_GCP_01 < (normal)

DpJ2eeDisableRestart

DpModState: buffer in state MBUF_PREPARED

NiBufSend starting

NiIWrite: hdl 2 sent data (wrt=110,pac=1,MESG_IO)

MsINiWrite: sent 110 bytes

MsIModState: change state to SHUTDOWN

DpModState: change server state from STARTING to SHUTDOWN

Switch off Shared memory profiling

ShmProtect( 57, 3 )

ShmProtect(SHM_PROFILE, SHM_PROT_RW

ShmProtect( 57, 1 )

ShmProtect(SHM_PROFILE, SHM_PROT_RD

DpWakeUpWps: wake up all wp's

Stop work processes

Stop gateway

killing process (5116) (SOFT_KILL)

Stop icman

killing process (1724) (SOFT_KILL)

Terminate gui connections

wait for end of work processes

wait for end of gateway

[DpProcDied] Process lives (PID:5116 HANDLE:748)

waiting for termination of gateway ...

NiWait: sleep (1000ms) ...

NiISelect: timeout 1000ms

NiISelect: maximum fd=873

NiISelect: read-mask is NULL

NiISelect: write-mask is NULL

Mon Oct 11 18:07:23 2010

NiISelect: TIMEOUT occured (1000ms)

[DpProcDied] Process died (PID:5116 HANDLE:748)

wait for end of icman

[DpProcDied] Process lives (PID:1724 HANDLE:760)

waiting for termination of icman ...

NiWait: sleep (1000ms) ...

NiISelect: timeout 1000ms

NiISelect: maximum fd=873

NiISelect: read-mask is NULL

NiISelect: write-mask is NULL

Mon Oct 11 18:07:24 2010

NiISelect: TIMEOUT occured (1000ms)

[DpProcDied] Process died (PID:1724 HANDLE:760)

DpStartStopMsg: send stop message (myname is >gccia-erpn02_GCP_01 <)

DpStartStopMsg: Write AD_STARTSTOP message with type= 0, name=gccia-erpn02_GCP_01 , sapsysnr= 1, hostname=gccia-erpn02

AdGetSelfIdentRecord: > <

AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0

AdCvtRecToExt: opcode 4 (AD_STARTSTOP), ser 0, ex 0, errno 0

DpConvertRequest: net size = 189 bytes

NiBufSend starting

NiIWrite: hdl 2 sent data (wrt=562,pac=1,MESG_IO)

MsINiWrite: sent 562 bytes

send msg (len 110+452) to MSG_SERVER, type 1

DpStartStopMsg: stop msg sent to message server o.k.

NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)

NiBufIIn: NIBUF len=274

NiBufIIn: packet complete for hdl 2

NiBufReceive starting

MsINiRead: received 274 bytes

MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -

DpHalt: received 164 bytes from message server

NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)

NiBufIIn: NIBUF len=274

NiBufIIn: packet complete for hdl 2

NiBufReceive starting

MsINiRead: received 274 bytes

MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -

DpHalt: received 164 bytes from message server

NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)

NiBufIIn: NIBUF len=274

NiBufIIn: packet complete for hdl 2

NiBufReceive starting

MsINiRead: received 274 bytes

MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -

DpHalt: received 164 bytes from message server

NiIRead: hdl 2 recv would block (errno=EAGAIN)

NiIRead: read for hdl 2 timed out (0ms)

DpHalt: no more messages from the message server

DpHalt: send keepalive to synchronize with the message server

NiBufSend starting

NiIWrite: hdl 2 sent data (wrt=114,pac=1,MESG_IO)

MsINiWrite: sent 114 bytes

send msg (len 110+4) to name MSG_SERVER, type 0, key -

MsSndName: MS_NOOP ok

Send 4 bytes to MSG_SERVER

NiIRead: hdl 2 recv would block (errno=EAGAIN)

NiIPeek: peek successful for hdl 2 (r)

NiIRead: hdl 2 received data (rcd=114,pac=1,MESG_IO)

NiBufIIn: NIBUF len=114

NiBufIIn: packet complete for hdl 2

NiBufReceive starting

MsINiRead: received 114 bytes

MSG received, len 110+4, flag 3, from MSG_SERVER , typ 0, key -

Received 4 bytes from MSG_SERVER

Received opcode MS_NOOP from msg_server, reply MSOP_OK

MsOpReceive: ok

MsSendKeepalive : keepalive sent to message server

NiIRead: hdl 2 recv would block (errno=EAGAIN)

Mon Oct 11 18:07:25 2010

NiIPeek: peek for hdl 2 timed out (r; 1000ms)

NiIRead: read for hdl 2 timed out (1000ms)

DpHalt: no more messages from the message server

DpHalt: sync with message server o.k.

detach from message server

***LOG Q0M=> DpMsDetach, ms_detach () [dpxxdisp.c 12829]

NiBufSend starting

NiIWrite: hdl 2 sent data (wrt=110,pac=1,MESG_IO)

MsINiWrite: sent 110 bytes

MsIDetach: send logout to msg_server

MsIDetach: call exit function

DpMsShutdownHook called

NiBufISelUpdate: new MODE -- (r-) for hdl 2 in set0

SiSelNSet: set events of sock 732 to: ---

NiBufISelRemove: remove hdl 2 from set0

SiSelNRemove: removed sock 732 (pos=2)

SiSelNRemove: removed sock 732

NiSelIRemove: removed hdl 2

MBUF state OFF

AdGetSelfIdentRecord: > <

AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0

AdCvtRecToExt: opcode 40 (AD_MSBUF), ser 0, ex 0, errno 0

blks_in_queue/wp_ca_blk_no/wp_max_no = 1/300/16

LOCK WP ca_blk 1

make DISP owner of wp_ca_blk 1

DpRqPutIntoQueue: put request into queue (reqtype 1, prio LOW, rq_id 16)

MBUF component DOWN

NiICloseHandle: shutdown and close hdl 2 / sock 732

NiBufIClose: clear extension for hdl 2

MsIDetach: detach MS-system

cleanup EM

EsCleanup ....

EmCleanup() -> 0

Es2Cleanup: Cleanup ES2

***LOG Q05=> DpHalt, DPStop ( 4136) [dpxxdisp.c 11045]

Good Bye .....

logs from dev_w0

trc file: "dev_w0", trc level: 1, release: "701"

ACTIVE TRACE LEVEL 1
ACTIVE TRACE COMPONENTS all, MJ

B Mon Oct 11 18:05:47 2010

B create_con (con_name=R/3)

B Loading DB library 'D:\usr\sap\GCP\DVEBMGS01\exe\dboraslib.dll' ...

B Library 'D:\usr\sap\GCP\DVEBMGS01\exe\dboraslib.dll' loaded

B Version of 'D:\usr\sap\GCP\DVEBMGS01\exe\dboraslib.dll' is "700.08", patchlevel (0.25)

B New connection 0 created

M sysno 01

M sid GCP

M systemid 562 (PC with Windows NT)

M relno 7010

M patchlevel 0

M patchno 32

M intno 20020600

M make: multithreaded, Unicode, 64 bit, optimized

M pid 3744

M kernel runs with dp version 241000(ext=110000) (@(#) DPLIB-INT-VERSION-241000-UC)

M length of sys_adm_ext is 576 bytes

M ***LOG Q0Q=> tskh_init, WPStart (Workproc 0 3744) [dpxxdisp.c 1348]

I MtxInit: 30000 0 0

M DpSysAdmExtCreate: ABAP is active

M DpSysAdmExtCreate: VMC (JAVA VM in WP) is not active

M DpShMCreate: sizeof(wp_adm) 28032 (1752)

M DpShMCreate: sizeof(tm_adm) 5912704 (29416)

M DpShMCreate: sizeof(wp_ca_adm) 24064 (80)

M DpShMCreate: sizeof(appc_ca_adm) 8000 (80)

M DpCommTableSize: max/headSize/ftSize/tableSize=500/16/552064/552080

M DpShMCreate: sizeof(comm_adm) 552080 (1088)

M DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0

M DpShMCreate: sizeof(slock_adm) 0 (104)

M DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0

M DpShMCreate: sizeof(file_adm) 0 (72)

M DpShMCreate: sizeof(vmc_adm) 0 (1864)

M DpShMCreate: sizeof(wall_adm) (41664/36752/64/192)

M DpShMCreate: sizeof(gw_adm) 48

M DpShMCreate: SHM_DP_ADM_KEY (addr: 000000000A600050, size: 6612384)

M DpShMCreate: allocated sys_adm at 000000000A600050

M DpShMCreate: allocated wp_adm at 000000000A602270

M DpShMCreate: allocated tm_adm_list at 000000000A608FF0

M DpShMCreate: allocated tm_adm at 000000000A609050

M DpShMCreate: allocated wp_ca_adm at 000000000ABAC8D0

M DpShMCreate: allocated appc_ca_adm at 000000000ABB26D0

M DpShMCreate: allocated comm_adm at 000000000ABB4610

M DpShMCreate: system runs without slock table

M DpShMCreate: system runs without file table

M DpShMCreate: allocated vmc_adm_list at 000000000AC3B2A0

M DpShMCreate: allocated gw_adm at 000000000AC3B320

M DpShMCreate: system runs without vmc_adm

M DpShMCreate: allocated ca_info at 000000000AC3B350

M DpShMCreate: allocated wall_adm at 000000000AC3B360

M rdisp/queue_size_check_value : -> off

M ThTaskStatus: rdisp/reset_online_during_debug 0

X EmInit: MmSetImplementation( 2 ).

X MM global diagnostic options set: 0

X <ES> client 0 initializing ....

X Using implementation view

X <EsNT> Using memory model view.

M <EsNT> Memory Reset disabled as NT default

X ES initialized.

M Mon Oct 11 18:05:48 2010

M ThInit: running on host gccia-erpn02

M calling db_connect ...

C Prepending D:\usr\sap\GCP\DVEBMGS01\exe to Path.

C Oracle Client Version: '10.2.0.4.0'

C Client NLS setting (OCINlsGetInfo): 'AMERICAN_AMERICA.UTF8'

C Logon as OPS$-user to get SAPSR3's password

C Connecting as /@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000457BFE0 0000000002D0D240 0000000002D0EA58

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000002D0E918,srvhp=0000000004583E38)

C Mon Oct 11 18:06:10 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

C Try to connect with default password

C Connecting as SAPSR3/<pwd>@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000457BFE0 0000000002D0D240 0000000002D0EA58

C Detaching from DB Server (con_hdl=0,svchp=0000000002D0E918,srvhp=0000000004583E38)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000002D0E918,srvhp=0000000004583E38)

C Mon Oct 11 18:06:31 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

B ***LOG BY2=> sql error 12170 performing CON [dbsh#2 @ 1208] [dbsh 1208 ]

B ***LOG BY0=> ORA-12170: TNS:Connect timeout occurred [dbsh#2 @ 1208] [dbsh 1208 ]

B ***LOG BY2=> sql error 12170 performing CON [dblink#3 @ 431] [dblink 0431 ]

B ***LOG BY0=> ORA-12170: TNS:Connect timeout occurred [dblink#3 @ 431] [dblink 0431 ]

M ***LOG R19=> ThInit, db_connect ( DB-Connect 000256) [thxxhead.c 1449]

M in_ThErrHandle: 1

M *** ERROR => ThInit: db_connect (step 1, th_errno 13, action 3, level 1) [thxxhead.c 10563]

M Info for wp 0

M pid = 3744

M severity = 0

M status = 0

M stat = WP_RUN

M waiting_for = NO_WAITING

M reqtype = DP_RQ_DIAWP

M act_reqtype = NO_REQTYPE

M rq_info = 0

M tid = -1

M mode = 255

M len = -1

M rq_id = 65535

M rq_source =

M last_tid = 0

M last_mode = 0

M semaphore = 0

M act_cs_count = 0

M csTrack = 0

M csTrackRwExcl = 0

M csTrackRwShrd = 0

M mode_cleaned_counter = 0

M control_flag = 0

M int_checked_resource(RFC) = 0

M ext_checked_resource(RFC) = 0

M int_checked_resource(HTTP) = 0

M ext_checked_resource(HTTP) = 0

M report = > <

M action = 0

M tab_name = > <

M attachedVm = no VM

M *****************************************************************************

M *

M * LOCATION SAP-Server gccia-erpn02_GCP_01 on host gccia-erpn02 (wp 0)

M * ERROR ThInit: db_connect

M *

M * TIME Mon Oct 11 18:06:31 2010

M * RELEASE 701

M * COMPONENT Taskhandler

M * VERSION 1

M * RC 13

M * MODULE thxxhead.c

M * LINE 10783

M * COUNTER 1

M *

M *****************************************************************************

M PfStatDisconnect: disconnect statistics

M Entering TH_CALLHOOKS

M ThCallHooks: call hook >BtcCallLgCl< for event BEFORE_DUMP

M ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP

M *** ERROR => ThrSaveSPAFields: no valid thr_wpadm [thxxrun1.c 723]

M *** ERROR => ThCallHooks: event handler ThrSaveSPAFields for event BEFORE_DUMP failed [thxxtool3.c 261]

M Entering ThSetStatError

M ThIErrHandle: do not call ThrCoreInfo (no_core_info=0, in_dynp_env=0)

M Entering ThReadDetachMode

M call ThrShutDown (1)...

M ***LOG Q02=> wp_halt, WPStop (Workproc 0 3744) [dpnttool.c 334]

your SAP applications are not able to communicate to DB as you got ora-12170.

Is your DB and DI/CI are now on same node or different.

Also make sure that from DI you are able to telnet to your oracle listner port.

follow http://www.orafaq.com/wiki/ORA-12170 may help.

we have ASCS /DB /DI on one node and CI on another node.

The firewall is off at both nodes and initially after the installation the services were started.However after moving the cluster from one node to another failed the services to startup.

Also , the DB cluster is successfully moving from one node to another.

What else we can check , are there any thing major to be taken care of when we restart any of the node.

Thanks & Regards

just for interest can you please celebrate your system numbers? ASCS / DI and CI

also which instance you are not able to start is that CI or DI?

If you configured the cluster correctly as per installation guide then nothing you have to do manually. all cluster group should run on all cluster nodes.

we have ASCS 00 , CI 01 , DI 01

We are not able to start the CI which was installed on node 2. It starts for a while and after that disp+work stops. Also when we move SAP SID cluster service from one node to another it does not automatically starts up we have to do that manually for "SAP SID 00 service" and "SAP SID 00 instance" . Manually these two comes online.

We strictly followed the guide while doing the installation .

Regards

As the latest development the DI instance disp+work gets stops after starting up.

the log of dev_w0

trc file: "dev_w0", trc level: 1, release: "701"

ACTIVE TRACE LEVEL 1
ACTIVE TRACE COMPONENTS all, MJ

B Mon Oct 11 20:10:42 2010

B create_con (con_name=R/3)

B Loading DB library 'D:\usr\sap\GCP\D01\exe\dboraslib.dll' ...

B Library 'D:\usr\sap\GCP\D01\exe\dboraslib.dll' loaded

B Version of 'D:\usr\sap\GCP\D01\exe\dboraslib.dll' is "700.08", patchlevel (0.25)

B New connection 0 created

M sysno 01

M sid GCP

M systemid 562 (PC with Windows NT)

M relno 7010

M patchlevel 0

M patchno 32

M intno 20020600

M make: multithreaded, Unicode, 64 bit, optimized

M pid 3144

M kernel runs with dp version 241000(ext=110000) (@(#) DPLIB-INT-VERSION-241000-UC)

M length of sys_adm_ext is 576 bytes

M ***LOG Q0Q=> tskh_init, WPStart (Workproc 0 3144) [dpxxdisp.c 1348]

I MtxInit: 30000 0 0

M DpSysAdmExtCreate: ABAP is active

M DpSysAdmExtCreate: VMC (JAVA VM in WP) is not active

M Mon Oct 11 20:10:43 2010

M DpShMCreate: sizeof(wp_adm) 22784 (1752)

M DpShMCreate: sizeof(tm_adm) 5912704 (29416)

M DpShMCreate: sizeof(wp_ca_adm) 24064 (80)

M DpShMCreate: sizeof(appc_ca_adm) 8000 (80)

M DpCommTableSize: max/headSize/ftSize/tableSize=500/16/552064/552080

M DpShMCreate: sizeof(comm_adm) 552080 (1088)

M DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0

M DpShMCreate: sizeof(slock_adm) 0 (104)

M DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0

M DpShMCreate: sizeof(file_adm) 0 (72)

M DpShMCreate: sizeof(vmc_adm) 0 (1864)

M DpShMCreate: sizeof(wall_adm) (41664/36752/64/192)

M DpShMCreate: sizeof(gw_adm) 48

M DpShMCreate: SHM_DP_ADM_KEY (addr: 000000000A400050, size: 6607136)

M DpShMCreate: allocated sys_adm at 000000000A400050

M DpShMCreate: allocated wp_adm at 000000000A402270

M DpShMCreate: allocated tm_adm_list at 000000000A407B70

M DpShMCreate: allocated tm_adm at 000000000A407BD0

M DpShMCreate: allocated wp_ca_adm at 000000000A9AB450

M DpShMCreate: allocated appc_ca_adm at 000000000A9B1250

M DpShMCreate: allocated comm_adm at 000000000A9B3190

M DpShMCreate: system runs without slock table

M DpShMCreate: system runs without file table

M DpShMCreate: allocated vmc_adm_list at 000000000AA39E20

M DpShMCreate: allocated gw_adm at 000000000AA39EA0

M DpShMCreate: system runs without vmc_adm

M DpShMCreate: allocated ca_info at 000000000AA39ED0

M DpShMCreate: allocated wall_adm at 000000000AA39EE0

M rdisp/queue_size_check_value : -> off

M ThTaskStatus: rdisp/reset_online_during_debug 0

X EmInit: MmSetImplementation( 2 ).

X MM global diagnostic options set: 0

X <ES> client 0 initializing ....

X Using implementation view

X <EsNT> Using memory model view.

M <EsNT> Memory Reset disabled as NT default

X ES initialized.

M ThInit: running on host gccia-erpn01

M Mon Oct 11 20:10:44 2010

M calling db_connect ...

C Prepending D:\usr\sap\GCP\D01\exe to Path.

C Oracle Client Version: '10.2.0.4.0'

C Client NLS setting (OCINlsGetInfo): 'AMERICAN_AMERICA.UTF8'

C Logon as OPS$-user to get SAPR3's password

C Connecting as /@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000436A700 0000000004372250 0000000004373A68

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004373928,srvhp=0000000004374E28)

C Mon Oct 11 20:10:53 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12154

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12154'

[dbsloci.c 11175]

C Try to connect with default password

C Connecting as SAPR3/<pwd>@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000436A700 0000000004372250 0000000004373A68

C Detaching from DB Server (con_hdl=0,svchp=0000000004373928,srvhp=0000000004374E28)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004373928,srvhp=0000000004374E28)

C Mon Oct 11 20:11:02 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12154

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12154'

[dbsloci.c 11175]

B ***LOG BV3=> severe db error 12154 ; work process is stopped [dbsh#2 @ 1203] [dbsh 1203 ]

B ***LOG BY2=> sql error 12154 performing CON [dblink#3 @ 431] [dblink 0431 ]

B ***LOG BY0=> ORA-12154: TNS:could not resolve the connect identifier specified [dblink#3 @ 431] [dblink 0431 ]

M ***LOG R19=> ThInit, db_connect ( DB-Connect 000256) [thxxhead.c 1449]

M in_ThErrHandle: 1

M *** ERROR => ThInit: db_connect (step 1, th_errno 13, action 3, level 1) [thxxhead.c 10563]

M Info for wp 0

M pid = 3144

M severity = 0

M status = 0

M stat = WP_RUN

M waiting_for = NO_WAITING

M reqtype = DP_RQ_DIAWP

M act_reqtype = NO_REQTYPE

M rq_info = 0

M tid = -1

M mode = 255

M len = -1

M rq_id = 65535

M rq_source =

M last_tid = 0

M last_mode = 0

M semaphore = 0

M act_cs_count = 0

M csTrack = 0

M csTrackRwExcl = 0

M csTrackRwShrd = 0

M mode_cleaned_counter = 0

M control_flag = 0

M int_checked_resource(RFC) = 0

M ext_checked_resource(RFC) = 0

M int_checked_resource(HTTP) = 0

M ext_checked_resource(HTTP) = 0

M report = > <

M action = 0

M tab_name = > <

M attachedVm = no VM

M *****************************************************************************

M *

M * LOCATION SAP-Server gccia-erpn01_GCP_01 on host gccia-erpn01 (wp 0)

M * ERROR ThInit: db_connect

M *

M * TIME Mon Oct 11 20:11:02 2010

M * RELEASE 701

M * COMPONENT Taskhandler

M * VERSION 1

M * RC 13

M * MODULE thxxhead.c

M * LINE 10783

M * COUNTER 1

M *

M *****************************************************************************

M PfStatDisconnect: disconnect statistics

M Entering TH_CALLHOOKS

M ThCallHooks: call hook >BtcCallLgCl< for event BEFORE_DUMP

M ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP

M *** ERROR => ThrSaveSPAFields: no valid thr_wpadm [thxxrun1.c 723]

M *** ERROR => ThCallHooks: event handler ThrSaveSPAFields for event BEFORE_DUMP failed [thxxtool3.c 261]

M Entering ThSetStatError

M ThIErrHandle: do not call ThrCoreInfo (no_core_info=0, in_dynp_env=0)

M Entering ThReadDetachMode

M call ThrShutDown (1)...

M ***LOG Q02=> wp_halt, WPStop (Workproc 0 3144) [dpnttool.c 334]

>1. host-erpn01 ( Have ASCS , Database instance, Enqueue and Dialog Instance installed)

>2. host-erp02 ( Have Central Instance, Dialog Instance and Enqueue installed)

>we have ASCS 00 , CI 01 , DI 01

the above scenario only work without cluster failure. How come you have CI and DI as 01 instance number as they are on clustered servers. This will only work if you have DI on different host as you have used 01 instance number.

Can you please stop DI and perform cluster failover with ASCS, DB and CI, I think it should be OK then. If possible you can also un-install DI if possible to avoid confusion.

Hi Sunil ,

Apologizes for any misleading , here is the scenario.

MSCS Node1 : Have ASCS , Database Instance , Dialog Instance and Enqueue instances ( Created Oracle Failover group on this host)

MSCS Node2 : Have central Instance and enqueue instance.

The DIalog Instance and Central Instance have same instance number " 01" as per the guide .

Guide: "SAP ERP 6.0 - EHP4 Ready SR1 ABAP on Windows: Oracle"

Page 160 : "Central instance and dialog instance you enter the same instance number."

As of latest development i changed the system environment variable DBMS_TYPE=ORA at both nodes and started the services. At that time system started working fine and i was able to login into the system.

After couple of hours the Dialog instance (disp+work ) again went to yellow state and the system was not working.

Right now we have DB cluster role on node 2 and SAP cluster role on node1 .

I checked the dev_w0 log file of dispacher and found that these errors ...i am still not confirm about the issue ..

as after this issue i tried to connect DB through sqlplus and was not able to at host1 , when i moved the DB role to host1 we were able to connect to DB through sqlplus .

What can be the issue ?

Try to connect with default password

C Connecting as SAPSR3/<pwd>@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000496CB90 00000000049746E0 0000000004975EF8

C Detaching from DB Server (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Tue Oct 12 09:47:37 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

C Tue Oct 12 09:47:42 2010

C Logon as OPS$-user to get SAPSR3's password

C Connecting as /@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000496CB90 00000000049746E0 0000000004975EF8

C Detaching from DB Server (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Tue Oct 12 09:48:03 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

C Try to connect with default password

C Connecting as SAPSR3/<pwd>@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000496CB90 00000000049746E0 0000000004975EF8

C Detaching from DB Server (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Tue Oct 12 09:48:24 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

B Reconnect failed for connection:

B 0: name = R/3, con_id = 000000000 state = DISCONNECTED, perm = YES, reco = YES, timeout = 000, con_max = 255, con_opt = 255, occ = NO

M *** ERROR => ThHdlReconnect: db_reconnect failed (2048) [thxxhead.c 17238]

B db_con_reconnect disconnecting connection:

B 0: name = R/3, con_id = 000000000 state = DISCONNECTED, perm = YES, reco = YES, timeout = 000, con_max = 255, con_opt = 255, occ = NO

B db_con_reconnect performing the reconnect for con:

B 0: name = R/3, con_id = 000000000 state = DISCONNECTED, perm = YES, reco = YES, timeout = 000, con_max = 255, con_opt = 255, occ = NO

C Logon as OPS$-user to get SAPSR3's password

C Connecting as /@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000496CB90 00000000049746E0 0000000004975EF8

C Detaching from DB Server (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Tue Oct 12 09:48:45 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

C Try to connect with default password

C Connecting as SAPSR3/<pwd>@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000496CB90 00000000049746E0 0000000004975EF8

C Detaching from DB Server (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Tue Oct 12 09:49:06 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

C Tue Oct 12 09:49:11 2010

C Logon as OPS$-user to get SAPSR3's password

C Connecting as /@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000496CB90 00000000049746E0 0000000004975EF8

C Detaching from DB Server (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Tue Oct 12 09:49:33 2010

C *** ERROR => OCI-call 'OCIServerAttach' failed with rc=12170

[dboci.c 4447]

C *** ERROR => CONNECT failed with sql error '12170'

[dbsloci.c 11175]

C Try to connect with default password

C Connecting as SAPSR3/<pwd>@GCP on connection 0 (nls_hdl 0) ... (dbsl 700 151208)

C Nls CharacterSet NationalCharSet C EnvHp ErrHp ErrHpBatch

C 0 UTF8 0 000000000496CB90 00000000049746E0 0000000004975EF8

C Detaching from DB Server (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

C Attaching to DB Server GCP (con_hdl=0,svchp=0000000004975DB8,srvhp=00000000049772B8)

Also , when i check the sap services of other node through mmc , they are in red symbol marked and throws error

cannot connect sap service on SAPSID

web service error 20

unknown error

tcp connect failed in tcp_connect()

DCOM interface error : 800706BA

the RPC server is unavailable.

Regards

hm! I have gone through the instalaltion docs and yes, you can use the same instance number. let me go through the logs which have provided.

can you please pate listrener trace file logs?

Hi Sunil ,

On node 1 there is no listener.trc at /oracle_home/network/trace folder , here is the log of listener.log file in case if it is helpful.

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 10:37:37

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=3116

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCP.WORLDipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCPipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=gccia-erpn01.gccia.com.sa)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 11:59:37

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=5036

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCP.WORLDipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCPipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

10-OCT-2010 12:00:31 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=60592)) * establish * GCP * 0

10-OCT-2010 13:01:19 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61336)) * establish * GCP * 0

10-OCT-2010 13:01:37 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61340)) * establish * GCP * 0

10-OCT-2010 13:08:27 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61485)) * establish * GCP * 0

10-OCT-2010 13:08:42 * (CONNECT_DATA=(SID=GCP)(GLOBAL_NAME=GCP.WORLD)(CID=(PROGRAM=D:\oracle\OFS\SRV\fs\fssvr\bin\FsSurrogate.exe)(HOST=GCCIA-ERPN01)(USER=csrvadmin))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=61489)) * establish * GCP * 0

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 13:09:57

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=2336

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCP.WORLDipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCPipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 13:14:34

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=4948

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCP.WORLDipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCPipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 13:38:12

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=2456

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCP.WORLDipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCPipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 14:03:35

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=2756

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCP.WORLDipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCPipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 10-OCT-2010 14:10:42

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=4812

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCP.WORLDipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=
.\pipe\GCPipc)))

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 11-OCT-2010 09:34:05

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=1920

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

TNSLSNR for 64-bit Windows: Version 10.2.0.4.0 - Production on 11-OCT-2010 21:12:29

System parameter file is D:\oracle\GCP\102\network\admin\listener.ora

Log messages written to D:\oracle\GCP\102\network\log\listener.log

Trace information written to D:\oracle\GCP\102\network\trace\listener.trc

Trace level is currently 0

Started with pid=1952

Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.11.13)(PORT=1527)))

Listener completed notification to CRS on start

TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE

hm! nothing much in listener log.

just to refresh

>Right now we have DB cluster role on node 2 and SAP cluster role on node1

you have now DB on node2

SAP on node1 (ASCS and CI)

and DI on node2

and you have issues with DI not stable is this correct now?

HI Sunil ,

Now all other issues has been resolved by setting up the correct DB parameters and the sap services are working fine

However i have same initial issue.

i.e

From cluster management tool when i tries to move the "SAP SID" cluster service to another node , the resources "SAP SID 00 Service" and "SAP SID 00 Instance" gets failed. When i manually bring them up they works fine.

In windows event i have

"Generic application 'SAP GCP 00 Service' could not be brought online (with error '3') during an attempt to start the service. Possible cause: the specified service parameters might be invalid."

"Cluster resource 'SAP GCP 00 Service' in clustered service or application 'SAP GCP' failed."

"The Cluster service failed to bring clustered service or application 'SAP SID' completely online or offline. One or more resources may be in a failed state. This may impact the availability of the clustered service or application."

Whats can be the issue , should i manually create the SAP SID 00 Service or what ....any ideas?

Regards

Edited by: Tech GCCIA on Oct 12, 2010 2:02 PM

Great to hear the some of the issues has been resolved.

Now you have 2 cluster groups (SAP and DB), under SAP you may have ASCS and CI resources? and under DB cluster group you have DB resources. is this correct?

Yes you are correct.

Regards

OK, now you want to perform cluster test. follow as:

Bring up SAP cluster group on node1

Bring up DB cluster group on node2

and Dialog instance on node2

This is now working OK right?

Now perform as below:

Move DB cluster group to node1, this should also work.

and only option left is moving SAP to node2 which I will inform you what can be done if you do above tests first.

I have DB GROUP on NODE1 , SAP GROUP on NODE1 ,

How to move Dialog Instance ?

The DB GROUP is moving successfully from one node to another.

Regards

You can't move your DI as a part of your cluster and this is not installed under cluster.

Refer installation guide page 159, You install the dialog instance on a host outside of MSCS.

Yes we have done same as per guide we have installed the DI on node1 of our environment.

Please advice how can we resolve this issue of automatically startup of the SAP SID service while moving it up .

I have moved the DB to another node as per your instructions and its working fine.

Regards

Ok, this mean you have only issue when you move SAP cluster group (ASCS & SAP) to node2 right?

Yes thats right.

Can you stop DI manually on node 2 and then perform cluster move (SAP & DB) from node 1 to node 2 and check the results.

No use , the SAP SID 00 Service goes first " offline pending" and after that to "failed" state.

However , it starts manually .

The start parameters of the SAPSID_00 service is

O:\usr\sap\SID\ASCS00\exe\sapstartsrv.exe" pf="O:\usr\sap\SID\SYS\profile\START_ASCS00_SAPGCP"

Can there be any issue with "SAP SID 00 Instance" , maybe the instance is not going offline completely . Also the service is running by DOMAIN\SAPServiceSID

Regards

Edited by: Tech GCCIA on Oct 13, 2010 8:38 AM

The issue got resolved by adding "cluster disk" resource to the dependency of SAP SID 00 service in failover cluster management tool.

Now the switchover is working fine.

Regards

Ask a Question

Top Q&A Solution Author

User

Count