cancel
Showing results for 
Search instead for 
Did you mean: 

prd got down

former_member286941
Participant
0 Kudos

Hi,

The CI was running fine but the other application servers were not accessible from sm51, were giving the below message,

-


"Connection could not be established to local gateway (server name, gateway name"

-


-


Neither i was able to go to application server using sm51 nor using direct sap gui logon.

tcode SMGW was not running, same local host not accessible.

when i cheked on unix the gateway was showing up-----

sending some more logs :

-


1. Name: dev_icm.old

[Thr 01] Wed May 20 04:14:54 2009

[Thr 01] *** WARNING => IcmMsgProcess: connection to WP 9 broken (rc=-8,nirc=-6)

[Thr 01] *** WARNING => IcmMsgProcess: connection to WP 29 broken (rc=-8,nirc=-6)

[Thr 01] Wed May 20 04:14:56 2009

-


2. trc file: "dev_w12", trc level: 1, release: "640"

M Wed May 20 04:15:34 2009

M nihsl-getServName: found port number 0C.E4/3300 in cache

M nihsi-getServName: port 0C.E4/3300 = servicename 'sapgw00'

M NiSetStat: state hdl 5 NI_CONN_WAIT

M NiIClosehandle called while waiting for connection for hdl 5

M NiICloseHandle: shutdown and close hdl 5 / socket 12

M *** ERROR => GwConnectSapWp: GwConnect to localhost / sapgw00 failed (rc=NIECONN_PENDING) [gwxx.c 1846]

M ***LOG S0T=> GwConnectSapWp, GwConnect (-0012) [gwxx.c 1852]

M ***LOG S0R=> GwConnectSapWp, GwConnect () [gwxx.c 1857]

M ***LOG S0S=> GwConnectSapWp, GwConnect (sapgw00) [gwxx.c 1861]

M ThInitCpicStack: init cpic stack

-


3.Name: dev_disp.old

Wed May 20 05:14:55 2009

DpHdlDeadWp: restart wp (pid=1061092) automatically

      • ERROR => DpRqCheck: T230 in stat TM_SLOT_FREE [dpxxdisp.c 5993]

***LOG Q0G=> DpRqBadHandle, bad_req ( DIA) [dpxxdisp.c 4797]

      • ERROR => BAD REQUEST - Reason: DpRqCheck failed (line 5383): [dpxxdisp.c 4799]

-IN sender_id DISPATCHER tid 230 wp_ca_blk 6 wp_id 40

-IN action SEND_TO_WP uid 26089 appc_ca_blk -1 type DIA

-IN new_stat WP_WAIT mode 1 len 568 rq_id 60095

-IN req_info MSG_WITH_REQ_BUF MSG_WITH_OH

Wed May 20 05:20:47 2009

-


4.Dev_rd.old

Wed May 20 04:14:57 2009

***LOG Q0I=> NiPRead: recv (73: Connection reset by peer) [niuxi.c 928]

***LOG S23=> GwIDisconnectClient, client disconnected (369) [gwxxrd.c 11200]

***LOG S74=> GwIDisconnectClient, client disconnected ( bashahp4) [gwxxrd.c 11211]

***LOG S0R=> GwIDisconnectClient, client disconnected () [gwxxrd.c 11227]

***LOG S0I=> GwIDisconnectClient, client disconnected ( hsserver) [gwxxrd.c 11248]

                                                            • ****************************** *****************

  • LOCATION SAP-Gateway on host bascop02 / sapgw00

  • ERROR connection to partner broken

  • TIME Wed May 20 04:14:57 2009

  • RELEASE 640

  • COMPONENT NI (network interface)

  • VERSION 37

  • RC -6

  • MODULE niuxi.c

  • LINE 928

  • DETAIL NiPRead (20.60.111.114/2845, hdl 63)

  • SYSTEM CALL recv

  • ERRNO 73

  • ERRNO TEXT Connection reset by peer

  • COUNTER 2552

Wed May 20 06:11:14 2009

GwIFreeDiscConn: delete conversation 05189055 (conn=57) in state DISCONNECT

Wed May 20 06:14:57 2009

***LOG Q0I=> NiPRead: recv (73: Connection reset by peer) [niuxi.c 928]

***LOG S23=> GwIDisconnectClient, client disconnected (476) [gwxxrd.c 11200]

***LOG S74=> GwIDisconnectClient, client disconnected ( bashahp3) [gwxxrd.c 11211]

***LOG S0R=> GwIDisconnectClient, client disconnected () [gwxxrd.c 11227]

***LOG S0I=> GwIDisconnectClient, client disconnected ( hsserver) [gwxxrd.c 11248]

                                                            • ****************************** *****************

  • LOCATION SAP-Gateway on host bascop02 / sapgw00

  • ERROR connection to partner broken

  • TIME Wed May 20 06:14:57 2009

  • RELEASE 640

  • COMPONENT NI (network interface)

  • VERSION 37

  • RC -6

  • MODULE niuxi.c

  • LINE 928

  • DETAIL NiPRead (20.60.111.113/4356, hdl 98)

  • SYSTEM CALL recv

  • ERRNO 73

  • ERRNO TEXT Connection reset by peer

  • COUNTER 2812

                                                            • ****************************** *****************

Wed May 20 06:18:52 2009

GwIFreeDiscConn: delete conversation 09468221 (conn=54) in state DISCONNECT

Wed May 20 06:23:18 2009

GwIFreeDiscConn: delete conversation 09299858 (conn=25) in state DISCONNECT

GwIFreeDiscConn: delete conversation 09805173 (conn=111) in state DISCONNECT

caught SIGINT (2)

***LOG S30=> GwStopGateway, gateway stopped () [gwxxrd.c 13853]

any idea what could be the root cause for this, it would be a gr8 help.

-


Regards - MJ

Accepted Solutions (0)

Answers (9)

Answers (9)

former_member286941
Participant
0 Kudos

Upgraded the kernel.

former_member286941
Participant
0 Kudos

There is no error related to memeory pool in dev_disp.

kindly suggest what kind of error can come in this.

many thanks for reply.

MJ

Former Member
0 Kudos

Hi Munish

Have you checked the SAP note, as per error text in your post, this seems to be a relevant note that i found.

Ravi

Former Member
0 Kudos

Hi,

Whether your application system is running or not?

Problem 1. Both systems are started well but unable show in sm51 only.

problem 2. Your Application Server is down.

For first problem concentrate on hosts file entry, domain resolving, network problem and port problem see port 32<nn> nn-Instance *Comunication should be done from both side. same should be checked for Services file.

Problem 2 is entirely different you understand.

Regards

DK

former_member286941
Participant
0 Kudos

That seems the right solution, but the system is still running with the same parameters without problem, therefore its bit difficult to change these parameters without any valid proof.

Regds, MJ

ashish_mishra2
Contributor
0 Kudos

This is very well required to allocate the memory area to its shared pool.

There will not be any harm if you change the parameter to the recommended value.

If you will check the value, these are the value which your sappfpar check is recommending.

You do not touch ur CI profile , you can modify the parameter of ur APP server's profile, which is not coming up.

Regards,

Ashish

Former Member
0 Kudos

Hi

Check dev_disp for any error related to memory pool, besides refer SAP Note 212902 - Calculated shared memory pool is too small. Update the thread thereafter.

Ravi

former_member286941
Participant
0 Kudos

When checking profile status using sappfpar, the below result came, with this it seems shared memeory problem.

sappfpar=>sapparam( 2): fopenU("PBS_ DVEBMGS00_ bascop"," r"): No such file or directory

sappfpar=>No Profile used.

sappfpar=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

============ ========= ========= ========= ========= ========= ========= ========= =====

== Checking profile: <no_profile>

============ ========= ========= ========= ========= ========= ========= ========= =====

***ERROR: Size of shared memory pool 10 too small

============ ========= ========= ========= ========= ========= =======

SOLUTIONS: (1) Locate shared memory segments outside of pool 10

with parameters like: ipc/shm_psize_ <key> =0

SOLUTION: Increase size of shared memory pool 10

with parameter: ipc/shm_psize_ 10 =132000000

***ERROR: Size of shared memory pool 40 too small

============ ========= ========= ========= ========= ========= =======

SOLUTIONS: (1) Locate shared memory segments outside of pool 40

with parameters like: ipc/shm_psize_ <key> =0

SOLUTION: Increase size of shared memory pool 40

with parameter: ipc/shm_psize_ 40 =112000000

Shared memory disposition overview

============ ========= ========= ========= ========= ========= =======

Shared memory pools

Key: 10 Pool

Size configured.. ...: 19880000 ( 19.0 MB)

Size min. estimated.: 127925422 ( 122.0 MB)

Advised Size........ : 132000000 ( 125.9 MB)

Key: 40 Pool for database buffers

Size configured.. ...: 14250000 ( 13.6 MB)

Size min. estimated.: 108776088 ( 103.7 MB)

Advised Size........ : 112000000 ( 106.8 MB)

-


-


-


-


-


-


Swap space requirements estimated

============ ========= ========= ========= =========

Shared memory...... ......... .....: 736.6 MB

..in pool 10 19.0 MB, 643% used !!

..in pool 40 13.6 MB, 763% used !!

..not in pool: 502.0 MB

Processes... ......... ......... ...: 25.9 MB

Extended Memory ............ .....: 4092.0 MB

-


-


-


-


-


Total, minimum requirement. ......: 4854.6 MB

Process local heaps, worst case..: 1907.3 MB

Total, worst case requirement. ...: 6761.9 MB

Errors detected.... ......... .....: 2

any idea what could be the root cause for this, it would be a gr8 help.

Regds, MJ

ashish_mishra2
Contributor
0 Kudos

Hi ,

You can see , there are 2 Errors in this output.

Errors detected..................: 2

Warnings detected................: 0

modify the below parameters :

change the value of ipc/shm_psize_10 to 132000000 and

the value of ipc/shm_psize_40 to 112000000

these two parameters would already present in instance profile. so just you have to modify the

parameter's value. and then again check the sappfpar output, it should show 0 error over there.

Also .. would like to ask you to check permission of /etc/host file. make it 640 and then try.

then you SAP system should come up .

Regards,

Ashish

former_member286941
Participant
0 Kudos

continue of previous thread:-

-


4.Dev_rd.old

Wed May 20 04:14:57 2009

***LOG Q0I=> NiPRead: recv (73: Connection reset by peer) [niuxi.c 928]*

***LOG S23=> GwIDisconnectClient, client disconnected (369) [gwxxrd.c 11200]

***LOG S74=> GwIDisconnectClient, client disconnected ( bashahp4) [gwxxrd.c 11211]

***LOG S0R=> GwIDisconnectClient, client disconnected () [gwxxrd.c 11227]

***LOG S0I=> GwIDisconnectClient, client disconnected ( hsserver) [gwxxrd.c 11248]

                                                            • ****************************** *****************

  • LOCATION SAP-Gateway on host bascop02 / sapgw00

_ ERROR connection to partner broken_*

  • TIME Wed May 20 04:14:57 2009

  • RELEASE 640

_ COMPONENT NI (network interface)_*

  • VERSION 37

  • RC -6

  • MODULE niuxi.c

  • LINE 928

  • DETAIL NiPRead (20.60.111.114/2845, hdl 63)

  • SYSTEM CALL recv

  • ERRNO 73

  • ERRNO TEXT Connection reset by peer

  • COUNTER 2552

Wed May 20 06:11:14 2009

GwIFreeDiscConn: delete conversation 05189055 (conn=57) in state DISCONNECT

Wed May 20 06:14:57 2009

_*LOG Q0I=> NiPRead: recv (73: Connection reset by peer) [niuxi.c 928]_

***LOG S23=> GwIDisconnectClient, client disconnected (476) [gwxxrd.c 11200]

***LOG S74=> GwIDisconnectClient, client disconnected ( bashahp3) [gwxxrd.c 11211]

***LOG S0R=> GwIDisconnectClient, client disconnected () [gwxxrd.c 11227]

***LOG S0I=> GwIDisconnectClient, client disconnected ( hsserver) [gwxxrd.c 11248]

                                                            • ****************************** *****************

Wed May 20 06:18:52 2009

GwIFreeDiscConn: delete conversation 09468221 (conn=54) in state DISCONNECT

Wed May 20 06:23:18 2009 GwIFreeDiscConn: delete conversation 09299858 (conn=25) in state DISCONNECT

GwIFreeDiscConn: delete conversation 09805173 (conn=111) in state DISCONNECT

Wed May 20 06:24:21 2009

GwIFreeDiscConn: delete conversation 08173503 (conn=47) in state DISCONNECT

Wed May 20 06:30:02 2009

caught SIGINT (2)

***LOG S30=> GwStopGateway, gateway stopped () [gwxxrd.c 13853]

-


former_member286941
Participant
0 Kudos

Hi Gurus,

Sending my queries again in more clear way, pls see

-


The CI was running fine but the other application servers were not accessible from sm51, were giving the below message,

-


"Connection could not be established to local gateway (server name, gateway name"

-


Neither i was able to go to application server using sm51 nor using direct sap gui logon.

tcode SMGW was not running, same local host not accessible.

when i cheked on unix the gateway was showing up.

-


Sending some more logs :

-


1. Name: dev_icm.old

[Thr 01] Wed May 20 04:14:54 2009

[Thr 01] *** WARNING => IcmMsgProcess: connection to WP 9 broken (rc=-8,nirc=-6)

[Thr 01] *** WARNING => IcmMsgProcess: connection to WP 29 broken (rc=-8,nirc=-6)

[Thr 01] Wed May 20 04:14:56 2009

-


2. trc file: "dev_w12", trc level: 1, release: "640"

M Wed May 20 04:15:34 2009

M nihsl-getServName: found port number 0C.E4/3300 in cache

M nihsi-getServName: port 0C.E4/3300 = servicename 'sapgw00'

M NiSetStat: state hdl 5 NI_CONN_WAIT

M NiIClosehandle called while waiting for connection for hdl 5

M NiICloseHandle: shutdown and close hdl 5 / socket 12

M *** ERROR => GwConnectSapWp: GwConnect to localhost / sapgw00 failed (rc=NIECONN_PENDING) [gwxx.c 1846]

M ***LOG S0T=> GwConnectSapWp, GwConnect (-0012) [gwxx.c 1852]

M ***LOG S0R=> GwConnectSapWp, GwConnect () [gwxx.c 1857]

M ***LOG S0S=> GwConnectSapWp, GwConnect (sapgw00) [gwxx.c 1861]

M ThInitCpicStack: init cpic stack

-


3.Name: dev_disp.old

Wed May 20 05:14:55 2009

DpHdlDeadWp: restart wp (pid=1061092) automatically

      • ERROR => DpRqCheck: T230 in stat TM_SLOT_FREE [dpxxdisp.c 5993]

***LOG Q0G=> DpRqBadHandle, bad_req ( DIA) [dpxxdisp.c 4797]

      • ERROR => BAD REQUEST - Reason: DpRqCheck failed (line 5383): [dpxxdisp.c 4799]

-IN sender_id DISPATCHER tid 230 wp_ca_blk 6 wp_id 40

-IN action SEND_TO_WP uid 26089 appc_ca_blk -1 type DIA

-IN new_stat WP_WAIT mode 1 len 568 rq_id 60095

-IN req_info MSG_WITH_REQ_BUF MSG_WITH_OH

Wed May 20 05:20:47 2009

-


Attaching some more logs in next thread, pls continue ---

*Please do not use large fonts

Edited by: Juan Reyes on May 22, 2009 9:18 AM

former_member286941
Participant
0 Kudos

Kindly see the sappfpar results, i am not sure why this is showing "No Profile used"

sappfpar=>No Profile used.

sappfpar=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

-


================================================================================

== Checking profile: <no_profile>

================================================================================

-


***ERROR: Size of shared memory pool 10 too small

================================================================

SOLUTIONS: (1) Locate shared memory segments outside of pool 10

with parameters like: ipc/shm_psize_<key> =0

SOLUTION: Increase size of shared memory pool 10

with parameter: ipc/shm_psize_10 =132000000

****ERROR: Size of shared memory pool 40 too small

================================================================

SOLUTIONS: (1) Locate shared memory segments outside of pool 40

with parameters like: ipc/shm_psize_<key> =0

SOLUTION: Increase size of shared memory pool 40

with parameter: ipc/shm_psize_40 =112000000

-


Shared memory disposition overview

================================================================

Shared memory pools

Key: 10 Pool

Size configured.....: 19880000 ( 19.0 MB)

Size min. estimated.: 127925422 ( 122.0 MB)

Advised Size........: 132000000 ( 125.9 MB)

Key: 40 Pool for database buffers

Size configured.....: 14250000 ( 13.6 MB)

Size min. estimated.: 108776088 ( 103.7 MB)

Advised Size........: 112000000 ( 106.8 MB)

-


Kinldy suggest any possible ways to find the exact root cause of this problem,

Thanks in advance.

Reds, MJ

Former Member
0 Kudos

did you run sappfpar with profile path ?

former_member286941
Participant
0 Kudos

Hi Gurus,

but, I am not able to find what exactly caused the prd application servers to get down, any valid proof which can confirm it its network problem or shared memeory problem.

It would help in RCA, so that the this thing may not happen again.

thanks for your concerned answers.

Regds, MJ

ashish_mishra2
Contributor
0 Kudos

Hi Munish,

  1. Check the content,permission and ownership of /etc/hosts file.

  2. IP addresses given in sqlnet.ora file -> App server IP address should be present.

  3. This issue occurs because of SHM segment also.

run the report sappfpar and check all the segments are set correctly.in case of any error correct the same. in the sappfpar check output , check pool 10 and 40 are available as well as no error should be reported.

format

from sidadm user

sappfpar check pf=/usr/sap/<sid>/DV*/profiles/<instance profile>.pfl

Former Member
0 Kudos

we faced similar kind of issue at once, and system restart solved our problem, it might be due to network error.

Former Member
0 Kudos

check network card/s of server. (i guess)