SAP Enterprise Portal Response Time is High...

[Former Member]

Dear All,

Yesterday morning onwards our portal response time is increasing 500 ms per each our on server process 1. For server process 0 showing normal values. If i check the portal --> System Administration --> Monitoring --> Portal --> Request Over view --> I found one of component always taking Largest Net time and Largest CPU time by the Administrator User.

Request Name - NW:J2E:SRV:HTTP

Component With Largest Nettime - NW:J2E:SRV:WEB_REQUEST

Component with largest CPU Time - NW:J2E:SRV:WEB_REQUEST

If i check the dump stack trace file i found the following errors. Plese let help us if any one knows the solution to this problem

"SAPEngine_Application_Thread[impl:3]_3" prio=5 tid=0x05304708 nid=0x18bc in Object.wait() [0x05bae000..0x05bafd94]

at java.lang.Object.wait(Native Method)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:320)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponseLDT(ResponseImpl.java:336)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.httpserver.server.ResponseImpl.sendResponse(ResponseImpl.java:278)

- locked <0x2dc01270> (a java.lang.Object)

at com.sap.engine.services.servlets_jsp.server.runtime.client.ServletOutputStreamImpl.flush(ServletOutputStreamImpl.java:440)

at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:410)

at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)

- locked <0x33bbcc10> (a java.io.OutputStreamWriter)

at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)

at java.io.PrintWriter.flush(PrintWriter.java:120)

- locked <0x33bbcc10> (a java.io.OutputStreamWriter)

at com.sap.engine.services.servlets_jsp.server.runtime.client.PrintWriterImpl.flush(PrintWriterImpl.java:164)

at com.sap.engine.services.servlets_jsp.server.runtime.client.HttpServletResponseFacade.flushBuffer(HttpServletResponseFacade.java:1629)

at com.sap.engine.services.servlets_jsp.server.runtime.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:380)

at com.sap.engine.services.servlets_jsp.server.jsp.PageContextImpl.forward(PageContextImpl.java:199)

at com.sap.engine.services.servlets_jsp.server.jsp.PageContextImpl.handleErrorPage(PageContextImpl.java:715)

at com.sap.engine.services.servlets_jsp.server.jsp.PageContextImpl.handlePageException(PageContextImpl.java:702)

at jsp_SystemInfo1223756339593._jspService(jsp_SystemInfo1223756339593.java:65535)

at com.sap.engine.services.servlets_jsp.server.jsp.JspBase.service(JspBase.java:112)

at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.runServlet(HttpHandlerImpl.java:401)

at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.handleRequest(HttpHandlerImpl.java:266)

at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:386)

at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:364)

at com.sap.engine.services.httpserver.server.RequestAnalizer.invokeWebContainer(RequestAnalizer.java:1039)

at com.sap.engine.services.httpserver.server.RequestAnalizer.handle(RequestAnalizer.java:265)

at com.sap.engine.services.httpserver.server.Client.handle(Client.java:95)

at com.sap.engine.services.httpserver.server.Processor.request(Processor.java:175)

at com.sap.engine.core.service630.context.cluster.session.ApplicationSessionMessageListener.process(ApplicationSessionMessageListener.java:33)

at com.sap.engine.core.cluster.impl6.session.MessageRunner.run(MessageRunner.java:41)

at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)

at java.security.AccessController.doPrivileged(Native Method)

at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:102)

at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:172)

"SAPEngine_Application_Thread[impl:3]_1" prio=5 tid=0x04f9f008 nid=0x18b4 in Object.wait() [0x05b2f000..0x05b2fd94]

at java.lang.Object.wait(Native Method)

at java.lang.Object.wait(Object.java:429)

at com.sap.engine.lib.util.WaitQueue.dequeue(WaitQueue.java:238)

- locked <0x1aa19ca0> (a com.sap.engine.lib.util.WaitQueue)

at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:149)

Kind Regards

Sudhakar A

[Former Member]

Hi NIIT Basis,

I suggest you to check the Thread dumps on the server and the dispatcher processes.

Check the number of threads for the server and dispatcher through

config tool -> Cluster Data| Global Server Configuration -> Managers -> Thread Manager and

Cluster Data -> Global Dispatcher Configuration -> Managers ->| Thread Manager.

u2022 ChangeRQSizeStep 200 (the request queue is increased in ChangeRQSizeStep

increments until the MaxRQSize is reached while serving incoming requests)

u2022 MaxRQSize 10000 (maximum size of the request queue; if reached, the next

starting thread is blocked until a request is completed)

u2022 InitialRQSize 1000 (initial size of the request queue until which the threads can be

used to process the requests)

u2022 InitialThreadCount 100 (initial number of threads in the thread pool, which are in

the waitQueue, until a RunnableObject uses it for processing)

u2022 MaxThreadCount 200 (maximum number of threads available for processing;

required to avoid overload due to a large number of threads serving requests)

u2022 MinThreadCount 100

For opengin a Thread dump you can check notes SAP Note 1044373, 1020246.

Refer to note 742395 for High CPU usage on the system.

IMP: Exercise caution when changing J2EE engine configuration parameters. Do not change these unless you know what you are doing. You need a restart of the server if you change these values to take effect.

Try accessing J2EE engine by portal component through http://<server>:<port>/irj/servlet/prt/portal/com.sap.portal.navigation.portallauncher.default

The problem may exist in one of the components in the web infrastructure such as a web server or load balancer.

Is it connecting to a backend system ?

Regards

Puneet

[symon_braunbaer]

Hello,

I have a similar problem, but I don't have the Request Overview option under System Administration --> Monitoring --> Portal --> Request Over view. We are running the portal on NetWeaver 7.31 EHP 1.

Furthermore, the jcmon link to the SAP help pages is dead as well:

http://help.sap.com/saphelp_nw04/helpdata/EN/d3/4d074147c1f06fe10000000a1550b0/frameset.htm

[hofmann]

Hi,

looks like your server node is hanging (locked). Judging from the component name (web_request) and the output (SendRequest) your server node is trying to send something but can't.

Can you do a restart of your server node (or instance)?

br,

Tobias

[Former Member]

Dear Mr.Tobias,

We got the same issue on July 20th, That time we restarted the instance and resolved the problem. Is there any alternate solution to unlock server01 process with out restart. Please let us know.

Kind Regards

Sudhakar

[hofmann]

Hi,

the first thing to do is to check your system, apply the recomendations of SAP Notes (GC, Java parameters, AS configuration, etc), analyze the Early Watch notifications for your system and do a thread dump analyzis to find out, what exaclty is happening (SAP app or custom app is causing the problem).

To restart just 1 server node, use jcmon.

SAP Help: http://help.sap.com/saphelp_nw04/helpdata/EN/d3/4d074147c1f06fe10000000a1550b0/frameset.htm

br,

Tobias

[Former Member]

Hello NIIT Basis

We have face similar issue in our portal. Kindly provide your solution which help us lots more to fix the issue.

It is very very urgent for. Kindly help me

Regards

Keshari