Host & Java monitoring returns grey entries solman 7.1 SP10

[Former Member]

Hello SAP,

We have configured system monitoring in Solution Manager 7.1 SP10 system.

Monitoring data for database and ABAP technical instance is getting captured and available in system/database monitoring in technical monitoring work center.

But host monitoring not working for both ABAP and Java systems and technical instance of Java data not capturing, shows grey entries.

We have followed and implemented the SAP note:

1853160 - Grey host metrics in Technical Monitoring

1855064 - Error occurred when calling the DPC Push web service

1638655 - Errors with requests coming from ICM

But still the host monitoring and Java system details are not working in Technical Monitoring.

We have noticed below error message on Diagnostic Agent log files:

Jul 18, 2014 2:23:40 AM [Thread[ExRun:dpc:job_3,5,dpc:job:ExecTG]] Error      [DPCServicePushMetricJob.pushSimpleEvents] Error occurred when calling the DPC Push web service. (http://xxxxxxxx:8000/sap/bc/srt/scs/sap/e2e_dpc_push?sap-client=030).
[EXCEPTION]
java.rmi.RemoteException: Service call exception; nested exception is:
        com.sap.engine.services.webservices.jaxrpc.exceptions.InvalidResponseCodeException: Invalid Response Code: (404) Not found. The request URL was:"http://xxxxxxxx:8000/sap/bc/srt/scs/sap/e2e_dpc_push?sap-client=xxx". Check, whether the URL is correct, and the Web Service that you are requesting is up and running.
        at com.sap.smd.agent.wsclients.dpc.BindingStub.e2eDpcPushMetrics(BindingStub.java:85)
        at com.sap.smd.agent.wsclients.dpc.BindingStub.e2eDpcPushMetrics(BindingStub.java:96)
        at sun.reflect.GeneratedMethodAccessor39576.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.sap.smd.api.util.SynchronizedProxy$SyncHandler.invoke(SynchronizedProxy.java:32)
        at $Proxy37.e2eDpcPushMetrics(Unknown Source)
        at com.sap.smd.dpc.job.DPCServicePushMetricJob.pushSimpleEvents(DPCServicePushMetricJob.java:122)
        at com.sap.smd.dpc.job.DPCServicePushMetricJob.run(DPCServicePushMetricJob.java:66)
        at com.sap.smd.server.exec.TaskRunner.run(TaskRunner.java:46)
        at com.sap.smd.server.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:785)
        at java.lang.Thread.run(Thread.java:722)
Caused by: com.sap.engine.services.webservices.jaxrpc.exceptions.InvalidResponseCodeException: Invalid Response Code: (404) Not found. The request URL was:"http://xxxxxxxx.:8000/sap/bc/srt/scs/sap/e2e_dpc_push?sap-client=xxx". Check, whether the URL is correct, and the Web Service that you are requesting is up and running.
        at com.sap.engine.services.webservices.jaxrpc.wsdl2java.soapbinding.MimeHttpBinding.handleResponseMessage(MimeHttpBinding.java:998)
        at com.sap.engine.services.webservices.jaxrpc.wsdl2java.soapbinding.MimeHttpBinding.call(MimeHttpBinding.java:1452)
        at com.sap.smd.agent.wsclients.dpc.BindingStub.e2eDpcPushMetrics(BindingStub.java:78)
        ... 11 more

Request you to look into this and advice us to correct the issue

Thanks,
Vamshi

[Former Member · Accepted solution]

Hi Vamshi,

do you see any data of the managed system at Wily IS manager?

Best regards,

Mike

[Former Member · Accepted solution]

Thank you All for your kind updates.

I have worked with SAP and my issue had resolved by following the below steps:

The metrics are missing due to an invalid/missing WebService configuration for the SMD Agent push operation.

Please re-execute steps 5.2 and 5.3 within System Preparation, especially you need to set the checkmarks in step 5.2.  After this has been done, the WebService E2E_DPC_PSUH should be configured properly and the metrics will be available within Solution Manager.

Thanks,
Vamshi

[divyanshu_srivastava3 · Accepted solution]

Hi Vamshi,

If you can see above, I asked you to do same.

Divyanshu

[Former Member]

Hi,

Could you check the ownership and permissions of the host agent files comparing with some other systems

Regards,

Nikhil

[Former Member]

Thank You Divyanshu and Nikhil.

As stated earlier I have verified a correctly completed all the tasks in SAP notes 1854525, 1916180, 1921592, and 1618858.

The ownership and permissions of the host agent files looks good in comparison with other systems.

Just to let you know system preparation, Basic configuration and Managed system configurations are Green.

Monitoring data for database and ABAP technical instance is getting captured and available in system/database monitoring in technical monitoring work center.

But host monitoring not working for both ABAP and Java systems and technical instance of Java data not capturing, shows gray entries.

Thanks,

Vamshi

[Former Member]

Hi Vamshi ,

If host monitoring is not working than there is problem with Diagnostic agent residing on the host .
Kindly check if the agent is in running status , also is it connected & registered with Solman... ?

If not registered than please execute script to complete it  -

smdsetup.sh managingconf hostname="FQDN" port:"5<00>04" user:"USER" pwd:"Password"

FQDN = fully qualified domain name of Solman

USER =  User id solman admin

Password=  Password of solman admin

Regards,

Nash

[Former Member]

Hello Nash,

Host Agent is up and running fine. Even upgraded to latest version. It is connected and registered with Solman.

Monitoring data for database and ABAP technical instance is getting captured and available in system/database monitoring in technical monitoring work center. But host monitoring not working for both ABAP and Java systems and technical instance of Java data not capturing, shows grey entries.

Hello Divyanshu,

SICF service  /default_host/sap/bc/srt/ is active and scs/sap/e2e_dpc_push doesn't exists.

Request you to look into this and advice us to correct the issue

Thanks,
Vamshi

[divyanshu_srivastava3]

Hi,

Please follow 1819848 - All metrics are grey in Technical Monitoring

and re-run those steps.

Divyanshu

[roland_hennessy]

Hi Vamsi,

Please check the managed system configuration, and the step 7

'Configure Automatically' and 'Instroscope Host Adapter' if showing a

yellow led, indicating it needs to be re-run?

If so please re-run along with any other steps that are yellow? Also please check the Basic Config and make sure all step are performed in configure automatically as well. All these step need to executed automatically and not manually.

Kind regards,

Roland

[Former Member]

Hello Roland,

Thank you for your reply.

Both the mentioned configurations are Green and I have rerun again as suggested but the issue still exists.

Monitoring data for database and ABAP technical instance is getting captured and available in system/database monitoring in technical monitoring work center. But host monitoring not working for both ABAP and Java systems and technical instance of Java data not capturing, shows gray entries.

Please advice.

Thanks,
Vamshi

[Former Member]

Hello All,

Can someone please help us in resolving the above mentioned Technical Monitoring issue.

Host monitoring not working for both ABAP and Java systems and technical instance of Java data not capturing, shows gray entries.

Thanks,
Vamshi

[divyanshu_srivastava3]

Hi Vamshi,

Please share the host agent logs and profile.

Divyanshu

[Former Member]

Thank you Divyanshu,

Please find the Host agent logs and profile details .

1. Host Agent logs:
---------------------------------------------------
trc file: "dev_saphostexec", trc level: 3, release: "720"
---------------------------------------------------
--------------------
SAPHOSTAGENT information
--------------------

kernel release                720

kernel make variant           720_REL

compiled on                   AIX 2 5 00092901D600 for rs6000_64

compiled for                  64 BIT

compilation mode              Non-Unicode

compile time                  Jan 25 2014 06:32:12

patch number                  178

latest change number          1470842

---------------------
supported environment
---------------------

operating system
AIX 2 5
AIX 3 5
AIX 1 6
AIX 1 7

[Thr 01] Sat May 17 08:31:13 2014
[Thr 01] update env : PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/sap/<SID>/DVEBMGSxx/exe/sapjvm_4/bin:/etc:/usr/ucb:/usr/bin/X11:/
usr/java14/jre/bin:/usr/java14/bin:/usr/local/bin:/etc:/usr/ucb:/home/<SID>adm/bin:/usr/bin/X11:/usr/sap/<SID>/SYS/exe/uc/rs6000_
64:/usr/sap/<SID>/SYS/exe/run:/home/<SID>adm:.:/db2/db2<SID>/sqllib/bin:/db2/db2<SID>/sqllib/adm:/db2/db2<SID>/sqllib/misc:
[Thr 01] update env : LIBPATH=/usr/sap/hostctrl/exe
[Thr 01] _=/usr/sap/hostctrl/exe/hostexecstart
[Thr 01] MANPATH=/opt/tectia/man:
[Thr 01] LANG=en_US
[Thr 01] WSM_WS_CMD="startsrc -s http4websm"
[Thr 01] LOGIN=<SID>adm
[Thr 01] PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/sap/<SID>/DVEBMGSxx/exe/sapjvm_4/bin:/etc:/usr/ucb:/usr/bin/X11:/usr/java14/jr
e/bin:/usr/java14/bin:/usr/local/bin:/etc:/usr/ucb:/home/<SID>adm/bin:/usr/bin/X11:/usr/sap/<SID>/SYS/exe/uc/rs6000_64:/usr/sap/S
MP/SYS/exe/run:/home/<SID>adm:.:/db2/db2<SID>/sqllib/bin:/db2/db2<SID>/sqllib/adm:/db2/db2<SID>/sqllib/misc:
[Thr 01] RES_RETRY=2
[Thr 01] RSEC_SSFS_KEYPATH=/usr/sap/<SID>/SYS/global/security/rsecssfs/key
[Thr 01] LC__FASTMSG=true
[Thr 01] CLASSPATH=:/db2/db2<SID>/sqllib/java/db2java.zip:/db2/db2<SID>/sqllib/java/runtime.zip:.
[Thr 01] LOGNAME=<SID>adm
[Thr 01] RES_TIMEOUT=2
[Thr 01] dbs_db6_user=sap<SID>
[Thr 01] MAIL=/usr/spool/mail/<SID>adm
[Thr 01] DSMI_CONFIG=/usr/tivoli/tsm/client/api/bin64/dsm.opt
[Thr 01] LOCPATH=/usr/lib/nls/loc
[Thr 01] PS1=uahris12:<SID>adm> 
[Thr 01] DIR_LIBRARY=/usr/sap/<SID>/SYS/exe/run
[Thr 01] WSM_DOC_DIR="/usr/websm/http/com.ibm.websm.http.server_1.0.0"
[Thr 01] USER=<SID>adm
[Thr 01] AUTHSTATE=compat
[Thr 01] DSM_LOG=/db2/db2<SID>/errors
[Thr 01] IBM_JAVA_OPTIONS=-Xmx256M -Xj9
[Thr 01] SHELL=/usr/bin/ksh
[Thr 01] ODMDIR=/etc/objrepos
[Thr 01] CASHCOMP=/opt/CA/SharedComponents
[Thr 01] JAVA_HOME=/usr/sap/<SID>/DVEBMGSxx/exe/sapjvm_4
[Thr 01] TIMEOUT=900
[Thr 01] TMOUT=900
[Thr 01] DSMI_LOG=/db2/db2<SID>/errors
[Thr 01] dbs_db6_schema=sap<SID>
[Thr 01] RSEC_SSFS_DATAPATH=/usr/sap/<SID>/SYS/global/security/rsecssfs/data
[Thr 01] HOME=/home/<SID>adm
[Thr 01] DB2INSTANCE=db2<SID>
[Thr 01] TERM=vt100
[Thr 01] MAILMSG=[YOU HAVE NEW MAIL]
[Thr 01] DSM_DIR=/usr/tivoli/tsm/client/ba/bin
[Thr 01] rsdb_ssfs_connect=0
[Thr 01] DB2DBDFT=<SID>
[Thr 01] PWD=/home/<SID>adm
[Thr 01] TZ=EST5EDT
[Thr 01] dbms_type=db6
[Thr 01] AIXTHREAD_SCOPE=S
[Thr 01] WSM_CGI_DIR=
[Thr 01] SAPSYSTEMNAME=<SID>
[Thr 01] DSM_CONFIG=/usr/tivoli/tsm/client/ba/bin/dsm.opt
[Thr 01] DSMI_DIR=/usr/tivoli/tsm/client/api/bin64
[Thr 01] INSTHOME=/db2/db2<SID>
[Thr 01] A__z=! LOGNAME="*TMOUT
[Thr 01] NODNSSAPTRANSHOST=1
[Thr 01] LIBPATH=/usr/sap/hostctrl/exe
[Thr 01] **** Create a deamon **********

---------------------------------------------------
trc file: "dev_saphostexec", trc level: 3, release: "720"
---------------------------------------------------

[Thr 01] Sat May 17 08:31:13 2014
[Thr 01] ##############################################################
[Thr 01] saphostexec restarted as daemon with tracelevel 1
[Thr 01] ##############################################################
[Thr 01] SigISetDefaultAction : default handling for signal SIGCHLD
[Thr 01] Setup autoupgrade with following path:
[Thr 01] source='/usr/sap/hostctrl/exe'
[Thr 01] upgrade='/usr/sap/hostctrl/exe/../new'
[Thr 01] * Environment of /usr/sap/hostctrl/exe/sapstartsrv process *
[Thr 01] _=/usr/sap/hostctrl/exe/hostexecstart
[Thr 01] MANPATH=/opt/tectia/man:
[Thr 01] LANG=en_US
[Thr 01] WSM_WS_CMD="startsrc -s http4websm"
[Thr 01] LOGIN=<SID>adm
[Thr 01] PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/sap/<SID>/DVEBMGSxx/exe/sapjvm_4/bin:/etc:/usr/ucb:/usr/bin/X11:/usr/java14/jr
e/bin:/usr/java14/bin:/usr/local/bin:/etc:/usr/ucb:/home/<SID>adm/bin:/usr/bin/X11:/usr/sap/<SID>/SYS/exe/uc/rs6000_64:/usr/sap/S
MP/SYS/exe/run:/home/<SID>adm:.:/db2/db2<SID>/sqllib/bin:/db2/db2<SID>/sqllib/adm:/db2/db2<SID>/sqllib/misc:
[Thr 01] RES_RETRY=2
[Thr 01] RSEC_SSFS_KEYPATH=/usr/sap/<SID>/SYS/global/security/rsecssfs/key
[Thr 01] LC__FASTMSG=true
[Thr 01] CLASSPATH=:/db2/db2<SID>/sqllib/java/db2java.zip:/db2/db2<SID>/sqllib/java/runtime.zip:.
[Thr 01] LOGNAME=sapadm
[Thr 01] RES_TIMEOUT=2
[Thr 01] dbs_db6_user=sap<SID>
[Thr 01] MAIL=/usr/spool/mail/<SID>adm
[Thr 01] DSMI_CONFIG=/usr/tivoli/tsm/client/api/bin64/dsm.opt
[Thr 01] LOCPATH=/usr/lib/nls/loc
[Thr 01] PS1=uahris12:<SID>adm> 
[Thr 01] DIR_LIBRARY=/usr/sap/<SID>/SYS/exe/run
[Thr 01] WSM_DOC_DIR="/usr/websm/http/com.ibm.websm.http.server_1.0.0"
[Thr 01] USER=sapadm
[Thr 01] AUTHSTATE=compat
[Thr 01] DSM_LOG=/db2/db2<SID>/errors
[Thr 01] IBM_JAVA_OPTIONS=-Xmx256M -Xj9
[Thr 01] SHELL=/usr/bin/ksh
[Thr 01] ODMDIR=/etc/objrepos
[Thr 01] CASHCOMP=/opt/CA/SharedComponents
[Thr 01] JAVA_HOME=/usr/sap/<SID>/DVEBMGSxx/exe/sapjvm_4
[Thr 01] TIMEOUT=900
[Thr 01] TMOUT=900
[Thr 01] DSMI_LOG=/db2/db2<SID>/errors
[Thr 01] dbs_db6_schema=sap<SID>
[Thr 01] RSEC_SSFS_DATAPATH=/usr/sap/<SID>/SYS/global/security/rsecssfs/data
[Thr 01] HOME=/home/sapadm
[Thr 01] DB2INSTANCE=db2<SID>
[Thr 01] TERM=vt100
[Thr 01] MAILMSG=[YOU HAVE NEW MAIL]
[Thr 01] DSM_DIR=/usr/tivoli/tsm/client/ba/bin
[Thr 01] rsdb_ssfs_connect=0
[Thr 01] DB2DBDFT=<SID>
[Thr 01] PWD=/home/<SID>adm
[Thr 01] TZ=EST5EDT
[Thr 01] dbms_type=db6
[Thr 01] AIXTHREAD_SCOPE=S
[Thr 01] WSM_CGI_DIR=
[Thr 01] SAPSYSTEMNAME=<SID>
[Thr 01] DSM_CONFIG=/usr/tivoli/tsm/client/ba/bin/dsm.opt
[Thr 01] DSMI_DIR=/usr/tivoli/tsm/client/api/bin64
[Thr 01] INSTHOME=/db2/db2<SID>
[Thr 01] A__z=! LOGNAME="*TMOUT
[Thr 01] NODNSSAPTRANSHOST=1
[Thr 01] LIBPATH=/usr/sap/hostctrl/exe
[Thr 01] USERNAME=sapadm

_____________________________________________________________________________

2. Host Agent profile:

SAPSYSTEMNAME = SAP
SAPSYSTEM = 99
service/porttypes = SAPHostControl NwaManagement SAPCCMS SAPOscol
#service/porttypes = SAPHostControl SAPOscol SAPCCMS
service/admin_users = diaadm
DIR_LIBRARY = /usr/sap/hostctrl/exe
DIR_EXECUTABLE = /usr/sap/hostctrl/exe
DIR_PROFILE = /usr/sap/hostctrl/exe
DIR_GLOBAL = /usr/sap/hostctrl/exe
DIR_INSTANCE = /usr/sap/hostctrl/exe
DIR_HOME = /usr/sap/hostctrl/work
service/logfile_000 = /tmp/s
service/logfile_001 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/dispatcher/log/applications*.log
service/logfile_002 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/dispatcher/log/applications.*.log
service/logfile_003 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/dispatcher/log/defaultTrace*.trc
service/logfile_004 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/dispatcher/log/defaultTrace.*.trc
service/logfile_005 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/server0/log/applications*.log
service/logfile_006 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/server0/log/applications.*.log
service/logfile_007 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/server0/log/applications/com.sap.xi/xi*.log
service/logfile_008 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/server0/log/applications/com.sap.xi/xi.*.log
service/logfile_009 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/server0/log/defaultTrace*.trc
service/logfile_010 = /usr/sap/<SID>/DVEBMGSxx/j2ee/cluster/server0/log/defaultTrace.*.trc

Thanks  Vamshi.

[divyanshu_srivastava3]

Goto agent directory and last sort the logs, share last 4 logs. Also, the dpc logs.

Meanwhile check: 1855064 - Error occured when calling the DPC Push web service

Divyanshu

[divyanshu_srivastava3]

"Enable Web Services from System Preparation and activity Setup DPC/DCC Web Service URL from Basic Configuration step Solution Manager Internal Connectivity"

This is what should be done correctly for above error.

For host info, you oscol and hostagent should be generating logs. Have a look.

Connect your hostagent in MSC for your techincal system.

Check your have specified correct name of host.

Also, Can you share the the screenshot.

Divyanshu

[divyanshu_srivastava3]

Check:

1916180 - Grey Java metrics in Technical Monitoring with EM version 9.1

1921592 - Java Monitoring grey metrics

1618858 - Grey metrics in Technical System Monitoring UI

[divyanshu_srivastava3]

Hi Vamshi,

Did you see this "Check, whether the URL is correct, and the Web Service that you are requesting is up and running."

Goto SICF and then goto this /default_host/sap/bc/srt/ activate the service scs.

Regards,

Divyanshu