cancel
Showing results for 
Search instead for 
Did you mean: 

BI 4.1 CMC Servers/Users and groups tabs take too much time to load

0 Kudos

Hello colleagues,

We have a Production environment in BI4.1 recently updated from SP2 Patch 2 to SP7 Patch 1, composed of 4 nodes, two Intelligence nodes with (RAM 75G each) with CMS server and other two for processing(RAM 65G each), and CMS and Audit DB in Oracle 11g. Both CMS nodes have Tomcat running.

I'm not completely sure if after the update came the issue we are currently experiencing.

We login to the CMC web page without any issues or spinning concerns, but once we are inside the CMC, we are noticing that when we move to "Servers" or "Users and Groups" tabs take too long to load them completely, Servers tab takes 50 seconds to display all Nodes information(I must mention once is fully loaded, we can move through nodes, metrics and properties options without delay or concerns, the spinning is only occurring every time we move to one of these tabs; for Users and groups is taking 1 minute to populate every time we move there, in our DEV and QA environments take less than 10 sec for each tab. Also we have tried using other Tomcat environment(QA) with the same result, All of the other tabs load correctly without delays.

After update we just rebuilt the Paltform Search Index due was not working before update, it was been going slow and due some high usage of the system we just stopped the Platform Search indexing, but the Servers/Users and Groups(Almost 3000 users only) tabs still have the same behavior.

Can you consider this is a normal behavior? Have you experienced this before? Where could be the malfunctioning here is case is something wrong?

I'll appreciate your responses and feedback.

Thanks and regards!

Roy Garcia

Accepted Solutions (1)

Accepted Solutions (1)

0 Kudos

Hello all,

For the last months, we have been looking for the solution to resolve this issue and I found that the high amount of objects in the CMS DB were failed instances at the system. We found that there were over 7 million of failed instances, caused by the DST bug:

1448881 - Multiple instances spawned after daylight savings time change.

So, we have been cleaning the failed instances with the script provided by SAP in the note attached. And currently we have over 700,000 so the amount have been decreasing considerably, last weekend after we have reduced the high amount of failed instances that means reduced the amount of objects in CMS DB, we performed a table reorg for all CMS tables, that action decreased the size of the CMS schema from 35Gb to 4Gb, and increased the MaxObjects in CMS cache to 300,000 per CMS. These highly imrpoved the performance for the issue we were having.

Currently the "Servers" and "Users and groups" tabs take over 15 sec to load, we truely believe, once we completely removed the high amount of failed instances in the system, the slowness stills present will be gone.

Thanks all for your help and support!

Roy

Answers (5)

Answers (5)

0 Kudos

Hello all,

Also I have validated that it's performing slow when a query goes to the CMS DB looking for SI_ID=16(Servers), SI_ID=19(Users) and SI_ID=20(Groups), almost 1 minute for each when querying using the Query Builder: "select SI_ID from CI_SYSTEMOBJECTS where SI_ID = 16"

Regards!

Roy

0 Kudos

Hello all,

We have performed the system restart last weekend, and also rebuilt the Catalina folder to clean the cache for Tomcat WebApp, but still the same. Untill now we have noted we currently have over 5 million objects inside CMS sytem DB and almost 100,000 at CMS system cache objects, when in our QA landscape is less than 700,00 objects into the CMS system DB, not sure if this could be the cause of the slowness presenting in the system.

Also we are noticing that each CMS process(2 within the environment) at OS level are cosuming over 9G of RAM(we don't have enabled -Xmx parameters for these servers), again, not completely sure if it's normal.

Thanks for your comments and feedback!

Regards!

Roy Garcia

Former Member
0 Kudos

Hello Roy,

If the number of objects in CMS cache is less than number of objects in CMS DB then we can have performance issues. In this case you can add the switch -maxobjectsincache 500000. This might improve the performance. Adding this switch will allow the CMS cache to increase upto 500000 objects as the default value is 100000.

Also, check size of "Logging"  and "Auditing" folder in the installation directory. If the size is very large you might need to clear these folders. And also the LDAP and SAP authentication group updates can be crucial in your case. Try setting the schedule to a frequency like every 8 hours or 10hours.

-Check CMC==>Applications==>platform search==>properties and if the crawling frequency is set to "continuous crawling" change it to "scheduled crawling"

🙂

Thanks,

Kapil

0 Kudos

Hello Kapil,

Thanks for the suggestion to increase the -maxobjectsincache to improve the CMS DB performance. I haven't applied it since it's a PRD system and should go thorugh a whole proccess but I will confirm it with SAP and suggest it to the customer. Logging and Auditing folders have low size so no problems with those. We currently have disabled the indexing functionality. For the LDAP and SAP authetication updates I will propose it to the customer to schedule it less frequest.

Thank you!

Roy

0 Kudos

Hello all,

In addition to this I'm noticing at each CMS server metrics, the "Machine Name" and "Server Name" is not showing correctly, I don't think this is normal, have you seen it before? Or what could be the cause?

Also we have noticed the Promotion Management it's performing very slow, while trying to add objects or exporting them takes too much time, almost 30 min for one single report, And I'm leading all of the slowness presenting in the system to the same cause, that is unknown right now.

Regards!

Roy Garcia

former_member185603
Active Contributor
0 Kudos

Can you assign the Hostname or IP address instead of Auto Assign option?

0 Kudos

Hi Jawahar,

I can try that, but the main thing is that in the other environments(DEV, QA) is currently set to auto assign and its showing the Machine Name(host) and the "Name Server"(host:port) correctly. So I was thinking what could happened that caused these metrics don't show correctly anymore.

Regards!

Roy Garcia

Former Member
0 Kudos

Hello there,

You can try following things:

-Clear tomcat cache.

-In CMC Homepage==>Preferences, check the number of objects per page and set it to 50 if it is set to  a higher value.

-Check value of "Auditing events in the queue" in CMS metrics. If it is too high, you might need to clear "Auditing" folder in the installation directory.

-In case you are using any third party authentication like Windows AD or LDAP etc. then you can check if the plugin  schedule frequency is not too less like every hour. Set it to run after every 12 hours.

Regards,

Kapil

0 Kudos

Hi Kapil,

This weekend we'll have a planned restart where I will clean Tomcat's cache though in that too, also the numer of objects per page we currently have set to 50, the auditing events in queue for CMS is less tan 5 objects. For the third party authentication, we have enabled the LDAP and SAP auth. and is scheduled to update every hour, I'll check with the customer if it's viable to change it at least for every 6 hours.

Thank you so much for your comments.

Regards!

Roy García

Former Member
0 Kudos

Hello Roy,

Please review KBA-2174292 regarding best practices for LDAP auth. it says that normally the plugin update should take place once per day. This applies to any third party authentication.

Also KBA-1867416 would be helpful.

Thanks!!

-Kapil

former_member205064
Active Contributor
0 Kudos

Stop the CMS

add the

-maxobjectsincache 400000

in command line argument to the CMS command line

restart the cms and check the behavior.

0 Kudos

Hello Raunak,

Actually we currently have almost 100,000 objects in CMS system cache for one CMS server and 102,000 objects in the other one. So I don't think setting to 400,000 would help, also it's  a Production environment and it should go through a whole process to change a parameter for CMS server, what does that parameter will do? that can imrpove the CMS functionality.

Regards!

Roy Garcia

Former Member
0 Kudos

Have you done APS Configuration as per SAP Note?

0 Kudos

Hello Kashif,

We already have sized our APS services according to the Best Practices for SAPBO BI 4.0 Adaptive Processing Servers document. So every APS have been tuned correctly.

Thank you.

Roy García