Question about SAP + SLES10 + vSphere and the truth about memory

[Former Member]

Hi,

we are using vSphere and SLES 10 for a SolutionManager 7.0 on MaxDB 7.6.

The system is configured with 2 GB RAM in the VM. The database is configured to use 125000 pages as cache (100000 KB).

Yes, I know the system is configured very small for this usage, but it is a demo system...

But the system is really slow. I can't figure out whats the problem.

The RAM is configured with 2 GB in the VM a "free" in linux shows me that there is 500 MB used for file system chache

total used free shared buffers cached Mem: 2076908 2002704 74204 0 28936 578212 -/+ buffers/cache: 1395556 681352 Swap: 6185016 3037744 3147272

so this shoud be ok, for me it is close to the limit...

But VMware infrastruture client shows me, that there are only 300 MB of the 2GB is used ... but the database should use 1 GB and SAP should use some RAM , too.

So who tells the truth? Where does the RAM goes to if it is not used in VMware? Could this be the performance problem?

An other problem is, that the show dev_io of the database shows me very slow access to the files ...

i.e.

I/O via Device Processes: UNIX Devspace Read avg_read Write avg_write tid name count time count time 28596 /sapdata/SYS/sapdata/DISKD01 6532 0.0294 101 0.2077

But VMWare shows that there is nearly no usage of DISK ... average is shown as 150 KBps...

So again, who tells the truth?

What else could be the problem?

Thank you.

Best regards

Christian

[markus_doehr2 · Accepted solution]

> we are using vSphere and SLES 10 for a SolutionManager 7.0 on MaxDB 7.6.

> The system is configured with 2 GB RAM in the VM. The database is configured to use 125000 pages as cache (100000 KB).

2 GB is way too less.

On a 64bit installation the Java instance alone (which is installed as part of SolMan) uses 2 GB heap memory fixed, so all your memory is allocated by the Java process. Then you have the database (with parameter CACHE_SIZE) plus the ABAP part plus the operating system.

I would not run a SolMan with less than 6 GB of memory and carefully configuring how much is used by each of the system parts (OS, database, ABAP and Java).

Markus

[Former Member · Accepted solution]

Hello Markus,

I know 2 GB is less, but the system is still 32 bit Linux, so 6 GB is "impossible".

During the long weekend the system is switched to an other VMServer I am going to request more RAM. What a pitty that 32 bit does not support more than 3 GB....

But there is still the question whats correct? The linux variant of free or the VMware infrastructure client monitoring the RAM?

What could slow down the Database IO performance?

USE_OPEN_DIRECT we are already using ...

Thank you.

Best regards

Christian

[markus_doehr2 · Accepted solution]

> I know 2 GB is less, but the system is still 32 bit Linux, so 6 GB is "impossible".

Why that? The 32bit limit is a per process limit, not a limit for the amount of total supported memory.

> During the long weekend the system is switched to an other VMServer I am going to request more RAM. What a pitty that 32 bit does not support more than 3 GB....

I don't know who told you that but it's wrong

> But there is still the question whats correct? The linux variant of free or the VMware infrastructure client monitoring the RAM?

> What could slow down the Database IO performance?

> USE_OPEN_DIRECT we are already using ...

Are you sure it's the I/O peformance? A couple of questions arise then:

- What filesystem is underneath?

- How are your data files distributed?

- What are your I/O times for a single I/O (you can switch that on using DB50 - TAsk manager - and press on the "clock", the time will then be displayed in the database analyzer.

If you use VMWare on a recent processor you can also switch nicely to a 64bit Linux.

Markus

[nelis · Accepted solution]

I know 2 GB is less, but the system is still 32 bit Linux, so 6 GB is "impossible". 

See http://en.wikipedia.org/wiki/Physical_Address_Extension

Nelis

[markus_doehr2 · Accepted solution]

> I know 2 GB is less, but the system is still 32 bit Linux, so 6 GB is "impossible".

> See http://en.wikipedia.org/wiki/Physical_Address_Extension

PAE is for a per process allocation of more than 2 GB memory, not for the whole machine.

And:

Linux
The Linux kernel includes full PAE mode support starting with version 2.3.23,[3] 
enabling access of up to 64 GB of memory on 32-bit machines.

Our main ERP server was running on 32bit with 16 GB main memory for a LONG time.

Markus

[Former Member · Accepted solution]

From your 'free' output one sees that 3 GB swap of 6 GB is used. So using 2 GB RAM for that system is really not a good idea. Nevertheless, you can investigate your slow I/O performance by using esxtop:

http://kb.vmware.com/kb/1008205

[Former Member]

Hi all,

thanks for your reply.

Sorry, my fault, I always had in mind, that 32 bit OS could not allocate more than 3 GB (I remember SAP notes about this for Windows NT) .... but if 32 bit limits are only for each process, I am going to keep this in mind.

I am going to request more RAM .... !!

I am going to figure out what kind of SAN we are using .. at the moment a reboot of the system sped up the access ...

But there is still the question who tells the truth:

VMWare (or esxtop): showing me that the system uses 300 MB RAM

free on linux SLES10: showing me that the system uses 2GB RAM (and 3 GB SWAP)

please keep in mind that just the database is configured to use 1 GB RAM, so VMWare should be wrong...

But why shows VMWare such "nonsense" RAM usage?

If there is further information I am goin to post it.

Best regards

Christian

[Former Member]

Hi Christian,

VMWare (or esxtop): showing me that the system uses 300 MB RAM

I'd need the name of the parameter where you saw this value (or screenshot or similar). The ESX server has several parameters indicating RAM usage.

In general, [Understanding Memory Resource Management in VMware ESX Server|http://www.vmware.com/resources/techresources/10062] is a good guide for such questions.

Kind regards,

Matthias

[Former Member]

Hello Matthias,

sorry, but I missed the feature how to add a (local) screenshot to the post, so I try to describe it in words.

I am looking on my VMware Infrastructure client -> tab performance (for my sap system) -> there is a chart showing the memory (maybe use "switch to") to me -> I am talking about the values

Memory Usage (Average) in KB latest : 524288 maximum: 1078576 minimum: 524288 average: 676912 Memory Granted (Average) in KB latest : 2089608 maximum: 2089608 minimum: 2089608 average: 2089608

For me that means, at max. 1 GB of 2 GB was used from the virtual maschine, at the moment there are ca. 500 MB are used of the virtual maschine, even if the database cache should use 1 GB ...

What do I miss?

Best regards

Christian

[Former Member]

Hello Christian,

sorry, but I don't know the "Memory Usage (Average) in KB" parameter. There is a "Memory Usage (Average) in Percent". Perhaps you mean "Memory Consumed (Average) in KB" or "Memory Active (Average) in KB". It would be indeed bogus if "memory consumed" or "memory active" does not somehow reflect the real RAM consumption of the Virtual Machine.

- What ESX version is this?

- Is the ESX part of a vCenter configuration?

- If yes, what's the version of the vCenter?

- Are the VMware Tools installed inside the Guest OS?

- What Guest OS version is it?

- Did you configure share values for the assigned RAM of the Virtual Machine?

- Is there a memory overcommittment situation on the ESX server?

Kind regards,

Matthias

[Former Member]

Hello Matthias,

sorry, my fault (...because of copy and paste error...).

The parameter I see in default configuration is "Memory Active (Average) in KB" this parameter, thats the one I mean.

But I checked the other parameters as well, it seems that "Memory Consumed (Average)" is the one showing the value a free in linux shows me.

So this helped me understand the parameters

[http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1002604]

and

[http://craniumtales.blogspot.com/2009/03/vmware-memory-explained.html]

In a conclusion I am going to add RAM, and check the SAN to get the system faster.

Thank you.

Best regards

gnurl