how to increase ASE Max Memory on Linux?

I have a linux with ASE 12_5. On the machine, there 17G memory, 2G for ramdisk. Sybase max memory configued as 4000000(2K)=7.63G.

If reserve 1G for OS, then I think there are 17-2-1 = 14G available and I want to ASE can utilize this 14G memory.

What I did is:

1. Increase max shared memory(shmmax):

modify /etc/sysctl.conf to set shmmax = 14738890752 (this is roughly 13.72G)

run command:

echo "kernel.shmmax=14738890752" >> /etc/sysctl.conf

2. Reboot Linux with command reboot

3. Modify myserver.cfg to chang max memory for sybase:

max memory = 7196724

the start sybase ase, and I got error(see my ticket http://scn.sap.com/thread/3530977), but I can't start sybase. I try to increase max memory to different number for example 5000000, and not success. Only original 4000000 working.

Not sure why. I populate all info as below. How to resolve this problem?

[sybase@myserver ASE-12_5]$ free -g
             total       used       free     shared    buffers     cached
Mem:            17          7          9          0          0          7
-/+ buffers/cache:          0         17
Swap:            3          0          3

[sybase@myserver ASE-12_5]$ df -h
Filesystem            Size Used Avail Use% Mounted on
/dev/sda2              12G 2.6G 8.8G 23% /
/dev/sda1             251M   19M 220M   8% /boot
none                  8.7G     0 8.7G   0% /dev/shm
/dev/sdb1              30G 8.4G   20G 30% /home
tmpfs                 2.1G   20M 2.1G   1% /db/oe/tempdb

[sybase@myserver ASE-12_5]$ cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 1

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

# kernel.shmmax = 10737418240
kernel.shmmax= 14738890752
kernel.shmmni = 4096
kernel.shmall = 2097152

[sybase@myserver ASE-12_5]$ cat /proc/sys/kernel/shmmax
14738890752

Sybase: max memory = 4000000

here is the output of sp_configure memory:

additional network memory 0 0 0 0 bytes dynamic

allocate max shared memory 0 0 1 1 switch dynamic

compression memory size 0 152 0 0 memory pages(2k) dynamic

engine memory log size 0 2 0 0 memory pages(2k) dynamic

heap memory per user 4096 0 4096 4096 bytes dynamic

lock shared memory 0 0 0 0 switch static

max memory 45056 8000000 4000000 4000000 memory pages(2k) dynamic

memory alignment boundary 2048 0 2048 2048 bytes static

memory per worker process 1024 66 4096 4096 bytes dynamic

messaging memory 400 0 400 400 memory pages(2k) dynamic

shared memory starting address 0 0 0 0 not applicable static

total logical memory 32768 7620028 3810014 3810074 memory pages(2k) read-only

total physical memory 0 8000000 0 4000000 memory pages(2k) read-only

SAP Managed Tags:
SAP Adaptive Server Enterprise

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Looks like there are 2 ramdisks:

none 8.7G 0 8.7G 0% /dev/shm ----this is set by system as default, 50% of all RAM???

tmpfs 2.1G 20M 2.1G 1% /db/oe/tempdb ---this is setup manually

Not sure about /dev/shm. Should I reduce it to small, say 128M to release memory? there is nothing under /dev/shm(run cmd ls /dev/shm)

Another question: if sybase ase max memory always get memory from kernel.shmmax?

Hi Mark, here is the showing from ipcs -a:

[sybase@myserver ASE-12_5]$ ipcs -a

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0xf402f74d 65536 sybase 600 8192000000 2

------ Semaphore Arrays --------
key semid owner perms nsems

------ Message Queues --------
key msqid owner perms used-bytes messages

A quick google search would show that /dev/shm is Linux's way of managing shared memory, so in your case it would appear that you have 8.7GB of memory set aside for shared memory usage.

This in turn would explain why your attempt to have Sybase use 10GB (max memory=5000000) fails, and why Sybase can use 8GB (max memory=4000000) ... the former is > 8.7GB while the latter is < 8.7 GB.

So the next question would be ... why doesn't /dev/shm show ~14GB?

NOTE: I'm not a linux admin so at this point it's probably quicker to do some google searching as opposed to waiting for me to do some google searching, eh.

When I try to change max memory from 4000000 to 5000000 with sp_configure, I got error message:

00:00000:00007:2014/04/08 12:01:57.74 server The configuration option 'max memory' has been chang ed by 'sa' from '4000000' to '5000000'.

00:00000:00007:2014/04/08 12:01:57.74 kernel Allocating a shared memory segment of size 204800000 0 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel os_create_region: shmget(0xf402f74f): No space left on device

00:00000:00007:2014/04/08 12:01:57.74 kernel kbcreate: couldn't create server region 0.

00:00000:00007:2014/04/08 12:01:57.74 kernel Retrying shared memory allocation with smaller size 1024000000 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel Allocating a shared memory segment of size 102400000 0 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel os_create_region: shmget(0xf402f74f): No space left on device

00:00000:00007:2014/04/08 12:01:57.74 kernel kbcreate: couldn't create server region 0.

00:00000:00007:2014/04/08 12:01:57.74 kernel Retrying shared memory allocation with smaller size 512000000 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel Allocating a shared memory segment of size 512000000 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel os_create_region: shmget(0xf402f74f): No space left on device

00:00000:00007:2014/04/08 12:01:57.74 kernel kbcreate: couldn't create server region 0.

00:00000:00007:2014/04/08 12:01:57.74 kernel Retrying shared memory allocation with smaller size 268435456 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel Allocating a shared memory segment of size 268435456 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel Allocating a shared memory segment of size 177956454 4 bytes.

00:00000:00007:2014/04/08 12:01:57.74 kernel os_create_region: shmget(0xf402f751): No space left on device

00:00000:00007:2014/04/08 12:01:57.74 kernel kbcreate: couldn't create server region 1.

01:00000:00007:2014/04/08 12:01:57.75 kernel Attempt to attach shared memory (id 65537) at addres s 0x2c7e12a000, with flags 0 failed

01:00000:00007:2014/04/08 12:01:57.75 kernel Cannot attach shared memory : error = 22 (Invalid ar gument)

01:00000:00007:2014/04/08 12:01:57.75 kernel os_attach_region: shmat(65537): Invalid argument

01:00000:00007:2014/04/08 12:01:57.75 kernel kbattach: couldn't attach to server region 0

If I try to change max memory from 4000000 to 4500000, then I got following:

00:00000:00007:2014/04/08 12:10:36.39 server The configuration option 'max memory' has been changed by 'sa' from '4000000' to '4500000'.

00:00000:00007:2014/04/08 12:10:36.39 kernel Allocating a shared memory segment of size 1024000000 bytes.

00:00000:00007:2014/04/08 12:10:36.39 kernel os_create_region: shmget(0xf402f752): No space left on device

00:00000:00007:2014/04/08 12:10:36.39 kernel kbcreate: couldn't create server region 0.

00:00000:00007:2014/04/08 12:10:36.39 kernel Retrying shared memory allocation with smaller size 512000000 bytes.

00:00000:00007:2014/04/08 12:10:36.39 kernel Allocating a shared memory segment of size 512000000 bytes.

00:00000:00007:2014/04/08 12:10:36.39 kernel os_create_region: shmget(0xf402f752): No space left on device

00:00000:00007:2014/04/08 12:10:36.39 kernel kbcreate: couldn't create server region 0.

00:00000:00007:2014/04/08 12:10:36.39 kernel Retrying shared memory allocation with smaller size 268435456 bytes.

00:00000:00007:2014/04/08 12:10:36.39 kernel Allocating a shared memory segment of size 268435456 bytes.

00:00000:00007:2014/04/08 12:10:36.39 kernel Allocating a shared memory segment of size 755564544 bytes.

00:00000:00007:2014/04/08 12:10:36.39 kernel os_create_region: shmget(0xf402f753): No space left on device

00:00000:00007:2014/04/08 12:10:36.39 kernel kbcreate: couldn't create server region 1.

01:00000:00007:2014/04/08 12:10:36.41 kernel Attempt to attach shared memory (id 98305) at address 0x2c7e12a000, with flags 0 failed

01:00000:00007:2014/04/08 12:10:36.41 kernel Cannot attach shared memory : error = 22 (Invalid argument)

01:00000:00007:2014/04/08 12:10:36.41 kernel os_attach_region: shmat(98305): Invalid argument

01:00000:00007:2014/04/08 12:10:36.41 kernel kbattach: couldn't attach to server region 0

on most linux systems, /dev/shm (shared memory) is mounted with 50% of the total available physical memory by default. This _can_ be changed carefully by a sysadmin by simply adding a line in the /etc/fstab file for /dev/shm with the tmpfs option "size=<n>%" for example (you can also specify size in terms of Kb or Gb). Display the man page for "mount" and see the "tmpfs" mounting options for a full discussion of the topic:

# man mount

For example. My Ubuntu linux box has 3Gb of physical memory. My /etc/fstab entry for /run/shm looks like:

# <file system> <mount point> <type> <options> <dump> <pass>

none /run/shm tmpfs rw,nosuid,nodev,size=70% 0 0

as a result, my /run/shm (notice that on Ubuntu, it's /run/shm, not /dev/shm, different path but same concept) is sized:

# df -h /run/shm

Filesystem Size Used Avail Use% Mounted on

none 2.1G 0 2.1G 0% /run/shm

I'm happy to report that 2.1G is 70% of 3.0G.

Thanks. I have tried to change size for /dev/shm to smaller, 256M or 2G or lager 14G(by modify /etc/fstab) then reboot linux, then start SYBASE ASE, then run sp_configure to change max memory from 4000000 to 5000000, I got same error(see posted above), if I change it from 4000000 to 3000000, I get message said max memory should be larger than logical memory....

So I am so confused. because I have 17G memory, ramdisk took 2G for tempdb. So I should have 14G available, but only 7G available for sybase?

even I run free -g to check, it shows there are 9G free

sybase@mysyb ~]$ free -g

total used free shared buffers cached

Mem: 17 7 9 0 0 7

-/+ buffers/cache: 0 17

Swap: 3 0 3

I try to check doc like sag1,sag2, ... but can't figure it out. how to resolve this problem?

Kent. You don't seem to be interested in reading about the /dev/shm concept. Grab a manual for your OS and understand concepts around memory, and all of the other resources you are dealing with. I'll try to explain again using more explicit terms.

Firstly, forget about the "free" command. It says nothing about shared memory. The "shared" column is obsolete. The "man" page for the "free" command probably says as much.

/dev/shm is simply a linux pool of memory set aside as _available_ "shared memory". If you want your ASE instance to be a 10Gb instance on a 17G physical box, then you need to size /dev/shm for 10Gb or higher (but careful, not too much higher). Again, that's because by default, your Linux installation only makes available 50% (8.5Gb in your case) of your physical memory for shared memory access. Let's set your /dev/shm (again, your _available_ shared memory pool) to 11Gb to allow headroom for some growth.

your /etc/fstab entry will be something like:

none /dev/shm tmpfs rw,nosuid,nodev,size=11G 0 0

After a reboot, your "max memory" configuration setting for ASE can be set to 5000000 ( which is a bit less than 10Gb). Double check your shared memory _limit_ setting:

grep "kernel.shmmax" /etc/sysctl.conf

Again, this is a MAXIMUM allowed limit. It doesn't state how much shared mem is AVAILABLE to use. That comes from your mounting option for /dev/shm above. Obviously this shmmax setting should be at least 11G in out example. It's okay if it's more.

So, in summary:

kernel.shmmax setting is a limit for the size of a single shared memory segment. In most cases it can just be set to the amount of physical memory you have. It's a limit. If there isn't availabe memory, the user will get an error. So, it simplifies the discussion here to just set this to the total amount of physical memory on the box. There are other shared memory limits (like the NUMBER of shared memory segments that one process can create, and the TOTAL shared memory a single process can request), but let's move on.

/dev/shm size is the amount of memory available to be used as shared memory. By default on Linux, it mounts with 50% of the physical memory on the box (THIS IS YOUR PROBLEM). It can be increased as necessary to allow more memory on the box to be used as shared memory. Use /etc/fstab to change size settings. Run the "mount" command by itself (no options) to see how this filesystem is mounted and take care to use the same options in /etc/fstab (plus adding the "size" option). Care should be taken in the sizing though so as not to starve other processes of the memory on the box.

max memory is the size (in 2k byte units) in the .cfg file that ASE requests from the operating system (if "allocate max memory" setting is on, one memory segment this size will be allocated at startup, if not, then "total logical memory" size will be allocated at startup and multiple additional memory segments may be requested during the life of ASE runtime as memory requirements are changed). This is the absolute limit of the amount of shared memory this ASE instance will request from the operating system.

/dev/shm must be greater than or exactly equal to "max memory".

EDIT: I see above that you only have 9Gb available anyway. What other processes on this box use up 8b of memory? Anyway, I told you above to size for 11Gb, but obviously you are constrained by what is available after you start everything else on this server. Apparently, you really only have around 9Gb of memory for ASE. So, size above parameters accordingly. Also, as Jeff noted in other thread, quit messing with tmpfs for your tempdb devices. Just use normal filesystem files and free up that memory for your caches.

Message was edited by: Kevin Sherlock

Hi Kevin, thanks for help. I double to check the info and redo again, still can't change max memory for sybase. here is what I did:

1. change size of /dev/shm to 14G

vi /etc/fstab

2. remount /dev/shm

mount -o remount /dev/shm

3. change shmmax:

/sbin/sysctl -w kernel.shmmax=17674797056

/sbin/sysctl -p

4. Reboot linux:

init 6

After reboot, here is info:

[root@mysyb ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda2 12G 2.6G 8.8G 23% /

/dev/sda1 251M 19M 220M 8% /boot

none 14G 0 14G 0% /dev/shm

/dev/sdb1 30G 8.4G 20G 30% /home

tmpfs 2.1G 0 2.1G 0% /db/oe/tempdb

[root@mysyb ~]# grep "kernel.shmmax" /etc/sysctl.conf

# kernel.shmmax = 10737418240

# kernel.shmmax=14738890752

kernel.shmmax = 17674797056

[root@mysyb ~]# cat /etc/sysctl.conf

# Kernel sysctl configuration file for Red Hat Linux

# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and

# sysctl.conf(5) for more details.

# Controls IP packet forwarding

net.ipv4.ip_forward = 0

# Controls source route verification

net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing

net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel

kernel.sysrq = 1

# Controls whether core dumps will append the PID to the core filename.

# Useful for debugging multi-threaded applications.

kernel.core_uses_pid = 1

# kernel.shmmax = 10737418240

kernel.shmmni = 4096

kernel.shmall = 2097152

# kernel.shmmax=14738890752

kernel.shmmax = 17674797056

[root@mysyb ~]# ipcs -la

------ Shared Memory Limits --------

max number of segments = 4096

max seg size (kbytes) = 17260544

max total shared memory (kbytes) = 8388608

min seg size (bytes) = 1

------ Semaphore Limits --------

max number of arrays = 128

max semaphores per array = 250

max semaphores system wide = 32000

max ops per semop call = 32

semaphore max value = 32767

------ Messages: Limits --------

max queues system wide = 16

max size of message (bytes) = 8192

default max size of queue (bytes) = 16384

5. start sybase ASE. max memory is 4000000 at this point

6. change max memory:

isql

1> sp_configure memory

2> go

Msg 17411, Level 16, State 1:

Server 'MYSYB', Procedure 'sp_configure', Line 214:

Configuration option is not unique.

Parameter Name Default Memory Used Config Value

Run Value Unit Type

------------------------------ ----------- ----------- ------------

----------- -------------------- ----------

additional network memory 0 0 0

0 bytes dynamic

allocate max shared memory 0 0 1

1 switch dynamic

compression memory size 0 152 0

0 memory pages(2k) dynamic

engine memory log size 0 2 0

0 memory pages(2k) dynamic

heap memory per user 4096 0 4096

4096 bytes dynamic

lock shared memory 0 0 0

0 switch static

max memory 45056 8000002 4000001

4000001 memory pages(2k) dynamic

memory alignment boundary 2048 0 2048

2048 bytes static

memory per worker process 1024 66 4096

4096 bytes dynamic

messaging memory 400 0 400

400 memory pages(2k) dynamic

shared memory starting address 0 0 0

0 not applicable static

total logical memory 32768 7620028 3810014

3810074 memory pages(2k) read-only

total physical memory 0 8000002 0

4000001 memory pages(2k) read-only

(1 row affected)

An additional 379854 K bytes of memory is available for reconfiguration. This is

the difference between 'max memory' and 'total logical memory'.

(return status = 1)

then try to change max memory

28> sp_configure 'max memory', 4000100

29> go

Parameter Name Default Memory Used Config Value

Run Value Unit Type

------------------------------ ----------- ----------- ------------

----------- -------------------- ----------

max memory 45056 80002000 40001000

40001000 memory pages(2k) dynamic

(1 row affected)

Configuration option changed. The SQL Server need not be rebooted since the

option is dynamic.

Changing the value of 'max memory' does not increase the amount of memory

Adaptive Server uses.

(return status = 0)

message:

00:00000:00007:2014/04/08 14:55:22.39 server Configuration file '/opt/sybase/ASE-12_5/LIFESYB.cfg' has been written and the previous version has been renamed to '/opt/sybase/ASE-12_5/LIFESYB.398'.

00:00000:00007:2014/04/08 14:55:22.39 server The configuration option 'max memory' has been changed by 'sa' from '5000000' to '4000001'.

00:00000:00007:2014/04/08 14:55:39.56 server Configuration file '/opt/sybase/ASE-12_5/LIFESYB.cfg' has been written and the previous version has been renamed to '/opt/sybase/ASE-12_5/LIFESYB.399'.

00:00000:00007:2014/04/08 14:55:39.57 server The configuration option 'max memory' has been changed by 'sa' from '4000001' to '4000100'.

00:00000:00007:2014/04/08 14:55:39.57 kernel Allocating a shared memory segment of size 202752 bytes.

01:00000:00007:2014/04/08 14:55:39.58 kernel Attempt to attach shared memory (id 131073) at address 0x2c7e12b000, with flags 0 failed

01:00000:00007:2014/04/08 14:55:39.58 kernel Cannot attach shared memory : error = 22 (Invalid argument)

01:00000:00007:2014/04/08 14:55:39.58 kernel os_attach_region: shmat(131073): Invalid argument

01:00000:00007:2014/04/08 14:55:39.58 kernel kbattach: couldn't attach to server region 0

Try bumping up your shmall value.

Current setting of 2097152 (kernel.shmall) is multiplied by 4k to get your 'max total shared memory (kbytes) = 8388608'.

So if you want to set it to 14GB then divide by 4k to get the new value for kernel.shmall (eg, shmall=3500000 => 14336 MB).

changed shmall=3500000 and reboot, now it allow me to change max memory to 5000000, but not 6000000.

then change shmall=7500000, then 6000000 allowed for max memory, but not 8630272 (14G)

then I increase shmall = 30000000, then 7000000(2k) = 13.35G max memory allowed.

Thank you so much, Kevin!!! question is: how to calc shmall and how sybase use shared memory and segment?

This isn't a question of how ASE uses it, but rather the typical "desktop" configuration issue of Linux. From my sysctl, check out the comments:

# shmall is the total amount of shared memory allowed (in 4k pages) – period
# attempts to allocate shared memory above this will fail. Multiply by 4096
# to see bytes
kernel.shmall = 4294967296

# shmmax is the largest shared mem segment size (in bytes) (64GB) ASE, Backup
# server and other Sybase processes can use multiple shared memory segments –
# each segment can be up to shmmax in size (but may be smaller due to free
# contiguous memory space). Unless a small box, set in even GB.

# however, when ASE boots, it will try to grab all of the memory in a single

# shared memory segment, so make sure this value is larger than the largest

# memory setting for any ASE on the host.
kernel.shmmax = 68719476736

#maximum number of shared memory segments system wide
kernel.shmmni = 4096

Accepted Solutions (0)

Answers (0)

Ask a Question

Top Q&A Solution Author

User

Count