Global allocation limit on HANA secondary node

[Former Member]

hi

I am trying to find out if there is some calculation to set the global allocation limit on a Seondary HANA node for System replication, we set the limit to 0 for the initial full load and now we are trying to figure out what to set the memory limit on the secondary for the delta replications as the box where the secondary resides also share another HANA instance .

Thank you

Jonu Joy

[former_member192508]

Hi All,

I think the vital piece of information missing from this thread is that the Global Allocation Limit (GAL) for the DR instance needs to be 64GiB or the level of row data +20GiB, which ever is greater. The reason being that row data has to be in memory, as do the HANA services, column data is optional.

We monitor this important metric with Monitiq, as discussed in https://www.monitiq.com/monitoring-hana-replication-dont-forget-the-gal/

This combined with the setting to prevent pre-load of data, will provide space to house your non-prod environments using the unused resources on the second site. Remember not to exceed the 97% of physical memory when adding up the combined GAL's

Regards,

Robin Webster.

[Former Member]

@All:

This was a nice discussion. Thanks Joy for intating. And finally  want to end up with the soution.

on your QAS/PRD replication box -> let say if it is 1 TB

for

QAS -> keep global_alocation_limit to 700 GB

PRD replication box -> keep the rest.

Joy- I belive you are also having same configuration of HANA Box. so this should work in your case

Note: During Takeover you need to ensure that in your

PRD replication box should change the value of global_alocation_limit to more value .

Regards,

Pavan Gunda

[deepak_chodha]

Hi Jonu Joy,

Please note you shall only restrict the memory of secondary site until that site is not actively used by applications.

How to do:

1. login to secondary site Linux server using sidadm user of HANA. example <hdbadm>
2. Goto location: /hana/shared/<SID>/global/hdb/custom/config/
3. You will find a file global.ini.
4. Edit that file with sidadm user.
5. Add 2 lines to that file:

              [memorymanager]

               global_allocation_limit = <memory amount in MB>

               example 200GB limit:

               [memorymanager]

               global_allocation_limit = 200000

     6.  Save and quit the file.

There is no restart of HANA required after this and you are all set to go.

After takeover there shall not be any other HANA system running on secondary site.

Happy HANA

Deepak Chodha.

[Former Member]

thx deepak, i am trying to figure out how much memory should be set on secondary, not trying to find how to set it .

[deepak_chodha]

Hi Jonu Jay,

Now that is tricky.

You need to make sure HANA secondary site shall remain active with pre-loaded data and also has some resources to keep accepting the data. So that is completely on your production server.

You need to take care of:

Current database size(because that will be there in-memory)

+

growth of database(because it will keep increasing depending upon the throughput of primary)

+

Resources required to do the replication.

There is no such measurement tool which SAP provides to calculate the "how much it should be set to after the initial load is completed."

In your scenario I suggest is you shall restrict your DEV/QA systems and not secondary HANA system which is being replicated.

Happy HANA

Deepak Chodha.

[HayBouten]

@Deepak, I think that is not what Jonu is asking for.

The secondary system in the replication setup is in 'shadow operation' so that one can have limited resources as it doesn't keep the tables in memory. This is about 10% of the resources in the system.

The DEV and QA can be actively used and they should not use the full resources (max. 90%) as the other 10% is for the 'Shadow operation' replicated PRD system.

@Juno, the documents that I linked explain more on how it works. So to make it work, you give 10% to PRD and 45% to DEV/QAS. Maybe you need to play around a bit with these values.

[Former Member]

thx much Hay

[HayBouten]

@Jonu, So what you want is "Using Secondary Servers for Non-Production systems". Please check:

• the SAP HANA Administration guide page 495
• SAP Node 1999880 - FAQ: SAP HANA System Replication
• The document:

You need:

• The replicated systems still needs about 10% system resources
• The DEV/QA need their own storage.
• Preload of tables must be switched off on the Secondary
• The global allocation limit needs to be set so that it works for all the systems.

So it's a bit more then only setting the global allocation limit parameter.

[Former Member]

Hay  ,

this is exaclty what i am trying to find out as to how much it should be set to after the initial load is completed.

The global allocation limit needs to be set so that it works for all the systems

[Former Member]

Hi Jonu,

Why do you want to set it ? you can set it as 0. I believe there is no harm in setting it like that

Regards,

Pavan Gunda

[Former Member]

we want to set it as there are multiple HANA instances running on the same box .

[Former Member]

Is this a Single node system or distributed system?

What is the current memory available on Production and on System Replication hardware?

We had setup the SAP HANA system replication and had some learnings from it, I can share the same after you answer the above questions

[Former Member]

this is a distributed system with 6 nodes with 1TB mem on each box for both primary and secondary .

[Former Member]

We have a 3 Node Distributed system with 512 GB RAM on each node in Production, the Global Allocation limit set to 460 GB(under the Global.ini and applicable to all underlying nodes)

We started with a 3 Node Distributed system with 512 GB RAM on each node in Non Prod

System Replication instance was sharing resources with another non prod system and we split the Global Allocation Limit(GAL) into half for 230 GB for each instance. The rest 52 GB was left for the system resources on each node

We had repeated issues with System Replication as it would struggle to do a Full Replica copy and keep restarting

We had to eventually add 512 GB RAM on each secondary node(making it 1 TB on each node) and set the same GAL of 460 GB for the System Replication instance leaving the rest to the Non prod instance

If you have 3 instances running on Non prod shared distributed system you will need to do some permutations and combinations to keep a  safe amount of GAL for System Replication instance

We have learnt it is always safe to allocate the same GAL for system replication as on PROD instance based on our experience

Note:

1. Make sure you leave Preload tables option turned OFF for the System Replication Instance

2. The above observations were found for both System Replication mode SYNCMEM and ASYNC

[Former Member]

thx much Sunil, so after the initial load is completed do you still have 460 GB set on the secondary systems, we thought that after the initital load is done we dont need that much memory for the delta replication .

[Former Member]

Yes we had to leave it as 460 GB, there are some huge Delta Replica on our system during the Daily loads and we did not want to change the GAL as it may effect the System replication

As mentioned before please test the System Replication thoroughly and arrive at the right GAL accordingly, in our case we decided to go with additional memory based on our tests