cancel
Showing results for 
Search instead for 
Did you mean: 

cleaning up log files in $(DIR_INSTANCE)/backup/log

Former Member
0 Kudos


There has ben huge number of logs that are getting generated under $(DIR_INSTANCE)/backup/log directory. Is there a way to dlete this automatically after some days ?

Like script or parmater in HANA studio ?

I know the below

"Each time the log backup job runs, a log segment snapshot or series of files are created in the $(DIR_INSTANCE)/backup/log directory. During the log file backup processes, new files are created each time the job runs. The existing files are not automatically deleted or overwritten. In short, the files will grow indefinitely until they are deleted by the SAP HANA DBA"

Script:

find /usr/sap/../../backup/log/* -mtime +5 -exec rm {} \;

I am looking for somekind of parmater in HANA . Is it possible?

Regards,

Pavan Gunda

Accepted Solutions (0)

Answers (2)

Answers (2)

richard_bremer
Advisor
Advisor
0 Kudos

In addition, you should understand what you're doing. The Script your are quoting which blindly deletes all log backups will render your backup setup practically unusable (at least for point-in-time recovery, including restoring the DB to the latest consistent state).

If you really want this behavior, you should disable the automatic log backup, and set the log mode to overwrite. See the Admin guide or Lars' and my book.

If, on the other hand, you want to set up a useful mechanism that gives you a backup retention time of a few days or week, including management of the data and log backups, the script mentioned in http://scn.sap.com/community/hana-in-memory/blog/2014/11/27/new-and-final-version-of-backup- script is one option to look at. You should in any case make sure you understand the data and log backup mechanisms of HANA, again see the mentioned documentation.

Best,

Richard


Former Member
0 Kudos

Hi Both,

Why there is no parameter defined for it in HANA studio ? Any idea ?

Regards,

Pavan Gunda

lbreddemann
Active Contributor
0 Kudos

Well, I'd say because it really doesn't belong there.

Look, professional users of DBMS systems don't backup just to disk.

They ensure that the log backups are copied at least once to a safe other place, maybe onto tape, another data center or some "cloud storage".

Very often they use backup management tools for this. In fact these backup management tools are able to manage the backup demands of the whole IT infrastructure and handle this aspect from a single central point.

And that's a good thing. If you think about it, most of the times, your IT system landscape does not consist of just the DBMS. There's application servers, OS installations, LDAP servers.. a whole set of other systems. Those need to be backed up, too. And instead of doing this system by system these system allow you to cope with the backup-topic in unified way. Nice, ain't it?

Coming back to the first point: the database cannot really know your backup strategy or retention policy. It cannot know when it is safe to delete the log backups because it doesn't know when the files had been copied.

SAP HANA comes with an interface API (BACKINT) that allows those aforementioned backup tools to go an 'tell' SAP HANA about the state of the backup activity and they will also do the "clean-up" then.

But without such a tool, it doesn't make any sense to auto-delete the log backups.

This actually is one of the few cases where a binary thinking can be applied.

Either you need to be able to restore and recover the  system. Then you cannot just delete the backups, but only when it is sure that the files are stored somewhere safe.

Or you simply don't care about backups, really.

You accept that there is a good chance to loose data because you may loose some log backups.

For these cases you can actually avoid log backup writing altogether (log mode OVERWRITE).

That's my take on this.

I am well aware that other systems bring their own backup-management systems "on-board"; however I don't know of any that would allow you to automatically delete un-safed log backups.

- Lars

Former Member
0 Kudos

I feel that there should be a parameter like log_deletion_xyz=enable/disable

if enable it should ask for number of days . As customer knows about their backup policies they can easily set the above values based on there requirement.

But, without having anything..the /backup/log is just getting filled up and system is going into hung sate. (seen in many customer landscapes).

later after troubleshooting ..we will come to know that log directory is full and hence hung state.

later on we delete it. And moreover keeping logs for long days will doesnt make any sense because there will be complete data backups for hourly/daily/weelly/monthly backup will happen.

and

1. i went through the document that you ppl have mentioned , but that scripts will not clear any logs that instance/backup/logs --feels very complicated.

2. @Richard bremer:

i feel that below is easy way of achiving my requirement.  Do you find any harm in this ?

find /usr/sap/../../backup/log/* -mtime +5 -exec rm {} \; -> lsrs, if you have idea, can you also please comment on it


i will make above script to run for every 5 days. Becoz we have daily complete data backup. and OS backup


Regards,

Pavan Gunda

richard_bremer
Advisor
Advisor
0 Kudos

Adding to Lars' point, SAP HANA offers you a way to delete old data and log backups from disk - based on a given backup ID - that is, you can select the oldest data backup that you want to keep, and then delete all data and log backups that are older than this one.

What SAP HANA does not offer is a scheduling mechanism for this - for the reasons that Lars has outlined above.

See the description of the "BACKUP CATALOG DELETE" command in the SAP HANA administration guide; there is a UI on top of that in the SAP HANA studio.

Richard

richard_bremer
Advisor
Advisor
0 Kudos

I have a pretty clear idea about this way of deletion indeed: it is a really bad idea. The reason is that you need to look at the dependencies between data and log backups. The only clean way of deleting log backups is to figure out what is the oldest data backup you want to be able to recover from; then, based on this data backup, you can determine all log backups that are not needed anymore. That's what my shell script provides.

On any database, you have to either disable log backups, or implement a backup management strategy. If customers don't do this, they will run into problems. That is not the fault of the backup system, but of the implementation team. Since the beginning of HANA, you could disable log backups. The script that explains how you have to and how you could manage your log backups exists since about two or three years now, I cannot remember. The mechanism in HANA studio to manage data and log backups is also around two years old, now.

Best regards,

Richard

Former Member
0 Kudos

I think i should be more clear on this

1. yes, i can understtand that in HANA studio-> backup catalog if we select the catalog ->right click and then delete option is there. -> I am not at all concerend about this.

2. I am not bothered about the data & log backup's becoz every customer will keep it external disk/tapes.

3. What i am bothered is

There are many logs(transaction logs) that are getting generated for every 5 mins (savepoint_tim = 300sec not sure about the correct parameter name but, it will be like that.). These are getting stored from many long time under /$dir-instance/backup/log---- I am bothered about these logs.

richard_bremer
Advisor
Advisor
0 Kudos

but if you use the functionality in 1., and choose to delete from disk, it will remove the things bothering you in 3.

Former Member
0 Kudos

I dont think that will do that. Becoz our backups will goes to tapes, but those transaction logs are still in $instance/backup/log

however, let me try to save the backup to disk and delete it. And let me see it delete the logs in /$disr_instance/backup/log

I will test it in my test system and will reply back on Monday.

BTW congrats @Richard/lars for SAP HANA Admin book

richard_bremer
Advisor
Advisor
0 Kudos

Even if your data backups are on tape, it will work - HANA only deletes those backup files (data or log) that are still in the original location. If you move data backups from disk to stage, then they will not get deleted (how could HANA do this). So your problem is solved.

If you write your backups to disk anyway, you really want to check if BACKINT is an option for you, there are a lot of professional backup solutions that have implemented HANA's BACKINT interface.

Best,

Richard

Former Member
0 Kudos


The above dint worked. We need to have an external script to clean up the logs that are present in /backup/logs.

I belive that there is no other option in SAP HANA for doing that apart from manual scripts.

P.S: you need to run it for every 5 days(may be depends on requirement) .

Regards,

Pavan Gunda

Former Member
0 Kudos


in additon My reqiremetn is

1. clean the 5 days older logs that are present in /backup/log directory

srinivasan_vinayagam
Active Contributor
0 Kudos

Hi Pavan,

Make sure the automatic log backup is enabled.

Try this.

In Hana Studio open the SQL editor and execute the command "alter system reclaim log;" and take Backup.

What is your log_mode , Is it value is Normal?

Regards,

V Srinivasan

Former Member
0 Kudos

Hi Srini,

Yes, my log_mode is normal

"alter system reclaim log;" -> I belive this will delete the logs and needs to execute manually.

But, i want automated one. something like the below. 

find /usr/sap/../../backup/log/* -mtime +5 -exec rm {} \;

srinivasan_vinayagam
Active Contributor
0 Kudos

Hi Pavan,

No automation process may be as per below thread.

Reply from SAP support,They said that the automatic delete backup catalog is not supported now.

Regards,

V Srinivasan

richard_bremer
Advisor
Advisor
0 Kudos

Hi Pavan,

you have me totally confused: If you write your log backups to tapes, how can they be in ${DIR_INSTANCE}/backup/log?

Any log backup that gets created on the file system (the actual directory doesn't matter) can be deleted via the BACKUP CATALOG DELETE statement, as long as it is maintained in the backup catalog - unless you manually move the backup to some other location. If you have managed your log backups incorrectly, it can be that a deletion fails (e.g.: moved log backups away from original location; then deleted from catalog; then put log backups back into original location and try to clean up using BACKUP CATALOG DELETE).

If you have not messed up your system yet, and if now log backup deletion is not working on your system, it would be a bug and you would have to open a ticket with SAP support, and in this ticket describe with as much detail as you can provide the situation. Parameter values and screenshots might be helpful, or providing system access.

Scheduling of backup deletion would be a feature request - for those SAP customer messages would be the wrong forum.

I believe you are either not describing your situation correctly, or you testing wrongly, or there is a bug in the software version you are using.

Best,

Richard

Former Member
0 Kudos

Hi Everyone,

I saw this post today. I am able to understand what Pavan was trying explain, because I faced this issue twice. Pavan in my case data & log backup goes to tape using 3rd party tool & hence uses backint. First lets try to understand what does log backup in HAN do? It backs up log segment from log area (disk = /$dir-instance/backup/log) to the tape & then it auto clears the log segments from the disk (that means it auto reclaims the space from disk). However, this functionality (delete operation of log segments from disk) doesnt work as usual when there are issues with backup at 3rd party backup tool. WHen the backup mechanism disturbs, delete operation doesnt work & even if you see successfull log backups, you will not see successfull log reclaims (this can be a problem at HANA product level or 3rd party backup tool).

You can try restarting HANA & see if behaviour continues. In our case there was issue at 3rd party backup tool which they corrected. Also we were on rev74 & there is a known bug in rev74 which is resolved in 74.01 & higher (refer below note). We upgraded the HANA to rev 82 & do not see this problem.

SAP Note 2021160 - logbackups are hanging in running state when backint solution is used

Regards

Sachin

Former Member
0 Kudos

Yes, Even i feel that it is a product bug(may be with rev 74). I need to try by upgrading the version and need to check if the logs are getting reclaimed or not.  Thanks for understanding @sachin_bhatt. I know this scenario is tough to explain/understand as long as they face the same issue.

Former Member
0 Kudos

Hi Pavan,

Thanks. if you are on 74.00 upgrading to next maintenance or revision patch is suggested by SAP notes as there is a known bug in rev74.00. Also if your original query is answered please mark this thread as answered.

Best Regards

Sachin Bhatt

Former Member
0 Kudos

HI Sachin,

After upgrading and performing a test, i will mark it as answered.

Regards,

Pavan Gunda

lbreddemann
Active Contributor
0 Kudos

You could use the script created by () to do the clean up for you.

- Lars