Hi,
Recently, backups on several servers started to fail. I schedule backups from dbacockpit. There program log shows:
*** DBA Action starts:
Timestamp: 20131117200023 Function: sda Object: DATA
*** SAVE/RESTORE request:
SAVE DATA QUICK TO '/mnt/backup/data/2_completeDataBackup.fcs' FILE PACKED BLOCKSIZE 8 NO CHECKPOINT MEDIANAME '2_completeDataB
*** SAVE/RESTORE request accepted:
OK
*** SAVE/RESTORE result:
ERR
-24988,ERR_SQL: SQL error
-903,Host file I/O error
3,Data backup failed
1,Backupmedium #1 (/mnt/backup/data/2_completeDataBackup.fcs)
Backup error occured, Error code 3700 "hostfile_error"
*** SAVE/RESTORE request:
SAVE DATA CANCEL
*** SAVE/RESTORE request accepted:
OK
*** SAVE/RESTORE result:
ERR
-24988,ERR_SQL: SQL error
-104,DBM command impossible at this time
*** DBA Action ends:
Timestamp: 20131117232752 State: Failure
While in Database Studio the following message popups:
The backup history file /sapdb/FCS/data/wrk/FCS/dbm.knl is corrupted in line 132
-24763,ERR_BHIST_INVALID_ENTRY: The line "52776432026B|DAT_000000048|SAVE WARM|2013-11-04 10:09:06|2013-11-04 10:09:06|2013-11-04 10:09:07|2013-11-04 11:58:18| 17380677| |NO |2_completeDataBackup | 17904640| 1| -903| " is an invalid line for the backup history
-24757,ERR_FORMAT_SEPARATORNOP: The separator '|' cannot be found 15 times within string "52776432026B|DAT_000000048|SAVE WARM|2013-11-04 10:09:06|2013-11-04 10:09:06|2013-11-04 10:09:07|2013-11-04 11:58:18| 17380677| |NO |2_completeDataBackup | 17904640| 1| -903| "
-24762,ERR_FORMAT_SEPARATORMISSING: String " " does not contain the expected separator '|'
The backup history file /sapdb/FCS/data/wrk/FCS/dbm.knl is corrupted in line 132
-24763,ERR_BHIST_INVALID_ENTRY: The line "52776432026B|DAT_000000048|SAVE WARM|2013-11-04 10:09:06|2013-11-04 10:09:06|2013-11-04 10:09:07|2013-11-04 11:58:18| 17380677| |NO |2_completeDataBackup | 17904640| 1| -903| " is an invalid line for the backup history
-24757,ERR_FORMAT_SEPARATORNOP: The separator '|' cannot be found 15 times within string "52776432026B|DAT_000000048|SAVE WARM|2013-11-04 10:09:06|2013-11-04 10:09:06|2013-11-04 10:09:07|2013-11-04 11:58:18| 17380677| |NO |2_completeDataBackup | 17904640| 1| -903| "
-24762,ERR_FORMAT_SEPARATORMISSING: String " " does not contain the expected separator '|'
MaxDB version on this system is 7.9.08.12.
When I open dbm.knl on OS level I can see that some lines mentioned in DatabaseStudio and some other lines contain 'garbage' characthers like:
52891247009D|DAT_000000066|SAVE WARM|2013-11-17 20:00:23|2013-11-17 20:00:23|2013-11-17 20:00:29|2013-11-17 23:11:46| 19487304| |NO |2_completeDataBackup | 17917200| 1| -903|^A^@^@^@^@^@^@^@<A5>^<8E>^@^@^@^@^@pΚ<DF>^]^?^@^@wΚ<DF>^]^?^@^@<80>Κ<DF>^]^?^@^@|
After repeating same DB backup, I could notice that backup size is about the same size as previous successful backups (almost exactly the same size) and it seems backup fails when it tries to record activity in dbm.knl. I tried to remove grabaged lines from dbm.knl and first immediate backup works but later garbaged lines are again introduced to dbm.knl and database backups start to fail again.
This happens on several systems with 7.9 maxdb.
Any idea how to resolve this issue?