db2 archiving struck

[Former Member]

Hello All,

I have limited knowledge on db2 database

PRD system 22TB database size

Linux OS

In our Production system ,  /db2/PRD/log_dir/NODE0000/LOGSTREAM0000/

has 700 files accumalated.

/db2/PRD/log_dir  is used by 99% ,

The number of Primary log files configured is 256 files.

since 12 hours the number of files  /db2/PRD/log_dir/NODE0000/LOGSTREAM0000/  remain 700

the rate of archiving is 60 to 80mb/second

questions

1. is it a db2 issue ? as claimed by Back up and Recover team

2. we suspect archiving is not happening , TSM is not moving the file   - we ERP Team claim this

TSM has to archive these files , is this understanding correct ?

The rate of Generation of log files by db2 is more , compared to archiving , hence we see 700 files since 12 hours.

Please suggest how to come our of this situation , if we leave system like this for next 4hours,

system will go'es  down  it will be huge impact

Thanks in advance

[former_member182505]

Hi Sidhart

Could you please give the db2diag.log file details,  file location is /db2/SID/db2dump

mean while check the below note may help you,

495297 - DB6: Monitoring transaction log

2141933 - DB6: How to initially trouble shoot transaction log full issue

Thanks

Sadiq

[Former Member]

Hi Iqbal

Thanks for the response ,find the log extract from db2diag log

PID     : 11141828             TID : 15165          PROC : db2sysc 0

INSTANCE: db2PRD               NODE : 000           DB   : PRD

HOSTNAME: aeswdsbi007

EDUID   : 15165                EDUNAME: db2loggr (PRD) 0

FUNCTION: DB2 UDB, oper system services, sqlowrite, probe:70

MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."

          DIA8312C Disk was full.

CALLED  : OS, -, write

OSERR   : ENOSPC (28) "No space left on device"

DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes

DATA #4 : String, 105 bytes

Search for ossError*Analysis probe point after this log entry for further

self-diagnosis of this problem.

2015-10-22-08.10.53.095536+120 I39335783A3072       LEVEL: Error (OS)

PID     : 11141828             TID : 15165          PROC : db2sysc 0

INSTANCE: db2PRD               NODE : 000           DB   : PRD

HOSTNAME: aeswdsbi007

EDUID   : 15165                EDUNAME: db2loggr (PRD) 0

FUNCTION: DB2 Common, OSSe, ossErrorIOAnalysis, probe:100

CALLED  : OS, -, write

OSERR   : ENOSPC (28) "No space left on device"

[Former Member]

Hi  Iqbal  and hi all ,

parallel to this

is it a  db2 issue or TSM not able to move  log  files  ?

should we ask Bacak up and Recovery Team to check ,

please advise what needs to be checked from their end ?

Regards

[former_member182505]

Hi

1st add the space on FS to avoid the issue.

and give us the full file of db2diag log to analysis it.

also check the note i suggested if any log full is going on,

Thanks

Sadiq

[Former Member]

Hi Iqbal,

Currently FS usage  has come down to 90% automatically,

all the BI jobs are completed , but still we have 600 files in log_dir,

archiving speed is 60mb/second

I will check the Notes mentioned by you

one basic doubt:

once the files are in   db2/PRD/log_dir/NODE0000/LOGSTREAM0000 

it is the responsibility of TSM to move this file , is this understanding correct ?

Regards

[former_member182505]

Hi

This is the general behaviour of the db2  for log full,

you need to do the full analysis of log generation and accordantly you need to add the space,

yes if log is moving very slow then TSM team need to check it,

but for the log full we as SAP BASIS and DB2 has be check,

to handle the log full my note will help you, and you can kill the job which is causing the log full, before system get down,

Thanks

Sadiq