System Crash after transactional log filled filesystem

[Former Member]

Dear gurus,

We have an issue in our PRD system under FlexFrame platform. We SAP NW 7.4 (SP03) with ASE 15.7.0.042 (SuSe SLES 11 SP1) running as BW system.

While uploading data from ERP system, the transactional log was filled. We can see in <SID>.log:

Can't allocate space for object 'syslogs' in database '<SID>' because 'logsegment' segment is full/has no free extents. If you ran out of space in syslogs, dump the transaction log. Otherwise, use ALTER DATABASE to increase the size of the segment.

After this, we increase the transactional log (disk resize). Then, executed ALTER DATABASE <SID> log on <LOGDEVICE> = '<size>'

While ALTER is running the log filesystem was filled (100%), after this, <SID>.log began to grow tremendously.

We stopped Sybase and now, when we try start it all FF node will be down. The filesystem has free space (around 10 GB)

Could you help us?

Add: We think that a posible solution could be to delete the transactional log due to the fact that we understand that the failure is related to this log (maybe corrupted?)

Regards

[Former Member · Accepted solution]

Hi all,

Finally, we found the root cause, there was a snapshots, at cabinet level (setup for DRS) that filled the filesystem.

Stopping these snapshots the system is up&running fine

Regards

[Former Member]

Hi all,

Renaming <SID>.log file and start the system normally was enough to fix our mistake.

Is it possible that <SID>.log is loaded into memory at startup process?

Regards

[Former Member]

Hi all,

The issue have been reproduced again. I attached BPP.log (logs.zip)

Note: BPP.log file has again 300 GB, I attached 1700 first lines in f_1700.log and the last 1500 lines in l_1500.log.

timestamp: 2014/06/26 14:35:01.83

Is really a hardware error?

Regards

F_1700.txt.zip

L_1500.txt.zip

[former_member188958]

What caused the .log file to fill rapidly (was it just repeats of the "log segment is out of space" message, or something more?)

When you try to reboot ASE, does ASE actually complete booting or fail?

Could you provide more detail about what happens during boot?

In particular, what errors are shown in the ASE errorlog?

What is an "FF node"?

[Former Member]

Hi Bret,

The most repeated messages in the log are:

Update of syslogins failed

Error: 12205, Severity: 17, State: 2

Could not acquire a lock within the specified wait period. SERVER level wait period=1800 seconds, spid=447, lock type=exclusive row, dbid=4, objid=1456590538, pageno=954225, rowno=1. Aborting the transaction.

Also appears, "log segment is out of space" for sybmgmtdb database.

The logs mark ASE as started but startdb process runs for one hour (approx) until the physical server downs. ASE is available for isql, not for SAP system.

I attached the trace of startup process

FF node is FlexFrame node (could say similar to VMWare/Solaris Zone and BW is running into a VM/Zone)

Regards

Startup_trace.txt.zip

[tilman_model-bosch]

Dear Mr Sahagun,

The messages in the attched zip file indicate that your restarted ASE after a forfced shutdown (lots of aboted transactions that were rolled back)

But its seems crash recovery went fine and all should be ok in the databases now.

It does not tell anything about what went wrong eralier , though.

You are still on 15.7 ESD#4.2.

This version is outdated and not recommended anymore. There was a HOPT NEWS for defects in that version.
I stronly suggest to upgrade to 15.7 SP62 which is available on SMP for download.

With kind regards
Tilman Model-Bosch