Backup hanging Oracle 10.2.0.2 ECC600

[Former Member]

Hello,

We are running on Windows 64 bit with Oracle 10.2.0.2 installed SAP version ECC 600.

While backing up the Oracle database from DB13 the backups constantly hangs on the same specific point. It is while writing the BTABI.DATA2 from tablespace PSAPBTABI to our backup server. We backup to disk and exactly at the same point it hangs. First I used disk_copy_cmd cpio and later I changed it to dd but still the same error. I updated the BRTOOLS to its latest versions, and tried to backup this specific tablespace from brtools. Everytime the same problem. After a while the whole operating system then hangs.

Can anyone give me any advise where to look? Some log or trace files. I haven't got a clue at this moment. Could it be this file is corrupt or something? Did anyone run into this problem before? Any help would be appreciated.

Kind regards,

Dave Arends

[former_member246982 · Accepted solution]

Do you mean both back to tape and backup to disk hang at the same time?

try a DBVERIFY on the database first.

if that works, try an export of that tablespace.

Do you have any datafile belong to this tablespace exceeds 4GB? Most cpio and dd implementation has a limit of 4GB of file size.

[uwe_theisinger]

Hi,

try to check the file BTABI.DATA2 only

You can start dbverify command line with different parameter,

so you can check the file from block x to block y.

Through this procedure you can determine bad blocks,

or set the TS in Backup Mode and copy the file with ocopy

(this tool was available on Oracle 8/9 i never used this with Oracle 10g).

If the error occour, i think the datafile ist corrupt

Or try a fsck on OS

regards

Uwe

[uwe_theisinger]

Hello,

backing up an tablespace/datafile copies only the blocks to another location.

The backup utility dont know any Oracle internals like Index PCL2~0.

So use dbverify to check if the OS can read the complete file and then export the Tablespace like Alans proposal.

regards

Uwe Theisinger

[Former Member]

Hello Uwe,

I tried to do a dbverify also but this also failes at the same point.

The strange thing is I don't get an error or something it just hangs.

It looks like it is running but it is not.

Here an extract of my dbverify log:

<i>BR0395I Verifying K:\ORACLE\PRD\SAPDATA2\BTABD_4\BTABD.DATA4 using DBVERIFY...

BR0396I Verification of K:\ORACLE\PRD\SAPDATA2\BTABD_4\BTABD.DATA4 using DBVERIFY successful

BR0280I BRBACKUP time stamp: 2007-11-23 16.47.40

BR0063I 4 of 46 files processed - 39820.031 MB of 148176.359 MB done

BR0204I Percentage done: 26.87%, estimated end time: 21:23

BR0001I *************_____________________________________

BR0395I Verifying K:\ORACLE\PRD\SAPDATA2\BTABD_5\BTABD.DATA5 using DBVERIFY...

BR0396I Verification of K:\ORACLE\PRD\SAPDATA2\BTABD_5\BTABD.DATA5 using DBVERIFY successful

BR0280I BRBACKUP time stamp: 2007-11-23 16.57.15

BR0063I 5 of 46 files processed - 49760.039 MB of 148176.359 MB done

BR0204I Percentage done: 33.58%, estimated end time: 20:36

BR0001I *****************_________________________________

BR0395I Verifying K:\ORACLE\PRD\SAPDATA2\BTABD_6\BTABD.DATA6 using DBVERIFY...

BR0396I Verification of K:\ORACLE\PRD\SAPDATA2\BTABD_6\BTABD.DATA6 using DBVERIFY successful

BR0280I BRBACKUP time stamp: 2007-11-23 17.00.32

BR0063I 6 of 46 files processed - 53856.047 MB of 148176.359 MB done

BR0204I Percentage done: 36.35%, estimated end time: 20:20

BR0001I ******************________________________________

BR0395I Verifying K:\ORACLE\PRD\SAPDATA2\BTABD_7\BTABD.DATA7 using DBVERIFY...

BR0396I Verification of K:\ORACLE\PRD\SAPDATA2\BTABD_7\BTABD.DATA7 using DBVERIFY successful

BR0280I BRBACKUP time stamp: 2007-11-23 17.03.14

BR0063I 7 of 46 files processed - 57856.055 MB of 148176.359 MB done

BR0204I Percentage done: 39.05%, estimated end time: 20:05

BR0001I ********************______________________________

BR0395I Verifying K:\ORACLE\PRD\SAPDATA3\BTABI_1\BTABI.DATA1 using DBVERIFY...

BR0396I Verification of K:\ORACLE\PRD\SAPDATA3\BTABI_1\BTABI.DATA1 using DBVERIFY successful

BR0280I BRBACKUP time stamp: 2007-11-23 17.05.51

BR0063I 8 of 46 files processed - 60796.063 MB of 148176.359 MB done

BR0204I Percentage done: 41.03%, estimated end time: 19:57

BR0001I *********************_____________________________

BR0395I Verifying K:\ORACLE\PRD\SAPDATA3\BTABI_2\BTABI.DATA2 using DBVERIFY...</i>

[former_member246982]

Sounds like a Filesystem/storage issue then.

As Uwe said, backup software does not care about index name, it works on block level. So if you drop the index you think is problemmetic, and the backup works, it means that index used the corrupted blocks.

If that's the case (drop the index and backup works), try to create that index in another tablespace and then try the backup. If this test works. I suggest you do a database reorg as soon as possible and do a thorogh verify of your storage system.

[Former Member]

Issue solved.

It had something to do with a NIC that was deleted.

There were 2 NIC's in it in the past, we deleted one we didn't need and the system got unstable.

We have installed it back and now it is running fine.

I still find this strange behaviour of Windows but for now it solved my issue.

Kind regards,

Dave Arends

[Former Member]

So can you copy the datafile using just the Windows utilities?

Does that hang?

What do you see in the Windows event log viewer?

It would probably be easiest to eliminate 1 layer at a time to isolate the issue.

It's very suspicious when you say the whole OS eventually hangs.

[Former Member]

Thanks all for your replies.

I am a little further in my investigation.

It seems it is hanging on index PCL2~0.

I have dropped it and created it again but still the problem exists.

This index has 15616448 rows.

Is it too big maybe?

Thanks,

Dave

[former_member204746]

it just hangs... or do you get an error message?

how big is PSAPBTABI? a complete reorg of this tablespace might be a solution.