Oracle 10.2.0.4 to 11.2.0.2 fails with ORA-03113

[Former Member]

This is Oracle upgrade from 10.2.0.2 to Oracle 11.2 (on Windows platform)

Since 10202 to 11 is not SAP supported, we upgrade from 10.2.0.4 first before upgrade to 11

10202 to 10204 upgrade finished fine. All post upgrade sql files are run without any issues.

During 10204 to 11202 upgrade, at 42% complete the Database Upgrade Assistant gives

ORA-03113: end-of-file on communication channel

Upgrade failed due to running the step "Upgrading Oracle Server"

The upgrade was repeated and it always gets the same error at the same spot.

Once the error occurs, if I try to rerun the upgrade script, it complains that objects already exist.

Looking at the \oracle\cfgtoollogs\dbua\<SID>\upgrade1 directory, I see the log file which shows at the end of the file:

GRANT INSERT ON sys.impdp_stats TO PUBLIC

2 /

Grant succeeded.

GRANT DELETE ON sys.impdp_stats TO PUBLIC

2 /

Grant succeeded.

Rem ===================================

Rem End User stats table format change

Rem ===================================

Rem==========================================================================

Rem Call script to upgrade type dictionary tables from 8.0 image to

Rem 8.1 image format. If they are already in 8.1 image format, the

Rem script will do nothing.

Rem==========================================================================

set serveroutput on

EXECUTE dbms_objects_utils.upgrade_dict_image;

BEGIN dbms_objects_utils.upgrade_dict_image; END;

*

ERROR at line 1:

ORA-03113: end-of-file on communication channel

ERROR:

So the problem is at running dbms_objects_utils.upgrade_dict_image procedure

"This procedure upgrades the type dictionary images from 8.0 to 8.1. The dictionary tables that could be in 8.0 are kottd$, kottb$, kottbx$, kotad$ and kotmd$, which existed in 8.0 or 8.1, when 8.0 compatibility was possible, kotadx$ was created in 9iR2 when we required 8.1 minimum compatibility. The source code is fully exposed for both package header and body."

/rdbms/admin/dbmsobj.sql

If I try to run that same procedure within SQLPlus (after the upgrade failed), I get the error message ORA-00955: name is already used by an existing object. This tells me that the upgrade script actually created the newer version of the dictionary images, but for some reason failed to complete. And so when I re-run the upgrade script, it complains.

We've searched google, SAP Notes, sdn, etc. but havent gotten anywhere. Help or hints would be greatly appreciated.

Thanks

[Former Member]

Hi,

If you are upgrading your database from any previous version of the database to Oracle Database 11g and the configuration includes ASM, then you must upgrade the cluster synchronization services daemon (CSS) to Oracle Database 11g. Otherwise, the database upgrade procedure will fail with the following error:

ERROR at line 1:

ORA-03113: end-of-file on communication channel

ERROR:

If your configuration does not include ASM, then you should shut down the CSS daemon and delete the CSS service from the system by running the localconfig batch file with the delete option. For example:

SYSTEM_DRIVE :\oracle\product\11.1.0\db_1\bin\localconfig delete

If you do not know whether your configuration includes ASM or not, then run the following command:

select count(*) from v$asm_client where status = 'CONNECTED';

If this returns one or more rows, then the database is actively using an ASM disk group.

Regards,

Venkata S Pagolu

[Former Member]

Thanks folks

We've looked at those info, and chased them up. But so far, no luck.

Now, we are looking to see if upgrading 10202 to 10205. And then see if 10.2.0.5 to 11.2 works better.

We are still working on it.... any help or hints are still very much appreciated.

[Former Member]

Hi Martin,

It seems that you hit the bug on Oracle 11.2. Did you check the documents, on Oracle Metalink, below;

1) ORA-00600 [kodpunp-nulltds], ORA-00600 [kokeeiix1], [600] When Upgrading To 11GR2

2) Problem : ORA-00600 [kodpunp-nulltds], ORA-00600 [kokeeiix1], [600] When Upgrading To 11GR2

Best regards,

Orkun Gedik

[Former Member]

Hi Martin,

Please find the document as attachment . Here mention all this step.

[http://www.scribd.com/doc/49274082/Oracle-11g-Upgrade]

Thanks,

Manas

[Former Member]

HI,

Check below details from oracle Metalink.

Curtesy of Oracle Metalink

A1) Errors connecting as SYSDBA / Internal OR on startup nomount

There is something fundamental wrong with the software / environment

if you cannot connect to Server Manager as a DBA user.

The steps here cover errors such as ORA-3113, ORA-12547: TNS:lost contact

or similar errors connecting to Oracle or starting the instance NOMOUNT.

Check the following items:

A1.1) If possible reboot the server disabling any automatic

startup of Oracle before you do so. This may seem drastic

but helps make sure you are working from a consistent

starting point.

A1.2) Check your environment points at the expected ORACLE_HOME

and ORACLE_SID and that TWO_TASK is not set (Unix) or

LOCAL is not set (NT registry).

Check the USER_DUMP_DEST and BACKGROUND_DUMP_DEST and default

trace directories under this environment for any user trace

files or alert log entries generated. These may help indicate

the cause of the problem.

Eg: ORA-600[SKGMINVALID] may indicate a problem with the

shared memory Unix parameters on Unix systems.

Try to show that any trace file / alert log entry you

find is truely related to the "CONNECT" command by re-issuing

the "connect" and checking for a new trace file / alert entry

at the time of the error.

A1.3) Unix only:

Some Unix platforms need LD_LIBRARY_PATH to be set

correctly to resolve any dynamically linked libraries.

As the user with the problem:

% script /tmp/ldd.out

% id

% cd $ORACLE_HOME/bin

% ldd oracle

% exit

If the 'ldd' command does not exist go to the next step below.

Check that all lines listed show a full library file. If there

are any 'not found' lines reported contact Oracle support

with the output of /tmp/ldd.out .

A1.4) Unix only:

Your 'oracle' executable may be corrupt. Relink it thus:

Log in as the 'oracle' user.

% script /tmp/relink.out

% cd $ORACLE_HOME/rdbms/lib

% mv $ORACLE_HOME/bin/oracle $ORACLE_HOME/bin/oracle.dd.mon.yy

% rm -f ./oracle

% make -f ins_rdbms.mk ioracle

% exit

Prior to Oracle7.3 the relink command was:

make -f oracle.mk ioracle

If this reports any errors Oracle support will need to see

the contents of the file /tmp/relink.out .

A1.5) Have you installed the Parallel Server Option ?

ORA-3113 can occur if you have installed the Parallel

Server Option but do NOT have a Distributed Lock Manager

installed or running correctly.

Unix:

If the Parallel Server Option was installed by accident

then it can be de-installed by relinking.

Eg:

Shut down any Oracle instances

% script /tmp/relink.out

% cd $ORACLE_HOME/rdbms/lib

1. 'oracle' should not exist so delete it if it present

% rm -f oracle

% make -f ins_rdbms.mk no_parropt ioracle

% exit

NB: Do NOT deinstall the Parallel Server Option if the

database is using Parallel Server unless both nodes

are shut down otherwise database corruption could occur.

A1.6) If the error is on STARTUP NOMOUNT:

Check the init.ora file used to start the database.

This provides the configuration details used

configure the instance. To help isolate the problem

it may be useful use a very basic init.ora file

when starting the instance. If this works then

parameters can be increased / introduced one at a

time to see if there is a problem with a particular

setting.

A1.7) Check for server side trace files which may give more

indication what the underlying problem is.

See section C for details on how to check

for server trace files.

A1.8) Ensure there is free disk space in:

a. Your USER_DUMP_DEST and BACKGROUND_DUMP_DEST locations

b. Your AUDIT destination (Unix)

The default is $ORACLE_HOME/rdbms/audit

c. Your Oracle Trace directory if Oracle Trace is enabled

See <Note:45482.1>

A2) Errors Mounting the database

Check all the items in A1 first.

If an error occurs when mounting the database there may be problems

with the control-files or data files, or with resources required to

open these files.

A2.1) The location of the control files are specified in the

init.ora file. Try mounting using each controlfile in

turn.

eg: "Shutdown abort",

edit the init.ora to refer to ONE of the controlfiles only,

"startup nomount",

"alter database mount"

Repeat for each controlfile to see if any controlfile works.

A2.2) It is possible to re-create the controlfiles if you know the

location of all datafiles and online logs, or to restore an old

backup controlfile. Always back up the current controlfiles before

restoring any backup copies or issuing a CREATE CONTROLFILE

command.

The steps for this are not documented here.

A2.3) Unix:

Some unix platforms have a 'truss' command (or 'tusc').

If available this can be used to help trace how far Oracle

gets before the error occurs.

Eg:

% truss -o /tmp/truss.out -f svrmgrl

Keep the file /tmp/truss.out safe - Oracle Support MAY need to see it.

Regards,

Venkata S Pagolu

Edited by: Venkata Pagolu on Mar 10, 2012 12:10 PM

[Former Member]

good ...I have the same issue /// my oraarch directory was full and i ahve solve the issue now.

[former_member188883]

Hi Martin,

I could find a thread where similar problem has been faced. Possible root cause seems to be some patch not being applied properly.

Hope this helps.

Regards,

Deepak Kori

[Former Member]

Hi Martin,

It seems that the Oracle process is dying because of a reason, during the upgrade process. You may find the reason, in the trace file, under "usertrace" directory.

By the way, what is the OS that you are running on? Secondly, did you downloaded the Oracle 11g from SAP marketplace?

Best regards,

Orkun Gedik