cancel
Showing results for 
Search instead for 
Did you mean: 

ORA-07445 on Xen RHEL5.4 guest on Itanium Montecito (BL870c/BL860c) host

Former Member
0 Kudos

I am wandering if anyone had any experience with similar issue - it most likely related to something about Oracle on RHEL5.4 Xen host on an Itanium (Montecito core, to be specific, as this proved to make important difference - HP BL870c and BL860c servers). I didn't test this on same physical machine (Itanium/RHEL5.4). I am running Oracle 10.2.0.4 with CPUJul2009 (patch 19) and all the latest OS RHEL5.4 updates, and ECC6.0 on top. It works very good, but Oracle is getting ORA-07445 (evabtr) from time to time, which causes ORA-03113 (disconnect session) on the ABAP side (or whatever is connected to db, eg. imp or impdp, too) - something like SGEN can cause it (anything making load long enough). I did everything by the book, with all SAP recommendations and I was hopping I am was doing something wrong. Anyone ?

Zoran.

Accepted Solutions (0)

Answers (1)

Answers (1)

Former Member
0 Kudos

Hi,

Please paste the Oracle alert log and relevant trace .trc file generated by the ORA-07445.

Best regards,

Warren

Former Member
0 Kudos

Ok, you asked for it (IASDB database instance was just a DBCA general template created database, and this alert log is it's very first bringing up after creation, no special load applied even):

alert_IASDB.log:


Thu Oct  1 02:20:05 2009
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on. 
IMODE=BR
ILAT =18
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.4.0.
System parameters with non-default values:
  processes                = 150
  sga_target               = 692060160
  control_files            = /oracle/IASDB/IASDB/control01.ctl, /oracle/IASDB/IASDB/control02.ctl, /oracle/IASDB/IASDB/control03.ctl
  db_block_size            = 8192
  compatible               = 10.2.0.3.0
  log_archive_dest_1       = LOCATION=/oracle/IASDB/arch
  log_archive_format       = %t_%s_%r.dbf
  db_file_multiblock_read_count= 16
  db_recovery_file_dest    = /oracle/flash_recovery_area
  db_recovery_file_dest_size= 2147483648
  undo_management          = AUTO
  undo_tablespace          = UNDOTBS1
  remote_login_passwordfile= EXCLUSIVE
  db_domain                = 
  dispatchers              = (PROTOCOL=TCP) (SERVICE=IASDBXDB)
  local_listener           = LISTENER_IASDB
  job_queue_processes      = 10
  background_dump_dest     = /oracle/admin/IASDB/bdump
  user_dump_dest           = /oracle/admin/IASDB/udump
  core_dump_dest           = /oracle/admin/IASDB/cdump
  audit_file_dest          = /oracle/admin/IASDB/adump
  db_name                  = IASDB
  open_cursors             = 300
  pga_aggregate_target     = 229638144
PMON started with pid=2, OS id=7737
PSP0 started with pid=3, OS id=7739
MMAN started with pid=4, OS id=7741
DBW0 started with pid=5, OS id=7743
LGWR started with pid=6, OS id=7745
CKPT started with pid=7, OS id=7751
SMON started with pid=8, OS id=7783
RECO started with pid=9, OS id=7807
CJQ0 started with pid=10, OS id=7809
MMON started with pid=11, OS id=7811
Thu Oct  1 02:20:06 2009
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=12, OS id=7813
Thu Oct  1 02:20:06 2009
starting up 1 shared server(s) ...
Thu Oct  1 02:20:06 2009
ALTER DATABASE   MOUNT
Thu Oct  1 02:20:10 2009
Setting recovery target incarnation to 2
Thu Oct  1 02:20:10 2009
Successful mount of redo thread 1, with mount id 362979286
Thu Oct  1 02:20:10 2009
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Thu Oct  1 02:20:10 2009
ALTER DATABASE OPEN
Thu Oct  1 02:20:10 2009
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=16, OS id=7823
Thu Oct  1 02:20:10 2009
ARC0: Archival started
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=17, OS id=7825
Thu Oct  1 02:20:10 2009
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: /oracle/IASDB/IASDB/redo01.log
Successful open of redo thread 1
Thu Oct  1 02:20:10 2009
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu Oct  1 02:20:10 2009
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
Thu Oct  1 02:20:10 2009
ARC0: Becoming the heartbeat ARCH
Thu Oct  1 02:20:10 2009
SMON: enabling cache recovery
Thu Oct  1 02:20:11 2009
Successfully onlined Undo Tablespace 1.
Thu Oct  1 02:20:11 2009
SMON: enabling tx recovery
Thu Oct  1 02:20:11 2009
Database Characterset is WE8ISO8859P1
Opening with internal Resource Manager plan
where NUMA PG = 1, CPUs = 6
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=18, OS id=7827
Thu Oct  1 02:20:12 2009
Errors in file /oracle/admin/IASDB/udump/iasdb_ora_7821.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-12663: Services required by client not available on the server
ORA-36961: Oracle OLAP is not available.
ORA-06512: at "SYS.OLAPIHISTORYRETENTION", line 1
ORA-06512: at line 15
Thu Oct  1 02:20:12 2009
db_recovery_file_dest_size of 2048 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Thu Oct  1 02:20:12 2009
Completed: ALTER DATABASE OPEN
Thu Oct  1 03:20:13 2009
Errors in file /oracle/admin/IASDB/bdump/iasdb_m000_27853.trc:
ORA-07445: exception encountered: core dump [evabtr()+496] [SIGILL] [unknown code] [0x40000000081804F0] [] []
Thu Oct  1 13:00:56 2009
Thread 1 advanced to log sequence 2 (LGWR switch)
  Current log# 2 seq# 2 mem# 0: /oracle/IASDB/IASDB/redo02.log

... and the iasdb_m00027853.trc:


/oracle/admin/IASDB/bdump/iasdb_m000_27853.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Data Mining and Real Application Testing options
ORACLE_HOME = /oracle/ERC/102_64
System name:	Linux
Node name:	sap-test2
Release:	2.6.18-164.el5xen
Version:	#1 SMP Tue Aug 18 16:14:51 EDT 2009
Machine:	ia64
Instance name: IASDB
Redo thread mounted by this instance: 1
Oracle process number: 19
Unix process pid: 27853, image: oracle@sap-test2 (m000)

*** ACTION NAME:(Auto-DBFUS Action) 2009-10-01 03:20:12.840
*** MODULE NAME:(MMON_SLAVE) 2009-10-01 03:20:12.840
*** SERVICE NAME:(SYS$BACKGROUND) 2009-10-01 03:20:12.840
*** SESSION ID:(136.58) 2009-10-01 03:20:12.840
Registers:
  r0: 0x0000000000000000   r1: 0x600000000018f980   r2: 0x0000000000000001
  r3: 0x2000000002075e08   r4: 0x0000000000000000   r5: 0x0000000000000000
  r6: 0x0000000000000000   r7: 0x0000000000000000   r8: 0x60000fffffe1d922
  r9: 0x20000000022ab678  r10: 0x2000000002138508  r11: 0x0000000000000060
 r12: 0x60000fffffe1d910  r13: 0x2000000001a49f90  r14: 0x000000000000000a
 r15: 0x2000000002138508  r16: 0x0000000000000030  r17: 0x20000000021a0000
 r18: 0x200000000219fff8  r19: 0x20000000022ab638  r20: 0xffffffffffffffff
 r21: 0x0000000000000011  r22: 0x0000000000000020  r23: 0x000004002439dd70
 r24: 0x0000000000000020  r25: 0x000004002439dd74  r26: 0x0000000000000020
 r27: 0x2000000002075e2e  r28: 0x2000000002075e30  r29: 0x0000000000000000
 r30: 0x0000000000000002  r31: 0x0000000000000018   b0: 0x4000000008aa0b40
  b1: 0x0000000000000000   b2: 0x0000000000000000   b3: 0x0000000000000000
  b4: 0x0000000000000000   b5: 0x0000000000000000   b6: 0x200000000008dc00
  b7: 0x4000000008180300   pr: 0x0000000000656003   ip: 0x40000000081804f0
 cfm: 0x000000000000070f  rsc: 0x000000000000000f  bsp: 0x600007ffffe4dbd0
rnat: 0x0000000000000000  ccv: 0x2000000000000013 unat: 0x0000000000000000
fpsr: 0x0009804c8a76233f  bof: 0x600007ffffe4db58
 r32: 0x2000000002138470  r33: 0x200000000210ea50  r34: 0x60000fffffe1e2ec
 r35: 0x0000000000000000  r36: 0x20000000022aba60  r37: 0x0000000000000000
 r38: 0xc000000000001dc3  r39: 0x4000000008aa08d0  r40: 0x600000000018f980
 r41: 0x60000fffffe1d900  r42: 0x20000000022aba98  r43: 0x0000000000004000
 r44: 0x200000000210ea54  r45: 0x60000000001fdbb0  r46: 0x2000000002138470
 r47: 0x2000000002138478  r48: 0x0000000000000000  r49: 0x0000000000000000
Exception signal: 4 (SIGILL), code: 2 (unknown code), PC: [0x40000000081804f0, evabtr()+496]
*** 2009-10-01 03:20:13.731
ksedmp: internal or fatal error
ORA-07445: exception encountered: core dump [evabtr()+496] [SIGILL] [unknown code] [0x40000000081804F0] [] []
Current SQL statement for this session:
select name,intcol#,segcol#,type#,length,nvl(precision#,0),decode(type#,2,nvl(scale,-127/*MAXSB1MINAL*/),178,scale,179,scale,180,scale,181,scale,182,scale,183,scale,231,scale,0),null$,fixedstorage,nvl(deflength,0),default$,rowid,col#,property, nvl(charsetid,0),nvl(charsetform,0),spare1,spare2,nvl(spare3,0) from col$ where obj#=:1 order by intcol#
----- PL/SQL Call Stack -----
  object      line  object
  handle    number  name
0x4002343f050        45  procedure SYS.DBMS_FEATURE_XDB
0x4002343f308         1  anonymous block
0x40028c45f58      1204  package body SYS.DBMS_SYS_SQL
0x40028c4b730       323  package body SYS.DBMS_SQL
0x400234d5ce0       312  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
0x400234d5ce0       518  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
0x400234d5ce0       690  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
0x400234d5ce0       787  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
0x400234d6680         1  anonymous block
----- Call Stack Trace -----
calling              call     entry                argument values in hex      
location             type     point                (? means dubious value)     
-------------------- -------- -------------------- ----------------------------
ksedst()+64          ????     ksedst1()            000000000 ?
                                                   2000000001F47040 ?
ksedmp()+1344        ????     ksedst()             000000001 ?
                                                   C000000000000C1E ?
                                                   40000000003D0300 ?
                                                   000000000 ?
                                                   2000000001F47040 ?
                                                   C000000000000185 ?
ssexhd()+1248        ????     ksedmp()             000000003 ?
                                                   2000000001F473E0 ?
                                                   C000000000000899 ?
                                                   40000000024CB560 ?
                                                   600000000018F980 ?
                                                   000000000 ? 000000000 ?
                                                   000000001 ?
evabtr()+496         ????     0xA0000000000107E0   2000000002075E28 ?
__PGOSF326_opifcr()  ????     evabtr()             2000000002138470 ?
+2496                                              200000000210EA50 ?
                                                   60000FFFFFE1E2EC ?
                                                   000000000 ?
                                                   20000000022ABA60 ?
                                                   000000000 ?
                                                   C000000000001DC3 ?
                                                   4000000008AA08D0 ?
qersoFetch()+800     ????     __PGOSF326_opifcr()  60000FFFFFE1ED90 ?
                                                   000000001 ?
                                                   C000000000000EA5 ?
                                                   40000000088750E0 ?
                                                   600000000018F980 ?
                                                   200000000210EA50 ?
                                                   200000000210EA80 ?
                                                   20000000021A0388 ?
opifch2()+6752       ????     qersoFetch()         4002439BCE8 ?
                                                   2000000001E066C0 ?
                                                   60000FFFFFE1ED90 ?
                                                   000000001 ?
                                                   C000000000001B3E ?
                                                   4000000008AA5C60 ?
                                                   600000000018F980 ?
                                                   4002439BD00 ?
....