Skip to Content
avatar image
Former Member

ORA-07445 on Xen RHEL5.4 guest on Itanium Montecito (BL870c/BL860c) host

I am wandering if anyone had any experience with similar issue - it most likely related to something about Oracle on RHEL5.4 Xen host on an Itanium (Montecito core, to be specific, as this proved to make important difference - HP BL870c and BL860c servers). I didn't test this on same physical machine (Itanium/RHEL5.4). I am running Oracle 10.2.0.4 with CPUJul2009 (patch 19) and all the latest OS RHEL5.4 updates, and ECC6.0 on top. It works very good, but Oracle is getting ORA-07445 (evabtr) from time to time, which causes ORA-03113 (disconnect session) on the ABAP side (or whatever is connected to db, eg. imp or impdp, too) - something like SGEN can cause it (anything making load long enough). I did everything by the book, with all SAP recommendations and I was hopping I am was doing something wrong. Anyone ?

Zoran.

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • avatar image
    Former Member
    Nov 06, 2009 at 02:02 AM

    Hi,

    Please paste the Oracle alert log and relevant trace .trc file generated by the ORA-07445.

    Best regards,

    Warren

    Add comment
    10|10000 characters needed characters exceeded

    • Former Member

      Ok, you asked 😊 for it (IASDB database instance was just a DBCA general template created database, and this alert log is it's very first bringing up after creation, no special load applied even):

      alert_IASDB.log:

      Thu Oct  1 02:20:05 2009
      Starting ORACLE instance (normal)
      LICENSE_MAX_SESSION = 0
      LICENSE_SESSIONS_WARNING = 0
      Picked latch-free SCN scheme 3
      Autotune of undo retention is turned on. 
      IMODE=BR
      ILAT =18
      LICENSE_MAX_USERS = 0
      SYS auditing is disabled
      ksdpec: called for event 13740 prior to event group initialization
      Starting up ORACLE RDBMS Version: 10.2.0.4.0.
      System parameters with non-default values:
        processes                = 150
        sga_target               = 692060160
        control_files            = /oracle/IASDB/IASDB/control01.ctl, /oracle/IASDB/IASDB/control02.ctl, /oracle/IASDB/IASDB/control03.ctl
        db_block_size            = 8192
        compatible               = 10.2.0.3.0
        log_archive_dest_1       = LOCATION=/oracle/IASDB/arch
        log_archive_format       = %t_%s_%r.dbf
        db_file_multiblock_read_count= 16
        db_recovery_file_dest    = /oracle/flash_recovery_area
        db_recovery_file_dest_size= 2147483648
        undo_management          = AUTO
        undo_tablespace          = UNDOTBS1
        remote_login_passwordfile= EXCLUSIVE
        db_domain                = 
        dispatchers              = (PROTOCOL=TCP) (SERVICE=IASDBXDB)
        local_listener           = LISTENER_IASDB
        job_queue_processes      = 10
        background_dump_dest     = /oracle/admin/IASDB/bdump
        user_dump_dest           = /oracle/admin/IASDB/udump
        core_dump_dest           = /oracle/admin/IASDB/cdump
        audit_file_dest          = /oracle/admin/IASDB/adump
        db_name                  = IASDB
        open_cursors             = 300
        pga_aggregate_target     = 229638144
      PMON started with pid=2, OS id=7737
      PSP0 started with pid=3, OS id=7739
      MMAN started with pid=4, OS id=7741
      DBW0 started with pid=5, OS id=7743
      LGWR started with pid=6, OS id=7745
      CKPT started with pid=7, OS id=7751
      SMON started with pid=8, OS id=7783
      RECO started with pid=9, OS id=7807
      CJQ0 started with pid=10, OS id=7809
      MMON started with pid=11, OS id=7811
      Thu Oct  1 02:20:06 2009
      starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
      MMNL started with pid=12, OS id=7813
      Thu Oct  1 02:20:06 2009
      starting up 1 shared server(s) ...
      Thu Oct  1 02:20:06 2009
      ALTER DATABASE   MOUNT
      Thu Oct  1 02:20:10 2009
      Setting recovery target incarnation to 2
      Thu Oct  1 02:20:10 2009
      Successful mount of redo thread 1, with mount id 362979286
      Thu Oct  1 02:20:10 2009
      Database mounted in Exclusive Mode
      Completed: ALTER DATABASE   MOUNT
      Thu Oct  1 02:20:10 2009
      ALTER DATABASE OPEN
      Thu Oct  1 02:20:10 2009
      LGWR: STARTING ARCH PROCESSES
      ARC0 started with pid=16, OS id=7823
      Thu Oct  1 02:20:10 2009
      ARC0: Archival started
      ARC1: Archival started
      LGWR: STARTING ARCH PROCESSES COMPLETE
      ARC1 started with pid=17, OS id=7825
      Thu Oct  1 02:20:10 2009
      Thread 1 opened at log sequence 1
        Current log# 1 seq# 1 mem# 0: /oracle/IASDB/IASDB/redo01.log
      Successful open of redo thread 1
      Thu Oct  1 02:20:10 2009
      MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
      Thu Oct  1 02:20:10 2009
      ARC1: Becoming the 'no FAL' ARCH
      ARC1: Becoming the 'no SRL' ARCH
      Thu Oct  1 02:20:10 2009
      ARC0: Becoming the heartbeat ARCH
      Thu Oct  1 02:20:10 2009
      SMON: enabling cache recovery
      Thu Oct  1 02:20:11 2009
      Successfully onlined Undo Tablespace 1.
      Thu Oct  1 02:20:11 2009
      SMON: enabling tx recovery
      Thu Oct  1 02:20:11 2009
      Database Characterset is WE8ISO8859P1
      Opening with internal Resource Manager plan
      where NUMA PG = 1, CPUs = 6
      replication_dependency_tracking turned off (no async multimaster replication found)
      Starting background process QMNC
      QMNC started with pid=18, OS id=7827
      Thu Oct  1 02:20:12 2009
      Errors in file /oracle/admin/IASDB/udump/iasdb_ora_7821.trc:
      ORA-00604: error occurred at recursive SQL level 1
      ORA-12663: Services required by client not available on the server
      ORA-36961: Oracle OLAP is not available.
      ORA-06512: at "SYS.OLAPIHISTORYRETENTION", line 1
      ORA-06512: at line 15
      Thu Oct  1 02:20:12 2009
      db_recovery_file_dest_size of 2048 MB is 0.00% used. This is a
      user-specified limit on the amount of space that will be used by this
      database for recovery-related files, and does not reflect the amount of
      space available in the underlying filesystem or ASM diskgroup.
      Thu Oct  1 02:20:12 2009
      Completed: ALTER DATABASE OPEN
      Thu Oct  1 03:20:13 2009
      Errors in file /oracle/admin/IASDB/bdump/iasdb_m000_27853.trc:
      ORA-07445: exception encountered: core dump [evabtr()+496] [SIGILL] [unknown code] [0x40000000081804F0] [] []
      Thu Oct  1 13:00:56 2009
      Thread 1 advanced to log sequence 2 (LGWR switch)
        Current log# 2 seq# 2 mem# 0: /oracle/IASDB/IASDB/redo02.log
      

      ... and the iasdb_m00027853.trc:

      /oracle/admin/IASDB/bdump/iasdb_m000_27853.trc
      Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
      With the Partitioning, Data Mining and Real Application Testing options
      ORACLE_HOME = /oracle/ERC/102_64
      System name:	Linux
      Node name:	sap-test2
      Release:	2.6.18-164.el5xen
      Version:	#1 SMP Tue Aug 18 16:14:51 EDT 2009
      Machine:	ia64
      Instance name: IASDB
      Redo thread mounted by this instance: 1
      Oracle process number: 19
      Unix process pid: 27853, image: oracle@sap-test2 (m000)
      
      *** ACTION NAME:(Auto-DBFUS Action) 2009-10-01 03:20:12.840
      *** MODULE NAME:(MMON_SLAVE) 2009-10-01 03:20:12.840
      *** SERVICE NAME:(SYS$BACKGROUND) 2009-10-01 03:20:12.840
      *** SESSION ID:(136.58) 2009-10-01 03:20:12.840
      Registers:
        r0: 0x0000000000000000   r1: 0x600000000018f980   r2: 0x0000000000000001
        r3: 0x2000000002075e08   r4: 0x0000000000000000   r5: 0x0000000000000000
        r6: 0x0000000000000000   r7: 0x0000000000000000   r8: 0x60000fffffe1d922
        r9: 0x20000000022ab678  r10: 0x2000000002138508  r11: 0x0000000000000060
       r12: 0x60000fffffe1d910  r13: 0x2000000001a49f90  r14: 0x000000000000000a
       r15: 0x2000000002138508  r16: 0x0000000000000030  r17: 0x20000000021a0000
       r18: 0x200000000219fff8  r19: 0x20000000022ab638  r20: 0xffffffffffffffff
       r21: 0x0000000000000011  r22: 0x0000000000000020  r23: 0x000004002439dd70
       r24: 0x0000000000000020  r25: 0x000004002439dd74  r26: 0x0000000000000020
       r27: 0x2000000002075e2e  r28: 0x2000000002075e30  r29: 0x0000000000000000
       r30: 0x0000000000000002  r31: 0x0000000000000018   b0: 0x4000000008aa0b40
        b1: 0x0000000000000000   b2: 0x0000000000000000   b3: 0x0000000000000000
        b4: 0x0000000000000000   b5: 0x0000000000000000   b6: 0x200000000008dc00
        b7: 0x4000000008180300   pr: 0x0000000000656003   ip: 0x40000000081804f0
       cfm: 0x000000000000070f  rsc: 0x000000000000000f  bsp: 0x600007ffffe4dbd0
      rnat: 0x0000000000000000  ccv: 0x2000000000000013 unat: 0x0000000000000000
      fpsr: 0x0009804c8a76233f  bof: 0x600007ffffe4db58
       r32: 0x2000000002138470  r33: 0x200000000210ea50  r34: 0x60000fffffe1e2ec
       r35: 0x0000000000000000  r36: 0x20000000022aba60  r37: 0x0000000000000000
       r38: 0xc000000000001dc3  r39: 0x4000000008aa08d0  r40: 0x600000000018f980
       r41: 0x60000fffffe1d900  r42: 0x20000000022aba98  r43: 0x0000000000004000
       r44: 0x200000000210ea54  r45: 0x60000000001fdbb0  r46: 0x2000000002138470
       r47: 0x2000000002138478  r48: 0x0000000000000000  r49: 0x0000000000000000
      Exception signal: 4 (SIGILL), code: 2 (unknown code), PC: [0x40000000081804f0, evabtr()+496]
      *** 2009-10-01 03:20:13.731
      ksedmp: internal or fatal error
      ORA-07445: exception encountered: core dump [evabtr()+496] [SIGILL] [unknown code] [0x40000000081804F0] [] []
      Current SQL statement for this session:
      select name,intcol#,segcol#,type#,length,nvl(precision#,0),decode(type#,2,nvl(scale,-127/*MAXSB1MINAL*/),178,scale,179,scale,180,scale,181,scale,182,scale,183,scale,231,scale,0),null$,fixedstorage,nvl(deflength,0),default$,rowid,col#,property, nvl(charsetid,0),nvl(charsetform,0),spare1,spare2,nvl(spare3,0) from col$ where obj#=:1 order by intcol#
      ----- PL/SQL Call Stack -----
        object      line  object
        handle    number  name
      0x4002343f050        45  procedure SYS.DBMS_FEATURE_XDB
      0x4002343f308         1  anonymous block
      0x40028c45f58      1204  package body SYS.DBMS_SYS_SQL
      0x40028c4b730       323  package body SYS.DBMS_SQL
      0x400234d5ce0       312  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
      0x400234d5ce0       518  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
      0x400234d5ce0       690  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
      0x400234d5ce0       787  package body SYS.DBMS_FEATURE_USAGE_INTERNAL
      0x400234d6680         1  anonymous block
      ----- Call Stack Trace -----
      calling              call     entry                argument values in hex      
      location             type     point                (? means dubious value)     
      -------------------- -------- -------------------- ----------------------------
      ksedst()+64          ????     ksedst1()            000000000 ?
                                                         2000000001F47040 ?
      ksedmp()+1344        ????     ksedst()             000000001 ?
                                                         C000000000000C1E ?
                                                         40000000003D0300 ?
                                                         000000000 ?
                                                         2000000001F47040 ?
                                                         C000000000000185 ?
      ssexhd()+1248        ????     ksedmp()             000000003 ?
                                                         2000000001F473E0 ?
                                                         C000000000000899 ?
                                                         40000000024CB560 ?
                                                         600000000018F980 ?
                                                         000000000 ? 000000000 ?
                                                         000000001 ?
      evabtr()+496         ????     0xA0000000000107E0   2000000002075E28 ?
      __PGOSF326_opifcr()  ????     evabtr()             2000000002138470 ?
      +2496                                              200000000210EA50 ?
                                                         60000FFFFFE1E2EC ?
                                                         000000000 ?
                                                         20000000022ABA60 ?
                                                         000000000 ?
                                                         C000000000001DC3 ?
                                                         4000000008AA08D0 ?
      qersoFetch()+800     ????     __PGOSF326_opifcr()  60000FFFFFE1ED90 ?
                                                         000000001 ?
                                                         C000000000000EA5 ?
                                                         40000000088750E0 ?
                                                         600000000018F980 ?
                                                         200000000210EA50 ?
                                                         200000000210EA80 ?
                                                         20000000021A0388 ?
      opifch2()+6752       ????     qersoFetch()         4002439BCE8 ?
                                                         2000000001E066C0 ?
                                                         60000FFFFFE1ED90 ?
                                                         000000001 ?
                                                         C000000000001B3E ?
                                                         4000000008AA5C60 ?
                                                         600000000018F980 ?
                                                         4002439BD00 ?
      ....