DB2 Proceeses Terminated as a result of SGEN, and DBIF_REPO_INTERNAL_ERROR

[former_member524429]

Hello Friends,

I am facing one unexpected Problem as a result of SGEN or running any loaded work on SAP SOLMAN with DB6 9.1 FP5

Brief Information about the SAP system and DB2:

SAP System : SAP Solution Manager 7.0, SPs 15

SAP System Kernel:

(old) 700_185

(current) 701_15 (recently i have upgraded kernel from 700 to 710 release, with purpose of this problem resolution )

Operating System: Windows Server 2003 Enterprise Edition x64 Edition

Database : DB2 v9.1.500.555

(Old) FP4

(Current) FP5

C:\>db2level
DB21085I  Instance "DB2SMP" uses "64" bits and DB2 code release "SQL09015" with

level identifier "01060107".
Informational tokens are "DB2 v9.1.500.555", "s080512", "WR21410", and Fix Pack

"5".
Product is installed at "J:\db2\db2smp\db2_software\" with DB2 Copy Name
"SAPDB2SMP".

How Problem came into picture:

-> In starting phase , as a part of service Preparation Activity , I have applied ST-A/PI - 01L_CRM570 in Solution manager.

-> After that I run SGEN, It had run for some time, and suddenly SAP system was stuck and ended as DB2 related all process were killed at OS level. Means DB2 was stopped accidently as a result of SGEN

-> then I started DB2 using db2start command. It started normally.

-> I have seen that, last SGEN was still in ACTIVE state and was ended with DBIF_REPO_INTERNAL_ERROR & GEN_SOURCE_FORMAT Run-time Errors.

-> Then as a doubt, i patched DB2 and raised its FP level from FP4 to FP5, after following some SAP NOTES.

->Then again I started SGEN, but,it also ended with the same symptoms.

-> Then as a doubt, i patched SAP Kernel and raised its release level from 700(185) to 701(15), as normal kernel upgradation activity on windows system.

-> Again, i ran SGEN to get expected output, but still its ended with the same Symptoms. The Database was stopped accidently as a result of SGEN.

I am confused , why this is happening ?

Weather its pointing to Database Client's Doubtful functionality or not?

or due to Database inconsistency, this is happening?

My Regular Backup Activity is running successfully, without error and within its time.

May i know, where i am missing something?

Regards

Bhavik G. Shroff

[former_member524429]

Database Tablespace is corrupted. Run db2dart utility to get analysis report of your DB2 Database.

[former_member906139]

Hello,

You have mentioned as your SAP system is Solution Manager 7.0 SP15.

Hence your Kernel should be of release 700. You will need kernel of version 701 if you are moving at SP18 which will be EHP1 for Solution Manager.

I think this is main reason of why you are getting error. Please revert to Kernel 700 patch 185 or 189.

Regards,

Abhay

[former_member524429]

Hi,

Yes, you may be right. But, Previously my SAP SOLMAN 7.0 was running with 700 Kernel (185). As a doubt , i had upgrade its release to 701, recently for testing purpose. I can rollback it to the previous level, but I don't thinks so that It will not give the same symptoms after that.

I have doubt on FP5 of DB2. Something is pointing to incompatibility between the SAP Kernel(700_185) and FP5 of DB2 UDB. Because, after applying the FP5 to DB2 UDB, this problem came into picture.

May be i will have to rollback FP level.

What you say ?

Regards,

Bhavik G. Shroff

[former_member524429]

Hi,

I have rollback the SAP Kernel Release & Patch Level to 700 & 185 respectively. I also tried with Kernel 700 with 201 (which is latest for 700 Release), but same dumps are coming while execution of SGEN.

From Beginning I have faced the following Run time Errors and lost the Database Connection in mid of Processing.

04.07.2009	17:45:32	DBIF_REPO_INTERNAL_ERROR
04.07.2009	17:39:04	RAISE_EXCEPTION
04.07.2009	17:34:07	RAISE_EXCEPTION
04.07.2009	17:29:17	RAISE_EXCEPTION
04.07.2009	17:18:57	DBIF_REPO_SQL_ERROR
04.07.2009	17:13:07	GEN_SOURCE_FORMAT
04.07.2009	17:12:56	GEN_SOURCE_FORMAT
04.07.2009	17:12:48	GEN_SOURCE_FORMAT
04.07.2009	17:12:47	GEN_SOURCE_FORMAT
04.07.2009	17:12:46	GEN_SOURCE_FORMAT
04.07.2009	17:12:44	GEN_BRANCHOFFSET_LIMIT_REACHED
04.07.2009	17:10:46	DBIF_REPO_INTERNAL_ERROR

The Brief Short Text for all DBIF_REPO_INTERNAL_ERROR are showing same symptoms.

Internal error when accessing program "/1CRMGC/LDNL_CUST_CONDU03

By executing SDBI_REPO_CHECK, for /1CRMGC/LDNL_CUST_CONDU03, it completed with "1 errors detected" as output.

Any idea, how to solve this situation ?

Regards,

Bhavik G. Shroff

[former_member524429]

Hi,

Run-time Error

GEN_BRANCHOFFSET_LIMIT_REACHED

Short Text

Jump distance is too large and cannot be generated.

A jump distance is too large and cannot be generated.
A control structure or a routine with "CHECK" or "EXIT" contains
too many ABAP statements.

Error in the ABAP Application Program

Run-time Error

GEN_SOURCE_FORMAT

Short Text

Error when reading program "CL_CRM_IC_SOLSTEXT============CP " line 1.
Error when reading program "CL_CRM_IC_SOLSTEXT_CN00=======CCDEF " line 2.
Error when reading program "CL_CRM_IC_SOLSTEXT============CU " line 7.

Processing had to be terminated because an internal error
occurred when generating the ABAP/4 program "CL_CRM_IC_SOLSTEXT============CP" .......

Run-time Error

DBIF_REPO_SQL_ERROR

Short Text

SQL error "-99999" occurred when accessing program "MDM_GENERIC_EXTRACTOR "
Database error text........: "CLI0106E Connection is closed. SQLSTATE=08003"

[former_member524429]

Hi,

System environment
    SAP-Release 700

    Operating system..... "Windows NT"
    Release.............. "5.2"
    Hardware type........ "4x AMD64 Level"
    Character length.... 16 Bits
    Pointer length....... 64 Bits
    Work process number.. 6
    Shortdump setting.... "full"

    Database type..... "DB6"
    Database name..... "SMP"
    Database user ID.. "SAPSMP"

    Char.set.... "C"

    SAP kernel....... 700
    created (date)... "Nov 18 2008 22:53:36"
    create on........ "NT 5.2 3790 Service Pack 1 x86 MS VC++ 14.00"
    Database version. "DB6_81 "

    Patch level. 185
    Patch text.. " "

    Database............. "DB6 08.02.*, DB6 09.*"
    SAP database version. 700
    Operating system..... "Windows NT 5.0, Windows NT 5.1, Windows NT 5.2, Windows
     NT 6.0"

    Memory consumption
    Roll.... 16192
    EM...... 12569520
    Heap.... 0
    Page.... 0
    MM Used. 4193664
    MM Free. 4183168

Run time Error

RAISE_EXCEPTION

Short Text

Exception condition "DESTINATION_NOT_OPEN" raised.
The current ABAP/4 program encountered an unexpected
situation.

[former_member524429]

Hi,

Suspicious db2diag.log entry

04.07.2009	17:24:15	6.936	db2syscs.exe	common communication	MESSAGE  :Detected client termination
04.07.2009	17:24:15	6.936	db2syscs.exe	common communication	MESSAGE  :sqlcctest RC
04.07.2009	17:24:15	6.936	db2syscs.exe	base sys utilities	MESSAGE  :sqleqint2 iRC
04.07.2009	17:24:15	6.936	db2syscs.exe	catalog services	
04.07.2009	17:24:15	6.936	db2syscs.exe	catalog services	
04.07.2009	17:24:15	6.936	db2syscs.exe	catalog services	

[former_member524429]

Hi,

Right now, I have roll-back kernel level to 700_185 , and have updated DBSL Patch Level to 212. But, still the problem is not resolved and SGEN is terminated due to Error Code DBIF_REPO_INTERNAL_ERROR, which is telling "Internal error when accessing program /1CRMGC/LDNL_CUST_CONDU03".

This is some portion of Developer Trace of the WP_0, where the Dump is reported.

A
A  ABAP Program SDBI_REPO_CHECK                         .
A  Source SDBI_REPO_CHECK                          Line 85.
A  Error Code DBIF_REPO_INTERNAL_ERROR.
A  Module  $Id: //bas/700_REL/src/krn/runt/abdbif4.c#3 $ SAP.
A  Function readreport Line 957.
A  ** RABAX: level LEV_RX_STDERR completed.
A  ** RABAX: level LEV_RX_RFC_ERROR entered.
A  ** RABAX: level LEV_RX_RFC_ERROR completed.
A  ** RABAX: level LEV_RX_RFC_CLOSE entered.
A  ** RABAX: level LEV_RX_RFC_CLOSE completed.
A  ** RABAX: level LEV_RX_IMC_ERROR entered.
A  ** RABAX: level LEV_RX_IMC_ERROR completed.
A  ** RABAX: level LEV_RX_DATASET_CLOSE entered.
A  ** RABAX: level LEV_RX_DATASET_CLOSE completed.
A  ** RABAX: level LEV_RX_RESET_SHMLOCKS entered.
A  ** RABAX: level LEV_RX_RESET_SHMLOCKS completed.
A  ** RABAX: level LEV_RX_ERROR_SAVE entered.
A  ** RABAX: level LEV_RX_ERROR_SAVE completed.
A  ** RABAX: level LEV_RX_ERROR_TPDA entered.
A  ** RABAX: level LEV_RX_ERROR_TPDA completed.
A  ** RABAX: level LEV_RX_PXA_RELEASE_RUDI entered.
A  ** RABAX: level LEV_RX_PXA_RELEASE_RUDI completed.
A  ** RABAX: level LEV_RX_LIVE_CACHE_CLEANUP entered.
A  ** RABAX: level LEV_RX_LIVE_CACHE_CLEANUP completed.
A  ** RABAX: level LEV_RX_END entered.
A  ** RABAX: level LEV_RX_END completed.
A  ** RABAX: end no http/smtp
A  ** RABAX: end RX_GOTO_RABAX
A  Internal error when accessing program /1CRMGC/LDNL_CUST_CONDU03
A
M

So, Anyone have any suggesions for this Issue ?

I am waiting for your precious replies.

Regards,

Bhavik G. Shroff

[former_member524429]

Hi,

This is some description of DBIF_REPO_INTERNAL_ERROR Runtime Error.

Runtime Errors         DBIF_REPO_INTERNAL_ERROR
Date and Time          08.07.2009 14:18:55



 Short text
     Internal error when accessing program "/1CRMGC/LDNL_CUST_CONDU03 ".



 What happened?
     The current ABAP/4 program terminated due to
     an internal error in the database interface.



 What can you do?
     Note which actions and input led to the error.

     For further help in handling the problem, contact your SAP administrator
     .

     You can use the ABAP dump analysis transaction ST22 to view and manage
     termination messages, in particular for long term reference.




 Error analysis
     An unspecified internal error occurred in the SAP-internal
     database interface.
     For further analysis the SAP system log should be examined
     (transaction SM21).
     Check the entries in the developer trace of the relevant work process
     (transaction ST11).
     Attempted to access component "SRC " of program "/1CRMGC/LDNL_CUST_CONDU03 ".

Do i need to do any patch work at Database Level or at SAP Level ?

Regards

Bhavik G. Shroff

[former_member524429]

Hello Friends,

May I have any suggestions to resolve this matter ?

Regards,

Bhavik G. Shroff

[Frank-Martin]

Hi Bhavik,

did you take a look at the corresponding entries in the syslog and in the dev_w* trace file of the corresponding work process?

Regards

Frank

[former_member524429]

Hi Frank,

Thank you for your response.

In syslog, not a single error was reported for that time , when that Dump came, while SGEN was running.

Some Database Communication Errors are reported in the Work-Process's Trace. i am giving some snaps which are as followed.

X Wed Jul 01 15:45:46 2009
X  *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c       2214]
X
X Wed Jul 01 15:47:46 2009
X  *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c       2214]
X
X Wed Jul 01 15:51:46 2009
X  *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c       2214]
A
A Wed Jul 01 15:51:51 2009
A  *GENER* request remote generation: SAPLSBUF.
A  *GENER* request remote generation: SAPLSPBT.
A
A Wed Jul 01 15:51:52 2009
A  *GENER* request remote generation: %_CPBTMO.
A  *GENER* request remote generation: CL_ABAP_RANDOM================CP.
S  handle memory type is RSTSPROMMM
M
M Wed Jul 01 15:55:32 2009
M  *** WARNING => ThHdlReconnect: no reconnect in hold mode
M  *** WARNING => ThHdlReconnect: no reconnect in hold mode
C
C Wed Jul 01 15:55:34 2009
C  *** ERROR in prepareStatement[dbdb6.c, 12666] CON = 0 (BEGIN)
C  &+     DbSlBegReadDB6( SQLExtendedPrepare 😞 [IBM][CLI Driver] SQL30081N  A communication error has been detected.  Commu
C  &+     nication protocol being used: "TCP/IP".  Communication API being used: "SOCKETS".  Location where the error was de
C  &+     tected: "192.168.#.#".  Communication function detecting the error: "recv".  Protocol specific error code(s): "*"
C  &+     , "*", "0".  SQLSTATE=08001
C  &+
C  &+     SELECT * FROM "TCPSBUILD" WHERE "SEGID" = ? ORDER BY "SEGID" , "SPART"  WITH UR
C  &+       cursor type=NO_HOLD, isolation=UR, cc_release=YES, optlevel=5, degree=1, buffer_lobs=YES, op_type=18, reopt=0
C  &+       ABAP location info: 'RSPARAGENER8', 2016
C  &+
C  &+     ABAP location info 'RSPARAGENER8', 2016
C  &+
C  *** ERROR in prepareStatement[dbdb6.c, 12666] (END)
B  ***LOG BYM=> severe DB error -30081    ; work process in reconnect status [dbsh#1 @ 1123] [dbsh    1123 ]
B  ***LOG BY4=> sql error -30081 performing SEL on table TCPSBUILD  [dbtrtab#2 @ 3914] [dbtrtab 3914 ]
B  ***LOG BY0=> SQL30081N  A communication error has been detected.  Communication protocol being used: "TCP/IP".  Communication API
A  *** ERROR => rscpConvert rc=4 .
from=0100 adr=0000000143029CA0 len=85. to=4103 adr=0000000143029CA0 len=1024
ignore=1 replchar=U+0023. rscpRC=8 op=Activate slot=1 [abcnvrscp.c  793]

[Frank-Martin]

Hi,

SQL30081N points to a network problem. From the information I see I am not convinced that this is related to your initial problem. It may be better to investigate this through a OSS message.

Regards

Frank