Background job failing - ECC 6.0, Solution Manager 7.0

[Sudip1]

Dear All,

Four days back, I have installed the Solution Manager 7.0 and ECC 6.0 on

HPUX IA64 hardware with database as Oracle 10.2.0.2.0 (Unicode).

In both server, latest kernel patch has been applied.

The Support package versions are (both server):

KERNEL (Release 700) 146

SAP_BASIS 0014

SAP_ABA 0014

PI_BASIS(2005_1_700) 0014

ST-PI 0005

SAP_BW 0016

While checking the the jobs from SM37, in both the server,

I am observing that following jobs are always failing with

same reason:

SAP_COLLECTOR_FOR_PERFMONITOR

=============================

Date Time Message text Message class Message no. Message type

========== ======== ====================================================================== ============= =========== ============

15.05.2008 06:25:12 Job started 00 516 S

15.05.2008 06:25:12 Step 001 started (program RSCOLL00, variant , user ID BGDUSER) 00 550 S

15.05.2008 06:25:18 Clean_Plan:Cleanup of DB13 Plannings DB6PM 000 S

15.05.2008 06:25:18 Clean_Plan:started by RSDBPREV on server gcbeccd DB6PM 000 S

15.05.2008 06:25:18 Clean_Plan:Cleaning up jobs of system RD3 DB6PM 000 S

15.05.2008 06:25:18 Clean_Plan:finished DB6PM 000 S

15.05.2008 06:25:18 Database system not supported S1 101 S

15.05.2008 06:25:36 ABAP/4 processor: DBIF_DSQL2_SQL_ERROR 00 671 A

15.05.2008 06:25:36 Job cancelled 00 518 A

SAP_REORG_UPDATERECORDS

=======================

Date Time Message text Message class Message no. Message type

========== ======== ===================================================================== ============= =========== ============

14.05.2008 00:31:01 Job started 00 516 S

14.05.2008 00:31:01 Step 001 started (program RSM13002, variant SAP&001, user ID BGDUSER) 00 550 S

14.05.2008 00:31:01 Reorganization of update date not allowed in batch 15 100 A

14.05.2008 00:31:02 Job cancelled 00 518 A

SAP_WP_CACHE_RELOAD_FULL

========================

Date Time Message text Message class Message no. Message type

========== ======== ============================================================================================ ============= =========== ============

15.05.2008 00:30:11 Job started 00 516 S

15.05.2008 00:30:11 Step 001 started (program RWP_RUNTIME_CACHE_RELOAD, variant SAP&RELOAD_ALL, user ID BGDUSER) 00 550 S

15.05.2008 00:30:11 No component system chosen URL_GEN_MSGS 031 E

15.05.2008 00:30:11 Job cancelled after system exception ERROR_MESSAGE 00 564 A

Please help to resolve this issue.

Thanks in advance.

Regards

Sudip Ghosh

[markus_doehr2]

15.05.2008 06:25:36 ABAP/4 processor: DBIF_DSQL2_SQL_ERROR

Please check ST22 and post the content of the "analysis" section of the dump.

14.05.2008 00:31:01 Reorganization of update date not allowed in batch

Yes - this is no more allowed - you can delete that job. It´s still in the default in SM36 although no more supported (since 6.10+).

SAP_WP_CACHE_RELOAD_FULL

Please check note 6083 - Standard jobs, reorganization jobs and search for the jobname.

Markus

[Sudip1]

Hi Markus,

Thanks for the reply.

Below is the dump:

======

======

Runtime Errors DBIF_DSQL2_SQL_ERROR

Date and Time 15.05.2008 06:25:36

-

---

-

---

Short text

An SQL error occurred when executing Native SQL.

-

---

-

---

What happened?

The error 3113 occurred in the current database connection "DEFAULT".

-

---

-

---

What can you do?

Note down which actions and inputs caused the error.

To process the problem further, contact you SAP system

administrator.

Using Transaction ST22 for ABAP Dump Analysis, you can look

at and manage termination messages, and you can also

keep them for a long time.

-

---

-

---

How to correct the error

Database error text........: "ORA-03113: end-of-file on communication channel"

Database error code........: 3113

Triggering SQL statement...: "FETCH NEXT "

Internal call code.........: "[DBDS/NEW DSQL]"

Please check the entries in the system log (Transaction SM21).

If the error occures in a non-modified SAP program, you may be able to

find an interim solution in an SAP Note.

If you have access to SAP Notes, carry out a search with the following

keywords:

"DBIF_DSQL2_SQL_ERROR" " "

"RSORAVSH" or "RSORAVSH"

"FILL_DBVSE"

If you cannot solve the problem yourself and want to send an error

notification to SAP, include the following information:

1. The description of the current problem (short dump)

To save the description, choose "System->List->Save->Local File

(Unconverted)".

2. Corresponding system log

Display the system log by calling transaction SM21.

Restrict the time interval to 10 minutes before and five minutes

after the short dump. Then choose "System->List->Save->Local File

(Unconverted)".

3. If the problem occurs in a problem of your own or a modified SAP

program: The source code of the program

In the editor, choose "Utilities->More

Utilities->Upload/Download->Download".

4. Details about the conditions under which the error occurred or which

actions and input led to the error.

-

---

-

---

System environment

SAP-Release 700

Application server... "gcbeccd"

Network address...... "10.10.4.158"

Operating system..... "HP-UX"

Release.............. "B.11.23"

Hardware type........ "ia64"

Character length.... 16 Bits

Pointer length....... 64 Bits

Work process number.. 15

Shortdump setting.... "full"

Database server... "gcbeccd"

Database type..... "ORACLE"

Database name..... "RD3"

Database user ID.. "SAPSR3"

Char.set.... "C"

SAP kernel....... 700

created (date)... "Apr 5 2008 00:55:24"

create on........ "HP-UX B.11.23 U ia64"

Database version. "OCI_102 (10.2.0.1.0) "

Patch level. 146

Patch text.. " "

Database............. "ORACLE 9.2.0.., ORACLE 10.1.0.., ORACLE 10.2.0.."

SAP database version. 700

Operating system..... "HP-UX B.11"

Memory consumption

Roll.... 1117216

EM...... 0

Heap.... 0

Page.... 32768

MM Used. 695792

MM Free. 400816

-

---

-

---

User and Transaction

Client.............. 000

User................ "BGDUSER"

Language key........ "E"

Transaction......... " "

Transactions ID..... "482B8A4C89F0442BE10000000A0A049E"

Program............. "RSORAVSH"

Screen.............. "SAPMSSY0 1000"

Screen line......... 6

-

---

-

---

Information on where terminated

Termination occurred in the ABAP program "RSORAVSH" - in "FILL_DBVSE".

The main program was "RSORAVSH ".

In the source code you have the termination point in line 48

of the (Include) program "RSORAVSH".

The program "RSORAVSH" was started as a background job.

Job Name....... "SAP_COLLECTOR_FOR_PERFMONITOR"

Job Initiator.. "BGDUSER"

Job Number..... 05251100

-

---

-

---

Source Code Extract

-

---

Line

SourceCde

-

---

18	INCLUDE STRUCTURE v_system_event_struc.
19	DATA: END OF v_system_event_tab .
20
21	check sy-dbsys(3) = 'ORA' .
22
23	Get the new snapshots
24
25	PERFORM fill_dbvse TABLES v_system_event_tab.
26
27
28	Get the old snapshots from database table MONI
29
30	startday = sy-datum - days_to_keep .
31
32
33	Save new data now (otherwise these data will be selected twice)
34
35	CONCATENATE 'SYSEVENT' sy-datum sy-uzeit INTO monikey .
36
37	EXPORT sy-datum
38	sy-uzeit
39	v_system_event_tab
40	TO DATABASE moni(db) ID monikey .
41	&----
42	*& Form FILL_DBVSE
43	&----
44	FORM fill_dbvse TABLES v_system_event_tab.
45	date = sy-datum .
46	time = sy-uzeit .
47	EXEC sql performing append_v_system_event .
>>>>>	select event,
49	total_waits,
50	total_timeouts,
51	time_waited * 10,
52	average_wait * 10
53	into :v_system_event_struc
54	from v$system_event
55	ENDEXEC .
56	ENDFORM. " FILL_DBVSE
57
58	&----
59	*& Form APPEND_V_SYSTEM_EVENT
60	&----
61	FORM append_v_system_event.
62	v_system_event_tab-date = date .
63	v_system_event_tab-time = time .
64
65	MOVE-CORRESPONDING v_system_event_struc TO v_system_event_tab.
66	APPEND v_system_event_tab .
67	ENDFORM. " APPEND_V_SYSTEM_EVENT

-

---

-

---

Contents of system fields

-

---

Name

Val.

-

---

SY-SUBRC	0
SY-INDEX	0
SY-TABIX	16
SY-DBCNT	0
SY-FDPOS	0
SY-LSIND	0
SY-PAGNO	0
SY-LINNO	1
SY-COLNO	1
SY-PFKEY
SY-UCOMM
SY-TITLE	Report for Collecting Data from V$SYSTEM_EVENT Hourly
SY-MSGTY
SY-MSGID
SY-MSGNO	000
SY-MSGV1
SY-MSGV2
SY-MSGV3
SY-MSGV4
SY-MODNO	0
SY-DATUM	20080515
SY-UZEIT	062520
SY-XPROG	RSDBRUNT
SY-XFORM	%_INIT_PBO_FIRST

-

---

-

---

Active Calls/Events

-

---

No. Ty. Program Include Line

Name

-

---

2 FORM RSORAVSH RSORAVSH 48

FILL_DBVSE

1 EVENT RSORAVSH RSORAVSH 25

START-OF-SELECTION

-

---

-

---

Chosen variables

-

---

Name

Val.

-

---

No. 2 Ty. FORM

Name FILL_DBVSE

-

---

SY-REPID
RSORAVSH
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
5545455422222222222222222222222222222222
23F2163800000000000000000000000000000000
V_SYSTEM_EVENT_STRUC
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
22222222222222222222222222222222222222222222222222222222222222220000000000000000
0000000000000000000000000000000000000000000000000000000000000000000C000C000C000C
%_SPACE
0
0
2
0
SY
###############################################################################T######## ####
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000004000000
000000000000000000000000000000000000000100000000000000000000000000000000000000000000000D000400
000001000000000000000000000000000000000F000000000000000000000000000000000000010500000005220000
000000000000000000000000010001000000000E00000000000000000000000000000000000006040000000800000C

-

---

No. 1 Ty. EVENT

Name START-OF-SELECTION

-

---

SY-LDBPG
SAPDB__S
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
5454455522222222222222222222222222222222
31042FF300000000000000000000000000000000
%_DUMMY$$
0000
0000
2222
0000
SY-DBSYS+0(6)
ORA
000
000
454
F21
SYST
###############################################################################T######## ####
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000004000000
000000000000000000000000000000000000000100000000000000000000000000000000000000000000000D000400
000001000000000000000000000000000000000F000000000000000000000000000000000000010500000005220000
000000000000000000000000010001000000000E00000000000000000000000000000000000006040000000800000C
V_SYSTEM_EVENT_TAB[]
Table[initial]
V_SYSTEM_EVENT_TAB
00000000000000 ################
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
3333333333333322222222222222222222222222222222222222222222222222222222222222220000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000000000000C000C000C000C
SY-DATUM
20080515
00000000
00000000
33333333
20080515
DAYS_TO_KEEP
30
0001
000E
STARTDAY
00000000
00000000
00000000
33333333
00000000
MONIKEY
0000000000000000000000
0000000000000000000000
2222222222222222222222
0000000000000000000000
SY-UZEIT
062520
000000
000000
333333
062520
SYST-REPID
RSORAVSH
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
5545455422222222222222222222222222222222
23F2163800000000000000000000000000000000
MONI-CLUSTR
0
00
00
MONI
##################################################################
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
2222222222222222222222220022000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

-

---

-

---

Internal notes

The termination was triggered in function "ExecuteCall"

of the SAP kernel, in line 1190 of the module

"//bas/700_REL/src/krn/runt/abexsql.c#2".

The internal operation just processed is "DSQL".

Internal mode was started at 20080515062520.

Internal call code.........: "[DBDS/NEW DSQL]"

-

---

-

---

Active Calls in SAP Kernel

-

---

Lines of C Stack in Kernel (Structure Differs on Each Platform)

-

---

(0) 0x4000000003b2b450 CTrcStack + 0x1b0 at dptstack.c:227 [dw.sapRD3_DVEBMGS00]

(1) 0x4000000004d2c470 Z16rabaxCStackSavev + 0x1d0 [dw.sapRD3_DVEBMGS00]

(2) 0x4000000004d32160 ab_rabax + 0x3570 [dw.sapRD3_DVEBMGS00]

(3) 0x40000000047b51e0 Z10abdbdserriPKtiS0_P9DS_HEADERS0_iPv + 0x470 [dw.sapRD3_DVEBMGS00]

(4) 0x4000000003d777f0 Z11ExecuteCallv + 0x11f0 [dw.sapRD3DVEBMGS00]

(5) 0x4000000003d7e7b0 Z8abjdsqlv + 0x1b0 [dw.sapRD3_DVEBMGS00]

(6) 0x400000000402f190 Z8abextriv + 0x440 [dw.sapRD3_DVEBMGS00]

(7) 0x4000000003f53bc0 Z9abxeventPKt + 0x3c0 at abrunt1.c:339 [dw.sapRD3_DVEBMGS00]

(8) 0x4000000003d9f210 Z8abtriggv + 0x110 at abrunt1.c:58 [dw.sapRD3_DVEBMGS00]

(9) 0x4000000003d9ebe0 ab_run + 0xc0 [dw.sapRD3_DVEBMGS00]

(10) 0x4000000001caa820 N_ab_run + 0x20 at dymainstp.c:4739 [dw.sapRD3_DVEBMGS00]

(11) 0x4000000001cb40f0 dynpmcal + 0x3f0 at dymainstp.c:2281 [dw.sapRD3_DVEBMGS00]

(12) 0x4000000001caf260 dynppai0 + 0xcb0 at dymainstp.c:1107 [dw.sapRD3_DVEBMGS00]

(13) 0x4000000001cb1ec0 dynprctl + 0x340 at dymainstp.c:358 [dw.sapRD3_DVEBMGS00]

(14) 0x4000000001c9dff0 dynpen00 + 0xac0 at dymain.c:1628 [dw.sapRD3_DVEBMGS00]

(15) 0x4000000001fea460 Thdynpen00 + 0x510 at thxxhead.c:4830 [dw.sapRD3_DVEBMGS00]

(16) 0x4000000001fb54e0 TskhLoop + 0x5520 at thxxhead.c:3945 [dw.sapRD3_DVEBMGS00]

(17) 0x4000000001faae40 ThStart + 0x460 at thxxhead.c:1164 [dw.sapRD3_DVEBMGS00]

(18) 0x4000000001569ec0 DpMain + 0x5f0 at dpxxdisp.c:1088 [dw.sapRD3_DVEBMGS00]

(19) 0x4000000002c10630 nlsui_main + 0x30 [dw.sapRD3_DVEBMGS00]

(20) 0x4000000002c105c0 main + 0x60 [dw.sapRD3_DVEBMGS00]

(21) 0xc00000000002be30 main_opd_entry + 0x50 [/usr/lib/hpux64/dld.so]

-

---

-

---

List of ABAP programs affected

-

---

Index

Typ

Program

Group

Date

Time

Size

Lang.

-

---

0	Prg	RSORAVSH	0	30.03.2005	10:21:59	19456	E
1	Prg	SAPMSSY0	1	17.12.2007	15:41:05	92160	E
2	Prg	SAPMSSYD	1	12.09.2006	11:33:31	21504	E
3	Prg	SAPFSYSCALLS	1	09.09.2004	14:18:32	8192	E
4	Prg	RSDBRUNT	0	17.12.2007	16:04:40	254976	E
5	Typ	RSSCR	0	30.03.2005	10:21:45	5120
6	Prg	RSDBSPBL	0	30.03.2005	10:21:58	72704	E
7	Prg	SAPDB__S	0	30.03.2005	10:22:01	19456	E
8	Prg	RSDBSPMC	0	12.09.2006	11:25:01	79872	E
9	Typ	DDSHDESCR	0	03.09.1997	03:05:16	4096
10	Typ	SPPARAMS	0	07.05.1997	13:10:38	2048
11	Prg	SAPLSABE	11	09.09.2004	14:18:36	13312	E
12	Prg	SAPLSECU	12	17.12.2007	15:31:45	87040	E
13	Typ	RSSUBINFO	0	14.10.1999	22:01:03	3072
14	Prg	SAPLSTUP	14	17.12.2007	15:29:58	74752	E
15	Prg	SAPLCNDP	15	11.09.2007	15:24:45	195584	E
16	Prg	SAPLSCNT	16	18.02.2005	14:16:06	30720	E
17	Prg	SAPSHDTV	16	05.01.2005	16:26:16	33792	E
18	Prg	SAPFGUICNTL	1	18.02.2005	14:15:08	24576	E
19	Prg	SAPLOLEA	19	11.09.2007	11:17:30	96256	E
20	Prg	SAPLSGUI	20	17.12.2007	15:41:05	84992	E
21	Prg	SAPLSTTM	21	05.07.2005	13:10:18	69632	E
22	Prg	SAPLSBDC	22	17.12.2007	15:30:54	44032	E
23	Prg	SAPLSFES	23	17.12.2007	16:04:40	260096	E
24	Prg	SAPLTHFB	24	17.12.2007	15:41:05	394240	E
25	Typ	WPINFO	0	26.02.1999	14:49:01	6144
26	Prg	SAPLURFC	26	17.12.2007	15:37:35	22528	E
27	Prg	SAPLSPLUGIN	27	09.09.2004	14:18:36	8192	E
28	Typ	SWCBCONT	0	15.11.2000	17:55:11	3072
29	Typ	OLE_VERBS	0	04.04.1995	16:02:20	2048
30	Typ	OLE_PA	0	04.04.1995	16:02:19	2048
31	Typ	SYST	0	09.09.2004	14:18:12	31744
32	Typ	MONI	0	31.03.2004	16:41:21	5120

-

---

-

---

Directory of Application Tables

-

---

Name Date Time Lngth

Val.

-

---

Program RSORAVSH

-

---

SYST 09.09.2004 14:18:12 00004612

\0\0\0\0\0\x0010\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

MONI . . : : 00002944

\0\0 \0\0\0\0\0\0\0\0\0\0\0\0\0\0

-

---

-

---

ABAP Control Blocks (CONT)

-

---

Index

Name

Fl

PAR0

PAR1

PAR2

PAR3

PAR4

PAR5

PAR6

Source Code

Line

-

---

71	PAR2	03	0000	0014	002B					RSORAVSH	37
73	PAR2	03	0000	0015	002D					RSORAVSH	37
75	PAR2	03	0000	0016	0026					RSORAVSH	37
77		00	0000							RSORAVSH	37
78	PERP	00	0002							RSORAVSH	44
79	PERP	1C	0000							RSORAVSH	44
80	PERP	04	0000							RSORAVSH	44
81	mvqk	10	0005	0044	04EC					RSORAVSH	45
83	mvqk	0C	0005	0054	052E					RSORAVSH	46
85	DSQL	00	0017							RSORAVSH	48
86	DSQL	01	0018							RSORAVSH	48
87	DSQL	04	0025							RSORAVSH	48
>>>>>	DSQL	16	0025							RSORAVSH	48
89	BRAN	05	0004							RSORAVSH	48
90	perf	00	000A							RSORAVSH	48
91	PERP	80	0000							RSORAVSH	48
92	BRAX	00	FFFC							RSORAVSH	48
93	DSQL	23	0000							RSORAVSH	48
94	ENDF	00	0000							RSORAVSH	56
95		00	0000							RSORAVSH	56

-

---

===========

===========

Please check and advise.

Also, I am trying to search for the note 6083 in market place, but not getting the same.

Please advise.

Thanks & Regards

Sudip

[markus_doehr2]

> | Database error text........: "ORA-03113: end-of-file on communication channel" |

> | Database error code........: 3113 |

This error means, that the Oracle shadow process died/crashed. Check your Oracle alert<SID>.log for errors. Also have a look at the syslog (SM21) when this error occurs.

Did you apply all 40+ interim patches of note 871096 - Oracle Database 10g: Patch sets/patches for 10.2.0

And sorry - the note number is 16083 (I missed the "1").

Markus

[Sudip1]

Hi Markus,

I have not installed the patches.........

Thanks for pointing my mistake.

Presently, my situation is:

01) I have installed the SAP ECC 6.0 (DVD 51033511_2, 51033505_7, 51033500) + Oracle 10.2.0 (DVD 51031671) + Oracle Patch 10.2.0.2.0( DVD 51031810) + Oracle Client (DVD 51033272)

02) I have configured STMS, scheduled background jobs, imported initial profiles / configured for our environment, created clients (T000 entries) and now performing client copies (sccl - running at present).

Can I now apply the patch? or I need to perform the installation from scratch?

Also I checked the opatch version - it is 10.2.0.2.0 and there is no interim patches applied in the oracle home.

Will I require to install new opatch version?

Please advise.

Thanks & Regards

Sudip

[markus_doehr2]

> I have not installed the patches.........

>

> Thanks for pointing my mistake.

No problem - no offense

> Can I now apply the patch? or I need to perform the installation from scratch?

Sure you can apply the patches, go through the note attentively, it describes a mechanism to download all the patches and apply them all at once instead of one-by-one (see note 1027012 - MOPatch - Install Multiple Oracle Patches in One Run)

I highly suggest checking that note often - maybe once in the week to see, if there are new patches released which will prevent corruption and/or other problems.

> Also I checked the opatch version - it is 10.2.0.2.0 and there is no interim patches applied in the oracle home.

>

> Will I require to install new opatch version?

OPatch 10.2.0.2.0 is the newest one (see note 839182 - Oracle database 10g: OPatch).

Take the main note (871096) and work through it. It will take quite some time but it will help you making your system stable and preventing many problems.

Markus

[Sudip1]

Hi,

I have applied all the patches referred by note 871096 and the system is now stable and not giving any problem.

Thanks for the help.

Regards

Sudip