Data packets/size - Generic extractor

[Former Member]

Hi all,

We built a custom function module based datasource and it is extracting data to BW in one big packet of 900,000+ records and the load is taking about 18 hours. We are trying to spilt the BW extraction into smaller data packets to improve performance but unable to do so. Following is our extraction program...

Please let me know where we are doing it wrong...

This Program fetches/build e_t_data. The issue is, program does not splitting into packets as the SAP standard program does.

*"----

---

""Local interface:

*" IMPORTING

*" VALUE(I_REQUNR) TYPE SRSC_S_IF_SIMPLE-REQUNR

*" VALUE(I_DSOURCE) TYPE SRSC_S_IF_SIMPLE-DSOURCE OPTIONAL

*" VALUE(I_MAXSIZE) TYPE SRSC_S_IF_SIMPLE-MAXSIZE OPTIONAL

*" VALUE(I_INITFLAG) TYPE SRSC_S_IF_SIMPLE-INITFLAG OPTIONAL

*" VALUE(I_READ_ONLY) TYPE SRSC_S_IF_SIMPLE-READONLY OPTIONAL

*" VALUE(I_DATAPAKID) TYPE SBIWA_S_INTERFACE-DATAPAKID OPTIONAL

*" TABLES

*" E_T_DATA STRUCTURE Z0333W OPTIONAL

*" I_T_SELECT TYPE SRSC_S_IF_SIMPLE-T_SELECT

*" I_T_FIELDS TYPE SRSC_S_IF_SIMPLE-T_FIELDS

*" EXCEPTIONS

*" NO_MORE_DATA

----

---

DATA: lr_range_name TYPE rsselect-fieldnm.

DATA: st_e_t_data TYPE z0333w.

STATICS: l_cursor TYPE cursor.

STATICS: called(1) TYPE c VALUE 'N'.

• Maximum number of lines for DB table

STATICS: l_maxsize TYPE sbiwa_s_interface-maxsize.

FIELD-SYMBOLS: <l_range> TYPE ANY,

<l_t_range> TYPE STANDARD TABLE.

IF i_initflag = 'X'.

************************************************************************

• Initialization: check input parameters

• buffer input parameters

• prepare data selection

************************************************************************

• Fill parameter buffer for data extraction calls

g_s_interface-requnr = i_requnr.

g_s_interface-isource = i_dsource.

g_s_interface-maxsize = i_maxsize.

g_s_interface-initflag = i_initflag.

g_s_interface-datapakid = i_datapakid.

g_flag_interface_initialized = sbiwa_c_flag_on.

REFRESH g_t_select.

REFRESH g_t_fields.

APPEND LINES OF i_t_select TO g_t_select.

APPEND LINES OF i_t_fields TO g_t_fields.

ELSE.

• first data package of first table -> open cursor

IF g_counter_datapakid = 0.

*--Get selection ranges

LOOP AT i_t_select.

………………………………………….

………………………………

MOVE-CORRESPONDING i_t_select TO <l_range>.

APPEND <l_range> TO <l_t_range>.

ENDLOOP.

l_maxsize = g_s_interface-maxsize.

• fetch plants for the company code

PERFORM get_plants.

• fetch mast data into internal table as we will be using MAST for validation

• of whether BOM exist or not.

SELECT * FROM mast INTO TABLE it_mast

WHERE stlan = '1' OR stlan = '6'.

SORT it_mast BY matnr werks stlan.

• Material BOM information

• First data package -> OPEN CURSOR

OPEN CURSOR WITH HOLD l_cursor FOR

SELECT mast~matnr

mast~werks

mast~stlnr

mast~stlan

mast~stlal

stko~stlty

FROM mast INNER JOIN stko

ON stko_{stlnr = mast}stlnr AND

stko_{stlal = mast}stlal

FOR ALL entries IN gt_werks

WHERE mast~matnr IN gr_matnr AND

mast~werks IN gr_werks AND

mast~stlan IN gr_stlan AND

mast~werks = gt_werks-werks AND

mast~stlal = '01' AND

stko~stlty = 'M' AND "Material BOM only

( mast_{stlan = '1' OR mast}stlan = '6' ).

ENDIF.

• Fetch records into interface table.

• named E_T_'Name of extract structure'.

REFRESH: gt_mat_bom,gt_mat_bom1.

FETCH NEXT CURSOR l_cursor

APPENDING CORRESPONDING FIELDS

OF TABLE gt_mat_bom

PACKAGE SIZE i_maxsize.

IF sy-subrc <> 0.

CLOSE CURSOR l_cursor.

RAISE no_more_data.

ELSE.

• get BOM data and fill E_T_DATA

PERFORM get_bom_data TABLES e_t_data.

ENDIF.

• Increment Package

g_counter_datapakid = g_counter_datapakid + 1.

ENDIF.

ENDFUNCTION

Thanks,

Anirudh.

[Former Member]

I'm not sure, but this might help:

* Fetch records into interface table.
* named E_T_'Name of extract structure'.

  DO.
    REFRESH: gt_mat_bom,gt_mat_bom1.
    FETCH NEXT CURSOR l_cursor
      APPENDING CORRESPONDING FIELDS
      OF TABLE gt_mat_bom
      PACKAGE SIZE i_maxsize.

    IF sy-subrc <> 0.
      EXIT.
    ELSE.

* get BOM data and fill E_T_DATA

      PERFORM get_bom_data TABLES e_t_data.
    ENDIF.

* Increment Package
    g_counter_datapakid = g_counter_datapakid + 1.
  ENDDO.

  CLOSE CURSOR l_cursor.
  RAISE no_more_data.

Rob