Loop over dynamic hashed table with a key

[former_member568535]

Hello,

I'm working on custom planning function type that implements IF_RSPLFA_SRVTYPE_IMP_EXEC_REF (planning on reference data).

The EXECUTE method of the implementing class has I_TH_REF_DATA TYPE HASHED TABLE as an 'importing' parameter. This table actually includes my reference data. The structure of the table is defined dynamically at runtime in accord with the structure of an aggregation level. It includes all characteristics and keyfigures that the aggregation level is comprised from.

To avoid full scan of the table I want to loop over it using some kind of 'partial' key that includes only limited set of characteristics. Product assistance suggests something like that:

DATA spfli_tab TYPE HASHED TABLE
OF spfli
WITH UNIQUE KEY primary_key
COMPONENTS carrid connid
WITH NON-UNIQUE SORTED KEY city_from_to
COMPONENTS cityfrom cityto
WITH NON-UNIQUE SORTED KEY city_to_from
COMPONENTS cityto cityfrom.

LOOP AT spfli_tab ASSIGNING <spfli> USING KEY city_from_to.

This, however. does not apply to my task, as the table I_TH_REF_DATA is defined somewhere inside planning engine and I have no control over its structure, keys included.

I know beforehand the structure of the aggregation level, and want to loop using a key that includes, let's say, characteristics 0FISCPER, 0CURRENCY, ZSCOPE and ZVERSION. I need something like

LOOP AT I_TH_REF_DATA ASSIGNING <fs_line> USING KEY COMPONENTS '0FISCPER' '0CURRENCY' 'ZSCOPE' 'ZVERSION'

However, it seems that ABAP syntax does not provide for that.

Please advise, how can I loop over a dynamic hashed table avoiding full table scan?

Thank you,

Val

[michael_piesche]

1) Using a dynamic keyname to access table by key sort order

So you cannot define the key while programming, but you can define it at runtime? And you want to use the sorted key of the table to loop through the records according to the keys sort order?

" dynmically set your keyname based on the definition based on aggregation level
DATA keyname TYPE c LENGTH 30.
CASE // IF.
  " ...
  keyname = 'MY_DYN_KEY_NAME'.
  " ...
ENDCASE // ENDIF.

LOOP AT I_TH_REF_DATA ASSIGNING <fs_line> USING KEY (keyname). " WHERE (restrictions). 
  " ...
ENDLOOP.

2) Getting keyname and keyattributes from table dynamically

What is unclear for me right now, is whether you already know the key at runtime or just the key components? But if you dont know the key, or the key components you can get them with reading the keys and components at runtime with class CL_ABAP_TABLEDESCR using method GET_KEYS.

DATA: lo_tabledescr TYPE REF TO cl_abap_tabledescr,
      lt_keys       TYPE abap_table_keydescr_tab.
FIELD-SYMBOLS:      <ls_keys>           LIKE LINE OF lt_keys,
                    <ls_key_components> TYPE abap_table_keycompdescr.

" get keys and key components of table by data
lo_tabledescr  ?= cl_abap_tabledescr=>describe_by_data( I_TH_REF_DATA ).

lt_keys = lo_tabledescr->get_keys( ).
LOOP AT lt_keys ASSIGNING <ls_keys> WHERE access_kind = 'S'  " sorted mode
                                         AND key_kind = 'K'. " with key components
   " check whether you want to use the key <ls_keys>-name with decision based on:
   " -is_primary     'X' yes // ' ' no
   " -access_kind    'H' hashed // 'S' sorted
   " -is_unique      'X' yes // ' ' no
   " -key_kind       'T' rowtype // 'K' key components
   " ...

   LOOP AT <ls_keys>-components ASSIGNING <ls_key_components>.
   " check whether you want to use the key component <ls_key_components>-name
   " ...       

   ENDLOOP.
ENDLOOP.

[former_member568535]

Hello Michael, thank you for the input.

My situation actually looks as follows:

- The hashed table I_TH_REF_DATA is defined by planning engine in accord with the structure of my aggregation level. It includes fields 0FISCPER, 0CURRENCY, ZSCOPE, ZVERSION, and more characteristics and keyfigures. The primary key for this table is also defined by planning engine and contains all characteristics.

- My custom planning function type accepts multiple single values for parameters 'Period' (0FISCPER), 'Currency' (0CURRENCY), 'Consolidation Scope' (ZSCOPE) and 'Version' (ZVERSION). So, my reference data (I_TH_REF_DATA) may contain multiple values for any of this characteristics.

- I have a custom logic implemented as a class method that accepts only single values of 'Period', 'Currency', 'Consolidation Scope' and 'Version'. I need to extract lines that belong to any unique combination of these characteristics from I_TH_REF_DATA, copy it to another dynamic table and pass this dynamic table to the custom logic as 'importing' parameter. To do this I need:

first - define all unique combinations of 'Period', 'Currency', 'Consolidation Scope' and 'Version' (this is done already);

second - for every unique combination of 'Period', 'Currency', 'Consolidation Scope' and 'Version' loop at I_TH_REF_DATA, selecting only lines that belong to this combination.

Doing it with simply LOOP AT - ASSIGNING - WHERE may probably have negative effect on performance, as this looks like plain and simple full table scan. So, what I need is an approach to properly scan I_TH_REF_DATA using its 'hashed' type to achieve better performance. I assume there has to be some instrument to define dynamic key and use it while looping at the table. Your first example looks very much like what I actually need. Could you please elaborate a bit more about defining dynamic key:

DATA keyname TYPE c LENGTH 30.
CASE // IF.
  " ...
  keyname = 'MY_DYN_KEY_NAME'.
  " ...
ENDCASE // ENDIF.

What this procedure may look like in my case?

Thank you,

Val

[mateuszadamus]

Hi Val,

There is no way to dynamically add a key to the already defined internal table. You need to create a new internal table with the required key and populated it with data. Then do your logic on the newly created internal table.

That's exactly what my below answer is showing.

Kind regards,
Mateusz

[michael_piesche]

Val Teem, so contrary to what I understood first, the way you need to access the table is not dynamic, but static, based on "'0FISCPER', '0CURRENCY', 'ZSCOPE', 'ZVERSION'", and the only thing that is dynamic, if at all, might be the hashed key components of I_TH_REF_DATA, correct?Why dont you do the following:(I havnt syntax checked it and not all variables are defined)

" 1) get keys and key components of table by data
DATA: lo_tabledescr TYPE REF TO cl_abap_tabledescr,
      lt_keys       TYPE abap_table_keydescr_tab.
FIELD-SYMBOLS:      <hash_key>          LIKE LINE OF lt_keys,
                    <ls_key_components> TYPE abap_table_keycompdescr.

lo_tabledescr  ?= cl_abap_tabledescr=>describe_by_data( I_TH_REF_DATA ).
lt_keys = lo_tabledescr->get_keys( ).
READ TABLE lt_keys ASSIGNING <hash_key> WITH KEY is_primary = abap_true.
IF sy-subrc <> 0.
  " something went wrong
ENDIF.
   
" 2) create new table based on your key fields and the hashed key components
data: 	structtype 	type ref to cl_abap_structdescr,
	tabletype 	type ref to cl_abap_tabledescr,
	comp_tab 	type cl_abap_structdescr=>component_table,
	new_comp_tab 	like comp_tab,
        new_key_tab     type ABAP_KEYDESCR_TAB,
	linetype 	type ref to cl_abap_structdescr,
	dref 		type ref to data.
field-symbols: <wa_comp> like line of comp_tab.
field-symbols: <table>   type any table.
field-symbols: <table_unique>   type any table.
field-symbols: <line>    type any.
field-symbols: <ls_key_components> TYPE abap_table_keycompdescr.

structtype ?= cl_abap_typedescr=>describe_by_data( I_TH_REF_DATA ).
comp_tab    = structtype->get_components( ).
loop at comp_tab assigning <wa_comp>.
	CASE <wa_comp>-name.
	  WHEN '0FISCPER' OR '0CURRENCY' OR 'ZSCOPE' OR 'ZVERSION'.
            " append those as key components as well as new struc (see below)
            INSERT <wa_comp> INTO TABLE new_key_tab.
          OTHERS.
            READ TABLE <hash_key>-components TRANSPORTING NO FIELDS WITH KEY name = <wa_comp>-name.
            CHECK sy-subrc = 0.
	ENDCASE.
        INSERT <wa_comp> INTO TABLE new_comp_tab.
endloop.

linetype 	 = cl_abap_structdescr=>create( new_comp_tab ).
tabletype 	 = cl_abap_tabledescr=>create( p_line_type  = linetype
                                               p_table_kind = cl_abap_tabledescr=>TABLEKIND_SORTED
                                               P_UNIQUE	    = abap_false
                                               p_key        = new_key_tab
                                               P_KEY_KIND   = cl_abap_tabledescr=>KEYDEFKIND_USER ).
create data dref type handle tabletype.
assign dref->* to <table>.

" unique table can be substituted by LOOP ... GROUP BY ... and LOOP AT GROUP ... statements.
tabletype 	 = cl_abap_tabledescr=>create( p_line_type  = linetype
                                               p_table_kind = cl_abap_tabledescr=>TABLEKIND_SORTED
                                               P_UNIQUE	    = abap_true
                                               p_key        = new_key_tab
                                               P_KEY_KIND   = cl_abap_tabledescr=>KEYDEFKIND_USER ).
create data dref type handle tabletype.
assign dref->* to <table_unique>.

create data dref type handle linetype.
assign dref->* to <line>.
create data dref type handle linetype.
assign dref->* to <line_unique>.

" 3) move input table to new internal dynamic table
LOOP AT I_TH_REF_DATA ASSIGNING <fs_line>.
  create data dref type handle linetype.
  assign dref->* to <line>.
  MOVE-CORRESPONDING <fs_line> to <line>.
  INSERT <line> INTO TABLE <table>.
  INSERT <line> INTO TABLE <table_unique>.
ENDLOOP.

" 4) create dynamic statements to access by keys
" where statement for known key access
lv_where = '0FISCPER = <line_unique>-0FISCPER AND 0CURRENCY = <line_unique>-0CURRENCY AND ZSCOPE = <line_unique>-ZSCOPE AND ZVERSION = <line_unique>-ZVERSION'.
" key statement for unkown (hashed) key access
LOOP AT <hash_key>-components ASSIGNING <ls_key_components>.
  lv_hashed_key = lv_hashed_key && ' ' && <ls_key_components>-name && ' = <line>-' && <ls_key_components>-name .
ENDLOOP.

" 5) actual logic 
loop at <table_unique> assigning <line_unique>.
  " aggregation level based on unique keys
  " ...

  LOOP at <table> assigning <line> WHERE (lv_where).
    " data level
    " ....

    READ TABLE I_TH_REF_DATA ASSIGNING <fs_line> WITH TABLE KEY ( lv_hashed_key ).
    IF sy-subrc = 0.
      " input data level
      " ...

    ENDIF.
  ENDLOOP.
endloop.

[mateuszadamus]

Definitely this will work.

However, why do you complicate this so much, when it does the same thing as the example code in my answer?

The TYPEDESCRIPTORs are good when you want to create the whole table structure dynamically from a list of fields. In this case the structure is known, the only change is the new key.

Kind regards,

Mateusz

[michael_piesche]

Mateusz Adamus, if everything is static, yes then 'your coding template' will work, with the necessary changes. But I still dont know what all is dynamic from the planning engine from BW. And with the above coding, the OP will have the most possible degree of freedom. And freedom has a price, in this case the 'complexity' or at least the more lines of code.

[mateuszadamus]

Hello Michael,

Your code will work. I can't disagree with that.

But I'm not sure if you're not bringing a cannon to kill a mosquito. There are some assumptions made, e.g.: method parameters, and I think the code could reflect these. Too much freedom isn't always a good thing.

Kind regards,
Mateusz

[former_member568535]

Hi Mateusz, thank you for the input.

Let me get into more details to make situation more clear.

Any ABAP class implementing IF_RSPLFA_SRVTYPE_IMP_EXEC_REF interface implements EXECUTE method, which typically contains something as follows:

DATA:          l_th_ref_data TYPE REF TO data.
FIELD-SYMBOLS: <fs_th_ref_data>type hashed table,
               <fs_line>type any.

CREATE DATA l_th_ref_data LIKE i_th_ref_data.
ASSIGN l_th_ref_data->* TO <fs_th_ref_data>.

loop at i_th_ref_data assigning <fs_line>.
" Some logic here
collect <fs_line> into <fs_th_ref_data>.
endloop.

I understand that I cannot define any new key on I_TH_REF_DATA. So I need to copy data to another internal table and define key as needed for this new table. <fs_th_ref_data> may serve as an example of this 'new table'. However, I cannot define anything on this table, as the 'CREATE DATA LIKE' construct creates just a 'replica' of the source table (I_TH_REF_DATA). Moreover, I only can use the field symbol <fs_th_ref_data> to manipulate this table. Trying to use l_th_ref_data in the code results in error "l_th_ref_data is not an internal table".

So, it seems unavoidable to use RTTI to create a new table and define its structure accordingly, including key(s). In this way what Michael suggests looks pretty reasonable. Or so it seems to me.

Best regards,

Val

[gregor_dieckmann]

Hi Val,

the parameter I_TH_REF_DATA is passed by value, so just do a dynamic sort (yes, hashed tables can be sorted) and do the usual group change logic. So in the loop at I_TH_REF_DATA you just fill a local table and at any new combination you just call your method with the collected records from the previous combination. You don't need the unique combination table here.

This also works without sorting but then you have to collect the records contained in one group first, e.g. in the non-key part of your unique combination table. This has the disadvantage that you have to copy the records (or use a reference).

I don't understand why you are afraid of the full table scan, the logic you explained anyway needs a full table scan.

I think the first option is simple and efficient; observe that SORT ... is one statement in the ABAP kernel, so it fast.

Regards,

Gregor

[mateuszadamus]

Heh, it never occurred to me, that a hash table can be sorted. I guess it's because I got burned on trying to sort a sorted table...

But indeed, it is possible and clearly stated in the documentation, too.

Kind regards,
Mateusz

[former_member568535]

Hi Gregor,

Thank you for your suggestion. I'll definitely try it, and see how it affects performace.

Best regards,

Val

[Sandra_Rossi]

If you want to create an index on a table, but you can't because this input table is typed generically, and you know the component names on which you want to define the key (you are sure that these components will exist forever), and you know that this table is standard or sorted, I'd suggest that you define statically a temporary internal table with a component being the table index corresponding to the line in the input table, initialize it from the input table, and use the order of this temporary table to access the lines of the input table.

Example:

TYPES : BEGIN OF ty_temp_table,
          0FISCPER  TYPE ...
          0CURRENCY TYPE ...
          ZSCOPE    TYPE ...
          ZVERSION  TYPE ...
          TABIX     TYPE sytabix,
        END OF ty_temp_table,
        tt_temp_table TYPE SORTED TABLE OF ty_temp_table
             WITH NON-UNIQUE KEY 0FISCPER, 0CURRENCY, ZSCOPE, ZVERSION.

temp_table = VALUE #( 
    FOR <i_th_line> IN <i_th_data> INDEX INTO i_th_line_tabix
    ( VALUE #( BASE CORRESPONDING #( <i_th_line> ) tabix = i_th_line_tabix ) ).

LOOP AT temp_table REFERENCE INTO DATA(temp_line) 
      WHERE 0FISCPER  = ...
        AND 0CURRENCY = ...
        AND ZSCOPE    = ...
        AND ZVERSION  = ...
  <i_th_line> = <i_th_data>[ temp_line->tabix ].
  ...

[former_member568535]

Hello Sandra,

I am not sure I follow this:

LOOP AT temp_table REFERENCE INTO DATA(temp_line) 
      WHERE 0FISCPER  = ...
        AND 0CURRENCY = ...
        AND ZSCOPE    = ...
        AND ZVERSION  = ...

Because I don't know the values of these characteristics beforehand. I just want to loop at source table of hashed type for all combinations of these four fields. Maybe I'm missing something in what you suggest here.

Thank you,

Val

[Sandra_Rossi]

Could you write a pseudo algorithm so that we can understand what you want to do?

[former_member568535]

Sandra,

Actually I described the algorithm that I need a bit earlier in the previous thread of this topic. mateuszadamus and michael.piesche, in turn, suggested solutions which I am going to explore in more details and (maybe) apply to my task.

Can we get back and talk the algorithm in more details after I'm done with the abovementioned research?

Thank you and best regards,

Val

[Sandra_Rossi]

My answer fits your algorithm, I don't understand what you don't understand. Can you clarify?

1. Input parameter: The hashed table I_TH_REF_DATA is defined by planning engine in accord with the structure of my aggregation level. It includes fields 0FISCPER, 0CURRENCY, ZSCOPE, ZVERSION, and more characteristics and keyfigures. The primary key for this table is also defined by planning engine and contains all characteristics. My custom planning function type accepts multiple single values for parameters 'Period' (0FISCPER), 'Currency' (0CURRENCY), 'Consolidation Scope' (ZSCOPE) and 'Version' (ZVERSION). So, my reference data (I_TH_REF_DATA) may contain multiple values for any of this characteristics.
2. define all unique combinations of 'Period', 'Currency', 'Consolidation Scope' and 'Version' (this is done already);
3. for every unique combination of 'Period', 'Currency', 'Consolidation Scope' and 'Version' loop at I_TH_REF_DATA, selecting only lines that belong to this combination. I have a custom logic implemented as a class method that accepts only single values of 'Period', 'Currency', 'Consolidation Scope' and 'Version'. I need to extract lines that belong to any unique combination of these characteristics from I_TH_REF_DATA, copy it to another dynamic table and pass this dynamic table to the custom logic as 'importing' parameter.

For part 3:

LOOP AT unique_combinations REFERENCE INTO DATA(combination).
  LOOP AT temp_table REFERENCE INTO DATA(temp_line) 
      WHERE 0FISCPER  = combination->0fiscper
        AND 0CURRENCY = combination->0currency
        AND ZSCOPE    = ...
        AND ZVERSION  = ...
    <i_th_line> = <i_th_data>[ temp_line->tabix ].

[former_member568535]

Thank you, Sandra, I'll take a note of that. This one definitely fits the scanario:

LOOP AT unique_combinations REFERENCE INTO DATA(combination).
  LOOP AT temp_table REFERENCE INTO DATA(temp_line) 
      WHERE 0FISCPER  = combination->0fiscper
        AND 0CURRENCY = combination->0currency

Best regards,

Val

[mateuszadamus]

Hello val.teem

Have a look at this example based on the VBAP table.

  DATA:
    ld_row         TYPE REF TO data,
    ld_table       TYPE REF TO data,
    lt_vbap_hashed TYPE HASHED TABLE OF vbap WITH UNIQUE KEY vbeln posnr.

  FIELD-SYMBOLS:
    <ls_row>   TYPE any,
    <lt_table> TYPE ANY TABLE.

  CREATE DATA ld_row LIKE LINE OF lt_vbap_hashed.
  ASSIGN ld_row->* TO <ls_row>.

  CREATE DATA ld_table LIKE SORTED TABLE OF <ls_row> WITH NON-UNIQUE KEY vbelv.
  ASSIGN ld_table->* TO <lt_table>.

  <lt_table>[] = lt_vbap_hashed[].

If the key field provided does not exist in the newly created table, then an exception CX_SY_TABLE_KEY_SPECIFICATION will be raised. If not handled it will result in a short-dump.

You'll probably need to provide the WHERE condition in a dynamic way, for example.

cond = 'col2 > dref->*'. 

LOOP AT itab INTO line FROM 10 TO 25 WHERE (cond). 
  APPEND CONV string( line-col2 ) TO output. 
ENDLOOP. 

https://help.sap.com/doc/abapdocu_751_index_htm/7.51/en-us/abaploop_at_itab_cond.htm#!ABAP_ADDITION_...

Kind regards,

Mateusz

Edit: Added the dynamic key definition, as it may not be clear.

DATA:
  lt_key TYPE TABLE OF string.

APPEND 'VBELV' TO lt_key.

CREATE DATA ld_table LIKE SORTED TABLE OF <ls_row> WITH NON-UNIQUE KEY (lt_key).

[mateuszadamus]

Hello val.teem

For some reason I'm not able to respond to your comment...

First of all, let me mark that I don't know your current scenario as well as you do. You choose what you thing is best for you and that's cool. You know your situation and needs best.

I don't like to have a complicated solution, when a simple one would suffice. Sometimes it (complicated solution) is necessary and gives a lot of added value to the system (e.g.: configuration). Sometimes something simpler will do just fine.

Here's an example code for the EXECUTE logic, for your consideration (based on the VBAP table). Notice, that except for the key field VBELV everything else is dynamic in the EXECUTE method. For this particular example the result of the method is the first row of the newly created sorted table (here a LOOP based on a dynamic WHERE clause could be used).

CLASS lcl_z DEFINITION.
  PUBLIC SECTION.
    INTERFACES:
      if_rsplfa_srvtype_imp_exec_ref.
ENDCLASS.

CLASS lcl_z IMPLEMENTATION.
  METHOD if_rsplfa_srvtype_imp_exec_ref~execute.
    DATA:
      ld_row         TYPE REF TO data,
      ld_table       TYPE REF TO data,
      lt_key         TYPE TABLE OF string.

    FIELD-SYMBOLS:
      <ls_row>   TYPE any,
      <lt_table> TYPE SORTED TABLE.

    CREATE DATA ld_row LIKE LINE OF i_th_ref_data.
    ASSIGN ld_row->* TO <ls_row>.

    APPEND 'VBELV' TO lt_key.

    CREATE DATA ld_table LIKE SORTED TABLE OF <ls_row> WITH NON-UNIQUE KEY (lt_key).
    ASSIGN ld_table->* TO <lt_table>.

    <lt_table>[] = i_th_ref_data[].

    READ TABLE <lt_table> ASSIGNING <ls_row> INDEX 1.
    INSERT <ls_row> INTO TABLE c_th_data.
  ENDMETHOD.
ENDCLASS.

" test execution in the test report
DATA:
  ls_tmp         TYPE vbak,
  lt_vbap_source TYPE HASHED TABLE OF vbap WITH UNIQUE KEY vbeln posnr,
  lt_vbap_result TYPE HASHED TABLE OF vbap WITH UNIQUE KEY vbeln posnr,
  lo_msg         TYPE REF TO if_rsplfa_msg,
  lo_param_set   TYPE REF TO if_rsplfa_param_set.

SELECT *
  INTO TABLE lt_vbap_source
  FROM vbap
  UP TO 100 ROWS.

BREAK-POINT. " just to see what's going on

DATA(lo_z) = NEW lcl_z( ).
lo_z->if_rsplfa_srvtype_imp_exec_ref~execute(
  EXPORTING
    i_r_param_set = lo_param_set " empty for test, because required
    i_th_ref_data = lt_vbap_source
    i_s_block_line = ls_tmp " empty for test, because required
    i_r_msg = lo_msg " empty for test, because required
  CHANGING c_th_data = lt_vbap_result ). 

Data on the start of the method:

Original data in the method (notice the key, blue columns):

Copied data in method (notice the key):

Data as a result:

That's it from me on this topic. As mentioned before, michael.piesche 's code will work just fine, too.

Kind regards,
Mateusz

[mateuszadamus]

I have an issue uploading the images.

Data on the start of the method:

Original data in the method (notice the key, blue columns):

Mateusz