Skip to Content
0

APD memory issue

Jun 22, 2017 at 06:29 AM

37

avatar image

Hi All,

We're trying to optimize an APD run which is failing due to memory shortage. After execution of the APD, a file is generated in an AL11 path. Now, we have tried to optimize the size of the data processed, using ABAP programs on either side of the APD execution to split the data into 3 (we have 3 valuation area), creating temporary files for each of them , and then combining them into one file after all APD runs are completed. But now we have been informed that we need to split the data even more, by company code and product type as well. Thing is, this split and then combine has to be dynamic, as we wont know beforehand the list of product types and company codes available (we did know that there are only 3 valuation areas, so we were OK there). Any suggestions would be most welcome. Thanks.

Here are the programs for valuation area:

REPORT ZTEST_RENAME_FILE.


types: BEGIN OF int_s_TMSMCONF,
DOMNAM type TMSDOMNAM,
SYSNAM type TMSSYSNAM,END OF int_s_TMSMCONF.
Parameters: val_area type c length 3.

DATA: itsysname TYPE TABLE OF int_s_TMSMCONF ,
wa_sysname type int_s_TMSMCONF,
dir_path type c length 60,
dir_path_file type c length 100.


clear itsysname.SELECT DOMNAM SYSNAM FROM tmsmconf INTO TABLE itsysname.clear wa_sysname.READ TABLE itsysname INTO wa_sysname INDEX 1.
concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US' into dir_path.

if val_area = '001'.

CALL METHOD cl_cts_language_file_io=>copy_files_localEXPORTING
im_source_file = 'SHP_US_BW_IMIS_ACC.TXT'
im_source_directory = dir_path
im_target_file = 'SHP_US_BW_IMIS_ACC_001.TXT'
im_target_directory = dir_path* im_overwrite_mode = SPACE* IMPORTING* ex_file_size =EXCEPTIONS
open_input_file_failed = 1
open_output_file_failed = 2
write_block_failed = 3
read_block_failed = 4
close_output_file_failed = 5others = 6.




elseif val_area = 'US0'.

CALL METHOD cl_cts_language_file_io=>copy_files_localEXPORTING
im_source_file = 'SHP_US_BW_IMIS_ACC.TXT'
im_source_directory = dir_path
im_target_file = 'SHP_US_BW_IMIS_ACC_US0.TXT'
im_target_directory = dir_path* im_overwrite_mode = SPACE* IMPORTING* ex_file_size =EXCEPTIONS
open_input_file_failed = 1
open_output_file_failed = 2
write_block_failed = 3
read_block_failed = 4
close_output_file_failed = 5others = 6.



elseif val_area = 'US2'.


CALL METHOD cl_cts_language_file_io=>copy_files_localEXPORTING
im_source_file = 'SHP_US_BW_IMIS_ACC.TXT'
im_source_directory = dir_path
im_target_file = 'SHP_US_BW_IMIS_ACC_US2.TXT'
im_target_directory = dir_path* im_overwrite_mode = SPACE* IMPORTING* ex_file_size =EXCEPTIONS
open_input_file_failed = 1
open_output_file_failed = 2
write_block_failed = 3
read_block_failed = 4
close_output_file_failed = 5others = 6.

endif.
concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC.TXT' into dir_path_file.
DELETE DATASET dir_path_file.

REPORT ZBW_IMIS_ACC_COMBINE_FILE.


types: BEGIN OF int_s_TMSMCONF,
DOMNAM type TMSDOMNAM,
SYSNAM type TMSSYSNAM,END OF int_s_TMSMCONF.


DATA: itsysname TYPE TABLE OF int_s_TMSMCONF ,
wa_sysname type int_s_TMSMCONF,
dir_path_file type c length 100.

DATA: file_001(120) type c,
file_US0(120) type c,
file_US2(120) type c,
file_final(120) type c,

result TYPE string,
temp TYPE standard table of string.



clear itsysname.SELECT DOMNAM SYSNAM FROM tmsmconf INTO TABLE itsysname.clear wa_sysname.READ TABLE itsysname INTO wa_sysname INDEX 1.
concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC_001.TXT' into dir_path_file.
file_001 = dir_path_file.clear dir_path_file.
concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC_US0.TXT' into dir_path_file.
file_US0 = dir_path_file.clear dir_path_file.
concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC_US2.TXT' into dir_path_file.
file_US2 = dir_path_file.clear dir_path_file.

concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC.TXT' into dir_path_file.
file_final = dir_path_file.clear dir_path_file.


CLEAR temp.


OPEN DATASET file_001 FOR INPUT IN TEXT MODEENCODING UTF-8 IGNORING CONVERSION ERRORS.
TRY.
DO.READ DATASET file_001 INTO result.IF sy-subrc = 0.APPEND result TO temp.ELSE.EXIT.ENDIF.ENDDO.
CATCH CX_SY_FILE_OPEN_MODE. "#EC NO_HANDLER
ENDTRY.
CLOSE DATASET file_001.


OPEN DATASET file_US0 FOR INPUT IN TEXT MODEENCODING UTF-8 IGNORING CONVERSION ERRORS.

TRY.
DO.READ DATASET file_US0 INTO result.IF sy-subrc = 0.APPEND result TO temp.ELSE.EXIT.ENDIF.ENDDO.
CATCH CX_SY_FILE_OPEN_MODE. "#EC NO_HANDLER
ENDTRY.
CLOSE DATASET file_US0.



OPEN DATASET file_US2 FOR INPUT IN TEXT MODEENCODING UTF-8 IGNORING CONVERSION ERRORS.

TRY.
DO.READ DATASET file_US2 INTO result.IF sy-subrc = 0.APPEND result TO temp.ELSE.EXIT.ENDIF.ENDDO.
CATCH CX_SY_FILE_OPEN_MODE. "#EC NO_HANDLER
ENDTRY.
CLOSE DATASET file_US2.


concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC_001.TXT' into dir_path_file.DELETE DATASET dir_path_file.clear dir_path_file.

concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC_US0.TXT' into dir_path_file.DELETE DATASET dir_path_file.clear dir_path_file.

concatenate '/SAPFTP/' wa_sysname-SYSNAM '/SHIP2/interface/OUTBOUND/AXA_US/SHP_US_BW_IMIS_ACC_US2.TXT' into dir_path_file.DELETE DATASET dir_path_file.clear dir_path_file.
clear result.
OPEN DATASET file_final FOR OUTPUT IN TEXT MODEENCODING UTF-8 IGNORING CONVERSION ERRORS.

TRY.
loop at temp INTO result.IF sy-subrc = 0.TRANSFER result TO file_final.ELSE.EXIT.ENDIF.endloop.
CATCH CX_SY_FILE_OPEN_MODE. "#EC NO_HANDLER
ENDTRY.
CLOSE DATASET file_final.
clear result.

apd
10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

0 Answers