Skip to Content

Running SAP BODS extract in background

Dec 20, 2017 at 09:10 PM


avatar image

We are experiencing timeouts when trying to run long running BODS extracts from SAP/ECC to SAP/BW. In order to overcome we tried switching the job to run in background by clicking on the option on the datastore. The degree of parallelism is set to 6. When running the job in dialog the processes can be seen in SM51 from the source system. However, when trying to run in batch only one process is seen and no data is returned, it seems to run forever. Eventually the batch version is abending with an error that there is no virtual memory available for execution - a BODS error.

Any thoughts would be appreciated.

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

4 Answers

Arun Sasi Dec 21, 2017 at 05:29 AM


As I understand, you are using Regular Data flows in your job with the option Execute in Background(batch).

I hope you are following the SAP note for the regular data flows

2036337 - How to set up a non-ABAP dataflow to execute in background - SAP Data Services

  • Enable the "Execute in background (batch)" checkbox inside of each SAP table in the non-ABAP dataflow to execute in background mode in the SAP side.
  • You should be able to verify the background jobs in the SAP side using the transaction SM50

SAP Note 2036337

Below is an excerpt from SAP Supplement Guide to improve the performance of the job when you are running the data flow in background. Read Section 4.3 Reading from SAP Tables.

Below is the process which happens when you run a data flow in Background

In batch execution mode, Data Services connects to SAP, submits the ABAP program as a background job in SAP, then disconnects from SAP after the program is submitted. Data Services schedules and executes the submitted program as a background job.
An ABAP program executed in batch mode is not subject to the dialog execution time constraint mentioned in that mode. However, there is a slight performance overhead due to background scheduling in SAP and execution status checking by Data Services.

SAP recommends ABAP data flow for time consuming processes.

Dirk Venken , Kindly provide your suggestions


Arun Sasi

sjwdw.png (50.6 kB)
10 |10000 characters needed characters left characters exceeded
Jaspreet Singh (Bedi) Dec 24, 2017 at 06:00 PM

All, we have regularly run SAP DS jobs using standard dataflows(i.e. non-ABAP) to extract data from SAP in background mode. It was a major issue for us when we first time ran into dialog mode time limite(it was 3000 seconds). We basically had to enable RFC streaming mode, setup "SAPDS" (could be any name but should match what we specify in datastore) RFC in SAP. It worked very easily and we did not see any more timeouts. Job trace logs also look different once you run the DS job in background mode. So, just to summarize, execute in background does apply to non-ABAP data flows.

10 |10000 characters needed characters left characters exceeded
Jaspreet Singh (Bedi) Dec 24, 2017 at 06:06 PM

Richard, please confirm if your issue is on the 'extract' side or also on the 'load' side. Loads to BW can also slow down the dataflow. Best way to troublesheet is by having a job which extracts from ECC but dumps data into a relational DB. Once confirmed that issue is on extract, you could enable background mode, ensure you extract only those ECC columns you really need, make sure your Query filters get passed to SAP(by reviewing the generated SQL), increasing array fetch size(there is a sweet spot but i have never gone beyond 10K, see what nunber gives you best runtime). If all fails, I have even broken down the job into multiple SAP extracts and using some key to split single extract into two extracts.

10 |10000 characters needed characters left characters exceeded
Dirk Venken
Dec 22, 2017 at 10:36 AM

In contradiction to SAP Note 2036337, I have (only recently) learnt that all normal data flows runs as dialog jobs, even if the execution mode in the datastore definition has been set to background.

The Execute in Background(batch).setting applies to ABAP data flows only.

10 |10000 characters needed characters left characters exceeded