APD shortdum - Join causes error TSV_TNEW_BLOCKS_NO_ROLL_MEMORY

[Former Member]

Hello Experts,

Our APD process is shortdumping in Production system due to high memory consumption - TSV_TNEW_BLOCKS_NO_ROLL_MEMORY.

This is happening despite unticking the box in "Performance settings"--> Process All Data in the Main Memory.

The issue is caused by JOIN transformation which first reads data from two sources, then sorts it for each case and then merge. It seems the whole data set is stored in the memory (heap memory + extended memory) which eventually runs out.

SAP recommends using InfoSets instead of JOINS as they operate on the database server rather than application server. However, we have some logic before we join data.

Is there anything we can do to improve APD performance / memory management?

Kind regards,

Roman

[Former Member · Accepted solution]

Hi,

Please check the OSS Note 1047255 .

-Vikram

[Former Member]

Hello Vikram,

We have checked this note but it didn't help us. It describes how JOIN works - which is exactly our problem. It needs to read the whole data set from both sources, sort it and then merge. It is consuming a lot of memory.

Any other ideas?

[Pravender]

You have to decrease the data size which you are processing in one iteration. You can try including aggregation block in APD and see if it helps.

[Former Member]

Hello All,

SAP came back to me regarding this issue. There are only way to eliminate this error is to:

- use InfoSet rather than JOIN

- increase Extended Memory or Heap Memory for Background WP.

We have doubled the memory per process (4-->8GB) and it works fine now.

Kind regards,

Roman

[Former Member]

Thank you very much Vikram!

Unfortunately we are not using queries, so partitioning is not really applicable for us. Data is fetched from DSOs, then filtered, processed in formulas, ABAP routines, etc. and then linked in JOINs. And JOINs are failing in this whole process.

Any other ideas?

[Former Member]

Hi,

There is one very interesting OSS note.

794257

Please check.

-Vikram