Back to Basics: Internal table

[Former Member]

Iam back to basics..

My internal table size exceeds 2 GB while extracting the data from MSEG and BSEG.

I want to extract all data ie size = 4GB at 1 shot. So guys tell me how to put 4GB size in my internal table.

1. Is it by field groups and LDBs.

2. Is it FETCH cursor.

3. Is it packet size.

I need it by step by step.

Thanks in advance.

Sri.

[christian_wohlfahrt]

Hi Sri!

I doubt you will be able to store 4GB in an internal table (simultanesly).

All your mentioned technics are avoiding to hold the whole data, but slicing this task into smaller chunks.

1. With field groups you go for storing in file system, maybe the slowest alternative (but can hold 4GB without dump)

2. Fetch cursor can be used to access your data line by line (like select - endselect, but whithout looking problems. Still you need to do your work per line and you won't be able to append the data to an internal table.

3. Using packets will be the fastests DB access, because you have less steps. But again: you have to handle each package on it's own and can't append all packages into one large table.

The only option to get 4GB into memory: change the system parameters (on a machine, which has enough power...). But here you can use a simple select ... into table.

What is the result of your task? A file? This could be writen step by step -> you don't need to store 4GB.

Regards,

Christian

[Former Member]

Hi.

Here iam going to schedule my program in back ground.

So i need to store the data in my internal table..

[christian_wohlfahrt]

Hi,

if your program is running in foreground or background has nothing to do with the logic (or the available memory).

In a background job you still can make a lot of write-statements in a loop (and each page will be put into the spool without holding all pages in memory);

you can very easily create application server files (open dataset);

you can make DB changes - e.g. change a Z-append field of MSEG line for line (or by packages...

so you can have many ways to work with the data by lines, not holding the whole table.

Which step you think you have to do for the whole internal table?

[Former Member]

Background or foreground will make no difference for storage. I strongly doubt if you can accomplish what you are trying to do. I think you should re-think your requirements.

Rob

[Former Member]

Hi,

The program will endup with a short dump due to unavailable resources like insufficient memory.

It is better to go with the PACKAGE SIZE option. Or another option, which is very unlikly to happen, is your program should have the whole system for itself. Then since no other process is taking up the resources, may be it can run.

Regards,

Sumant.

[former_member183804]

Hello Srinivas,

Even a memory requirement of 2 GB is say an oddity. Looking for more memory is not a proper solution. Any algorythm should be scaleable in terms of resource and time requirements.

Maybe you can break down the extraction into smaller chunks and perform them in a loop. For sample doing the extraction for 12 months instead of a year.

Regards,

Klaus

[Former Member]

Hi,

One alternative is that you can use "Open Cursors"

for collecting data in small packets...the logic is widely used in "Extracors" in SAP-BW to enhance performance and avoid overusage of memory leading to Dumps because as suggested by all above size is a concern,if not now...later

Regards

Bx

[VinayPrasad_PM]

Hi Sri,

Before Release 4.0A, ABAP stored the content of internal tables in a combination of main memory and file space. This means that the maximum size of all internal tables of all programs running on such an application server at one time is about 2 GB. With Release 4.0A or greater, this size decreases to about 500 MB. (Note that those values aren't fixed, but this is a good guide. This minimum of 500 MB is the lowest limit of the real value, which varies among different operating systems and even among different releases of the same operating system.)

It may sound strange that a newer release has a higher restriction on capacity. But it's a consequence of the fact that the contents of internal tables move from a reserved file to shared memory. When you process internal tables this way in Release 4.0A or greater, you pay for much better performance with a smaller potential size.

Captured in the 2-GB trap of a 32-bit operating system, you have to redesign your program in a way that the complete data is split into different logical parts that each, on its own, can be processed by ABAP.

http://www.sapinsideronline.com/archive/volume_01_(2000)/issue_01_(june)/v1i1a12.cfm?session=

Regards,

Vinay

[Former Member]

I agree with Edmana the concept of OPEN CURSOR, FETCH CURSOR and CLOSE CURSOR in BIW extractors so that we can fetch the data in packets.

Sri.