Huge data itab and table updation

[rnb86]

Hello Experts,

I have a requirement where i need to fetch records placed in 2 FTP paths into SAP with my ABAP program and update a custom table. Data is picked up later from that custom table through sequence of background jobs and all those records will be archived and post archival, those records are deleted from that custom table. (Only 9500 records can be picked up for archiving jobs at once)

The total records in those 2 text files in FTP path are in the range of 170000.

The issue is: Once those records are in my internal table from FTP

1. It is not ideal to loop thorough all those records at once. So, how many records should i pick at once to update the custom table to avoid memory overflow? is there a limit?

This picking of files from FTP is a daily activity and the frequency of data that will be in those files will be same as i mentioned above. Data size is huge.

Thanks in advance.

[Former Member]

170,000 records shouldn't be too large for an internal table. But you haven't said how large each record is.

As for looping, you have to read each record eventually, why not all at once?

Rob

[Former Member]

hi, if you think 170 000 entries are to much, may be it's possible to split the ftp files directly on creation?

Are the files on the application server where you cann open them with open dataset?

If so, you can open the file read a certain amount of lines, do something with it and then read the next package..and so on.

[rnb86]

Thanks Rob and Daniel for your suggestions.

Each record(5 fields) will be 75 chars wide. I will update all the itab records into the table at once as you told, Rob.