What is the file size limit when using the FTP adapter in SAP Integration Suite ?

[glenmisquith]

What is the file size limit when using the FTP adapter in SAP Integration Suite ?

[alex_bundschuh · Accepted solution]

Hi,

in theory, there is no file size limit because FTP and SFTP adapter support streaming, however when using streaming you need to ensure that all flow steps within your integration low support streaming, see https://help.sap.com/docs/CLOUD_INTEGRATION/368c481cd6954bdfa5d0435479fd4eaf/396941a8c12441ab999f0f3...

There is however a general recommendation to keep the payload size below 40 MB for an optimized overall performance although larger sizes may be supported, however this always depend on your specific integration flow design and requirement, see https://help.sap.com/docs/CLOUD_INTEGRATION/368c481cd6954bdfa5d0435479fd4eaf/491c80d16c3547c3b124cf3...

Alex

[benjaminnehring · Accepted solution]

Hi alexander.bundschuh,

can you give a bit more detail on what it means for the SFTP sender adapter to be streaming enabled? Let's say we have a large text file on a SFTP server, but the lines of the file can be processed independently. If we now configure an Iterating Splitter after the SFTP sender adapter with the following configuration:

Expression Type: Line Break

Grouping: 50

Streaming: enabled

Parallel Processing: disabled

Is my understanding then correct that the SFTP sender adapter would start reading the first 50 lines of the file and hands it over to the Iterating Splitter. After the splitter is finished with processing this batch of data, the SFTP sender adapter reads lines number 51 to 100 and hands it over again to the Iterating Splitter for processing. This cycle continues until the file is completely read. Is that what is happening in the streaming case in CPI?

BR, Benjamin

[alex_bundschuh · Accepted solution]

Hi Benjamin,

the outcome is the same like without streaming, with streaming however the runtime already starts processing even if the whole message hasn't been fully stored in the memory, so streaming is less memory-intensive

Alex

[benjaminnehring · Accepted solution]

Hi alexander.bundschuh,

ok, thanks for the clarification. So by using a streaming enabled splitter, the complete file would also be loaded into the memory. If we want to process arbitrary large files, this is only possible when all IFlow steps support streaming.

BR, Benjamin

[benjaminnehring · Accepted solution]

Hi alexander.bundschuh,

in the PO SFTP sender adapter there is the option to split a large csv file with File Content Conversion and the parameter Recordsets per Message into multiple messages. According to note

2663129 - Performanceverbesserung für den Modus des SFTP-Adapters zur Konvertierung des Dateiinhalts

this can also be configured to be memory efficient, meaning the file is always only partially read into the memory. Is such a feature also planned to be made available for the CPI SFTP sender adapter?

BR, Benjamin

[yogananda]

Hi glenmisquith

10MB is the size limit as far as I have tried..

[glenmisquith]

Hi Yogananda,

Will the FTP adapter throw an exception if the file size is above 10 Mb ? Is this size limit mentioned in help.sap.com or some other content ? I wasn't able to find any reference of this.

[yogananda]

glenmisquith

yes it would indicate in message monitoring if the file size is exceeded for your iflow with failed status. Its not mentioned in help documentation about the file size limit.