Skip to Content
avatar image
Former Member

Incomplete File Processing

Hi

I have a File 200MB. I can neither break it into chunks nor increase the poll interval, which has to be 60 seconds. How do I ensure that file is processed completely ? Msecs to wait would not work here as my scenario is FTP and that parameter works only for NFS.

What are the settings that I have to ensure so that the file would not be processed incompletely ?

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

3 Answers

  • Best Answer
    avatar image
    Former Member
    Apr 06, 2010 at 08:34 AM

    Hi Midhun,

    Since this is a FTP, the best solution will be to let the process which create the file to rename it to any other extension in the end or move to some other directory.

    If this is not possible you may need to write an adapter module which will check the file size at every poll interval and if there is no increase then pick it else need to wait for next polling cycle

    Regards

    Suraj

    Add comment
    10|10000 characters needed characters exceeded

  • avatar image
    Former Member
    Apr 06, 2010 at 08:58 AM

    Hi Midhun,

     I have a File 200MB. I can neither break it into chunks nor increase the poll interval

    200 MB is quite big for PI,and processing it every 60 secs ... its will be a great performance hit for PI server.I guess even if you are allowed to increase the plloing time , this may lead to timeout again due to this huge size.

    If there are no mappings required, Why not give a try to Managed file transfer(RMFT etc), this will save PI from the burden of huge files to be processed every 1 min as you mentioned.

    Please refer this for analysing the performance which PI can handle.

    /people/sravya.talanki2/blog/2005/11/29/night-mare-processing-huge-files-in-sap-xi

    Regards,

    srinivas

    Add comment
    10|10000 characters needed characters exceeded

  • Apr 06, 2010 at 08:27 AM

    Hi,

    You can try this one. See if your file adapter is configured to pick up a file with the name say abc.txt then what you can do is that you can put the file with the name Zabc.txt. Then you can rename the file back to abc.txt using unix shell script and oss command.

    Thanks

    Prasanna

    Add comment
    10|10000 characters needed characters exceeded