Incomplete FTP read

[Former Member]

Hi,

In my File to Idoc scenario, when I am testing with big file, it gets failed sometimes. The reason being the file is read by the adapter even before it is completely ftped.

I have checked with "Msecs to wait before modification check" but it is not working in my case. I think it works only for NFS.

Pls let me know if there is any work around in XI for this?

regards

Anirudh.

[Former Member]

Hi,

Every file which has been written has eof i.e. end of file. So you can write an script and check for the end-of-file if found then only pick the file else do not pick.

Regard,

Sarvesh

[Former Member]

we can active/inactive the CC thr a link.. so after file gets ftp, then active the CC.

[Former Member]

Hi,

better to handle this within FTP Server level. Create a mother folder which holds all data and schedule a script which will transfer all files which are complete to the source dir from where xi picks up.

this solves the prob

if u know the File dump time then it is better to use CC schedule time

as said u need to adjust with the parties to go ahead

srini

[Former Member]

Hi Venkat,

How to check in shell script if the ftp is complete or not?

Can you suggest any code or link as I can't locate anything?

regards,

Anirudh.

[Former Member]

Hello Aniruddh,

I got this command from one of the blogs. Try to set it in advanced mode

'file.processingLocked=YES|NO'

you need to give the value as yes.

Am not sure whether this would be given in advanced tab or in file content conversion.

Try it in Advanced mode and add these value.

Regars

Krish

[Former Member]

Hi,

This is the best approach.

generally its better idea to get the file to a intermediate directory on same machine before actually moving to the directory being polled by the XI. This way once the entire file is written then it will be moved fast inside the same machine, so u dont get this kind of problems.

Createa shell script that moves from intermeditate drectory to actual directory, it should be fast and wont end up in a problem.

Regards

Krish

[Former Member]

Hi Krish,

I have tried this approach.

To my horror, it doesn't work as expected.

Suppose, CC is polling /tmp/BI

Now if I ftp the file to /tmp and in the middle of FTP if I mv the file to BI, it does gets moved and FTP continues without truncation to this new folder /tmp/BI.

I am now thinking of telling the sender party to ftp the file with tmp extension and then rename it to .txt after completing the ftp.

regards,

Anirudh.

[Former Member]

Hi,

Did you try this option.

'file.processingLocked=YES|NO'

[Former Member]

Hi ,

Check the size of the file continously and if the file size doesnot changed then copy it to the source directory from where xi picks up the file for ftp

see

http://www.unix.com/shell-programming-scripting/32809-have-shell-script-check-file-exist-before-proc...

hope it wud give u a idea. ...am not good at scripts

srini

[Former Member]

Hi Anirudh

You can write script to identify the end of file and then move it to another folder. Let the client system write file in one location and when it it complete you can write it to another folder where PI is polling

This even you can do for your .dat to .txt file script.

Refer

http://www.unix.com/shell-programming-scripting/44888-adding-new-line-end-file.html

Thanks

Gaurav

[prasad_ulagappan2]

Hi,

I have faced a similar kind of issue a long back. There is a function module in XI-ABAP Stack where you can increase the processing time for each file. Am not sure abt the module name, however you can search it in SE37.

[Former Member]

Hi,

You can also do one more thing.

Put the complete file with different naming convention. Change the file name into original file that should be picked by CC via a script.

These script you can specify in the Run OS command before processing.

So, by this the CC will pick up the complete file and the script will also rename it once the complete file is been processed.

Regards

Krish

[Former Member]

Hi,

I have a shell script which is moves .dat files to .txt file.

Now in CC I have specified *.txt file for processing. I have specified this script in this CC.

Now this shell script won't run till the adapter finds any .txt file. And I am ftping .dat files.

Nothing gets processed.

regards,

Anirudh

[Former Member]

Hi Krish,

I think it only works for NFS protocol.

How can XI file adapter with FTP connention execute a script on remote server?

crontab looks to be only alternative.

What if the ftp server is on windows? How to scedule the batch file then?

[Former Member]

Hi,

Necessarily it need not to be renaming.

Just when the adapter finds *.dat files, the script will run.

So, even a wait command inside the script or just ls-l or anyting would do.

Yes, you need to deploy the script in the FTP server, it wont run in your local XI.

If none of these are not working, why dont u schedule the CC and before scheduling deploy all those files.

Do let me know the volume and frequency of your interface.

Assuming it needs to be run twice make sure to activate the CC at that time and try to fix a time to put all those files.

Best strategy should be to agree on all these things with the relevant parties.

You cannot assume the files to be deployed at any time.

Regards

Krish

[Former Member]

The files has transaction data that will keep coming after every 15 mins.

I have used below script

bash-3.00# cat test*

for f in *.dat; do

base=`basename $f .dat`

mv $f $base.txt

done

But in this case also, if the file is in middle of being ftped, it changes the file name to .txt when this script is run.

It defeats the entire purpose

The files that I am ftping are a lot bigger. So ls -l may not help.

regards,

Anirudh.

[Former Member]

Hi,

Give me a broader view like when are you scheduling the communication channel and how often do u post the file.

Moreover, try to increase the interval length of FTP.

Regards

Krish