SFTP Duplicate file check not working

[harish_babu5]

HI all,

I have an SFTP to ECC scenario. I am trying to avoid duplicate file processing in the sender SFTP channel. I checked Duplicate File check under processing tab. I also added the parameter "duplicateCheckPersist" under Advanced Tab and entered the value 2160, for 2160 minutes which is 14 days.

I placed a file with the name "TEST-30Nov2017-1234.gpg". It got processed in a minute. I again placed the file, but it was also processed. I restarted the application com.sap.aii.adapter.sftp.app from the NWA and placed the same file and it got processed again!

I checked the SFTP adapter from system info and the details are 1000.1.0.4.18.20160129043500. So it's SP4 and patch 18. It should be ideally supporting this feature.

In the Advanced Tab, I have checked only Set Adapter-Specific Message Attributes and Filename. Directory, Filesize, Timstamp all are unchecked.

Also, the file is deleted after processed and archived in a folder. If it's in error state then it's kept in another folder.

Some other parameters.

SFTP server Timeout (ms) is 30,000.

Poll interval 60 seconds

interval msec for modification check 100 seconds

Any idea why this is not taking care of duplicate processing?

Harish

[former_member182412]

Hi Harish,

Check the below document for duplicate handling:

NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

In your case the last modified time stamp is changing every time you placing the file so the adapter is treating as new file and processing the file.

Regards,

Praveen.

[harish_babu5]

Hi Praveen

The name of the file placed is ABC_20180105_183029.gpg. After ten minutes, I placed the same file name with the timestamp value same and it is getting processed.

183029 is the timestamp value added to it.

Or do you mean to say that the next time we place a file, the timestamp of the new file will be different, which is not part of the file Name? And PI considers that? Then this will never work, I guess..

Any way I can solve this?

[harish_babu5]

HI Praveen

It's a multimapping scenario. I now understood why it is not working. It's an SFTP to Idoc scenario and also there are two mappings, both multimappings with 1:n. Dynamic configuration doesn't work on Multimappings and hence the SFTP duplicate check is not working

Harish

[former_member368318]

Hi, Is there a way not to compare the file timestamp when activating the check duplicate parameter? Just the filename and size?

Thanks.

[PRAGSMATIC]

what is the version of your SAP PI ?

[harish_babu5]

The PO is single stack, 7.4 SP 11

PIB2B SFTP version is 1000.1.0.4.18.20160129043500 - Service Pack 4 and patch level 18

Regards

[PRAGSMATIC]

Our version is 7.5 and Service Pack is 05

Duplicate Check works on our PI with SFTP. I tested it this morning. When a file is duplicate, it gives error in the channel morning that dupicate files is received.

[harish_babu5]

Since it's working for you can you pelaase tellme the configuratoions you did? Did you give any value for duplicateCheckPersist? in ASMA tab, which all did you check?

Harish

[Former Member]

Hi Harish,

Was this issue fixed for you?

I am also facing similar issue with Sender SFTP but we are using Advantco SFTP Adapter. We initially had all the 3 duplicate checks enabled in the channel viz. 1. File Path AND 2. File Modification Time AND 3. File Size.

But due to some issue with the SFTP itself if file got modified for any reason above check failed and file was processed again.

To fix this Advantco recommended to just keep the File path as duplicate check- it worked fine and gave 'Duplicate Found: Source File 'abcxyz.txt' is already processed ' error. But not for all the files and one file did get processed again within 2 minutes of file getting processed initially.

Please suggest how you fixed the issue at your end, i understand you are using SAP SFTP Adapter, i may consider that option as well for my interface.

Thank You!

Regards,

Ashwani

[juan_vasquez2]

Hi Harish

Please verify this parameters, check filename, directory, size, host, timestamp, host

other good idea from this URL

https://archive.sap.com/discussions/thread/3694522

Regards

[harish_babu5]

Hi Juan,

Thanks for the suggestion!

I checked the boxes File Name, Directory, File Size, and Timestamp, and placed a file named DC-2017Aug31-3114.gpg in the SFTP source folder. PO processed the file. After few minutes, I placed the same file in the folder and it was also processed. I have not checked the SFTP Host.

We took the file from archive folder and placed it there but duplicate check is not working.

Processing parameters.

File is deleted. Error or faulty files are archived in one folder and successfully processed files are kept in another folder.

duplicatecheckpersist.png

Any idea why it is not happening?

Regards

Harish

[harish_babu5]

praveen.kumar49

Can you suggest some inputs?