SFTP Duplicate File

[itabhishek9]

Hi SDNites,

I wanted to put a check on duplicate files being processed while using SFTP adapter. I have used the checkbox Duplicate file checking but have few queries,

1. If I process and archive the file in first instance and the file with the same name again, should Duplicate file checking should work.

2. If I process and do not archive the file in first instance then I will keep on getting the error untill someone manually deletes the file. So is it mandatorily required to use ATP in these scenarios (But errors cannot still be avoided).

3. Duplicate error is found at Adapter level and not at the message monitor level. Can I get it at the message monitor level or is there any other centralised monitoring approach.

Regards,

Abhi

[itabhishek9]

Hi Raghu,

As mentioned by you that file has been archived after processing is fine. The overall objective of Duplicate File check is if the same file has been processed earlier, it should not be processed again (I expect that once file is processed, its details should be stored somewhere internally by adapter engine). But in my case it is being processed again and again despite the fact that I have same file name and parameter duplicate file check selected.

Regards,

Abhi

[former_member182412]

Hi Abhishek,

Check the below document for more about duplicate message handling in File adapter, this also same for SFTP adapter.

The duplicate file is identified by the combination of the following file properties.

Fully qualified path including file name (with extension)
Size
Last modified time stamp

If any of these properties gets changed, the duplicate count also reset and considered as a new file

1) If you manually place the file again the last modified time stamp will be changed and the adapter consider as new file and process the file without raising an error.

2) If you do not archive the file then the file last modified time stamp will be always same and file adapter consider as duplicate and will throw an error.

3) Duplicate file check is done at adapter level only.

Regards,

Praveen.

[former_member186851]

Hello Abhishek,

To handle the scenario your looking you need a module bean or OS command.

[former_member186851]

Hello Abhishek

1. If I process and archive the file in first instance and the file with the same name again, should Duplicate file checking should work.

It wont work as the file with same name has been processed already.

2. If I process and do not archive the file in first instance then I will keep on getting the error untill someone manually deletes the file. So is it mandatorily required to use ATP in these scenarios (But errors cannot still be avoided).

You can try using Thershold option.

3. Duplicate error is found at Adapter level and not at the message monitor level. Can I get it at the message monitor level or is there any other centralised monitoring approach

By using thershold option you can stop the channel and trigger an alert

[itabhishek9]

Thanks Raghu for the response.

Point 1 - If the file has been processed and archived, and the file with the same name is obtained again, based on your inputs it will not work which is what I have observed as well. But if this is th case then the fundamental of duplicate check will not be fulfilled. Please let me know your thoughts.

Point 3 - If I do not want to stop the channel, is there any other way around.

Regards,

Abhi

[former_member186851]

Hello Abhishek,

Its not wrong as it checks whether the file has been processed or not,Since after processing only the file has been archived.

I guess other mechanism for error is not possible,may be some os level could might work.Any other expert can help you on this.