SAP SFTP adapter -- interval for modification check

[Former Member]

Hello

we are using the new SFTP adapter as sender. In the Processing tab, there is a mandatory field Interval(mSec) for modification check. I really don't understand what it means, from SAP help:

This parameter defines the sender channel to poll the directory at regular intervals and checks for file modifications. The SFTP adapter polls after every interval and checks for the last modified time and size of the file. The criteria for the adapter to process the file is given below: •If the file has been modified, then it is not processed. •If the file is not modified, then it is processed. The default value of interval is 100 ms.

why would we need this parameter? There is a parameter Poll Interval which indicates how often to poll the file. why would the adapter keep polling the directory every 100 ms for modification? for what business sense would this parameter be required? I just can't understand why it is needed.

In fact, my SFTP sender adapter failed because of this, I have no idea how to fix it as it is a mandatory parameter.

Thanks
Jayson

[vadimklimov · Accepted solution]

Hi Jayson,

SFTP and File/FTP adapters can use modification check in (polling) sender communication channels in order to ensure that PI doesn't attempt to poll the file that is still in progress of creation / update by the sender application. This may be especially valid for large files or for sender applications for which the complete file create / change operation takes some time. For example, the sender application starts creation of a large file and after few seconds after this, PI starts polling. The file is not completely created, it is still in progress of creation - so even though it already appeared in the directory from where PI polls, PI should not really retrieve this file since it is not yet complete. When the sender application completes creation of the file, last modified timestamp will not be changed anymore (assuming the file is not changed subsequently) and PI can now safely poll this file. Technically this is done in the following way: as soon as PI finds the file in the directory from where it polls, it checks its last modified timestamp and waits for time period specified in modification check configuration in the communication channel. After this time elapsed, it again checks last modified timestamp of the file and compares new value with the old. If these two timestamps are the same, this means the file wasn't changed modification check wait time, and PI can poll the file safely.

If we don't have modification check in place in the communication channel, PI will try to poll this file and may even try to poll the file in the middle of its creation by the sender application, right when PI polling attempt time comes - which may result errors.

Whereas polling interval specifies how often PI will execute the polling attempt for the given sender communication channel (for example, list directory and poll appropriate files), modification check specifies how much time PI shall wait after first finding the file and before attempting to poll the file.

Regards,

Vadim

[Former Member · Accepted solution]

Make perfect sense. Thanks