FILE - SFTP : Bulk files

[former_member309357]

Hello Team,

      Thank you so much for helping me so far. Please suggest me on the new issue below :

      I have implemented an ESR by pass scenario FILE (NFS) - SFTP in which we want to pick and place around 30000 files at a time.

      Please let me know do we face any issues with the existing default configuration or need to do any changes from sender and receiver ADAPTER's end in parameter tab.

      Kindly provide your valuable inputs and many thanks in advance.

Thanks

Ravi.

[former_member309357 · Accepted solution]

Dear All,

    The issue is resolved , please find below resolution :

    The problem persists with SFTP server, It can handle only upto a size of 4GB. Whatever the files sending beyond the size are failing in PI. Increased the size from their end and successfully re-sent all the files from PI.

Thanks

Ravi.

[former_member309357]

Hello Team,

     Could you please give your valuable inputs on the above SFTP receiver adapter issue. Even trying with password authentication getting the same error. I had tried re sending even after full cache refresh still the error.

Thanks

Ravi

[iaki_vila]

Hi Ravi,

The target system, where the SFTP is allocated, can disconnect the connection if this spends so much time. Therefore, as the problem is in Production environment and you can't test this problem with Filezilla or another application similar, you only way is to talk with the target system administrators if they have a firewall or similar that it makes the disconnection.

Hope this helps.

Regards.

[former_member183908]

Hi Ravi,

30K Files at a time?Are you using wildcard for the filename?What will be the size of the 30k files?Are you which version of SAP PI.As you know we have the limitation of handling file size in PI it can be with ESR or bypassing ESR.

May be all the 30K files are taking at a time into the Java memory and it can process only some files with in the memory limit and the sender channel is going for the error.

I have faced similar issue with ADVANTCO sender SFTP in 711 when i am trying process more than 100MB file size which has more than 1million records with ESR however the java cluster node bounces because of resources consumption.

Thanks

[former_member309357]

Hi Pavan,

    Yes we processed 30K files at one run. But initially only 5K files got success and rest were failed. Later after two day I tried sending rest of the files in which 8K got success and rest failed. And later two days i.e., today when I tried to re send rest of the files, getting the error which mentioned.

Delivering the message to the application using connection SFTP_http://sap.com/xi/XI/SFTP failed, due to: com.sap.engine.interfaces.messaging.api.exception.MessageException: javax.resource.ResourceException: 4: Putting/XXXXXXX failed"

Thanks

Ravi

[former_member257758]

Ravi,

have you tried to change(increase) the SFTP Channels properties under processing tab? Properties like Maximum Concurrency,Timeout,Time unit.

I hope it will help once you increase those parameters.

[former_member309357]

Hi Arun,

   I have maintained maximum concurrency, time out to 60 seconds but still the issue

Thanks

Ravi

[former_member184720]

AFAIK - No need to change anything.

If your FTP server is capable of handling  parallel connections then try increasing the maximum concurrency in case you face "maximum concurrency error" when you are processing 30000 files.

[former_member309357]

Hello Hareesh,

          As per the above request, we have tested by sending around 36000 files from ECC - PI - SFTP. In which around 6000 files were successfully sent to 3rd party and rest of all the messages were failed with the below error.

"Delivering the message to the application using connection SFTP_http://sap.com/xi/XI/SFTP failed, due to: com.sap.engine.interfaces.messaging.api.exception.MessageException: javax.resource.ResourceException: 4: Putting/XXXXXXX failed"

         The issue I am facing is in PRODUCTION system, could you please help me resolving this.

         Many thanks in advance.

Thanks

Ravi

[former_member184720]

What happens if you resend? Are they getting processed?

[former_member309357]

I had tried re-sending, still getting the same error.

[JaySchwendemann]

Activate XPI-Inspector for the receiver channel you are using (template 50, give channel name) and re-send one. Deactivate and examine log. Share your thoughts here 🙂

Cheers

Jens

[former_member184720]

May i know the file name? I noticed similar issues when the file name exceeds the length than allowed.

As suggested by Jens, Please make use of xpi inspector to trace down the issue further.

[former_member309357]

Hi Hareesh,

       The file is an excel file with 27 characters and we have pdf files too which were failed.

Thanks

Ravi

[former_member184720]

What does the xpi_inspector log say?

Did you try uploading the file manually with same name?

[former_member309357]

Hareesh,

    We did not use xpi_inspector.

    As it is in the production system, we cannot upload the same file again.

Thanks

Ravi