cancel
Showing results for 
Search instead for 
Did you mean: 

High volume - FTP sender adpater

Former Member
0 Kudos

Hi,

We are migrating our PI application and our server will be down for 2-3 days. However out third party is working & will accumulate the file for us.

Once the PI is live, we are seeing at least 15-20,000 XML  files  ( each containing good amount of data) coming via PI FTP adapter route.

Since this already live system, we will not be allowed to change any major setting change.

How should we handle this high volume coming into PI via File adapter, since one the channel is opened all 15K files would be rushing to PI

Also how does the file adapter behaves, if the lets say in poll time of 60 sec, it is not able to finish the job.

Would it fail, asking to reprocess or it would get delayed and wait till the batch finishes before starting new Polling.

I am thinking of increasing the polling interval but still not clear what will happen in the first polling.

Thanks & please advice.

Accepted Solutions (0)

Answers (2)

Answers (2)

Former Member
0 Kudos

Hello,

I hope u don't have to maintain any kind of serialization for those incoming files?

>>Since this already live system, we will not be allowed to change any major setting change

I would suggest you to increase the number of adapter threads so that more messages can be processed simultaneously - search SDN u will find how to increase that.

Thanks

Amit Srivastava

Former Member
0 Kudos

Hi,

Thanks for the reply, Is there a way i can speed up this process, like putting more threads etc..

Please advice.

Former Member
0 Kudos

Hello,

IMO, u should certainly increase threads and that's the option i can see as of now.

Thanks

Amit Srivastava

Harish
Active Contributor
0 Kudos
Former Member
0 Kudos

Thanks all for the reply. I am looking for more specific on File adapters. We are expecting more that 20K messages stuck up when we restart the server.. What would be best strategy in this case to avoid blockages etc  for File sender & Receiver channel ( FTP to NFS)

Former Member
0 Kudos

Hello,

AFAIK, at file adapter level apart from increasing worker threads u cannot do much, but do remember adapter is not the only step which process ur messages, there are different processing steps (Depending upon ur type of config like ICO/Classical config) and u think of tuning them also.

So u need to enhance ur search skills to find our relevant blogs and optimize ur interface.

Some of the pointers are given below:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/2016a0b1-1780-2b10-97bd-be3ac6221...

Thanks

Amit Srivastava

Harish
Active Contributor
0 Kudos

Hi,

Adding to Amit's reply, you can also tune/enhance the performance of File adapter with configuration. try the below setting (if it is applicable for your configuration) to enhacne the performance.

  1. General timeout = 30 to 60 seconds (must be much less than the polling interval).
  2. taskTimeout = approx 120 seconds (should lie between general timeout and polling interval).
  3. Per File mode should be the normal scenario.
  4. Passive mode.
  5. Maximum concurrency = 1.
  6. poolWaitingTime = 30000.
  7. clusterSyncMode = lock

regards,

Harish

former_member184720
Active Contributor
0 Kudos

I don't think file adapter(sender) will create some issue in processing the files. Instead you should think of your receiver adapter

If your receiver is JDBC/Soap/FTP then it might cause concurrency issues.

The process time may be a bit longer. They might wait in Queue's(ABAP)/ they will in delivering state on the adapter engine for longer duration which depends on your target system.

I also don't think increasing the polling intervel will address your issue. As soon as you start your file channel, then it starts processing all the files.

However if you want to go for control processing then, instead of placing all the 15k files altogether process in batches.