Performance File/FTP adapter

[Former Member]

Hello all,

I've got the following problem.

For a lot of interfaces I use the file/ftp adapter.

One of them processes periodically a lot of files in a short time period, for example 3000 files in 20 minutes.

This interfaces consists of 6 message types. Each message type has it's own communication channel (ftp).

The messages are appended into one file (plain text) on the FTP server with a fixed file naam. One file for each message type.

The processing of a lot of these files takes a lot of time, for example 90 minutes for one message.

They are waiting a lot of time in the adapter queue (messaging system) before they are processed by the ftp adapter..

I think this is caused by the 'append step' wthin ftp.

Question (problem) is how I can increase the performance.

I already know you can change (raise) the parameter 'Maximum Concurrency' in the communication channel. For example value 2 in stead of 1. Only I'm afraid this causes locking problems because the files have to be appended one by one. But I'm not sure about this.

My questions are:

- can I use the parameter Maximum Concurrency in this case? If yes, what value should it have to increase the performance substantially?

- what other options do I have to increase the performance of the file/ftp adapter? (adding an extra server node is no option)

I hope anyone can help me with this.

Thanks

Regards,

Marco

[Former Member]

Hi Ivan,

Sorry for my late reply I had to leave yesterday.

Thanks a lot for your help so far!!

I still have a question about the Maximum Concurrecy parameter in the communication channel.

The interface appends file to one file on the FTP server.

When I raise this parameter (1->2) to 'activate' parallel processing then I'm afraid theere will be a locking problem, because in one interface there are now two parallel FTP connections that append to the same file on the FTP server.

Is this a risk or not?

Other question: are there any guidelines on how to set the parameter Maximum Concurrency? For example technical limitations.

Regards,

Marco

[Former Member]

Hi Marco,

Pls, for more technical information see sap notes: 821267 and 1136790.

Regards

Ivan

[Former Member]

Hi Ivan,

Thanks again for your reply.

- What does the parameter Maximum Concurrency on the FTP server do?

- Is parallel processing a solution is this case or not? Because I have to append the files into 1 file one by one.

Is there a locking problem when choosing for parallel processing?

- you suggest to look at the performance of the FTP server itself? where should I look at?

Regards,

marco

[Former Member]

Hi,

1. Configure the maximum number of connections which can be opened simultaneously.

2. Effectively parallel processing is a good option for this case.

3. A locking problem could be generated in you FTP Server. But the server in only for PI, i think there is no problem.

4. Use a ftp client to test the performance of your ftp server(i.e Filezilla).

Regards

Ivan

[Former Member]

Hi,

please check the link for perfomance tuning:

https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/2016a0b1-1780-2b10-97bd-be3ac622...

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/423f5046-0a01-0010-2698-b2dc7c318...

Hope this helps!

Kai

[Former Member]

Hi Ivan,

Sorry something went wrong in my last reply, I send it quickly.

In the audit log of the message in the adapter engine there is a lot of time (about 90 minutes) between the steps:

message succesfully put in the queue AND

message succesfully retreived from the receive queue

The FTP process itself (including append action) takes about 5 / 6 seconds per message.

Regards,

Marco

[Former Member]

Hi Ivan,

Thanks for your fast reply.

Yes I evaluated this. This problem is in the adapter engine.

The audit log of the message in the adapter engine shows a big time period between the steps:

-

[Former Member]

Hi,

Not always the problem is in the PI, I advice you check the configuration of your FTP server. Maybe you need to increase the value if Maximum Concurrency in your FTP Server.

Then You can try use Maximum Concurrency you have in PI.

Regards

Ivan

[Former Member]

Hi,

Have you evaluated if the problem is in the adapter engine(CC) or in integration engine(MM)?

Regards

Ivan