Bottleneck in Large file processing

[Former Member]

Hi,

We are experiencing timeout and memory issues in large file processings. I want to know wheather J2EE adapter engine or Integration engine is the bottleneck in processing large messages like over 300 MB files without splitting the files.

Thanks

Steve

[moorthy]

HI,

Timeout can happen because of mnay reasons.btw where exactly timeout happens in your case ? Is it mapping/delivering?

1) As you mentioned huge message

2) Design approach(mappings used etc)

3) Memory settigns/Hardware sizing- for this refer-

https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/4dc9381e-0a01-0010-c692-f6b...

My personal opinion is to split the file and procedd further.

Regards,

Moorthy

[Former Member]

Moorthy,

Thanks for the reply. We are currently splitting the files and processing. We want to know what is the maximum file size XI can handle without using IR/Mapping.

Also we are trying to understand where is the bottleneck in this case. We think this is J2EE adapter engine, but we would like to hear from others to confirm.

Thanks

Steve

Message was edited by: Steve Holland

[moorthy]

Hi Steve,

Just look into this SAP Note- 807000

Hope this helps,

Regards,

Moorthy

[Former Member]

Hi Steve,

this is just an assumption:

Adapter Engine = JAVA (just byte code)

Integration Engine = ABAP (fully compiled)

So, I guess the Adapter engine is the bottle neck.

If you are able to make a test, here is suggestion:

Send an IDOC (IDOC adapter & http adapter reside on ABAP stack) nearly same size as your message, that you send over the J2EE-adapter engine and compare!

Regards Mario

[Former Member]

Hi Mario,

We are testing a scenario to find out what is the maximum file size that XI can handle based on the blog

( /people/william.li/blog/2006/09/08/how-to-send-any-data-even-binary-through-xi-without-using-the-integration-repository) without any mapping. Upto 20 MB it works Ok and after that we are getting timeout error .

Data from Moni:

com.sap.engine.services.httpserver.exceptions.HttpIOException: Read timeout. The client has disconnected or a synchronization error has occurred. Read [1704371] bytes. Expected [33353075]. at com.sap.engine.services.httpserver.server.io.HttpInputStream.read(HttpInputStream.java:186) at com.sap.aii.af.service.util.ChunkedByteArrayOutputStream.write(ChunkedByteArrayOutputStream.java:181) at com.sap.aii.af.ra.ms.transport.TransportBody.<init>(TransportBody.java:99) at com.sap.aii.af.ra.ms.impl.core.transport.http.MessagingServlet.doPost

This could be due to ICM timeout settings which we are planning to increase.

I would like to hear from others experience of maximum file size that they could process. Ofcouse I do know that it depends on the environment.

Thanks

Steve