Error with huge file

[Former Member]

Hi,

I have uploaded a fixed field length file (75MB). The file is getting picked from the directory but it doesn't post any message in Integration Engine.

In Adapter Engine monitor I see this error:

Transmitting the message to endpoint http://<host>:8000/sap/xi/engine?type=entry using connection File_http://sap.com/xi/XI/System failed, due to: com.sap.aii.af.ra.ms.api.RecoverableException: java.io.IOException: Error writing to server.

The same interface works perfect with small sized files.

Any help is highly appreciated.

Regards,

Anish

[ravi_raman2]

This might help.......

14. Memory Requirements

• Q: Which memory requirements does the File Adapter have? Is there a restriction on the maximum file size it can process?

• A: The maximum file size that can be processed by the File Adapter depends on a number of factors:

o The most important one is the size of the Java heap, which is shared among all messages processed at a certain point in time. In order to be able to process larger messages without an out of memory error (OOM), it is recommended to increase the size of the available Java heap and/or to reduce the concurrency in the system so that fewer messages are processed in parallel.

o Another factor negatively influencing the maximum message size in releases up to and including XI 3.0 SP 13 is an enabled charcter set (encoding) conversion if the message type is set to "Text".

o Using the transport protocol "File Transfer Protocol (FTP)" also uses more memory for processing than the transport protocol "File System (NFS)" (up to and including XI 3.0 SP 13).

o If the Message Protocol "File Content Conversion" is used in a File Sender channel, consider that not only the size of the input file affects the File Adapter's memory usage, but even more the size of the XML resulting from the conversion, which is usually a few factors larger than the original plain text file.

To reduce the memory consumption in this scenario, consider configuring the setting "Maximum Recordsets per Message" for the sender channel. This will cause the input file to be split into multiple smaller mesages.

Regards

Ravi Raman

Dont forget the points if it was helpful

[Former Member]

Anish,

What is your jvm heap size.........

Regards

Venkat

[Former Member]

Hi Venkat,

Where can I find the heap size?

Regards

Anish

[ravi_raman2]

Hi,

For this...i would check heap size on machine running xi client..ie your browser, and also check jvm heap size on xi server...

If windows try running java -X .........then i belive its

-Xms<size> set initial Java heap size

-Xmx<size> set maximum Java heap size

Regards

Ravi Raman

[Former Member]

What OS is running your XI Server...standard default is 1 gb for all except aix i think..........

Venkat

[Former Member]

Hi Venkat,

XI server is hosted on Linux server.

Regards

Anish

[Former Member]

Hi Ravi,

Could you tell me how to find the same on a Linux server?

I found a parameter for setting maximum heap size in RZ10 (vmcj/max_vm_heap_MB). That was set to 64MB.

I have raised it to 300MB. Waiting for the server to restart.

Regards,

Anish

[ravi_raman2]

Hi Anish.......

Linux..you have a couple of more issues..to keep an eye on....try the command below..

-Xms2g -Xmx2g..this bumps..the heap to 2gb...

There is also a kernel issue on some Linux machine where..a user process can't get more than 2gb memory, and the jvm has some overhead so 2Gb won't work either.

Regards

Ravi Raman

[Former Member]

Thanks Ravi.

I will try this.

changes in profile (RZ10) didn;t help.

Regards,

Anish

[ravi_raman2]

well..Changes in profile will not help..its a local os thingy.........might wanna check with basis on that..........

Couple of questions..Why are you changing values in profile parameters....if you do.....you want to do it in you visual admin..or on the OS itself as the jvm that allocates the heap runs on your linux box...right.......i can see how changing heap size in rz10 will not help you in any way............Try changing it at the OS level also check for your kernel patch......

Dont forget the points if it was helpful

Regards

Ravi Raman

[Former Member]

If I remember correctly, the vmcj/max_vm_heap_MB is related to the sizing of the integrated VM of the 7.x kernel (IPC is running in it for instance), so increasing it won't increase heap size for XI's J2EE (ConfigTool can be used to adjust corresponding settings, equivalent to -xms -xmx jvm settings) ...

You can also have a look at XI's ICM traces to see what happens (error codes, timeouts etc), that could help you finding the source cause of this issue

Anyway to reduce file size (I do not know whether it would be possible to implement a (g)zip compression at FTP adapter level, or similar) ?

Rgds

Chris

[Former Member]

Hi All,

I have changed the heap size and did some tweaking in mapping as well.

The interface works well.

Thanks All....

Anish

Edited by: Anish Abraham on May 8, 2009 1:00 PM

Edited by: Anish Abraham on May 8, 2009 1:01 PM

[Former Member]

Closing the thread!!

[Former Member]

Anish:

Are you XI as FTP client transferring files from one place to another using ID or using some mapping also. I think XI is suffering from the file size to transfer. I have done upto 20-30MB files with out any mapping and using ID but 75MB seems tough for XI. May be you can make chunks of file if you are not using the mapping or reduce the number of records per message if you are using mapping. Try also to increase the buffer size of XI with help of XI admins

[Former Member]

I'm using NFS

[Former Member]

Hi,

To give more information I'm working on PI 7.0 SP 13

Regards,

Anish