Performance problem due to large idocs

[Former Member]

Hi

We have an important problem of performance due to large idoc's sent to XI.

Many often XI process message with more than 10 Mbytes to 60 Mbytes.

As I saw in any SAP note, more than 10 Mbytes is not recommended.

This is an IDOC to FILE scenario , and split idoc and generate more than one file could be a good solution.

Do you know if there is a easy way to restrict number of segments per IDOC?

Can I split a too large IDOC in more than one IDOC?

Thanks, in advance

[ravi_raman2]

Carme,,

When you split an idoc ...how do you correlate between the different segments.....if there is an error.......................................note 1076808...

Take a look at these sap notes...for idoc performance

760993, 709400, 861721, 637391..

Regards

Ravi Raman

[LeonardoAraujo]

Have you considered using Data Filtering or IDOC reduction BD53?

See some documentation about it:

FILTER

Principle

A filter can be put in place on an interface. A filter is always placed at the distribution level (BD64 transaction), on a particular message type.

A filter is a series of values for one or many fields. When this filter is applied on a message, the segments are created only if the filtered fields contain those values.

Example :

Filter 1 : field MATNR = "123", "456"; field ATWRT = "FOO1".

Basic IDoc type ZIDOC containing a single segment Z1SEG.

The segment (MATNR = "123"; ATWRT = "FOO1"; FIELD1 = "VALUE") will be created.

The segment (MATNR = "123"; ATWRT = "VAL2"; FIELD1 = "VALUE") will NOT be created.

So a filter acts the following way: for an IDoc segment to be created, each filtered field has to contain one of the set values. It is a logical "OR" between each value of a field and a logical "AND" between each filtered field.

Be careful! If a segment has a field which do not pass the filter, it is deleted. Then, every segments of a lower hierarchical level are deleted too, as the higher hierarchical level segment also if the deleted segment has been set to be mandatory in the basic IDoc type.

A filter is often used to reduce the volume of sent data, because they are not all relevant for the interface to be built.

Just trying to reduce the size of your IDOC...

Leonardo De Araujo

[Former Member]

You could split the IDOC by modifying/extending the ABAP that generates it. If you try and split it in XI you will still have performance issues as it would share the same memory space. You may want to write code that calls an ABAP proxy instead, as you may find you get a lot more control over what is generated (and improve efficiency by reducing the xml message size). XML encapsulated IDOCs are huge.

If you scale your XI system with a minimum 2gb heap and keep mappings very simple, it might be able to handle 50mb messages. You need to make sure that trace and logging is kept to a minimum to reduce overhead as numerous copies of the document are kept in memory! You could try upping heap to 3gb but SAP dont recommend more than 2gb (garbage collection could be an issue).

I've hit problems with mapping large messages also, seems to be around the 30mb for me, but I have quite large and complex mappings which definitely has an impact.

Also do NOT have compression turned on in the RFC.

Cheers

James.

[Former Member]

Hi Carme,

As per my understanding if you first create the xml file, i mean let the IDOC data to be saved as xml file by using file port in R/3 and then finlly you can take this file in parts and send it to XI. For this you can take the help of below mentioned blog.

/people/sravya.talanki2/blog/2005/11/29/night-mare-processing-huge-files-in-sap-xi

Regards,

Sarvesh