How to process a large XML file as source??

[Former Member]

Hi,

i have requirement as source is XML file (having 10000 service orders) needs to be processed through XI and post idoc to CRM. Can i do it using direct file adapter?

will it cause any performance issue??

Note: each service order has header,trailer and 'n' number of item records for each order.

Thanks,

Satheesh

[nabendu_sen · Accepted solution]

Hi Satheesh,

Please find the below blog:

http://scn.sap.com/community/pi-and-soa-middleware/blog/2010/10/18/pixi-pi-73-processing-of-large-fi...

Large File Handling

In transaction code SXMB_ADM, Integration Engine Configuration, you can maintain parameter

EO_MSG_SIZE_LIMIT of category TUNING to process large messages in series. This applies to any

kind of messages however especially files usually exceed the best performing message size, and

hence this is mentioned here. Once set, all messages exceeding the specified value are processed in

series in a separate message queue.

Recommendation

Maintain the parameter in order to avoid that parallel processing of multiple large

messages exceeds main memory resources. Furthermore, since large messages are

processed in a separate queue the processing of smaller messages won't be affected or

even blocked.

For large text files containing multiple records, you can split the same into multiple messages in the

file/ftp adapter. This applies when File Content Conversion mode is chosen. In the communication

channel, you have to maintain parameter Recordsets per Message.

For more details, please refer to SAP Help Portal http://help.sap.com, navigate to SAP NetWeaver 7.0

 SAP NetWeaver 7.0 Library  SAP NetWeaver Library  SAP NetWeaver by Key Capability 

Process Integration by Key Capability  SAP NetWeaver Exchange Infrastructure  Runtime 

Connectivity  Adapters  File/FTP Adapter  Configuring the Sender File/FTP Adapter 

Converting File Content in a Sender Adapter.

Reference: http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/2016a0b1-1780-2b10-97bd-be3ac6221...

[Former Member · Accepted solution]

Thanks a lot for your quick response.

i read that blog. is that applies to XML source also?

Why i'm asking is i'm really new to SAP PI and this is first time i got a XML source.

Also we can process the XML file using just FILE adapter(without any FCC) right?

Thanks,

Satheesh

[nabendu_sen · Accepted solution]

Yes, you can go where Source File is XML. Just dont mention FCC details in File adapter. Thats all.

[Former Member · Accepted solution]

okay fine. Thank you so much.

i will try it out !!

Thanks,

Satheesh

[Former Member · Accepted solution]

one more clarification i need here is,

if i got two options for above requirement like,

     1. source as XML file

     2. source as Flat Text File arranged in below hierarchy,

          order looks like,

          Header....
               item1...
               item2...
               item3...
               .
               .
          Trailer

which source is best as it has 10000 orders and processing is once a day??

Thanks,

Satheesh

[nabendu_sen · Accepted solution]

Hi Satheesh,

XML is always best because you are not performing any transformation, so processing time would be better. Otherwise you can also go for FCC for Flat File but maintain "Recordset Per Message" and put less number messages like 40-50.

[Former Member · Accepted solution]

okay thank you !!