Problem while processing large files

[Former Member]

Hi

I am facing a problem while processing large files.

I have a file which is around 72mb. It has around more than 1lac records. XI is able to pick the file if it has 30,000 records. If file has more than 30,000 records XI is picking the file ( once it picks it is deleting the file ) but i dont see any information under SXMB_MONI. Either error or successful or processing ... . Its simply picking and igonring the file. If i am processing these records separatly it working.

How to process this file. Why it is simply ignoring the file. How to solve this problem..

Thanks & Regards

Sowmya.

[Former Member]

Sowmya,

72 MB file is a very heavy load. First you need to check if your XI hardware is properly tuned or sized to handle such heavy load.

Did you check Component monitoring or Adapter Monitoring in Runtime Workbench. The file picked would have errrored out at the Adapter engine.

Even after tuning/scaling XI server,as suggested you should consider reducing the size of the file by splitting it in to smaller chunks.

Regards,

Sudharshan N A

[Former Member]

Hi,

Chech these blogs...for dividing your large files...

Check this blog for huge file processing

/people/alessandro.guarneri/blog/2007/02/21/sap-xi-acting-as-a-huge-file-mover

/people/william.li/blog/2006/09/08/how-to-send-any-data-even-binary-through-xi-without-using-the-integration-repository

check this one also..

https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/4dc9381e-0a01-0010-c692-f6b...

Thanks,

Vijaya

[Former Member]

Hi,

IF U want to process large file..you have to spilt using operating system command(ex UNIX SCIRPTS).Then use ZIP concepts using Modules.Use Payloadzipbean to achive.

it is helpful to u

Regards

[Former Member]

Hi,

XI pickup the Fiel based on max. limit of processing as well as the Memory & Resource Consumptions of XI server.

PRocessing the fiel of 72 MB is bit higer one. It increase the Memory Utilization of XI server and that may fali to process at the max point.

You should divide the File in small Chunks and allow to run multiple instances. It will be faster and will not create any problem.

Refer

SAP Network Blog: Night Mare-Processing huge files in SAP XI

/people/sravya.talanki2/blog/2005/11/29/night-mare-processing-huge-files-in-sap-xi

/people/michal.krawczyk2/blog/2005/11/10/xi-the-same-filename-from-a-sender-to-a-receiver-file-adapter--sp14

File Limit -- please refer to SAP note: 821267 chapter 14

Thanks

swarup

Edited by: Swarup Sawant on Jun 26, 2008 7:02 AM