107 MB File - was not able to process thru File Adapter

[Former Member]

Hi All,

I tried to send one 107 MB file using file adapter.

But was getting error as "not able to write on the integration server".It seemed like XI was not able to process file with that much size...as when we did split up the file into smaller ones ..those were processed....

I want to know what is the file size limit of File adapter......is there any way to send files with that much size from file adapter?

Can we do any file sizing exercise on XI servers?

Thanks.

[Former Member]

20 MB should be the average file size for better performance. Exceeding this limit will certainly have some performance issues and cause memory leaks. If you are using PI(7.0) these issues have been addressed in the adapter configuration where you can specify the file size.

[Former Member]

Hi,

Yes it is failing at adapter engine and i.e java stack.

From the previous answers it sems that other people has also faced the same problem

but my question is ...does this mean that we can not transfer large files using file adapter?

or there are some settings changes or some tunig exercise needs to be done on XI servers?

How do people transfer large files in case of such requirement?

[bhavesh_kantilal]

One option, write a Shell Script that splits the file outside of XI and then drops it into XI's folders.

regards

Bhavesh

[Former Member]

Khade,

If you dont have any business logic then you can try to break the file into chunk of records. Check this weblog and see whether it helps:

/people/sravya.talanki2/blog/2005/11/29/night-mare-processing-huge-files-in-sap-xi

---Satish

[Former Member]

Hi Bhavesh

can u elaborate a bit about the approach you r referring to....

As far as my interpretion...u split the file and send those file thru multiple communication channels....then how are we going to append those ( file to file scenario)?

Secondly how are we going to trigger the shell script/batch file?

Regards

Akhil

[bhavesh_kantilal]

Akhil,

1. You use a single Sender File adapter with a QOS of EOIO ( if sequential processing needed ) and with Mask Characters for the filename which will be generated by the script.

2. The script should be scheduled at the OS level independent of XI.

Regards

Bhavesh

[Former Member]

Hi,

with Netweaver 03 -SP 20 in adapter parameters ,you can specilfy the file size(in Bytes) to be processed by adapter ,

try with it, if u have,

it may help you

[Former Member]

Hi Shweta,

Where exactly it is failing.

I hope it is at the Adapter Engine

In general, the file size is determined by lot of factors

1. Hardware

2. JVM Settings(Java Heap Size,Maximum no of threads,etc)

3. Number of parallel Queues of Integration Engine

4. Type of Mapping used

Please go through XI/PI File Adapter FAQ and JVM Settings/Tuning Notes in service market place.check SAP XI Tuning Guide also.

Regards,

Sudharshan N A

[Former Member]

Hi,

I am not using FCC but there is a mapping,so I can not zip the file.

Is there any way to avoid this?

[henrique_pinto]

actually there is a standard ZIP module adapter which you can use in your adapter to unzip/zip files within file adapter.

But again, it seems more like a performance issue. What is your system status during the processing? Does the pc has enough memory/processing?

Regards,

Henrique.

[bhavesh_kantilal]

Hi Henrique

Even the Zip Module doesnt help.

We had a similar case, when Xi was acting as a FTP. We thought we would use the Zip Module, etc but what we realise was even if you plan to use the Zip Module, the Entire File first needs to be loaded into the memory before the module can be called and the chances are that the j2EE stack goes down even before the module can be called.

We also explored the option of using the Jar command in the command line processing before message processing, even this has the same issue that , the Data is loaded into the memory of the j2ee stack and only then the command is executed and so even though the content was zipped, the actual content used by the file adapter was not the zipped content.

Regards

Bhavesh

[Former Member]

Hi Bhavesh,

<i>We had a similar case, when Xi was acting as a FTP</i>.

Just wondering why were you doing that?

Regards,

Sumit

[bhavesh_kantilal]

Sumit,

Client had a set of guidelines in place that stated any flow of data from One System to another has to happen via XI. Even if it is a mere FTP.

All systems just send data to XI, they need not know the details of the end system. There are multiple instances when though it doesnt make from XI's perspective such implementations do happen.

Regards

Bhavesh

[Former Member]

Bhavesh,

Thanks. Just wondering it must be costing them lotz of money :). To transfer files using XI.

Btw just found a t-code few dayz back..USMM. which shows the data transfer statistics.

Regards,

Sumit

[Former Member]

There is no file size limit for file adapter. Some servers process files of sizes bigger than 107 mbs too. If you are not using any FCC and no mapping then you can zip/uzip the file in the sender CC and receiver CC using the operating system level commands.

If you are using any FCC, then go through this link

/people/stefan.grube/blog/2007/02/20/working-with-the-payloadzipbean-module-of-the-xi-adapter-framework

Hope this helps,

Regards,

Ravi