File Adapter Not handling High Volume data!

[Senthilprakash1]

Hi,

I did a Flat file to File Scenario in dev server. It was working very fine. But when i moved it into PRD, I found that it was not picking up the flat file from the sender location. The sender File Adapter is not picking the file.

I guess the reason may be because of the size of the flat file. I found that the size of the file which the sender is giving is around 120 MB

Is File Adapter not capable of handling this much high load?

we are using PI 7.0 with SP13.

If the adapter is not able to accomodate this much volume. then whats the solution we have?

We have to configure i guess JAva Proxy for ddirectly putting the data to Integration Engine.

HELP:

1. Is there any solution to make my file adater pick this much volume of data?

2. If have to do Java Proxy can anyone help me as i am not a Java developer.

Thanks in Advance.

Regards,

Senthilprakash

[Former Member · Accepted solution]

Hi Senthil

HELP:
1. Is there any solution to make my file adater pick this much volume of data?
2. If have to do Java Proxy can anyone help me as i am not a Java developer.

You can increase the size of message

Go to SXMB_ADM ->Configuration of the integration engine -> Tuning ->

parameter : EO_MSG_SIZE_LIMIT

change the current value.

With this using File Content Conversion as the Message Protocol and then made an entry under Recordsets per Message to split an input file into several messages.

You can develop adapter module which can split file in equal chunks and pass the binary file data to PI as separate messages.

Thanks

Gaurav

[Former Member · Accepted solution]

Hi Masters,

I am working File to File Scenario, working fine for below 10 MB file. But I am getting 40 MB file, its failing.

Is there any chance to process the 40 MB file from Source to Target file..

I check the SDN form, its say that Record set per message, we can do.

I increase the size of message

Go to SXMB_ADM ->Configuration of the integration engine -> Tuning ->

parameter : EO_MSG_SIZE_LIMIT = 40000

But its not workingu2026.

Thanks,

[VijayKonam · Accepted solution]

In order to process the file, XI has to first read it. Then convert it to XML, then process. It takes time. May be it is trying to read the file and at the same time resources got full and the system might have entered a stalled status.. Check with your BASIS people to find out whats happenning with the system.

VJ

[henrique_pinto · Accepted solution]

If your input is flat file, have you considered to set the "Recordsets per Message" option with a proper value, so that the file adapter will create multiple xml files (which would not impact XI).

You can try to define the number of records which will give you around 10 MB per file.

Regards,

Henrique.

[Former Member]

Hi,

Have you tried the option Recordset Per Message in sender FCC as suggested by Henrique in his reply.

Regards,

Sarvesh

[Former Member]

Yes, I set up Record set per message = 4.

Its not working...

thanks,

ANU

[udo_martens]

Hi Senthilprakash,

the file adapter supports messages up to 2 Gig, but this is a very theoreticly value as the maximum is strongly dependend from hardware and Java stack configuration. I processed in praxis easyly 100 MB on a quite weak test machine without any problems (XI 3.0 / WAS with 2 Gig RAM).

You should try to find out the reason for that behaviour; if you have a performance issue Henriques tip to split the message with "Recordsets per Message" option is very helpful. Additional processing EOIO could help to avoid processing of high load in parallel.

Regards,

Udo

[justin_santhanam]

Senthil,

Did you went to Communication channel monitoring and checked what's the exact error thrown for this Sender Comm.channel?

raj.