Handling Large files in XI

[Former Member]

Hi,

I have a file-to-file scenario with some transformation rules in the mapping. both sides i have to do the content conversion as both input and output files are flat files. But the problem is file size, size is variaying between 300 MB to 4GB.

Could you guys suggest a better approach to deal with this situation.

I may not be able to go for "No of recordsets per message" option in content conversion as the files have a header and trailer and i need header information in the message mapping. Is client java proxy will be of use in this scenario.

Please share u r thoughts.

Pavan.

[prabhu_s2 · Accepted solution]

file size can be handeled in client java proxy and the message is parsed in xi. this will not have much of performance issue. but yes when the smae is handeled by xi. and yes this scenario can be efficiently handeled from java client proxy

[Former Member · Accepted solution]

Thanks for sharing your ideas. I have not worked with java proxies. I have the following doubts regarding java proxies:

if i implement a client java proxy, do i need to write a j2se application to invoke the java proxy ? can i schedule this application ?

if you say java proxy can handle the large files, how this is achieved ? do we send bunch of records each time from java proxy ?

Thanks

Pavan

[Former Member · Accepted solution]

Pavan,

yes you need to have java application to invoke the java proxy. java proxxy can handle large files .

If a standalone java system needs to send message to XI, client java proxy can be used.

In SLD, you need to create a techical system and business system for the standalone java system.

please refer following blogs.

Both the PDFs refers to Client java proxy.

https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/a068cf2f-0401-0010-2aa9-f5ae4b20...

https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/f272165e-0401-0010-b4a1-e7eb8903...

/people/prasad.ulagappan2/blog/2005/06/27/asynchronous-inbound-java-proxy

Regards

Sreeram.g.Reddy

[Former Member · Accepted solution]

Sreeram,

Thanks for the reply. my scenario is like: i will receive files (100 MB-4GB), need to process them and send them to target server. all these files will have different formats. here, there is no other application (like standalone java system) talking to XI.

my question is if i go for client java proxy (to split the file and process) how can i schedule the java program which invokes java proxy ? i need to have some kind of polling/listening to the specified file folder, so that's why i need a scheduling mechanism.

Thanks

Pavan.

[prabhu_s2 · Accepted solution]

u can have windows job scheduler to do this job i.e. scheduling or u can check this link:

http://www.theserverside.com/discussions/thread.tss?thread_id=28769

http://downloads.instantiations.com/CodeProDoc/continuous/latest/docs/doc/features/scheduler/schedul...

[former_member431549]

The "recordsets per message" can deal with headers and details in one file. That's what a recordset is, a header with it's associated details. But you will probably crash the AE with a 4 gb input file, so I would follow other's suggestion and look at Java proxy.

[Former Member]

Hi Pavan,

Check the following links:

https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/72e2bb90-0301-0010-e883-c39...

/people/pooja.pandey/blog/2005/10/17/number-formatting-to-handle-large-numbers

Amaresh

[Former Member]

Pavan,

then i would suggest to go JAVA proxies.