How to split a message into chunks in a file to proxy interface

[Former Member]

Hi, all

We have an file to proxy async interface. The content of the file is straight forward but with 400K lines and 50M in size. Each line is a record. Is there a way to split the file into multiple chunks and send the receiver system(ECC) multiple messages. Since there is no sequencing in the data, we can process the data in parallel in ECC if possible.

There is a feature called chunkmode for the file adapter. However, it seems to me that the receiver needs to be file adapter too. It also do a file combination in the end which is not what I want. I also tried the "recordsets per message" and it seems to do a combine when the message gets to the receiver.

Can you please advise?

Thanks,

Jonathan.

[former_member184720 · Accepted solution]

>>>I also tried the "recordsets per message" and it seems to do a combine when the message gets to the receiver.

This should work and this is the best possible option. what do you mean by combine? It should be executed as a separate message when it hit's receiver system proxy.

Is you FCC splitting the file into multiple messages in PI? If not please check your FCC configuration.

[Former Member · Accepted solution]

Hi, Hareesh.

Here is file adapter config:

The test file has about 1000 records, but from PI(PIMON) or ECC(SXMB_MONI), I only see one message created. I am expecting to see 20 different messages either in PI or ECC. It seems like somehow PI merges the data into one message before sending it over to ECC. Actually I am not sure about that as I see no proof PI has done any "splitting".

If I use chunkmode, the same thing happens and I have to be in EOIO with a queue. That is not what I want to begin with.

Thanks,

Jonathan.

[former_member184720 · Accepted solution]

As you have configured LEAVE,* all you records will be under 1st record-set. so technically you'll always have just one recordset.

What you can do is configure the record set structure as LEAVE,50 or something so that in each record set you'll have 50 LEAVE segments.

then set the recordset's per message to 1.

I assume this is what is happening... you can give a try and share your findings..

[Former Member · Accepted solution]

Hi, Hareesh.

It works! I have never thought of using those parameters that way. Thank you!

Regards

Jonathan.

[RaghuVamseedhar]

Jonathan,

Please check my wiki.

Input can be text. Output can be proxy. Tailor the mapping logic. Add replace logic before constructing XML replaceAll("<","&lt;").replaceAll("&","amp;"). Let me know, if you want guidance with splitting logic.

Multi-Mapping using Java Mapping - Process Integration - SCN Wiki

[Former Member]

Hi, Rahu

Thanks for the link and I will definitely try that out.

Regards

Jonathan.

[former_member752836]

Hi,

I loved your blog, but I'm having hard time to understand how I can utilize with my requirement. I am in need to split my large proxy data into chunks before sending it to receiver. This has to be a synchronous call. PROXY <-> REST.

My doubt is how to use Java mapping along with graphical mapping. I know there is another doc available in your blog but urfortunately I am not able to open that.

Regards,

Surabhi