How to break a huge payload into smaller chunks

[Former Member]

Hello PI gurus

I have an interface where I have used graphical mapping and using SOAP adapter.

Now the issue is the payload (thousands of records) that comes via this interface is too large and so the receiving application is having isssue.

So I have been requested to split this payload smaller chunks of 1000 records and send it as seperate messages one after another.

I was told that you can do it using XSLT mapping since there are some built in tools to split messages. However, I would like to know

if I can do the same thing in graphical mapping.

Your feedback or comment will be highly appreciated.

Thanks
Ram

[ambrish_mishra · Accepted solution]

Hi Ram,

Apart from the message mapping split suggested by Mika, if you provide details of the scenario like the source system/message (file, proxy), you can get more suggestions.

For example, if file, Recordset/message and with proxy, simply send chunks from the source system.

hope it helps!

Ambrish

[Former Member · Accepted solution]

Thanks Ambrish for the response. It is regular XML file sent via SOAP adapter (Both target and source use SOAP Adapters). Hope I answered your question.

[former_member207892 · Accepted solution]

Hi Ram,

Hope you are looking for splitting the source message to multiple messages to same target application.

Try by changing the target message signature in message mapping to 0..unbounded [multi mapping]

and using UDF, count the number of input records and split the same to 1000 records per message.

Step1:

source record node [context change to root node] -->count----->

                                                                                                      UDF2 --> Target message type

number of records to be per message split count     ------------->

Step2:

source record node [context change to root node] ------->

                                                                                             UDF2 --> Target record node

number of records to be per message split count -------->

UDF1:

==================================================================

public void SplitMessages(String[] RecordNode, String[] NoOfRecords, ResultList result, Container container) throws StreamTransformationException

{

int a = 0;

int b = 0;

a = (Integer.parseInt(RecordNode[0])) / (Integer.parseInt(NoOfRecords[0]));

b = (Integer.parseInt(RecordNode[0])) % (Integer.parseInt(NoOfRecords[0]));

     if(b > 0)

          {

                    a = a + 1;

          }

     for(int i =0;i<a;i++)

          {

               result.addValue(i+"");

          }

}

=================================================================

UDF2:

=================================================================

public void SplitRecords(String[] RecordNode, String[] NoOfRecords, ResultList result, Container container) throws StreamTransformationException

{

int x = 0;

int y = 0;

int z = 0;

int NoOfMsgCount = 0;

x = RecordNode.length / (Integer.parseInt(NoOfRecords[0]));

y = RecordNode.length % (Integer.parseInt(NoOfRecords[0]));

z= (Integer.parseInt(NoOfRecords[0]));

     for(int i =0;i<x;i++)

          {

                    for(int j = 0;j<z;j++)

                    {

                                   NoOfMsgCount++;

                                   result.addValue("");

                                   if(NoOfMsgCount >=z)

                                   {

                                             NoOfMsgCount = 0;

                                             result.addValue(ResultList.CC);

                                   }

                    }

          }

     for(int i =0;i<y;i++)

          {

                    NoOfMsgCount++;

                   result.addValue(NoOfMsgCount+"");

                              if(NoOfMsgCount>=z)

                              {

                                             NoOfMsgCount=0;

                                             result.addValue(ResultList.CC);

                              }

          }

}

=====================================================================

If the splitted messages need to be send to different receivers, based on the conditional routing rule in the interface determination along with additional receiver agreements the messages can be send to multiple receivers.

Please try and let me know if you have any issues.

Thanks!

Praveen.

[rajasekhar_reddy14 · Accepted solution]

Are you not experiencing any issues with JAVA stack while pushing data from WEB SERVICE.?

1)Best design is split xml file at sender system end and develop interface as pass through(no mapping).

2)If you have to implement mapping logic then use multi mapping concept using UDF you can slit records.

Thank you

[Former Member · Accepted solution]

Thanks everyone for the response

Praveen, your suggestion did the trick. I used your UDF code with your suggestion and that worked as per our requirement. Thanks a lot for the detailed instructions. I appreciate it.

Ram

[anupam_ghosh2 · Accepted solution]

Hi Praveen,

                   Brilliant code. Loving this.

Regards

Anupam

[Former Member]

Hi Ram,

I think you will find quite a many guides from SCN about how to do this.

Here's two links which I think will help you to do the splitting.

• http://scn.sap.com/people/claus.wallacher/blog/2006/06/29/message-splitting-using-the-graphical-mapp...
• http://scn.sap.com/thread/2072178

Cheers, Mika