Message-Split in a Sync Scenario

[jrg_neumann]

Hello Experts.

As i am quite new to Pi, i have some problems with an integration scenario we are about to implement. As i am not really into the topic, i do'nt even know, if the prefered solution can be implemented in PI or if our current approach is nonsense after all.

The currently preferred solution would look like this:

1.) The sender system collects the data and sends one message using a synchronous webservice call

2.) PI splits the message into n parts (one outgoing message per receiving system), sends the message-parts synchronously to the receiving systems, collects the response messages into one big message and sends this message as the response message for the webservice-call, the sender system made.

3.) The sender system receives the response message and updates the database.

The problem is, that i have no clue how to do a message-split in a Sync scenario. I already experimented with a message-split in Async scenarios and everything worked fine, but i have no idea, how to do the trick in a Sync scenario.

As the incoming message represents a table (only one kind of element; occurence 0...unbounded), i would have to do the following:

1.) Split the incoming message line by line (or childnode by childenode)

2.) Collect the lines for the single receiver systems (each receiver should get only one message)

3.) Send the messages

4.) receive the results

5.) Join the results into one response message / Handle timeouts, if a receiver could not be reached in time

6.) Send the response message to the sender system

When experimenting with the async scenario, i used BPM to split the message into parts, but an integration process seems to need asynchronous abstract interfaces. At least PI did not accept my synchronous message-Interface and even if it would do so, i would have no idea how to send the message parts synchronously and how to join the results.

Of course we could also send one record per message - in this case everything would be easy and clear, but unfortunatelly this is not the preferred solution, but the "Plan B"...

Does anybody know how to do what we want? Any suggestions would be highly appreciated.

And by the way: We are using PI 7.0.

Regards, Jörg

[stefan_grube · Accepted solution]

> 2.) PI splits the message into n parts (one outgoing message per receiving system), sends the message-parts synchronously to the receiving systems, collects the response messages into one big message and sends this message as the response message for the webservice-call, the sender system made.

Why you want to do this?

Send the message direct to the receiver and get one response.

Maybe you have to adjust the receiver for this, but trust me: This is the better solution.

Sync scenarios have to go smooth to avoid bottlenecks.

[jrg_neumann · Accepted solution]

@ Stefan: This is the solution i would prefer and as the sender and the receiver are not developed yet, this solution could still be implemented. We will discuss the design of the interface once more and i hope, we will end up in a sync scenario, in which PI is just doing some stupid routing (Message split would be done by the sender in this case).

@ Henrique: I experimented a little and now i can basically imagine, how a scenario like the currently preferred solution could be set up - Thanks a lot!

If i had to implement the solution, there would be something, that could cause me some headaches: How could i react, if at least one of the recevier systems could not be reached in time? As a return code for any passed record is required, i would need a way to create - let's call them - dummy entries with a predefined status code so the sender will be informed about the communication failure.

[henrique_pinto · Accepted solution]

Do these sync calls to the multiple receivers actually create documents there (are transactionally relevant) or just do search/queries? If they don't insert/update anything, you could just use a deadline branch in a block step (and include the other steps within this block) where the deadline would be less than the timeout period.

If they do create/update some entries in the receiver systems, it'll be harder because then you need somehow to control the transactional behavior of the systems (i.e. rollback the successful calls in case of 1 or more unsuccessful calls). You might overcome that by setting "Create New Transaction" on the send step(s).

http://help.sap.com/saphelp_nwpi711/helpdata/en/43/6211331c895f6ce10000000a1553f6/frameset.htm

http://help.sap.com/saphelp_nwpi711/helpdata/en/45/058023846b2460e10000000a1553f7/frameset.htm

BR,

Henrique.

[jrg_neumann · Accepted solution]

The sync calls are updating the database of the receiving systems.

The sender system is used to create accounts for our FI-Systems. The new accounts are transferred into our SAP-Systems using the interface. That's why the sender needs a return-code for every record - otherwise the sender would not know, if the new account was successfully created in the target system, which would be a problem, as new accounts are delivered as incremental updates.

As the single receivers are independent from each other, there is no need to implement a "all-or-nothing" scenario. If everything worked for receiver #1 and receiver #2 could not be reached, this would be no problem. But the requirement says, that there must be a returncode for every passed entry, so we would need dummy-entries for receiver #2.

[stefan_grube · Accepted solution]

This is a typical concept which implies that you cannot trust your middleware.

But what should happen, when the process fails? Should the users in sender system redo the work again?

I sync scenario means: somebody is waiting for the response and have to be active. But what should a clerk do, when there is an error? Report somebody? Wait for the error to be fixed and repost the stuff?

But how should the clark know, what was done successfully and what has to be reposted, should the clark use an Excel for this to remember?

You see, there is no good idea of using a sync scenario just for not trusting the middle-ware.

Do this in an async way. When there is an error, basis or PI experts will care about this and will make the receiving system getting the information that they need.

In our company we have no sync scenario at all. If there is an error, the message will be processed after the fix and it is fine.

[jrg_neumann · Accepted solution]

In theory you are right, but i am afraid, that this is not a technical, but a political issue.

I'll keep your async approach in mind and discuss the problem in the next meeting.

There is still one thing that would be very interesting: Is there any possibility to set up a mail-notification in PI if a message could not be send in a defined period of time (e.g. 2 hours after receiving the message) or if the processing of the message was stopped with an error?

The problem is, that the department, that takes care of the sending system is also providing the support for this interface. So if an enduser creates a new account and the message-processing was stopped with an error or the target system could not be reached because of e.g. a malconfiguration of the firewalls, they will be called and asked, why the new account has not yet reached the target FI system.

Thats why this department and the PI experts have to be informed if anything goes wrong.

As far as i remember it was possible to set up alerts in the runtime workbench, that could then somehow be used to create mail notifications. Creating an alert for a processing error should not be a problem but how could i do that for a defined timeout?

[stefan_grube · Accepted solution]

> In theory you are right, but i am afraid, that this is not a technical, but a political issue.

The PI specailist has to provide a proposal for a scenario and for error handling also.

> There is still one thing that would be very interesting: Is there any possibility to set up a mail-notification in PI if a message could not be send in a defined period of time (e.g. 2 hours after receiving the message) or if the processing of the message was stopped with an error?

Yes, you can do this with. CCSM and alerting.

But it is not possible to discuss the details in forum.

I think there is a how-to gude for ccms. Lookout for this.

CCMS reports long-run messages, alterting reacts on any error.

[jrg_neumann · Accepted solution]

CCMS reports long-run messages, alterting reacts on any error.

Thanks a lot - knowing what i have to google for makes things much easier

[henrique_pinto · Accepted solution]

Jorg,

if you're on PI 7.1x, you might want to consider to use Web Service Reliable Messaging (WS-RM) protocol for this scenario.

You can have async communication and still receive information on the sender side, in case the communication fails.

BR,

Henrique.

[jrg_neumann · Accepted solution]

Unfortunatelly we are still using PI 7.0. It is not planned to set up a PI 7.1 system, as the release of PI 7.3 is coming soon and we are planning to set up a PI 7.3 system in december.

As this scenario has to be implemented before that, we will have to do it on PI 7.0.

[henrique_pinto]

Jorg,

you could use a BPM with a blockstep in (par)foreach mode (par might be better for timing purposes) and a receiver determination step. For each receiver interface, create a mapping that will map the response into a same message type.

In the end, you'll have a multiline container with multiple instances of your response message, which you can then encapsulate to your sender system with a multimapping.

Alternatively, you could use a fork step for a predefined number of receivers, if you want to use different interfaces for each one. Define the fork in order to just finish when all branches are complete. For each send step, you can define a send context and use this send context (context object "ProcessStep" in condition editor) in order to define conditional routing in the receiver determination. After the fork, use a multimapping from the N response messages into the response message to the sender. Would be similar to this: http://help.sap.com/saphelp_nwpi711/helpdata/en/e8/8e2d4233c8db2ce10000000a1550b0/frameset.htm

BR,

Henrique.

[former_member181985]

Not possible in synchronous way. Better make Async scenario with BPM.

[jrg_neumann]

Thanks for your reply.

I was already afraid, that the preferred approach could be a dead end. If the described scenario is impossible to implement, i could immagine two alternative solutions:

1.) Our "Plan B":

-> Single records

-> One record per message

-> Synchronous communication

PI would only have to do a quite stupid routing based on a certain field in the message.

2.) Something totally different:

-> Asynchronous communication

-> The sender still send the data for n receivers in one message

-> PI splits the message and sends the n parts to n receivers

-> The receiver sends a response after processing the data

-> PI simply forwards this message to the sender

I think scenario #2 could have a better performance, as the total number of messages is limited to the necessary minimum, but as i'm not experienced in PI, i'm not actually sure about this.

Any recommendations?

regards, Jörg