Inbound messages are not getting updated in SAP due to ample number of records

[Former Member]

Hi,

We have the scenario Webservices to Proxy via PI which will updates prices of materials in SAP

We are facing issues when we have large number of records in the file (size is around 1.6MB) in SAP

When we have files like this, in SMQ2 , the message is being processed for long time ( we observed one time for 32 hours) which gave trouble to our customers.

Last week, we have same amount of file which contains 6000 records which are to be updated in SAP, the message passed PI immediately but we face long running state of message in SMQ2 of ECC system which ultimately stopped the other messages in same queue.

So finally we kill the message after some time, managed to process by splitting the XML message(taken from SXI_MONITOR in PI)) into smaller parts and pushing into IE manually..it worked.

but is there any way to split the message of this kind into smaller parts by PI before it is processed in PI and sends to SAP system so that they will not take much time like earlier rather than changing the Proxy code in ABAP side??

Please advice. waiting for the solution..!!

thanks,

Venkat

[rodrigoalejandro_pertierr]

have you think about a multimapping. as result you will send to ECC one message for each price update.

consider the next blog as reference to solve your issue.

http://scn.sap.com/community/pi-and-soa-middleware/blog/2013/04/09/multiple-idoc-segment-occurance-t...

Regards

Rodrigo

[Former Member]

Hi Rodrigo,

If I go for multimapping, then number of messages will be created as many as number of records I think..

would it be not possible to split the payload based on for suppose "Plant " meaning that if there are 1000 records in total , can't we do split the payload of 1000 records into smaller parts say 200 each (if there are 5 plants)

To be clear..

If there are some records for Plant1, then all records for plant1 can be processed as one message..

In this way , we can reduce the number of messages if we go for multimapping..??

Please do share your views..

[Former Member]

In my head, this is exactly the sort of scenario I'd be using BPM/ccBPM for, you could have configurable parameters to control payload size for fine-tuning - but then I'm a BPM guy and as mentioned, don't know lots about PI so am very likely wrong!

Gareth.

[rodrigoalejandro_pertierr]

If you go for multimapping you will send a meesage per plant.

but thinking your case a little bit more a think you need to meet a functional guy of the ECC side to analyze why the process tale so long to update. Maybe there is a customization of the transaction. some times is the system itself that take so long to update .

E.X. if you create a SAles Order (VA01). you complete all the date and save changes. if you want to edit the same order inmediatelly, you wont be able. SAP take about 30/45 seconds in background to create the order once you press the SAVE Button.

Regards

Rodrigo

[gagandeep_batra]

Hi Venkatarao,

for this you need to split the message into multiple using multimapping

check below thread

http://scn.sap.com/thread/516444

Regards

gAgan

[Former Member]

Hello Venkat,

have you done performance analysis on ECC side? Do you use BAPIs? Avoid unneccessary database queries and loops over big tables? In my experience this is the root cause in many cases.

Otherwise, sure you can split the message in PI, but it is not so simple. It is simpler to split it in the inbound proxy in ECC. I'd rather focus on improving performance in ECC, this is smarter than just creating thousands of messages with a single price change.

Regards,

Jörg

[Former Member]

I'd say Jörg's comments are spot on, however this is only applicable if you have access to the backend system, can make changes and have a suitable technical resource to deliver them.

I'd also say whoever built the backend components should have thought about performance in the first place anyway and if analysis here shows there are issues, these should be resolved first before anything is done in PI.

There is then a design decision to make - should the payload splitting logic be added into the ECC component or in PI?  For me, the answer is PI as this is one of the points of a middleware platform (assuming the backend component has been analysed as above first.)  I see the payload splitting in the ECC component as an unnecessary overhead that should be handled before that component is called.

Just my thoughts, not necessarily correct!

Gareth.

[Former Member]

Hi Gareth,

I have built some interfaces in two versions: One as a single-record interface, one as a mass load interface. The second one would contain a list of objects of the first. Then in the inbound proxy we would loop over those objects and call for each of them the function for the single record. We would collect errors and report them accordingly. This is what I had in mind. It is surprisingly simple and can be written quite generically. Since it avoids using the big amount of records outside this one loop, performance issues due to big load should be eliminated.

But there are usually many solutions to a problem.

Regards,

Jörg

[Former Member]

Hi Jörg,

Totally understand your point.  Do you mean two interfaces in PI, or two RFCs, where the second one simply accepts a table and loops through it, calling the first which accepts a single row?

Either way, if I were developing the backend RFC I'd try to create something that handled 1..* input records and worked as efficiently as possible however I'd highly unlikely add any logic in there to handle payload splitting as I'd expect the caller/consumer of the RFC to think/manage that problem.

Talking of simple/generic solutions, I don't think it would be too difficult to create a generic, reusable BPM that could do this sort of payload splitting for any type of payload (within certain limitations of course) that could then be re-used in various PI scenarios.  I can't say if this is the case with ccBPM as I don't know it unfortunately.

As you say, there are always multiple solutions to a problem and often the best one is the one most familiar to the developer (can you tell I prefer BPM by my answers?!)

So, I guess the big question back to is whether they have access to make changes in the backend system or have to use PI to resolve this issue?  If PI, can you use ccBPM or not?  If backend, is it RFC/IDOC/etc processing?  Where/how would you prefer to fix it?

Gareth.

[Former Member]

Hi Gareth,

We don't have any access to Backend SAP system.I wanted do it in PI system itself..

An ABAP can guy can change some modifications in Proxy but Why can't we send the total payload in parts from PI itself..this is my point..

Thanks,

Venkat

[Former Member]

Hi Venkat,

Ok - understood.  If you are stuck with the current solution in the backend then yes, you very much need a PI solution.

Can you use ccBPM for this?  Is that a suitable alternative?  If not, what about some of the other suggestions on this thread about multi-mapping?

Sadly, I'm at the limit of my PI knowledge now so probably can't help too much more!

Gareth.

[rodrigoalejandro_pertierr]

If you go for multimapping you will send a meesage per plant.

but thinking your case a little bit more a believe you need to meet a functional guy of the ECC side to analyze why the process tale so long to update.

Maybe there is a customization of the transaction or a custom RFC/Proxy. some times is the system itself that take so long to update data in the tables because te relationship.

[Former Member]

Hi Gareth,

to explain my point:

- The backend programmer defines a method that works efficiently with one record (or many, doesn't matter).

- Then the interface programmer calls this method from inbound proxy server class (could also be an RFC). But how he does it, is up to him, right? So he could hand over the big bulk or he could loop over the table and call the method for a single record on each loop step.

When I did this kind of stuff, we as PI people were responsible for the inbound proxies, so the backend methods would be called by us. I guess it depends on your development environment which possibilities are given to you.

This was a couple of years ago, and with new BPM there may be new efficient ways to do that. In this case it looks like it has to be PI, and multi-mapping sounds good to me, too.

Regards,

Jörg

[Former Member]

Hi Venkat,

I'm no PI consultant so please bear that in mind when reading this answer...

I'd have expected message splitting into pre-defined payload chunks to be an integral feature of PI - it is one of the reasons you'd want to use PI to handle messages.  I'm guessing that depending upon the version of PI you are using, you could either use a ccBPM or a BPM to build a pattern that splits the initial payload into smaller chunks and then calls the proxy in numerous batches.

Hope this helps,

Gareth.

[Former Member]

Hi Gareth,

Thanks for your quick answer...Yes..we can do it by BPM but I am trying to get a solution without using BPM ...

Please share any ideas on this..

Thanks,

Venkat

[markangelo_dihiansan]

Hello,

In ecc sxmb_adm -> Maintain Message Filter for Queue Prioritization, use XBTL (for inbound) and then set a size e.g 1024kb. This would ensure that all messages larger than 1024kb get passed to the XBTL* queue, in effect the smaller messages get processed by the regular queues.

Aside from that, depending on your PI version e.g 7.11 and above, you might want to try using Integrated Configuration for your SOAP to Proxy scenario by changing it to SOAP to SOAP (XI Protocol) scenario.

Hope this helps,

Mark

[Former Member]

I knew someone who really knows what they are doing would come along with a better answer than me!