Reliable option in SAP PI to deal with high volume data

[Former Member]

Hi Experts,

I have a requirement of transferring a huge volume of messages (nearly 100-200K per instance) from ECC to a Third party system using PI 7.4 (dual stack). I have been looking SCN blogs and documents to understand what adapters would be best to address this. What i have found is that to connect ECC, proxy connection would be convenient approach and to connect third party system, SOAP or WebService would be an option.

So would like to know are those a correct option to address this requirement? if yes then reason behind this which I was not able to found in those blogs.

Let me know if any further clarification is required.

Thanks in advance.

BR,

Praloy

[former_member187587]

SAP PI is an ESB design primarily for messaging integration style.

This means whenever a business transaction takes place in system 1 it will trigger an even and a message near real-time to update system 2.

This also allow a better distribution of messages along the buisness day.

Can you elaborate ,what is the business use case that requires all the messages to be accumulated in the ECC before they are send to the 3rd party system?

[Former Member]

Hi Nimrod,

In ECC, there is a batch program which runs on 30 mins interval, picks all the changed orders based on vendor number, and sent out to the third party system.

BR,

Praloy

[former_member187587]

If this is a Purchase Order change it can be handled immediately on event of change and send the Order IDoc with status "change" of some sort.if you will use an EOIO queue as well

you will be able to minimize the chance of data corruption.

Data corruption may occur when you choose batch processing .In this scenario you might get into a case where change1 of an order will get after change3  of the same order to the recieving system causing data corruption.

So , a change in the ECC to trigger the event "on-time" will be a much better design.

any other solution will get you into trouble.

[xavisanse]

+1

Use proxy technology with advanced adapter engine.

[former_member187587]

If there is no applicative reason , changes should be send on a real time basis.

[Former Member]

Hi Nimod,

Thanks for your response. In our case we can not go with immediate processing since upon saving a new or changed order gets picked up by the batch program which based upon validation checks, make the order in release state and data then gets pushed to PI.

So there is reason for choosing that option. ECC design already exist. Earlier the batch program used to generate some reports post completion but henceforth the program will also send data to PI and finally to Third party systems. Now since this integration part is an enhancement, so I am specifically looking for reasons to opt a solution between IDOC or Proxy to send the data to PI.

Kindly let me know for further clarification.

BR,

Praloy

[Former Member]

Hi Sebastian,

Thanks for your response. I am specifically looking for reason for opting proxy to send the data to PI.

Kindly share any info if you have on the same.

BR,

Praloy

[former_member187587]

I found it strange that the standard SAP ECC validation for the ordering process don't match your organization requirements...it is likely that you have a  buinsess implementaiton issue rather then a technical challange here..

[GauravKant]

Hi Praloy,

Proxy communication does not pass by adapter engine,it directly communicate with  integration engine.

Proxies communicate with the XI server by means of native SOAP calls over HTTP.RFC does not, so you have to convert from SOAP to RFC calls and vice versa. So XML conversion is required.

If you use ABAP Proxy , you can reduce the overhead calling the function again and again.

Proxy is develop in ECC and any validation,calculation or filtration can be easily done in ECC side by writing code.

As the result you dont have to do much calculation,mapping and filtration. And performance of the system of PI is high because ultimately mapping is converted into Java only.So performance is high.

Please have a look of below blog. if we want to pass jpg or pdf file we can pass it as attachment.

Regards,

Gaurav

[Former Member]

Hi Gaurav,

Thank you for sharing all these helpful information. I was looking such helpful inputs.

Regards,

Praloy

[manoj_khavatkopp]

Praloy,

Instead of check the no of messages , check the message size this is what really matters. (100-200k is not a big number)

You can connect to ECC via Proxy ,IDOC or RFC.

For IDOC you can have a IDOC packing functionality where a certain number of IDOC are sent as one message to PI system , same way for proxy they can write a report such that they can restrict the number of records to be sent to Pi system in one go

And for third party you need to check what type of system they host .

for eg:Third party may host a file server /AS2 server/Web server/Database do depending on what type of server your third party host you need to choose the adapter .

br

Manoj

[Former Member]

Hi Manoj,

<< Instead of check the no of messages , check the message size this is what really matters. (100-200k is not a big number) >>

Thanks for your response and pointing out the point which I missed to mention that as per the requirement, each message size will be close to 15MB. So at one instance PI will receive close to 1.5 TB data.

<<For IDOC you can have a IDOC packing functionality where a certain number of IDOC are sent as one message to PI system , same way for proxy they can write a report such that they can restrict the number of records to be sent to Pi system in one go>>

I am aware of the IDOC packaging or data packaging for Proxy and also understood as per diff SCN blogs that Proxy would be convenient option what i would like to know more precisely is that what benefits I will get out of proxy and why proxy is more convenient for addressing this approach.

If you have any info or blogs as such on this, kindly share the same.

Regards,

Praloy