Large amount of data best practices.

[roger_alluivall]

Hello Experts,

I have an scenario where i have to extract large volume of data from SAP system to a external database using SAP PI. The process has to extract about 400.000 rows from SAP and send it to this external database. I guess the best way to insert the data to the database is using JDBC adapter but i'm wondering what's the best adapter i can use to comunicate SAP R/3 and SAP PI? What's the best way to send a message of 400.000 rows to SAP PI? Files, idocs, proxies? Could you please tell me if there's any documentation on the topic?

Thank you in advance.

[Former Member]

Hi Roger ,

I am having same scenario . But having 200 tables in SAP .Every data has to push to ECC .In this Case PROXY-PI-JDBC will work ..

What is the limitation PROXY in handling the data ..

Thanks for In advance ..

[former_member463616]

HI,

In your case, ClientProxy to JDBC is the best for the performance.

Please see the link, it will explain you scenario (proxy to JDBC) in details.

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/e0ac1a33-debf-2c10-45bf-fb19f6e15...

Regards,

Rajesh

[udo_martens]

Hi,

aysnchronous proxy to jdbc is the best approach. Create first the message type for the inbound (database) interface and reuse it for the outbound (proxy) side to avoid use of mapping.

Regards,

Udo

[roger_alluivall]

Perfect! So proxy has no problems with large messages? I thought that maybe the it would be better the use of file - jdbc. But if you say so i prefer to use proxies and no message mapping.

Thank you very much!!!

[Former Member]

U can also make use of batch mode in JDBC in order to increase the performance but check batch mode might not work with few option.

chirag

[udo_martens]

Hi,

the most serious problem is (theoretically) the mapping. Only with proxy interface you can avoid a mapping, additional proxy has a pretty good performance (better than rfc).

Regards,

Udo

[former_member208856]

If data is giving problem while posting in JDBC System,

make packets of the data on ECC system while filling Sender Proxies.

For example pick data from data base upto 5000 rows & Flag in system & send it to PI --> then to JDBC System & Post.

Now pick data for unflaged rows from database & send it to PI System --> then to JDBC System for posting.

[roger_alluivall]

Thank you guys!

All the responses have been very useful!!!

Let me try to give you all points.

Regards.

[roger_alluivall]

Thank you guys!

All the responses have been very useful!!!

Let me try to give you all points.

Regards.

[former_member208856]

I will suggest you to use Proxies for this requirement & push the data from PI System to ECC System using HTTP Receiver Proxy adapter using HTTP Type RFC Destination.

At ECC system code, you can handle the data for posting.

[roger_alluivall]

Hello Sandeep,

I don't understand your recomendation. Could you please clarify it? The way data flows is as below:

SAP R/3 --> PI --> Database

Thank you very much.

[former_member208856]

As per your flow, your scenario will be

Proxies to PI System &

PI to JDBC.

Use Asynchronous proxies for data from ECC System.

Use Message mapping without Functions and calculation (1:1) mapping.

All calculation or function can be used during Filling Proxies at ECC system in ABAP Proxy Code.

If the message is in large scale, you can make packates in Proxy code using internal table at ECC System.