How to transfer 60000 records from SAP PI

[former_member6134]

Hello Experts,

            I am having a scenario RFC - PI - JCBC, where we need to transfer 60,000 records from SAP R/3 to MSSQL server. Can this be done at one go? Will PI take this as in one attempt when we tried ,we got the Inbound expired message.

Else can it be possible to spilt the source message table with 10,000 records each and send?

Can somebody share their view as to how they have achieved the large data transfer which is in the magnitude of more than 50,000 or 1 lakhs.

Thanks

Regards

Suk4023

[MichalKrawczyk · Accepted solution]

hi,

>>>>I am having a scenario RFC - PI - JCBC, where we need to transfer 60,000 records from SAP R/3 to MSSQL server. Can this be done at one go

depends on the message size (number of records is not important as you can have two fields in one record and you can have thirty...)

PI can easily for got 50Mb messages but best size is around 2-7 Mbs so try splitting those rows in messages with 2-7 Mb size,

Regards,

Michal Krawczyk

[former_member6134 · Accepted solution]

Hi Michal,

    I am not talking about no of fields in each record, I am talking purely of number of records. Each records can have any no of fields. But my query is if no of records are more than 50,000 or 1 lakhs can they be spilt at source end and transferred or is their any method or setting to transfer such huge no of records at one go?

Thanks

Regards

Suk4023

[Former Member · Accepted solution]

Hello,

IMO, u can use Proxy - PI -JDBC scenario.

So, what u can do is send the data in multiple chunks to SAP PI and then in the receiver side use SP and pass the entire received data (from Proxy) in a XML string to DB. And eventually let SP parse that input XML message and do the insert/update.

Check the reply posted by me in below thread:

http://scn.sap.com/thread/3202714

Thanks

Amit Srivastava

[former_member200386 · Accepted solution]

Hi ,

You can your ABAP team to built a code that can split 10000 records at a time. this is the best solution. In my previous project where i need to send 20 laks records to Oracle data base.(Proxy to JDBC) we handled risk at ECC end writing a code that can trigger 20000 records at a time.

This the ABAP Code we used to split the records.

FORM SEND_TO_PROXY_2.

DATA: LT_PROXYCUS TYPE ZPROXYDT_UPDATEHIGHVOL_SA_TAB2,
LS_PROXYCUS TYPE LINE OF ZPROXYDT_UPDATEHIGHVOL_SA_TAB2,
LO_CLSCUS TYPE REF TO ZPROXYCO_SOA_UPDATEHIGHVOL_SAP,
LT_DUMMYMAT TYPE ZPROXYDT_UPDATEHIGHVOL_SAP_TAB,
LT_DUMMYEMP TYPE ZPROXYDT_UPDATEHIGHVOL_SA_TAB1,
LV_CUSCNT TYPE I.

CLEAR:LS_INPUT,
LS_MT_UPDATEHIGHVOL,
LS_OUTPUT,
LO_SYS_EXCEPTION.


IF LT_CUSTMAST[] IS INITIAL.
RETURN.
ENDIF.

*-------------------------
* Transfer Customer Master
*-------------------------

LOOP AT LT_CUSTMAST INTO LS_CUSTMAST.
CLEAR LS_PROXYCUS.

CALL FUNCTION 'CONVERSION_EXIT_ALPHA_OUTPUT'
EXPORTING
INPUT  = LS_CUSTMAST-KUNNR
IMPORTING
OUTPUT = LS_CUSTMAST-KUNNR.


LS_PROXYCUS-CUST_ID = LS_CUSTMAST-KUNNR.
LS_PROXYCUS-CUST_NAME = LS_CUSTMAST-NAME1.
LS_PROXYCUS-CUST_ADDRESS = LS_CUSTMAST-STREET.
IF LS_CUSTMAST-LOEVM = ''.
LS_PROXYCUS-CUST_STATUS = 'A'.
ELSE.
LS_PROXYCUS-CUST_STATUS = 'I'.
ENDIF.

LS_PROXYCUS-CREDIT_PARTY = LS_CUSTMAST-CPARTY.

IF LS_CUSTMAST-KBETCU IS NOT INITIAL.
LS_CUSTMAST-KBETCU = LS_CUSTMAST-KBETCU  * -1.
LS_PROXYCUS-DISC_PERCENT = LS_CUSTMAST-KBETCU.
ELSE.
LS_CUSTMAST-KBETGR = LS_CUSTMAST-KBETGR * -1.
LS_PROXYCUS-DISC_PERCENT = LS_CUSTMAST-KBETGR.
ENDIF.

LS_PROXYCUS-SGROUP_CODE = LS_CUSTMAST-VKGRP.
LS_PROXYCUS-SROUTE_CODE = LS_CUSTMAST-LZONE.
LS_PROXYCUS-SDEPOT_CODE = LS_CUSTMAST-VKBUR.
LS_PROXYCUS-SORGA_CODE = LS_CUSTMAST-VKORG.
LS_PROXYCUS-SDIVI_CODE = LS_CUSTMAST-SPART.
LS_PROXYCUS-COMPANY_CODE = LS_CUSTMAST-BUKRS.
LS_PROXYCUS-SDIST_CODE = LS_CUSTMAST-BZIRK.
LS_PROXYCUS-SREGI_CODE = LS_CUSTMAST-KVGR1.
LS_PROXYCUS-SZONE_CODE = LS_CUSTMAST-KVGR2.




LS_PROXYCUS-CHANGE_FLAG = LS_CUSTMAST-CFLAG.
LS_PROXYCUS-ENTERED_BY = LS_CUSTMAST-ENAME.
LS_PROXYCUS-ENTRY_DATE = LS_CUSTMAST-EDATE.
LS_PROXYCUS-MODIFIED_BY = LS_CUSTMAST-MNAME.
LS_PROXYCUS-MODIFY_DATE = LS_CUSTMAST-MDATE.
APPEND LS_PROXYCUS TO LT_PROXYCUS.
LV_CUSCNT = LV_CUSCNT + 1.
IF LV_CUSCNT  = 50000.
LV_CUSCNT  = 0.
*>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
* Transfer Proxy structures for customer
*>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
LS_INPUT-MATERIAL_MASTER[] = LT_DUMMYMAT[].
LS_INPUT-EMPLOYEE_MASTER[] = LT_DUMMYEMP[].
LS_INPUT-CUSTOMER_MASTER[] = LT_PROXYCUS[].


LS_MT_UPDATEHIGHVOL-INPUT = LS_INPUT.
  LS_OUTPUT-MT_UPDATEHIGHVOL_SAP_TO_ORA = LS_MT_UPDATEHIGHVOL.

*-------------------------------
* Create Object reference
*-------------------------------
CREATE OBJECT LO_CLSCUS.

*--------------------------------
* Call Proxy method.
*--------------------------------

TRY.
CALL METHOD LO_CLSCUS->SOA_UPDATEHIGHVOL_SAP_TO_ORA
EXPORTING
OUTPUT = LS_OUTPUT.
COMMIT WORK.
CATCH CX_AI_SYSTEM_FAULT INTO LO_SYS_EXCEPTION.
ENDTRY.
FREE LO_CLSCUS.
CLEAR: LT_PROXYCUS,LS_INPUT,LS_MT_UPDATEHIGHVOL,LS_OUTPUT,LO_SYS_EXCEPTION.
WAIT UP TO 60 SECONDS.
ENDIF.
ENDLOOP.

*>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
* Call the Proxy for last set of data.
*>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
LS_INPUT-MATERIAL_MASTER[] = LT_DUMMYMAT[].
LS_INPUT-EMPLOYEE_MASTER[] = LT_DUMMYEMP[].
LS_INPUT-CUSTOMER_MASTER[] = LT_PROXYCUS[].

Thanks & Records,

Pavan