Parallel Processing with Open Cursor / Fetch Command
I am working on a project with BI/BW. I'm working with a program where the database has millions of records and could gradually increase over time.
The idea is to use parallel processing to speed up processing time. I can not select from the table without using "fetch" because it would cause a short dump. (That's how large the table is.)
So I read a blog about parallel processing:
I thought - perfect! That's what we need. However, when starting FM in a new task. <DUMP>. Reading the documentation on open cursor it looked like the dump could be solved by simply using the "with hold" addition. That didn't work either.
Here's the problem code:
OPEN CURSOR WITH HOLD lc_data FOR SELECT doc_number AS doc_num s_ord_item AS doc_item sched_line salesorg distr_chan div_head AS division sold_to material crm_trpid FROM (f_table) WHERE sold_to IN lr_customer. ASSIGN lt_result_copa TO <fs_odsdata>. DO. FETCH NEXT CURSOR lc_data APPENDING CORRESPONDING FIELDS OF TABLE lt_output_copa PACKAGE SIZE 2000. IF sy-subrc <> 0. CLOSE CURSOR lc_data. EXIT. ENDIF. LOOP AT lt_output_copa ASSIGNING <fs_out_copa>. l_tabix = sy-tabix. l_trpid = <fs_out_copa>-crm_trpid. REFRESH lt_output_tmp. MOVE-CORRESPONDING <fs_out_copa> TO ls_result_copa . ls_result_copa-/bic/zc_trpid = <fs_out_copa>-crm_trpid. APPEND ls_result_copa TO lt_output_tmp. CLEAR <fs_out_copa>-crm_trpid. DO. cal = l_called_jobs - l_recvd_jobs . " IF cal LE 4. CALL FUNCTION 'Z_GET_TERRITORY_NEW' STARTING NEW TASK taskname DESTINATION IN GROUP group PERFORMING copa_return_info ON END OF TASK EXPORTING salesorg = <fs_out_copa>-salesorg distr_chan = <fs_out_copa>-distr_chan division = <fs_out_copa>-division customer = <fs_out_copa>-customer material = <fs_out_copa>-material busgrp = <fs_out_copa>-g_cwwbus minor = <fs_out_copa>-g_cwwmin minorsub = <fs_out_copa>-g_cwwsub TABLES t_sorg = lt_sorg t_customer = lt_customer CHANGING copa = ls_result_copa EXCEPTIONS communication_failure = 1 system_failure = 2 resource_failure = 3 no_territory = 4 OTHERS = 5. IF sy-subrc = 0. l_called_jobs = l_called_jobs + 1. taskname = taskname + 1. EXIT. ELSEIF sy-subrc = 3. WAIT UNTIL l_recvd_jobs >= l_called_jobs UP TO '9' SECONDS. ENDIF. " ENDIF. ENDDO. WAIT UNTIL l_recvd_jobs >= l_called_jobs UP TO 30 SECONDS.
Thank you for the help!
I love it!
Read the documentation about SAP LUW, Database LUW and processes.
You may then find out, that every asynchronous function call as your
CALL FUNCTION 'Z_GET_TERRITORY_NEW' STARTING NEW TASK taskname
triggers an implicit database commit.
I can understand the database that it will cause a dump if FETCH NEXT CURSOR is called after the database commit just closed all open cursors.
I saw a blog about parallel processing recently and I remember SAP standard processing in contract accounting: They both do some kind of pre-selection of blocks before and then sending out the blocks in parallel processing.
I think there is no better way.
P.S.: If you need the links to relevant documentation, let me know. Too lazy right now.