Processing large volume of idocs using BPM Processing
I have a scenario in which SAP R/3 sends large volume say 30,000 DEBMAS Idocs to XI. XI then sends data to 3 legacy systems using jdbc adapter.
I created a BPM Process which waits for 4 hrs to collect all the idocs. This is what my BPM does:
1. Wait for 4 hrs Collect the idocs
2. For every idoc do a IDOC->JDBC Message transformation.
3. Append to a Big List
4. Loop at the Big list from step 4 and in the loop for
5. Start counter from 0 and increment. Append to a Small List.
6. if counter reaches 100 then send a Batch JDBC Message in send step.
7. Reset counter after every send.
8. Process remaining list i.e if there was an odd count of say 5300 idoc then the remaining 53 idocs will be sent in anther block.
After sending 5000 idocs to above BPM following problems are there:
1. I cannot read the workflow log as system does not respond.
2. In the For Each loop which loops through the big list of say 5000 idocs only first pass of 100 was processed after that the workflow item is not moving ahead. It remains in the status as "STARTED" but I do not see further processing.
Please tell me why certain Work Items are stuck is it becuase I have reached upper limit and is this the right approach? The Main BPM Process is also hanging from last 2 days.
I have concerns about using BPM for processing such high volume of idocs in production. Please advice and thanks in advance.