Huge volume of data not getting processed

[former_member190358]

Hello Everyone,

Its a single file to multiple idoc scenario. There is no mapping involved. But the problem is that the file is of size 50 MB having 50000 idocs . This idocs get divided and are sent to BW for reporting based on the message id . Now since the message id should remain same for all the 50000 idocs, so i cannot split the file.

But when i process this, it gives me Lock_Table_Overflow error . The function module IDOC_INBOUND_ASYNCHRONOUS is in error in SM58 . I have checked the enque/table_size and its 64000. i think its enough to process a 50 MB file.

Please let me know how to proceed further with this.

Regards,

Ravi

[former_member184681 · Accepted solution]

Hi Ravi,

I don't really think this is a problem of PI itself, especially that you get the error in IDOC_INBOUND_ASYNCHRONOUS. The enque/table_size equal 64 000 might not be enough for 50 000 IDocs - just think if each IDoc requires two locks.

Hopefully, you should be able solve the issue by setting the Queue Processing checkbox in your receiver IDoc adapter in PI. This will force IDocs being processed one by one, so that so many locks will not be created simultaneously. The only problem is that I cannot foresee, how big the overall increase in processing time will be.

But you will not know until you try and please let us know about the results, as there might be others to follow your path

Hope this helps,

Greg

[prateek]

As Greg mentioned, the problem appears to be at receiver system and not at PI.

>>Now since the message id should remain same for all the 50000 idocs

What sort of requirement is associated with this statement? Why do you need same PI message ID?

Regards,

Prateek Raj Srivastava

[former_member190358]

Hi Pratik and Greg,

As assumed correctly, the issue was with BW system and not PI . PI was processing it correctly but BW enque/tablesize parameter value was less and so it was not able to handle the files and the " Lock table overflow error was coming"

As a solution, we have convinced the BW team to split the files and processs it .

Thanks all

Closing the thread.

[baskar_gopalakrishnan2]

What PI version you use? PI 7.3 can handle message size 50 MB better than PI 7.1. PI 7.1 provides optimum performance upto size 5mb. You might want to split the file data into smaller size for handling this situation. If your deal fcc on the sender side,you can set

Recordsets per message some value . SO that messages are split with few records each.