Big size data transfer, Proxy to JDBC

[Former Member]

Hi all,

I am trying to transfer 650thousand records (around 400MB) from Proxy to a DB over JDBC in Batch mode. When I try this process sending 50thousand records per time the JAVA side starts throwing Java Heap size erros.

Also the adapter JDBC gives error like: Terminating queue worker thread, because of fatal error: java.lang.OutOfMemoryError: Java heap space (failed to allocate 537002008 bytes)

And at the end the Java system restarts itself.

Is it not possible to send 14 messages each around 30 MB to a DB without getting errors like this. Is this because of the Hardware or something to do with parameters?

Thanks.

[stefan_grube]

Do you send the messages with a time gap?

When you send all messages the same time, of course the memory will not be sufficient.

But I noticed that the JDBC adapter implementation seems to need more memory compared to other adapters.

So I process JDBC messages in small chunks with a time gap.

[Former Member]

Can I handle this in PI, or must I handle this (sending smaller messages with a gap) on the SAP side?

[Former Member]

Hi,

It will be easier for u to handle this in ECC side........as the report sending the data just needs to be scheduled as a job with some time gap to repeat in ECC........

[stefan_grube]

Besides using a BPM to set a time gap, I have no idea how this could be solved inside PI.

But I do not know how to solve this in ERP neither, so maybe you have to go for BPM?

[Former Member]

BPM looks like a interesting solution. BPM decrease the performance I think, but at least I may get the messages processed successfully.

Thanks for the advice.