PI 7.1 - Addressing Stability Concerns

[Former Member]

We have frequent problems with Out of Memory (OOM) conditions in our PI landscape (7.1, no EHP1). While we are making efforts to address these issues from a development design standpoint, we also want to do whatever we can to mitigate risks through considering changes in basis architecture/configuration. Ideally we would like to prevent a single bad development, code exception, or data issue from taking down all of PI.

We repeatedly have occurrences where one interface can cause the entire PI system to fail. We have attempted to address this with an additional server node, but this has not helped at all. Other than addressing the interface design issues, what can Basis do to make the system more bulletproof? Can you comment on some of the following questions?

1) When an interface takes down the system, it also takes down all management tools which run in java (nwa, all of the pi tools including the ability to shut off communication channels). Is there any known way to configure PI so that the admin tools run in a different jvm - which would mean that they would stay available even when a rogue interface blows up a different jvm?

2) When a server node goes down because of OOM caused by an interface, the entire app appears down and no other interfaces run properly. Having 2 sever nodes has not seemed to help. If one node remains up, why does the entire app seem still to be down? Is there something else we can do with node configurations to help address this so that the remaining node can be used until the other has completed its restart?

3) As I understand it, by Java's nature, threads within the JVM all share the same memory, so if one thread takes all the memory then all threads die also. Is there any way to compartmentalize the impacts of a single java thread so that we can keep it from taking down the entire app? Can this be accomplished through basis configuration or application architecture changes?

We have plans to add an additional dialog instance to help address a dramatic increase in interface throughput. Will web dispatcher load balancing help us in any way to prevent single interfaces from taking down the entire app?

4) In SAP JVM 1.5x which comes with PI 7.1, is there any way to kill a single thread within the JVM? So far it would appear that the answer is no u2013 which re-enforces the fact that 1 misbehaving thread can cause all of PI to crash and we have almost no way to deal with it quickly or efficiently as we would in the abap world. Ideas?

Thanks for your time.

[Former Member]

Short answer to PI OOM situations is:

- reduce your message size (10mb is very large)

- reduce the parallelism on those adapters

These notes might help:

[994433 - XI AF on J2EE Engine terminates with OutOfMemoryError|https://service.sap.com/sap/support/notes/994433]

[1253826 - Configuring Maximum Message Size Limits|https://service.sap.com/sap/support/notes/994433]

1) When an interface takes down the system, it also takes down all management tools

1) the problem is, that in a default setup, you have only one server node (one heap memory). So if this is exhausted, then the server restarts, and the whole J2ee is down. Adding more server nodes can improve the situation, but does not solve the root cause.

2) ...Having 2 sever nodes has not seemed to help.

2) Well it does help, but you will still experience failures of messages at the time one server node goes down. With all the restart jobs and queue restarts scheduled you should be able to automatically recover from these events (regardless if you have another server). But remember, OOM is very likely to reoccur after the restart, because you try to use more memory than available.

3) Is there any way to compartmentalize the impacts of a single java thread so that we can keep it from taking down the entire app? Can this be accomplished through basis configuration or application architecture changes? 

3) No, this is not possible, the only thing you can do, is to increase the heap size and change the layout of the heap. But for the SAP JVM there is not much documentation so far.

4) is there any way to kill a single thread within the JVM?

4) Again, this is not (yet) possible

Cheers Michael

[marksmyth]

Hi John

As you have not had a response to this query in the PI forum, I will send it to the Netweaver Administrator forum. This forum will have users with more java experience than the PI forum and you may get an answer to, at least, some of your questions.

Regards

Mark Smyth

XI/PI Moderator