Memory leak problem in XI servers

[Former Member]

In our XI landscape, w ehave two server XPD and XDV. Both server have total 10GB physical Memory. XPD is not heavy loaded for mesage process

but we are expecting almost double the flow of messages process from next month.

In both servers, we have one dispatchere and one server node.

In last couple of weeks, J2EE server crashes with "OutOfMemroy: Java heap size" and

"OutOfMemroy: unable to create new native thread" in XDV server.

As per oss note 534867, i have extended the nkthread and max_thread_proc to fix the "OutOfMemroy: unable to create new native thread".

We have assigned 2GB to maximum heap size.

Whenever we check the system for free memory , it never shows us more then 300-400MB available memory out of 10GB in both servers.

I downloaded the "thcnt_IA64.zip" as recommended by oss note 534867. I run the tool to check the thread/memory in XPD for four times. Following is the details:

1. *** Process summary: 257 processes, 962 threads running

• Physical memory ............... 10191 MB

• Free memory ................... 361 MB

• System memory ................. 1198 MB

• Shared memory ................. 7490 MB

• Process memory (RSS) .......... 3778 MB

2. *** Process summary: 261 processes, 971 threads running

• Physical memory ............... 10191 MB

• Free memory ................... 358 MB

• System memory ................. 1198 MB

• Shared memory ................. 7490 MB

• Process memory (RSS) .......... 3785 MB

3. *** Process summary: 262 processes, 973 threads running

• Physical memory ............... 10191 MB

• Free memory ................... 350 MB

• System memory ................. 1198 MB

• Shared memory ................. 7490 MB

• Process memory (RSS) .......... 3794 MB

4. *** Process summary: 265 processes, 965 threads running

• Physical memory ............... 10191 MB

• Free memory ................... 330 MB

• System memory ................. 1198 MB

• Shared memory ................. 7490 MB

• Process memory (RSS) .......... 3814 MB

I want to understand where is the rest of memory in our server. It is showing only free memory 330MB out of 10GB.

Thanks

Amar

[Former Member · Accepted solution]

Have you set the JVM parameter as per note 723909.

Can you list all the JVM paramters and the current values.

What is the OS?

[Former Member · Accepted solution]

Hi Shaji

Thanks for your information.

Today morning, i have created one additional server node (server1) in XDV server and set the max heap size (Xmx :1GB). For server0, i reduced the Xmx from 2GB to 1GB (as i mentioned earlier, system shows only 300mb free memory out of 10GB, so i reduced the Xmx from 2GB to 1GB and used this 1GB for additional server1).

Tomorrow morning, i will follow the oss note you mentioned for jvm setting.

I have following question:

1. Right now, XDV shows 300MB free available memory out of 10GB. Is there any affect on XDV server (java crash with out of memory etc), if i increase the Xmx setting from 1GB to 2GB in both server nodes? or do i need to have addtional 2GB memory is our XDV server (extend memory from 10GB to 12GB).

2. Max Perm Size is 256MB and Perm size setting is also 256MB in our system. As per oss note 723909, this value should be 384MB for both Max Perm size and Perm size parameter. If i update this parameter value from 256MB to 384MB, does it reduce the free available memory in XDV?

3. Is there any benefit to have more server nodes in XI to manage the memory?

Pls suggest the above mentioned point.

Thanks

Amar

[Former Member · Accepted solution]

I assume your H/W is 32bit if you had set Xmx to 1 GB.

What about the NewSize and MaxNewSize Parameter size. You also need to set the parameter related to garbage collection (GC).

10GB is good enough to add 2 more server nodes, but first try and bring up this Instance.

[Former Member · Accepted solution]

We are running XI on HP-Unix platform (64 bit platform).

Following are the parameter for server0 node and these are the same setting for server1 node:

-Xmx: 1024m

-Djava.security.policy=./java.policy

-Djava.security.egd=file:/dev/urandom

-Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy

-Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy

-Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy

-Djco.jarm=1

-Dsun.io.useCanonCaches=false

-XX:MaxPermSize=256m

-Djava.awt.headless=true

-XX:PermSize=192m

-XX:SurvivorRatio=2

-XX:TargetSurvivorRatio=90

-XX:SoftRefLRUPolicyMSPerMB=1

-Xms1024m

-XX:NewRatio=3

-verbose:gc

-XX:+PrintGCDetails

-XX:+PrintGCTimeStamps

-XX:+UseParNewGC

-XX:+HeapDumpOnCtrlBreak

-XX:HeapDumpPath=/XDV_logging

-Xloggc:GC.log

-Xverbosegc:file=gc_server0.vgc

-Drdbms.driverLocation=/oracle/client/10x_64/instantclient/classes12.jar

-Dsys.global.dir=/usr/sap/XDV/SYS/global/

both server node are up and working fine.

but as i told you earlier, if i increase the Xmx/Xms from 1Gb to 2GB, does it make any impact on XDV server? There is only 300MB free available memory left in XDV server.

Thanks

Amar

[Former Member · Accepted solution]

Set your parameter values as given below :

Xmx 2048m <both Xmx and Xms should have the same values)

-verbose:gc

-Xms2048M

-Djco.jarm=1

-XX:PermSize=512M

-XX:MaxPermSize=512M

-XX:NewSize=320M

-XX:MaxNewSize=320M

-XX:SurvivorRatio=2

-XX:TargetSurvivorRatio=90

-XX:+PrintGCDetails

-XX:+PrintGCTimeStamps

-XX:+UseTLAB

-XX:SoftRefLRUPolicyMSPerMB=1

-XX:+PrintClassHistogram

-XX:+ForceMmapReserved <- This parameter works only on HP-UX. I have added and found improvement in the memory utilization

> but as i told you earlier, if i increase the Xmx/Xms from 1Gb to 2GB, does it make any impact on XDV server? There is only 300MB free available memory left in XDV server.

>

2GB is a must on 64 bit, there won't be much impact. Too less Heap Size also will give you 'Out of Memory' issue.

Stop the Instance and clear the SHM using :

cleanipc <instance no> remove <- only for all instance nos which are stopped

Restart the system and let me know