Solman Java Restarts

[hasanajsh]

Hello gurus,

After updating Solman 7.1 from SP08 to SP10 we are having issues because the system is not stable.

The system starts and everything is OK but after that it keeps stopping and restarting continuously; disp+work oscillates between yellow and green.

From dev_disp:

*** ERROR => 05.05.2014 08:49:22.156 PID=4476, TID=4924    SapTimer error. SapTime differs from Windows time. ERRORS: 0, SapTime-WindowsUtcTime=-246 seconds. TotalTimeCorrection=+0 sec, QpcTime-WindowsUtcTime=-246 sec (*). [dpxxdisp.c   21558]

*** ERROR => 05.05.2014 08:49:22.156 PID=4476, TID=4924    SapTimer error. Time may be incorrect on your system. Windows time jumped approx +246 seconds.  Slept 1000 millisec, Qpc elapsed: +1000 millisec, WindowsUtc elapsed +247984 millisec.  SapTime-WindowsUtcTime=-246250 millisec. QpcTime-WindowsUtcTime=-246251 millisec (*). [dpxxdisp.c   21558]

From dev_server0:

: 1388913K->1388913K(1747968K), 1.2497387 secs] 1388913K->1388913K(2009856K), [CMS Perm : 159084K->159084K(524288K)], 1.2499567 secs] [Times: user=1.25 sys=0.00, real=1.25 secs]

java.lang.OutOfMemoryError: Java heap space

....

[Thr 5848] JLaunchIExitJava: exit hook is called (rc = 666)

[Thr 5848] **********************************************************************

[Thr 5848] *** ERROR => The Java VM terminated with a non-zero exit code.

[Thr 5848] *** Please see SAP Note 943602 , section 'J2EE Engine exit codes'

[Thr 5848] *** for additional information and trouble shooting.

[Thr 5848] **********************************************************************

[Thr 5848] JLaunchCloseProgram: good bye (exitcode = 666)

Kindly asking your help on this.

Best regards

[cathal_ohare · Accepted solution]

Hi,

Your memory parameters are fine you can increase them which will delay any future OOM crashes, if you have enough free OS memory you can increase it as far as 4096. It looks like a memory leak though on the server0 node.

To get to the root of the problem you need to analyse the oom.hprof.

Note 1883568 - How to self analyze a Heap Dump using MAT, this will produce a leak suspect showing exaclty what is using all the memory.

Kind regards,

Cathal

[hasanajsh]

Thanks to everybody for the valuable inputs and the information.

The problem is the memory although the parameters seemed to be acceptable.

Kind regards

[Former Member]

How did you solve it, by raising MaxHeapSize?

[hasanajsh]

Hi Daniel,

Sorry for late reply. After the analysis of SAP Support: the SLD was consuming a lot of memory so I had to temporarily increase the xmx and xms parameters.

Regards

[Former Member]

Hi Hasanaj,

To analyze what objects are consuming more memory you can use heap dump file and analyse this file with eclipse memory analyzer tool to know whcih objects are consuming more memory.

Hope this will help you.

Regards,

Jitendra

[Former Member]

Hi Hasanaj,

Also you can follow below wiki doc to know which parameters needs to be added in configtool to general heap dump and how can you analyze them using memory analyzer.

http://wiki.scn.sap.com/wiki/display/TechTSG/(JSTTSG)(P4)+Produce+Heap+Dump+in+Java+Server

Regards,

Jitendra

[Former Member]

Can you upload whole dev_server0 log file (preferably as attachment)?

[hasanajsh]

https://www.dropbox.com/s/lv5097yma8x87qq/dev_server0.rar

Thanks

[Reagan]

[Thr 8140] Mon May 05 21:50:19 2014
[Thr 8140] JLaunchIExitJava: exit hook is called (rc = 666)
[Thr 8140] **********************************************************************
[Thr 8140] *** ERROR => The Java VM terminated with a non-zero exit code.
[Thr 8140] *** Please see SAP Note 943602 , section 'J2EE Engine exit codes'
[Thr 8140] *** for additional information and trouble shooting.
[Thr 8140] **********************************************************************
[Thr 8140] JLaunchCloseProgram: good bye (exitcode = 666)

Exit code 666 is for OutOfMemory issue.

Restart the Windows server and then start the SAP system.

Make sure there is sufficient memory on the machine available.

[hasanajsh]

Hi Reagan,

Actually the server has 12 GB RAM and it was working just fine.

I have tried restarting the server and starting the SAP systems again but no luck.

Thanks for your help.

[divyanshu_srivastava3]

Hi,

Go to configtool and increase the size of

MaxHeapSize parameter to 3072.

Restart JAVA only and check the results.

Regards,

DV

[Sriram2009]

Hi

Could you refer the SAP Note for Error Msg "

java.lang.OutOfMemoryError: Java heap space Dumping heap to OOM.hprof"

1289240 - How to deal with out of memory errors

BR

SS

[former_member185239]

Hi Shkelen,

Exit code 666 means there is problem with heap memory.

Can you paste the value of below parameters.

-XX:MaxPermSize=

-XX:PermSize=

-Xms3072M

-XX:NewSize=

-XX:MaxNewSize=

With Regards

Ashutosh Chaturvedi

[hasanajsh]

-XX:MaxNewSize=341M

-XX:NewSize=341M

-XX:MaxPermSize=512M

-XX:PermSize=512M

Xmx2048M

-Xms2048M

-Xss2M

Thanks

[former_member185239]

Hi ,

Change the below values

-XX:MaxNewSize=341M

-XX:NewSize=341M

-XX:MaxPermSize=512M

-XX:PermSize=512M

Xmx2048M

-Xms2048M

-Xss2M

to

-XX:MaxNewSize=210M

-XX:NewSize=210M

-XX:MaxPermSize=410M

-XX:PermSize=410M

Xmx2048M

-Xms2048M

-Xss2M

Restart the java stack and paste the logs for dev_server0 and std_server0

With Regards

Ashutosh Chaturvedi

[divyanshu_srivastava3]

Hi,

Couple of thing..

1st. Check you ABAP time zone STZAC and java time zone in configtool and if required correct it.

2nd. Seems your JAVA systems GC algo is not working - Again check config tool. Check heap dump generated in server0 directory. Also, provide us your system configuration and memory parameter set in configtool.

Also, Please upload std_server0 and dispatcher.out files

Regards,

DV

[hasanajsh]

Dropbox - dispatcher and std server.rar

Thanks

[former_member188883]

Hi Hasan,

How much RAM do you have configured in your server ?

Please check your J2EE configuration as per SAP note 943602

Hope this helps.

Regards,

Deepak Kori

[said_shepl]

Dear Hasanaj,

       Kindly check the time zone at different level as the following:-

•Login to SAP system client 

•Run the report from Tcode SA38   -> TZCUSTHELP

•Run the report from Tcode SA38   -> RSDBTIME

•Run this function module in SE37   -> TZ_SYSTEM_GET_TZONE

•Run this report from Tocde SA38  -> TZONECHECK

I hope this issue help you

Said Shepl

[hasanajsh]

Hi Deepak.

12 GB

I checked Note 940893 but can not find any relevant problem.

Thanks

[hasanajsh]

Hi Said,

I've looked into this Saptimer problem but there seems to be nothing about that.

That is what I got:

System time zone seems to be correct

UTC timer seems to be OK.

Thanks