J2EE Engine down with RC -337

[Former Member]

Hi All,

SAP J2EE engine went down after performing database refresh from prod to non prod system in PI7.0. We have followed the sap note 1622083 - J2EE System fails with "exitcode = -337" not useful for our issue.Please find the detailed error message below.

Fri Apr 17 08:05:05 2015

(CompilerOracle read from file /usr/sap/FXS/DVEBMGS33/exe/sapjvm_4/jre/.hotspot_compiler)

[Thr 772] Fri Apr 17 08:05:07 2015

[Thr 772] JHVM_LoadJavaVM: Java VM created OK.

**********************************************************************

JHVM_BuildArgumentList: main method arguments of node [dispatcher]

**********************************************************************

[Thr 7198] Fri Apr 17 08:05:13 2015

[Thr 7198] JHVM_RegisterNatives: com.sap.bc.krn.perf.PerfTimes

[Thr 7198] JHVM_RegisterNatives: 8 of 9 methods in com.sap.bc.krn.perf.PerfTimes registered.

[GC 286016K->2420K(2065408K), 0.0971680 secs]

[Thr 7198] Fri Apr 17 08:05:14 2015

[Thr 7198] JHVM_RegisterNatives: com.sap.bc.proj.jstartup.JStartupFramework

[Thr 7198] JLaunchISetClusterId: set cluster id 339479400

[Thr 3856] JLaunchIExitJava: exit hook is called (rc = -337)

[Thr 3856] **********************************************************************

[Thr 3856] *** ERROR => The Java VM terminated with a non-zero exit code.

[Thr 3856] *** Please see SAP Note 943602 , section 'J2EE Engine exit codes'

[Thr 3856] *** for additional information and trouble shooting.

[Thr 3856] **********************************************************************

[Thr 3856] SigISetIgnoreAction : SIG_IGN for signal SIGCHLD

[Thr 3856] JLaunchCloseProgram: good bye (exitcode = -337)

stdout/stderr redirect

--------------------------------------------------------------------------------

node name   : dispatcher

pid         : 19202134

system name : FXS

system nr.  : 33

started at  : Fri Apr 17 07:18:52 2015

[Thr 01] MtxInit: 10000 0 2

(CompilerOracle read from file /usr/sap/FXS/DVEBMGS33/exe/sapjvm_4/jre/.hotspot_compiler)

SAP J2EE Engine Version 7.00 SP 29   PatchLevel 121396.450 is starting...

Loading: LogManager ... 671 ms.

Loading: PoolManager ... 3 ms.

Loading: ThreadManager ... [GC 286016K->2415K(2065408K), 0.0368430 secs]

133 ms.

Loading: IpVerificationManager ... 12 ms.

Loading: ConnectionsManipulator ... 37 ms.

Loading: ClassLoaderManager ... 29 ms.

Loading: ClusterManager ... [Framework -> criticalShutdown] Exiting Listener Loop. This requires a restart of the node. Possible reason is an interrupted reconnect sess

ion to the message server.

Apr 17, 2015 7:18:57 AM             com.sap.engine.core.Framework [SAP J2EE Engine|MS Socket Listener] Fatal: Critical shutdown was invoked. Reason is: Exiting Listener

Loop. This requires a restart of the node. Possible reason is an interrupted reconnect session to the message server.

Please advise us to fix this issue.

Regards,

MK

[Reagan]

>>SAP J2EE engine went down after performing database refresh from prod to non prod system in PI7.0<<

How was the refresh done? Did you perform an export and import of the J2EE stack? Is the ABAP up and running?

[Former Member]

Hi All,

Thanks for your suggestion. Issue fixed now.

Regards,

MK

[Former Member]

Could you please share the solution with us.

thanks,

pavan

[AtulKumarJain]

Hi MK,

Please confirm  database is up and running ,

Configtool opening and you are able to connnecto configtool.

Please show , default trace, server0 log

BR

AKJ

[Former Member]

Hi akj,

db and Config tool able to login. Please find the default trace and server log details below.

Default Trace:

#

#1.#061FC79DB61000010000000301DE01F8000513E6CF9C08CB#1429256930068#com.sap.engine.core.cluster.impl6.ms.MSListener##com.sap.engine.core.cluster.impl6.ms.MSListener.run

()#######SAP J2EE Engine|MS Socket Listener##0#0#Error#1#/System/Server#Plain###MSLib: Exiting the listener loop. Cluster communication is not working anymore. This is

due to an interrupted reconnect session. Node will reboot.#

#1.#061FC79DB61000010000000501DE01F8000513E6CF9C0FA6#1429256930070#com.sap.engine.core.Framework##com.sap.engine.core.Framework#######SAP J2EE Engine|MS Socket Listene

r##0#0#Fatal#1#/System/Server/Critical#Plain###Critical shutdown was invoked. Reason is: Exiting Listener Loop. This requires a restart of the node. Possible reason is

an interrupted reconnect session to the message server.#

#1.#061FC79DB61000000000000701DE01F8000513E6CF9C5970#1429256930089#com.sap.engine.core.cluster.impl6.ClusterManagerImpl##com.sap.engine.core.cluster.impl6.ClusterManag

erImpl#######Thread[Thread-1,5,main]##0#0#Error#1#/System/Server/Critical#Plain###Cannot attach to the Message Server#

#1.#061FC79DB61000010000000001DE01FC000513E6D131225B#1429256956615#com.sap.engine.core.cluster.impl6.ms.MSRawConnection##com.sap.engine.core.cluster.impl6.ms.MSRawConn

ection.receiveRawMessage()#######SAP J2EE Engine|MS Socket Listener##0#0#Error##Plain###java.io.IOException: Error reading length field of message. EOF has been reached

.#

#1.#061FC79DB61000010000000101DE01FC000513E6D1313DDC#1429256956624#com.sap.engine.core.cluster.impl6.ms.MSRawConnection##com.sap.engine.core.cluster.impl6.ms.MSRawConn

ection.receiveRawMessage()#######SAP J2EE Engine|MS Socket Listener##0#0#Error##Plain###java.io.IOException: Error reading length field of message. EOF has been reached

.

        at com.sap.engine.core.cluster.impl6.ms.MSMessageHeader.read(MSMessageHeader.java:442)

        at com.sap.engine.core.cluster.impl6.ms.MSMessageObjectImpl.readHeader(MSMessageObjectImpl.java:142)

        at com.sap.engine.core.cluster.impl6.ms.MSRawConnection.receiveRawMessage(MSRawConnection.java:1674)

        at com.sap.engine.core.cluster.impl6.ms.MSListener.run(MSListener.java:86)

        at com.sap.engine.frame.core.thread.Task.run(Task.java:64)

        at com.sap.engine.core.thread.impl5.SingleThread.execute(SingleThread.java:83)

        at com.sap.engine.core.thread.impl5.SingleThread.run(SingleThread.java:156)

#

#1.#061FC79DB61000010000000301DE01FC000513E6D1315AD1#1429256956630#com.sap.engine.core.cluster.impl6.ms.MSListener##com.sap.engine.core.cluster.impl6.ms.MSListener.run

()#######SAP J2EE Engine|MS Socket Listener##0#0#Error#1#/System/Server#Plain###MSLib: Exiting the listener loop. Cluster communication is not working anymore. This is

due to an interrupted reconnect session. Node will reboot.#

#1.#061FC79DB61000010000000501DE01FC000513E6D1319958#1429256956644#com.sap.engine.core.Framework##com.sap.engine.core.Framework#######SAP J2EE Engine|MS Socket Listene

r##0#0#Fatal#1#/System/Server/Critical#Plain###Critical shutdown was invoked. Reason is: Exiting Listener Loop. This requires a restart of the node. Possible reason is

an interrupted reconnect session to the message server.#

Server Login:

**********************************************************************

JHVM_BuildArgumentList: main method arguments of node [server0]

**********************************************************************

[Thr 7198] Fri Apr 17 08:49:14 2015

[Thr 7198] JHVM_RegisterNatives: com.sap.bc.krn.perf.PerfTimes

[Thr 7198] JHVM_RegisterNatives: 8 of 9 methods in com.sap.bc.krn.perf.PerfTimes registered.

Fri Apr 17 08:49:15 2015

16.055: [GC 16.056: [ParNew: 409600K->2447K(512000K), 0.0903670 secs] 409600K->2447K(1994752K), 0.0907680 secs] [Times: user=0.10 sys=0.00, real=0.09 secs]

[Thr 7198] JHVM_RegisterNatives: com.sap.bc.proj.jstartup.JStartupFramework

[Thr 7198] Fri Apr 17 08:49:16 2015

[Thr 7198] JLaunchISetClusterId: set cluster id 339479450

Heap

par new generation   reserved 614400K, committed 614400K, used 35660K [0x0000000031010000, 0x0000000056810000, 0x0000000056810000)

  eden space 409600K,   8% used [0x0000000031010000, 0x000000003307f258, 0x000000004a010000)

  from space 102400K,   2% used [0x0000000050410000, 0x0000000050673dd0, 0x0000000056810000)

  to   space 102400K,   0% used [0x000000004a010000, 0x000000004a010000, 0x0000000050410000)

concurrent mark-sweep generation reserved 1482752K, committed 1482752K, used 0K [0x0000000056810000, 0x00000000b1010000, 0x00000000b1010000)

concurrent-mark-sweep perm gen reserved 524288K, committed 524288K, used 9048K [0x00000000b1010000, 0x00000000d1010000, 0x00000000d1010000)

[Thr 3856] JLaunchIExitJava: exit hook is called (rc = -337)

[Thr 3856] **********************************************************************

[Thr 3856] *** ERROR => The Java VM terminated with a non-zero exit code.

[Thr 3856] *** Please see SAP Note 943602 , section 'J2EE Engine exit codes'

[Thr 3856] *** for additional information and trouble shooting.

[Thr 3856] **********************************************************************

[Thr 3856] SigISetIgnoreAction : SIG_IGN for signal SIGCHLD

[Thr 3856] JLaunchCloseProgram: good bye (exitcode = -337)

Regards,

MK

[AtulKumarJain]

Hi MK,

Please check below wiki link and verify cluster communication paramater in configtool

Cluster Communication Parameters in Process Integration - Process Integration - SCN Wiki

there is some parameter still pointing to prduction box

BR

AKJ

[Former Member]

Hi ,

Still, We are facing the same issue.Pls do the needful.

Regards,

MK

[AtulKumarJain]

Hi Mk,

Did you check all the parameter related to message server host and port number.

You need to verify parameter should not point anywhere production server.

Check profile parameter.

Check message server log in work directory.please verify is there any chage in port due to instance number chnage.

BR

AKJ

[Former Member]

Hi AKJ,

Our Pre-prod and production same instance number and same message port number is in different host.I have checked in profiles not find any issues. Still issue exist.

Regards,

MK

[Former Member]

Did you follow/complete the post-system copy steps for Java syste, like edit instance.properties file in config tools for host name cahnge etc..

[Former Member]

Hi,

Yes.. I have changed some parameter pointing to production system.But dispatcher started successfully and server node stopped with below error detail.

Fri Apr 17 21:17:38 2015

Heap

par new generation   reserved 614400K, committed 614400K, used 171386K [0x0000000031010000, 0x0000000056810000, 0x0000000056810000)

  eden space 409600K,  41% used [0x0000000031010000, 0x000000003b76b638, 0x000000004a010000)

  from space 102400K,   0% used [0x0000000050410000, 0x00000000504131f0, 0x0000000056810000)

  to   space 102400K,   0% used [0x000000004a010000, 0x000000004a010000, 0x0000000050410000)

concurrent mark-sweep generation reserved 1482752K, committed 1482752K, used 15708K [0x0000000056810000, 0x00000000b1010000, 0x00000000b1010000)

concurrent-mark-sweep perm gen reserved 524288K, committed 524288K, used 26265K [0x00000000b1010000, 0x00000000d1010000, 0x00000000d1010000)

[Thr 3856] JLaunchIExitJava: exit hook is called (rc = -11114)

[Thr 3856] **********************************************************************

[Thr 3856] *** ERROR => The Java VM terminated with a non-zero exit code.

[Thr 3856] *** Please see SAP Note 943602 , section 'J2EE Engine exit codes'

[Thr 3856] *** for additional information and trouble shooting.

[Thr 3856] **********************************************************************

[Thr 3856] SigISetIgnoreAction : SIG_IGN for signal SIGCHLD

[Thr 3856] JLaunchCloseProgram: good bye (exitcode = -11114)

Regards,

MK

[Sriram2009]

Hi MK

Check this SAP KBA

1733352 - J2EE System fails with "exitcode = -11114"

BR

SS