Server0 not starting CRM Dual Stack

[former_member196664]

Hi All,

Configured ADS and restarted production CRM Dual Stack. Server) process is dying. Initially JAVA did not start at all. J2EE Status uvavailable. But after server reboot, only Server0 process is not starting. Please help ASAP. This is production system.

trc file: "G:\usr\sap\CRP\DVEBMGS01\work\dev_server0", trc level: 1, release: "700"
---------------------------------------------------
node name   : ID11343650
pid         : 1596
system name : CRP
system nr.  : 01
started at  : Sun Nov 16 19:57:23 2014
arguments   :
   arg[00] : G:\usr\sap\CRP\DVEBMGS01\exe\jlaunch.exe
   arg[01] : pf=\\sapcrmprd\sapmnt\CRP\SYS\profile\CRP_DVEBMGS01_sapcrmprd
   arg[02] : -DSAPINFO=CRP_01_server
   arg[03] : pf=\\sapcrmprd\sapmnt\CRP\SYS\profile\CRP_DVEBMGS01_sapcrmprd
   arg[04] : -DSAPSTART=1
   arg[05] : -DCONNECT_PORT=64999
   arg[06] : -DSAPSYSTEM=01
   arg[07] : -DSAPSYSTEMNAME=CRP
   arg[08] : -DSAPMYNAME=sapcrmprd_CRP_01
   arg[09] : -DSAPPROFILE=\\sapcrmprd\sapmnt\CRP\SYS\profile\CRP_DVEBMGS01_sapcrmprd
   arg[10] : -DFRFC_FALLBACK=ON
   arg[11] : -DFRFC_FALLBACK_HOST=localhost

[Thr 440] Sun Nov 16 19:57:23 2014
[Thr 440] *** WARNING => INFO: Unknown property [instance.box.number=CRPDVEBMGS01sapcrmprd] [jstartxx.c   841]
[Thr 440] *** WARNING => INFO: Unknown property [instance.en.host=sapcrmprd] [jstartxx.c   841]
[Thr 440] *** WARNING => INFO: Unknown property [instance.en.port=3200] [jstartxx.c   841]
[Thr 440] *** WARNING => INFO: Unknown property [instance.system.id=1] [jstartxx.c   841]

**********************************************************************
JStartupReadInstanceProperties: read instance properties [G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\instance.properties]
-> ms host    : sapcrmprd
-> ms port    : 3900
-> OS libs    : G:\usr\sap\CRP\DVEBMGS01\j2ee\os_libs
-> Admin URL  :
-> run mode   : NORMAL
-> run action : NONE
-> enabled    : yes
**********************************************************************

**********************************************************************
Used property files
-> files [00] : G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\instance.properties
**********************************************************************

**********************************************************************
Instance properties
-> ms host    : sapcrmprd
-> ms port    : 3900
-> os libs    : G:\usr\sap\CRP\DVEBMGS01\j2ee\os_libs
-> admin URL  :
-> run mode   : NORMAL
-> run action : NONE
-> enabled    : yes
**********************************************************************

**********************************************************************
Bootstrap nodes
-> [00] bootstrap            : G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\instance.properties
-> [01] bootstrap_ID11343600 : G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\instance.properties
-> [02] bootstrap_ID11343650 : G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\instance.properties
**********************************************************************

**********************************************************************
Worker nodes
-> [00] ID11343600           : G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\instance.properties
-> [01] ID11343650           : G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\instance.properties
**********************************************************************

[Thr 440] JLaunchRequestQueueInit: create named pipe for ipc
[Thr 440] JLaunchRequestQueueInit: create pipe listener thread
[Thr 5504] JLaunchRequestFunc: Thread 5504 started as listener thread for np messages.
[Thr 4620] WaitSyncSemThread: Thread 4620 started as semaphore monitor thread.
[Thr 440] NiInit3: NI already initialized; param 'maxHandles' ignored (1;202)
[Thr 440] CPIC (version=700.2011.08.22)
[Thr 440] [Node: server0] java home is set by profile parameter
Java Home: C:\j2sdk1.4.2_16-x64
[Thr 440] JStartupICheckFrameworkPackage: can't find framework package G:\usr\sap\CRP\DVEBMGS01\exe\jvmx.jar

**********************************************************************
JStartupIReadSection: read node properties [ID11343650]
-> node name          : server0
-> node type          : server
-> node execute       : yes
-> jlaunch parameters :
-> java path          : C:\j2sdk1.4.2_16-x64
-> java parameters    : -Djco.jarm=1 -XX:MaxPermSize=256M -XX:PermSize=256M -XX:NewSize=171M -XX:MaxNewSize=171M -XX:+DisableExplicitGC -verbose:gc -Xloggc:GC.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Djava.awt.headless=true -Dsun.io.useCanonCaches=false -XX:SoftRefLRUPolicyMSPerMB=1 -XX:SurvivorRatio=2 -XX:TargetSurvivorRatio=90 -Djava.security.policy=./java.policy -Djava.security.egd=file:/dev/urandom -Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy -Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy -Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy -XX:+JavaMonitorsInStackTrace -Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine. services.ts.jts.ots.PortableInterceptor.JTSInitializer
-> java vm version    : 1.4.2_16-b05
-> java vm vendor     : Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)
-> java vm type       : server
-> java vm cpu        : amd64
-> heap size          : 2048M
-> init heap size     : 1024M
-> root path          : G:\usr\sap\CRP\DVEBMGS01\j2ee\cluster\server0
-> class path         : .\bin\boot\boot.jar;.\bin\boot\jaas.jar;.\bin\system\bytecode.jar;.
-> OS libs path       : G:\usr\sap\CRP\DVEBMGS01\j2ee\os_libs
-> main class         : com.sap.engine.boot.Start
-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework
-> registr. class     : com.sap.bc.proj.jstartup.JStartupNatives
-> framework path     : G:\usr\sap\CRP\DVEBMGS01\exe\jstartup.jar;G:\usr\sap\CRP\DVEBMGS01\exe\jvmx.jar
-> shutdown class     : com.sap.engine.boot.Start
-> parameters         :
-> debuggable         : no
-> debug mode         : no
-> debug port         : 50121
-> shutdown timeout   : 120000
**********************************************************************

[Thr 440] JLaunchISetDebugMode: set debug mode [no]
[Thr 5696] JLaunchIStartFunc: Thread 5696 started as Java VM thread.
[Thr 5696] *** WARNING => Using non-standard VM option 'services.ts.jts.ots.PortableInterceptor.JTSInitializer' [jhvmxx.c     1019]
[Thr 5696] [JHVM_PrepareVMOptions] use java parameters set by profile parameter
Java Parameters: -Xss2m

**********************************************************************
JHVM_LoadJavaVM: VM Arguments of node [server0]
-> stack   : 1048576 Bytes
-> arg[  0]: exit
-> arg[  1]: abort
-> arg[  2]: vfprintf
-> arg[  3]: -Djco.jarm=1
-> arg[  4]: -XX:MaxPermSize=256M
-> arg[  5]: -XX:PermSize=256M
-> arg[  6]: -XX:NewSize=171M
-> arg[  7]: -XX:MaxNewSize=171M
-> arg[  8]: -XX:+DisableExplicitGC
-> arg[  9]: -verbose:gc
-> arg[ 10]: -Xloggc:GC.log
-> arg[ 11]: -XX:+PrintGCDetails
-> arg[ 12]: -XX:+PrintGCTimeStamps
-> arg[ 13]: -Djava.awt.headless=true
-> arg[ 14]: -Dsun.io.useCanonCaches=false
-> arg[ 15]: -XX:SoftRefLRUPolicyMSPerMB=1
-> arg[ 16]: -XX:SurvivorRatio=2
-> arg[ 17]: -XX:TargetSurvivorRatio=90
-> arg[ 18]: -Djava.security.policy=./java.policy
-> arg[ 19]: -Djava.security.egd=file:/dev/urandom
-> arg[ 20]: -Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy
-> arg[ 21]: -Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy
-> arg[ 22]: -Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy
-> arg[ 23]: -XX:+JavaMonitorsInStackTrace
-> arg[ 24]: -Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine.
-> arg[ 25]: services.ts.jts.ots.PortableInterceptor.JTSInitializer
-> arg[ 26]: -Dsys.global.dir=G:\usr\sap\CRP\SYS\global
-> arg[ 27]: -Dapplication.home=G:\usr\sap\CRP\DVEBMGS01\exe
-> arg[ 28]: -Djava.class.path=G:\usr\sap\CRP\DVEBMGS01\exe\jstartup.jar;G:\usr\sap\CRP\DVEBMGS01\exe\jvmx.jar;.\bin\boot\boot.jar;.\bin\boot\jaas.jar;.\bin\system\bytecode.jar;.
-> arg[ 29]: -Djava.library.path=C:\j2sdk1.4.2_16-x64\jre\bin\server;C:\j2sdk1.4.2_16-x64\jre\bin;C:\j2sdk1.4.2_16-x64\bin;G:\usr\sap\CRP\DVEBMGS01\j2ee\os_libs;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Veritas\Cluster Server\bin;G:\usr\sap\CRP\SYS\exe\uc\NTAMD64
-> arg[ 30]: -Dmemory.manager=2048M
-> arg[ 31]: -Xmx2048M
-> arg[ 32]: -Xms1024M
-> arg[ 33]: -DLoadBalanceRestricted=no
-> arg[ 34]: -Djstartup.mode=JCONTROL
-> arg[ 35]: -Djstartup.ownProcessId=1596
-> arg[ 36]: -Djstartup.ownHardwareId=B1806349065
-> arg[ 37]: -Djstartup.whoami=server
-> arg[ 38]: -Djstartup.debuggable=no
-> arg[ 39]: -Xss2m
-> arg[ 40]: -DSAPINFO=CRP_01_server
-> arg[ 41]: -DSAPSTART=1
-> arg[ 42]: -DCONNECT_PORT=64999
-> arg[ 43]: -DSAPSYSTEM=01
-> arg[ 44]: -DSAPSYSTEMNAME=CRP
-> arg[ 45]: -DSAPMYNAME=sapcrmprd_CRP_01
-> arg[ 46]: -DSAPPROFILE=\\sapcrmprd\sapmnt\CRP\SYS\profile\CRP_DVEBMGS01_sapcrmprd
-> arg[ 47]: -DFRFC_FALLBACK=ON
-> arg[ 48]: -DFRFC_FALLBACK_HOST=localhost
-> arg[ 49]: -DSAPSTARTUP=1
-> arg[ 50]: -DSAPSYSTEM=01
-> arg[ 51]: -DSAPSYSTEMNAME=CRP
-> arg[ 52]: -DSAPMYNAME=sapcrmprd_CRP_01
-> arg[ 53]: -DSAPDBHOST=SAPCRMPRDDB
-> arg[ 54]: -Dj2ee.dbhost=SAPCRMPRDDB
**********************************************************************

Unrecognized option: services.ts.jts.ots.PortableInterceptor.JTSInitializer
[Thr 5696] *** ERROR => JHVM_LoadJavaVM: Cannot create Java VM (rc=-1) [jhvmxx.c     559]
[Thr 5696] *** ERROR => Cannot load Java VM (server) (rc=-1) [jlnchxxi.c   752]
[Thr 5696] **********************************************************************
*** ERROR => Java VM initialization failed.
*** Please see SAP Note 943602 , section 'Java VM initialization issues'
*** for additional information and trouble shooting.
**********************************************************************
[Thr 5696] JLaunchCloseProgram: good bye (exitcode = -1)

Thanks,

Kavitha Rajan.

[Reagan · Accepted solution]

Could you supply the std_server log file ?

-> arg[ 31]: -Xmx2048M
-> arg[ 32]: -Xms1024M

Also set the Initial and Max Heap sizes to 2048M and restart the system.

[former_member196664 · Accepted solution]

Thank you so much Reagan. std_server showed the issue. Issue resolved.

Thanks,
Kavitha Rajan.

[hemanth2 · Accepted solution]

Hi Kavitha,

TWO points:

1)
It would be a good idea to check ALL the VM parameters:

ERROR:
"

Unrecognized option: services.ts.jts.ots.PortableInterceptor.JTSInitializer
"

SAP NOTE 723909 has all details.

2)

Other 3rd party JDKs are out of support now. SAP recommends users  to upgrade to the latest SAPJVM as soon as possible. Please see:

SAP Note No. 1920326 - Oracle JDK support agreement terminated

"
The JDK support contract between Oracle and SAP was terminated by Oracle (effective 31.12.2013). Due to Oracle's termination of the agreement, SAP cannot provide Oracle JDK support coverage starting from January 2014.
"

SAP JVM 4 is the standard JDK included in the follow-up SAP NetWeaver release. As of today appropriate performance for SAP applications, comparable to currently used vendor JDKs.

Below will help:

SAP Note No. 1367498 - SAP JVM installation prerequisites

SAP Note No. 1442124 How to download a SAP JVM patch from the SMP

SAP Note No. 1133020 How to import a SAP JVM patch into an AS Java

SAP Note No. 1603093- SAP JVM 4.1 parameters NetWeaver 2004 and 7.0

____________

Kind Regards,

Hemanth

SAP AGS

[former_member196664 · Accepted solution]

Hemanth,

Issue resolved. First point was the issue.

Thanks,

Kavitha Rajan.

[hemanth2 · Accepted solution]

Great Kavitha. However please make sure that you implement POINT 2 that I mentioned asap .

Kind Regards,

Hemanth

[former_member196664 · Accepted solution]

Hemanth,

Yes, I will sure change to SAP JVM ASAP. Thanks a lot.

Kavitha Rajan.