Enterprise Portal - jlaunch process Killed.

[Former Member]

Hi Gurus,

We are facing the following problem in Enterprise Portal Version 7 with SP 9.

The portal suddenly stops / not responding while working. From the output of OS,

ps -ef | grep SAP, I noticed that jlaunch was killed.

Kindly let me know how to rectifiy.

Output of ps -ef | grep sap when system was properly working

root 5895 5894 0 2008 ? 00:47:08 bin/dna -Djava.rmi.server.hostname=sap13 -Xrs -Xmx256m -Djava.class.path=classes

:../classes/diacommon.jar:classes/dna.jar -Djavax.net.ssl.keyStore=../config/mykey -Djavax.net.ssl.keyStorePassword=unicenter -D

javax.net.ssl.trustStore=../config/mykey -Djava.security.policy=file:../config/rmipolicy.txt -DCA_DIA_HOME=/opt/CA/SharedCompone

nts/ccs com/ca/dia/dna/DNA

root 25755 1 0 11:32 ? 00:00:00 /usr/sap/CPM/SYS/exe/run/saposcol

cpmadm 25871 1 0 11:32 ? 00:00:00 /usr/sap/CPM/SCS01/exe/sapstartsrv pf=/usr/sap/CPM/SYS/profile/START_SCS01_sap13

-D

cpmadm 25876 1 0 11:32 ? 00:00:00 /usr/sap/CPM/SCS01/exe/sapstart pf=/usr/sap/CPM/SYS/profile/START_SCS01_sap13

cpmadm 25890 25876 0 11:32 ? 00:00:00 ms.sapCPM_SCS01 pf=/usr/sap/CPM/SYS/profile/CPM_SCS01_sap13

cpmadm 25891 25876 0 11:32 ? 00:00:01 en.sapCPM_SCS01 pf=/usr/sap/CPM/SYS/profile/CPM_SCS01_sap13

cpmadm 25920 1 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/sapstartsrv pf=/usr/sap/CPM/SYS/profile/START_JC00_sap13 -

D

cpmadm 25924 1 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/sapstart pf=/usr/sap/CPM/SYS/profile/START_JC00_sap13

cpmadm 25941 25924 0 11:32 ? 00:00:00 jc.sapCPM_JC00 pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13

cpmadm 25942 25924 0 11:32 ? 00:00:00 ig.sapCPM_JC00 -mode=profile pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13

cpmadm 25946 25942 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/igsmux_mt -mode=profile -restartcount=0 pf=/usr/sap/CPM/SY

S/profile/CPM_JC00_sap13

cpmadm 25947 25942 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/igspw_mt -mode=profile -no=0 -restartcount=0 pf=/usr/sap/C

PM/SYS/profile/CPM_JC00_sap13

cpmadm 25948 25942 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/igspw_mt -mode=profile -no=1 -restartcount=0 pf=/usr/sap/C

PM/SYS/profile/CPM_JC00_sap13

cpmadm 26027 25941 4 11:33 ? 00:00:18 /usr/sap/CPM/JC00/exe/jlaunch pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13 -DSAPIN

FO=CPM_00_dispatcher -nodeId=0 -file=/usr/sap/CPM/JC00/j2ee/cluster/instance.properties -syncSem=144310293 -nodeName=ID8711500 -

jvmOutFile=/usr/sap/CPM/JC00/work/jvm_dispatcher.out -stdOutFile=/usr/sap/CPM/JC00/work/std_dispatcher.out -locOutFile=/usr/sap/

CPM/JC00/work/dev_dispatcher -mode=JCONTROL pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13

cpmadm 26028 25941 75 11:33 ? 00:05:02 /usr/sap/CPM/JC00/exe/jlaunch pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13 -DSAPIN

FO=CPM_00_server -nodeId=1 -file=/usr/sap/CPM/JC00/j2ee/cluster/instance.properties -syncSem=144310293 -nodeName=ID8711550 -jvmO

utFile=/usr/sap/CPM/JC00/work/jvm_server0.out -stdOutFile=/usr/sap/CPM/JC00/work/std_server0.out -locOutFile=/usr/sap/CPM/JC00/w

ork/dev_server0 -mode=JCONTROL pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13

cpmadm 26029 25941 2 11:33 ? 00:00:10 /usr/sap/CPM/JC00/exe/jlaunch pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13 -DSAPIN

FO=CPM_00_sdm -nodeId=2 -file=/usr/sap/CPM/JC00/SDM/program/config/sdm_jstartup.properties -syncSem=144310293 -nodeName=sdm -jvm

OutFile=/usr/sap/CPM/JC00/work/jvm_sdm.out -stdOutFile=/usr/sap/CPM/JC00/work/std_sdm.out -locOutFile=/usr/sap/CPM/JC00/work/dev

sdm -mode=JCONTROL pf=/usr/sap/CPM/SYS/profile/CPMJC00_sap13

cpmadm 27741 26880 0 11:39 pts/1 00:00:00 grep sap

Output of ps -ef | grep sap when System was NOT properly working

root 5895 5894 0 2008 ? 00:47:08 bin/dna -Djava.rmi.server.hostname=sap13 -Xrs -Xmx256m -Djava.class.path=classes

:../classes/diacommon.jar:classes/dna.jar -Djavax.net.ssl.keyStore=../config/mykey -Djavax.net.ssl.keyStorePassword=unicenter -D

javax.net.ssl.trustStore=../config/mykey -Djava.security.policy=file:../config/rmipolicy.txt -DCA_DIA_HOME=/opt/CA/SharedCompone

nts/ccs com/ca/dia/dna/DNA

root 25755 1 0 11:32 ? 00:00:01 /usr/sap/CPM/SYS/exe/run/saposcol

cpmadm 25871 1 0 11:32 ? 00:00:00 /usr/sap/CPM/SCS01/exe/sapstartsrv pf=/usr/sap/CPM/SYS/profile/START_SCS01_sap13

-D

cpmadm 25876 1 0 11:32 ? 00:00:00 /usr/sap/CPM/SCS01/exe/sapstart pf=/usr/sap/CPM/SYS/profile/START_SCS01_sap13

cpmadm 25890 25876 0 11:32 ? 00:00:00 ms.sapCPM_SCS01 pf=/usr/sap/CPM/SYS/profile/CPM_SCS01_sap13

cpmadm 25891 25876 0 11:32 ? 00:00:03 en.sapCPM_SCS01 pf=/usr/sap/CPM/SYS/profile/CPM_SCS01_sap13

cpmadm 25920 1 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/sapstartsrv pf=/usr/sap/CPM/SYS/profile/START_JC00_sap13 -

D

cpmadm 25924 1 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/sapstart pf=/usr/sap/CPM/SYS/profile/START_JC00_sap13

cpmadm 25942 25924 0 11:32 ? 00:00:00 ig.sapCPM_JC00 -mode=profile pf=/usr/sap/CPM/SYS/profile/CPM_JC00_sap13

cpmadm 25946 25942 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/igsmux_mt -mode=profile -restartcount=0 pf=/usr/sap/CPM/SY

S/profile/CPM_JC00_sap13

cpmadm 25947 25942 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/igspw_mt -mode=profile -no=0 -restartcount=0 pf=/usr/sap/C

PM/SYS/profile/CPM_JC00_sap13

cpmadm 25948 25942 0 11:32 ? 00:00:00 /usr/sap/CPM/JC00/exe/igspw_mt -mode=profile -no=1 -restartcount=0 pf=/usr/sap/C

PM/SYS/profile/CPM_JC00_sap13

cpmadm 29166 28625 0 12:04 pts/1 00:00:00 grep sap

sap13:cpmadm 2> ls

[Former Member]

Hi,

We too are facing similar issue.

Symptoms are as follows

Portal response time increases and availability goes down.

CPU utilization becomes 100% on application servers.

We try to restart the application instances 1 after another

And we get following errors in std_serverX.out files.

while restarting the server 13 we are getting the Fatal Error :

Loading: ClusterManager ... [Framework -> criticalShutdown] Error getting confirmation reply during attach to cluster.

Jan 11, 2010 4:20:11 PM com.sap.engine.core.Framework [Thread[Thread-1,5,main]] Fatal: Critical shutdown was invoked. Reason is: E

rror getting confirmation reply during attach to cluster.

Then we have to stop all application servers.

Then we restart the SCS instance.

After SCS restart,Portal starts normally.

This happens only when there is performance issue.

if we try to restart the portal application servers 1 after another on week ends then it does not throw above error.

Can you please help on following points

1 : Did you faced similar issue in your system ??

2 : If yes, kindly provide details on the solution like which SAP note did you refer, any other fixes applied on this.?/

Regards

Girish

[Former Member]

Hi Vijay,

Check the SAP services whether they are running or stopped.

regards

ambicasony

[Former Member]

sap service is running

[Former Member]

Have you tried restarting the process from jcmon and see, also please provide the defaulttrace.trc

[Former Member]

Kinldy help

defaultTrace.0.trc

Please find the defaultTrace

#

#1.5#00145E7A639400000000001200006B9100045FE1626A9E73#1231321530277#com.sap.engine.core.Framework##com.sap.engine.core.Framework#######SAPEngine_System_Thread[impl:5]_26#

#0#0#Fatal#1#/System/Server#Plain###Critical shutdown was invoked. Reason is: Core service security failed. J2EE Engine cannot be started.#

#1.5#00145E7A639400000000000100006CFB00045FE164A8A147#1231321567895#com.sap.engine.core.Framework##com.sap.engine.core.Framework#######Thread[Thread-2,5,main]##0#0#Fatal#

1#/System/Server#Plain###Critical shutdown was invoked. Reason is: Error getting confirmation reply during attach to cluster.#

#1.5#00145E7A639400000000000100006D6D00045FE166C2005B#1231321603113#com.sap.engine.core.Framework##com.sap.engine.core.Framework#######Thread[Thread-2,5,main]##0#0#Fatal#

1#/System/Server#Plain###Critical shutdown was invoked. Reason is: Error getting confirmation reply during attach to cluster.#

#1.5#00145E7A639400000000000100006DDF00045FE168DAF2DB#1231321638302#com.sap.engine.core.Framework##com.sap.engine.core.Framework#######Thread[Thread-2,5,main]##0#0#Fatal#

1#/System/Server#Plain###Critical shutdown was invoked. Reason is: Error getting confirmation reply during attach to cluster.#

sap13:cpmadm 82>

[Former Member]

please check if J2EE_GUEST and J2EE_ADMIN users whether they are and with proper authorizations.

[Former Member]

it was the problem with portal and java license. i have installed the license as per sap notes

[Former Member]

Hello Vijay,

Provide the log files defaulttrace.trc and dev_jrfc.trc. Jlaunch.exe may also terminate due to parameter settings. Kindly go through the following notes:

Note Number: 716604

Note Number: 723909

Regards,

Anandh