cancel
Showing results for 
Search instead for 
Did you mean: 

START Java fails on SUM upgrade - Execution phase

Former Member
0 Kudos

We're using SUM SP12 to upgrade ECC6 system, and we're having some issues during the execution phase, with the startup of the java instance.

LOG FILES:

START_AS_JAVA LOG

<!--LOGHEADER[START]/-->

<!--HELP[Manual modification of the header may cause parsing problem!]/-->

<!--LOGGINGVERSION[2.0.7.1006]/-->

<!--NAME[/usr/sap/SUM/sdt/log/SUM/START-AS-JAVA_05.LOG]/-->

<!--PATTERN[START-AS-JAVA_05.LOG]/-->

<!--FORMATTER[com.sap.tc.logging.TraceFormatter(%d [%6s]: %m)]/-->

<!--ENCODING[UTF8]/-->

<!--LOGHEADER[END]/-->

Feb 23, 2015 10:37:42 AM [Info  ]: Start service for instance 0 on host EXPSPRD ...

Feb 23, 2015 10:37:42 AM [Info  ]: Process ID 445, name sapstartsrv has been started.

Feb 23, 2015 10:37:42 AM [Info  ]:   Command line: /usr/sap/NP3/SYS/exe/run/sapstartsrv pf=/usr/sap/NP3/SYS/profile/START_DVEBMGS00_EXPSPRD -D

Feb 23, 2015 10:37:42 AM [Info  ]:   Standard out: /usr/sap/SUM/sdt/log/SUM/INSTALL_SERVICE_00_06.OUT

Feb 23, 2015 10:37:43 AM [Info  ]: Process ID 445 has been started.

Feb 23, 2015 10:37:43 AM [Info  ]: Waiting for process ID 445, name sapstartsrv to finish.

Feb 23, 2015 10:37:43 AM [Info  ]: Process ID 445, name sapstartsrv has been finished, exit code 0.

Feb 23, 2015 10:37:43 AM [Info  ]: Wait for service for instance 0 on host EXPSPRD to start...

Feb 23, 2015 10:37:45 AM [Info  ]: Process ID 446, name sapcontrol has been started.

Feb 23, 2015 10:37:45 AM [Info  ]:   Command line: /usr/sap/NP3/SYS/exe/run/sapcontrol -nr 0 -host EXPSPRD -prot NI_HTTP -function GetProcessList

Feb 23, 2015 10:37:45 AM [Info  ]:   Standard out: /usr/sap/SUM/sdt/tmp/SAPCONTROL_GETPROCESSLIST_0_11.OUT

Feb 23, 2015 10:37:45 AM [Info  ]: Process ID 446 has been started.

Feb 23, 2015 10:37:45 AM [Info  ]: Waiting for process ID 446, name sapcontrol to finish.

Feb 23, 2015 10:37:45 AM [Info  ]: Process ID 446, name sapcontrol has been finished, exit code 0.

Feb 23, 2015 10:37:45 AM [Info  ]: The sapcontrol service on host EXPSPRD and instance 0 is started.

Feb 23, 2015 10:37:45 AM [Info  ]: Service for instance 0 on host EXPSPRD has been started.

Feb 23, 2015 10:37:45 AM [Info  ]: Starting instance number 0 on host EXPSPRD with timeout of 7200 seconds.

Feb 23, 2015 10:37:45 AM [Info  ]: Process ID 447, name sapcontrol has been started.

Feb 23, 2015 10:37:45 AM [Info  ]:   Command line: /usr/sap/NP3/SYS/exe/run/sapcontrol -nr 0 -host EXPSPRD -prot NI_HTTP -function GetProcessList

Feb 23, 2015 10:37:45 AM [Info  ]:   Standard out: /usr/sap/SUM/sdt/tmp/SAPCONTROL_GETPROCESSLIST_0_12.OUT

Feb 23, 2015 10:37:45 AM [Info  ]: Process ID 447 has been started.

Feb 23, 2015 10:37:45 AM [Info  ]: Waiting for process ID 447, name sapcontrol to finish.

Feb 23, 2015 10:37:45 AM [Info  ]: Process ID 447, name sapcontrol has been finished, exit code 0.

Feb 23, 2015 10:37:45 AM [Info  ]: Starting instance number 0 on host EXPSPRD with timeout of 7200 seconds.

Feb 23, 2015 10:37:45 AM [Info  ]: Triggering an instance start by sapcontrol...

Feb 23, 2015 10:37:45 AM [Info  ]: Process ID 448, name sapcontrol has been started.

Feb 23, 2015 10:37:45 AM [Info  ]:   Command line: /usr/sap/NP3/SYS/exe/run/sapcontrol -nr 0 -host EXPSPRD -prot NI_HTTP -function Start

Feb 23, 2015 10:37:45 AM [Info  ]:   Standard out: /usr/sap/SUM/sdt/tmp/SAPCONTROL_START_0_05.OUT

Feb 23, 2015 10:37:46 AM [Info  ]: Process ID 448 has been started.

Feb 23, 2015 10:37:46 AM [Info  ]: Waiting for process ID 448, name sapcontrol to finish.

Feb 23, 2015 10:37:49 AM [Info  ]: Process ID 448, name sapcontrol has been finished, exit code 0.

Feb 23, 2015 10:37:49 AM [Info  ]: An instance start was triggered.

Feb 23, 2015 10:37:49 AM [Info  ]: Sent the start command to instance number 0 on host EXPSPRD with timeout of 7200 seconds.

Feb 23, 2015 10:37:49 AM [Info  ]: Process ID 449, name sapcontrol has been started.

Feb 23, 2015 10:37:49 AM [Info  ]:   Command line: /usr/sap/NP3/SYS/exe/run/sapcontrol -nr 0 -host EXPSPRD -prot NI_HTTP -function WaitforStarted 7200 5

Feb 23, 2015 10:37:49 AM [Info  ]:   Standard out: /usr/sap/SUM/sdt/tmp/SAPCONTROL_WAITFORSTARTED_0_05.OUT

Feb 23, 2015 10:37:49 AM [Info  ]: Process ID 449 has been started.

Feb 23, 2015 10:37:49 AM [Info  ]: Waiting for process ID 449, name sapcontrol to finish.

Feb 23, 2015 10:37:59 AM [Info  ]: Process ID 449, name sapcontrol has been finished, exit code 2.

Feb 23, 2015 10:37:59 AM [Error ]: Return code condition success evaluated to false for process sapcontrol for action wait for started.

Feb 23, 2015 10:37:59 AM [Error ]: The following problem has occurred during step execution: com.sap.sdt.util.diag.DiagException: Could not start SAP instance with number 0.

Could not check if the instance number 0 on host EXPSPRD is started.

Detected instance wait for started has an GATEWAY feature. It cannot be handled by SUM and will be ignored.

Return code condition success evaluated to false for process sapcontrol for action wait for started.

DETECT_SYSTEM_COMPONENTS

Feb 23, 2015 8:20:13 AM [Info  ]: Detecting deployable AS Java components...

Feb 23, 2015 8:20:13 AM [Info  ]: Obtaining connection to the table with system component versions...

Feb 23, 2015 8:20:13 AM [Info  ]: A valid connection to the table with system component versions has obtained.

Feb 23, 2015 8:20:13 AM [Info  ]: Access to the table with system component versions is available.

Feb 23, 2015 8:20:13 AM [Info  ]: Synchronizing table with system component versions and SDM repository...

Feb 23, 2015 8:20:13 AM [Error ]: com.sap.sdm.apiimpl.remote.client.APIRemoteExceptionImpl: Server EXPSPRD did not accept login request as admin on port 50018. Details: ERROR: Could not establish connection to server EXPSPRD at port 50018: Connection refused (port 50018 to address 10.100.0.161 (EXPSPRD.PECITY.CO.ZA))

Feb 23, 2015 8:20:13 AM [Error ]: com.sap.sdm.apiimpl.remote.client.APIRemoteExceptionImpl: Server EXPSPRD did not accept login request as admin on port 50018. Details: ERROR: Could not establish connection to server EXPSPRD at port 50018: Connection refused (port 50018 to address 10.100.0.161 (EXPSPRD.PECITY.CO.ZA))

Feb 23, 2015 8:20:13 AM [Error ]: Could not synchronize the table with system component versions with SDM repository.

Feb 23, 2015 8:20:13 AM [Error ]: Could not detect deployed components.

SDM port [50018] is available and listening, user [J2EE_ADMIN] and password has been set in secure store of configtool.

When manually trying to synchronize the SDM repository, it works:

./sdm.sh systemcomponentstate "mode=sync"                                  

Starting '/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/

java' '-Xmx2048M' -jar '/usr/sap/NP3/DVEBMGS00/SDM/program/bin/SDM.jar' syste

mcomponentstate 'sdmhome=/usr/sap/NP3/DVEBMGS00/SDM/program' mode=sync     

Starting SDM - Software Deployment Manager...                              

tc/SL/SDM/SDM/sap.com/SAP AG/7.0216.20140407133538.0000                    

Initializing Network Manager (50017)                       

Checking if another SDM is running on port 50018           

Synchronized all Deployments with the system component state.

Successfully executed. Return code: 0

I've also attached dev_dispatcher and dev_server0 logs

Regards,

Yogesh

Accepted Solutions (1)

Accepted Solutions (1)

former_member185326
Active Participant
0 Kudos

Hi Yogesh,

Now that you have already ran a manual synch, you may try this:

1. Bring your Java instance out of "Safe Mode" through Config Tool.

2. Restart your SAPSTARTSRV service using below commands

stopsap -t startupsrv

startsap -t startupsrv

3. Manually start Java

Regards,

Shahid Ahmed

Former Member
0 Kudos

Hi Shahid

I tried the commands, the START_AS_JAVA logs still gives the same error:

Feb 23, 2015 12:57:00 PM [Error ]: The following problem has occurred during step execution: com.sap.sdt.util.diag.DiagException: Could not start SAP instance with number 0.

Could not check if the instance number 0 on host EXPSPRD is started.

Detected instance wait for started has an GATEWAY feature. It cannot be handled by SUM and will be ignored.

Return code condition success evaluated to false for process sapcontrol for action wait for started.

Former Member
0 Kudos

hi yogesh,

Try to do it below method:-

Kill your sapstartsrv and sapcontrol process if any running. Then do it below way and start the JAVA system.

Note:-Before doing this do stop all JAVA process and nodes.

1) setenv SHLIB_PATH or LD_LIBRARY_PATH $SHLIB_PATH:<new_kerenel directory> --> normally its under your sum directory

setenv SHLIB_PATH $SHLIB_PATH::/usr/sap/<SID>/SUM/abap/exe

2) /usr/sap/<SID>/SUM/abap/exe/sapstartsrv pf=<instance Profile>  -D

3)/usr/sap/<SID>/SUM/abap/exe/sapcontrol -nr <instance number> -function StartService

4)/usr/sap/<SID>/SUM/abap/exe/sapcontrol -nr <instance number> -function RestartService

--> to get the Servicestarted as per the sapcontrol mechanism

5)/usr/sap/<SID>/SUM/abap/exe/sapcontrol -nr <instance number> -function GetSystemInstanceList

--> to check the system process status and information

6)/usr/sap/<SID>/SUM/abap/exe/sapcontrol -nr <instance number> -function StartSystem

after this Your system will be up and then retry in SUM for the same phase.

Regards,

Ram

Former Member
0 Kudos

Hi Ramkrishna

I've tried the steps as you suggested, and still the end result is the same:

former_member185326
Active Participant
0 Kudos

Hi Yogesh,

Check if you have FQDN (fully qualified domain name) of your Java instance present in the /etc/hosts file (Unix) and /hosts file in case of Windows OS. If its not there, then create an entry in /hosts file mapping your system IP with FQDN hostname of your system. Then re-try..

Regards,

Shahid Ahmed

Answers (11)

Answers (11)

manumohandas82
Active Contributor
0 Kudos

Hi Yogesh ,

Correction in my last post  .  I did not mean the following

3 ) Remove the parameter

DIR_CT_RUN = /usr/sap/NP3/SYS/exe/run



I meant


service/protectedwebmethods = NONE


You should be placing this in the instance profile


Note : Take a backup of the profile directory


Thanks ,

Manu

Former Member
0 Kudos

hi Yogesh,

Hope you had resolved issue let us know if the issue is still there.

Regards,

Ram

manumohandas82
Active Contributor
0 Kudos

Thanks Yogesh .

Please also Post your Start profile . Or more better check your start profile .

The SAPCPE should be copying from /usr/sap/NP3/SYS/exe/run  or /sapmnt/SID/exe  to /usr/sap/<SID>/<Instance>/exe



Thanks ,

Manu

Former Member
0 Kudos

Hi Manu

START profile below:

#-----------------------------------------------------------------------

# Global variables

#-----------------------------------------------------------------------

SAPSYSTEMNAME = NP3

INSTANCE_NAME = DVEBMGS00

DIR_EXECUTABLE = /usr/sap/NP3/SYS/exe/run

DIR_LIBRARY = /usr/sap/NP3/SYS/exe/run

SAPSYSTEM = 00

SAPGLOBALHOST = EXPSPRD

DIR_PROFILE = /usr/sap/NP3/SYS/profile

DIR_GLOBAL = /usr/sap/NP3/SYS/global

DIR_CT_RUN = /usr/sap/NP3/SYS/exe/run

DIR_INSTANCE = /usr/sap/NP3/DVEBMGS00

#APJVM_VERSION = 4.1.049

#AUTOSTART = 0

#SCN suggestion

service/protectedwebmethods = NONE

_PF = /usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

Execute_00 = immediate $(DIR_CT_RUN)/sapcpe$(FT_EXE) pf=$(_PF)

Execute_01 = immediate $(DIR_CT_RUN)/sapcpe$(FT_EXE) pf=$(_PF) list:$(DIR_CT_SAPJVM)/sapjvm_4.lst source:$(DIR_CT_SAPJVM)

#-----------------------------------------------------------------------

# Start sapcpe

#-----------------------------------------------------------------------

#-----------------------------------------------------------------------

# Start Java Dispatcher

#-----------------------------------------------------------------------

_JC = $(DIR_EXECUTABLE)$(DIR_SEP)jcontrol

Start_Program_03 = local $(_JC) pf=$(DIR_PROFILE)$(DIR_SEP)NP3_DVEBMGS00_EXPSPRD -DSAPSTART=1

SAPJVM_VERSION = 4.1.050

DIR_LIBRARY_RMT = $(DIR_EXECUTABLE)

Former Member
0 Kudos

START Profile for shadow instance:

# *** TEMP 702 CHANGE ***

SETENV_00 = DIR_LIBRARY=$(DIR_EXECUTABLE)

# Entries for stand-alone enqueue server

Start_Program_00 = local $(_EN) pf=$(DIR_PROFILE)$(DIR_SEP)NP3_DVEBMGS02_EXPSPRD

#.******************************************************************************************************************************

#.*                                                                                                                            *

#.*       Start profile START_DVEBMGS00_EXPSPRD                                                                                *

#.*                                                                                                                            *

#.*       Version                 = 000006                                                                                     *

#.*       Generated by user = SVANDYK                                                                                          *

#.*       Generated on = 19.02.2015 , 14:59:41                                                                                 *

#.*                                                                                                                            *

#.******************************************************************************************************************************

#parameter created                          by: SVANDYK      19.02.2015 14:59:34

PATH = $(DIR_EXECUTABLE):%(PATH)

#parameter created                          by: SVANDYK      19.02.2015 14:59:11

DIR_LIBRARY = $(DIR_LIBRARY)

SAPSYSTEMNAME = NP3

# Modify instance number during downtime

#SAPSYSTEM = 00

SAPSYSTEM = 02

# Modify instance number during downtime

#INSTANCE_NAME = DVEBMGS00

INSTANCE_NAME = DVEBMGS02

DIR_CT_RUN = $(DIR_EXE_ROOT)/run

DIR_EXECUTABLE = /usr/sap/NP3/SYS/exe/run

DIR_INSTALL = /usr/sap/NP3/SYS

DIR_PROFILE = $(DIR_INSTALL)/profile

#_PF = $(DIR_PROFILE)/NP3_DVEBMGS00_EXPSPRD

_PF = $(DIR_PROFILE)/NP3_DVEBMGS02_EXPSPRD

#-----------------------------------------------------------------------

# Initialize the execution environment

#-----------------------------------------------------------------------

#Execute_00 = local $(DIR_EXECUTABLE)/cleanipc 00 remove

Execute_00 = local $(DIR_EXECUTABLE)/cleanipc 02 remove

#Execute_01 = local CLRLIB LIB(R3WRK00)

Execute_01 = local CLRLIB LIB(R3WRK02)

Execute_02 = local $(DIR_EXECUTABLE)/sapmscsa pf=$(_PF) -n

#xecute_03 = local $(DIR_EXECUTABLE)/saposcol$(FT_EXE)

#-----------------------------------------------------------------------

# Cleanup after shutdown

#-----------------------------------------------------------------------

#Stop_Program_00 = local $(DIR_EXECUTABLE)/cleanipc 00 remove

Stop_Program_00 = local $(DIR_EXECUTABLE)/cleanipc 02 remove

#Stop_Program_01 = local CLRLIB LIB(R3WRK00)

Stop_Program_01 = local CLRLIB LIB(R3WRK02)

#-----------------------------------------------------------------------

# Start SAP message server

#-----------------------------------------------------------------------

_MS = $(DIR_EXECUTABLE)/msg_server$(FT_EXE)

Start_Program_01 = local $(_MS) pf=$(_PF)

#-----------------------------------------------------------------------

# Start application server

#-----------------------------------------------------------------------

_DW = $(DIR_EXECUTABLE)/disp+work$(FT_EXE)

Start_Program_02 = local $(_DW) pf=$(_PF)

#-----------------------------------------------------------------------

# Start syslog collector daemon

#-----------------------------------------------------------------------

# Change for release 702

#_CO = $(DIR_EXECUTABLE)/rslgcoll

# Change for release 702

#Start_Program_02 = local $(_CO) pf=$(_PF) -F

#-----------------------------------------------------------------------

# Start syslog send daemon

#-----------------------------------------------------------------------

# Change for release 702

#_SE = $(DIR_EXECUTABLE)/rslgsend

# Change for release 702

#Start_Program_03 = local $(_SE) pf=$(_PF) -F

#-----------------------------------------------------------------------

# Start internet graphics server

#-----------------------------------------------------------------------

# Start IGS

#_IG = $(DIR_EXECUTABLE)/igswd_mt

_IG = ig.sap$(SAPSYSTEMNAME)_$(INSTANCE_NAME)

# Start IGS

#Start_Program_04 = local $(_IG) -mode=profile pf=$(_PF)

# Start IGS

# Start IGS

#Start_Program_05 = local $(_IG) -mode=profile pf=$(DIR_PROFILE)$(DIR_SEP)NP3_DVEBMGS00_EXPSPRD

Execute_04 = local

# Change for release 702

#Execute_05 = immediate $(DIR_CT_RUN)$(DIR_SEP)sapcpe$(FT_EXE) pf=$(DIR_PROFILE)$(DIR_SEP)NP3_DVEBMGS00_EXPSPRD list:$(DIR_CT_SAPJVM)$(DIR_SEP)sapjvm_4.lst source:$(DIR_CT_SAPJVM)

Execute_05 = immediate $(DIR_CT_RUN)$(DIR_SEP)sapcpe$(FT_EXE) pf=$(DIR_PROFILE)$(DIR_SEP)NP3_DVEBMGS02_EXPSPRD list:$(DIR_CT_SAPJVM)$(DIR_SEP)sapjvm_4.lst source:$(DIR_CT_SAPJVM)

# Change for release 702

#Execute_06 = immediate $(DIR_CT_RUN)$(DIR_SEP)sapcpe$(FT_EXE) list:$(DIR_GLOBAL)$(DIR_SEP)bootstrap$(DIR_SEP)bootstrap.lst source:$(DIR_GLOBAL)$(DIR_SEP)bootstrap target:$(jstartup/DIR_CLUSTER)$(DIR_SEP)bootstrap pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

Execute_06 = immediate $(DIR_CT_RUN)$(DIR_SEP)sapcpe$(FT_EXE) list:$(DIR_GLOBAL)$(DIR_SEP)bootstrap$(DIR_SEP)bootstrap.lst source:$(DIR_GLOBAL)$(DIR_SEP)bootstrap target:$(jstartup/DIR_CLUSTER)$(DIR_SEP)bootstrap pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS02_EXPSPRD

# Entries for stand-alone enqueue server

Execute_07 = local QSH CMD('rm -f $(_EN)')

# Entries for stand-alone enqueue server

Execute_08 = local QSH CMD('ln -s -f $(DIR_EXECUTABLE)$(DIR_SEP)enserver$(FT_EXE) $(_EN)')

# Start IGS

# Start IGS

#Execute_07 = local QSH CMD('rm -f $(_IG)')

# Start IGS

# Start IGS

#Execute_08 = local QSH CMD('ln -s -f $(DIR_EXECUTABLE)/igswd_mt $(_IG)')

DIR_LIBRARY_RMT = $(DIR_EXECUTABLE)

#APJVM_VERSION = 4.1.049

# Entries for stand-alone enqueue server

_EN = en.sapNP3_DVEBMGS02

SAPJVM_VERSION = 4.1.050

manumohandas82
Active Contributor
0 Kudos

Cleanup / rename the directory /usr/sap/NP3/DVEBMGS00/exe  and leave onle the sapcpe exe file

then

Do you have the following  in the START profile

1 ) DIR_CT_RUN = /usr/sap/NP3/SYS/exe/run  ( Check whether the link to exe directory is right )


If not can you make sure that it  ( DIR_CT_RUN ) points to the correct exe directory


2 )

Change the following line from


Execute_01 = immediate $(DIR_CT_RUN)/sapcpe$(FT_EXE) pf=$(_PF)


to


Execute_01 = immediate $(DIR_INSTANCE)/exe/sapcpe$(FT_EXE) pf=$(_PF)



3 ) Remove the parameter


DIR_CT_RUN = /usr/sap/NP3/SYS/exe/run


Restart the SAP system . Kill all process under <sid>adm . Cleanipc


Thanks ,

Manu

Former Member
0 Kudos

Hi Yogesh,

Whats the outcome of suggestion of Manu. also Do not Upgrade the Kernel while doing this as it can bring in inconsistencies.

Also can you provide the output of the below command:-

And you can get the start profiles using below command:-

if your instance is 00 then

sapcontrol -nr 00 -function GetStartProfile | grep protectedwebmethods

Also as your in execution phase, can you let us know if the kernel switch has been completed. if yes to maintain the new kernel in your exe directories.

Let us know the outcome.

Regards,

Ram

manumohandas82
Active Contributor
0 Kudos

Hi Yogesh ,

Please post the SAPCPE log file from the work directory .

Thanks ,

Manu

Former Member
0 Kudos

Hi Manu

SAPCPE log file:

2015/02/24 14:39:26

sapcpe version UC: @(#) $Id: //bas/721_REL/src/ccm/sapstart/sapxxcpe.c#22 $ SAP

Command line:

  0: /usr/sap/NP3/SYS/exe/run/sapcpe

  1: pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

Trace level (profile): 1

** INFO:    Try to find file table sapcpeft (default:sapcpeft)

** INFO:    Try to find file table in the central directory /usr/sap/NP3/SYS/exe/run/sapcpeft

** INFO:    Found file table: /usr/sap/NP3/SYS/exe/run/sapcpeft

*************************************************

source: /usr/sap/NP3/SYS/exe/run

target: /usr/sap/NP3/SYS/exe/run

*************************************************

*** ERROR: identical source/target directory: /usr/sap/NP3/SYS/exe/run /usr/sap/NP3/SYS/exe/run, 2, No such file or directory, [sapxxcpe.c:1655]

Former Member
0 Kudos

Hello Yogesh,

could you attach the content of file /usr/sap/NP3/SYS/exe/run/sapcpeft ?

Thank You

Daniele

Former Member
0 Kudos

I've found the following information in various logs:

SAPCPE

2015/02/23 16:18:51

sapcpe version UC: @(#) $Id: //bas/721_REL/src/ccm/sapstart/sapxxcpe.c#21 $ SAP

Command line:

  0: /usr/sap/NP3/SYS/exe/run/sapcpe

  1: pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

Trace level (profile): 1

** INFO:    Try to find file table sapcpeft (default:sapcpeft)

** INFO:    Try to find file table in the central directory /usr/sap/NP3/SYS/exe/run/sapcpeft

** INFO:    Found file table: /usr/sap/NP3/SYS/exe/run/sapcpeft

*************************************************

source: /usr/sap/NP3/SYS/exe/run

target: /usr/sap/NP3/SYS/exe/run

*************************************************

*** ERROR: identical source/target directory: /usr/sap/NP3/SYS/exe/run /usr/sap/NP3/SYS/exe/run, 2, No such file or directory, [sapxxcpe.c:1655]

SAPSTART

SAP-R/3-Startup Program Rel 721 V2.0 (2008/11/15)

-------------------------------------------------

Starting at 2015/02/23 16:18:50

Startup Profile: "/usr/sap/NP3/SYS/profile/START_DVEBMGS00_EXPSPRD"

Execute Pre-Startup Commands

----------------------------

(4049) Local: /usr/sap/NP3/SYS/exe/run/sapcpe pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

(4043) Failed!

(4043) system(/usr/sap/NP3/SYS/exe/run/sapcpe pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD) returns 3

(4054) Local: /usr/sap/NP3/SYS/exe/run/sapcpe pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD list:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.049/sapjvm_4.lst source:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.049

(4043) Okay!

Starting Programs

-----------------

(4043) Spawning program "/usr/sap/NP3/SYS/exe/run/jcontrol pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD -DSAPSTART=1" asynchronously...

(4043) Okay, child's pid=4099.

(4043) Waiting for Child Processes to terminate.

STD_DISPATCHER


Feb 23, 2015 4:20:32 PM  ...tup.impl1.dispatcher.R3StartupService [SAPEngine_System_Thread[impl:6]_17] Fatal: Couldn't complete initial communication!

Exception is : The socket name is not available on this system.

Feb 23, 2015 4:20:32 PM  ...tup.impl1.dispatcher.R3StartupService [SAPEngine_System_Thread[impl:6]_17] Fatal: StackTrace is :

java.net.NoRouteToHostException: The socket name is not available on this system.

    at java.net.PlainSocketImpl.socketConnect(Native Method)

    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:320)

    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:186)

manumohandas82
Active Contributor
0 Kudos

Hi Yogesh,


1 )  Check whether any soft links exist  from /usr/sap/SID/<instanc>/exe to /sapmnt/<SID>/exe


If there are any softlinks  delete them and copy the filesfrom /sapmnt/SID/exe to /usr/sap/ID/<instanc>/exe

or use sapcpe


if you are using sapcpe use the  command


/sapmnt/NP3/exe /sapcpe pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD ( the files under /usr/sap/NP3/<INSTANCE>/exe  should be cleared )




2 ) check whether the link


/usr/sap/NP3/SYS/exe/run points to the actual /sapmnt/SID/exe directory



Thanks ,

Manu

Former Member
0 Kudos

Hi Yogesh,

Also do check for any links in your profile directory too cause if during any HA or any other configuration if you had created any links for the profiles you need to remove those too.

like of

START_DVEBMGS00_<Physical hostname> to START_DVEBMGS00_<virtual hostname>

Let us know the outcome.

Regards,

Ram

Former Member
0 Kudos

Hi Yogesh,

Also do check for any links in your profile directory too cause if during any HA or any other configuration if you had created any links for the profiles you need to remove those too.

like of

START_DVEBMGS00_<Physical hostname> to START_DVEBMGS00_<virtual hostname>

Let us know the outcome.

Regards,

Ram

Sriram2009
Active Contributor
0 Kudos

Hi Yogesh

1. On IBM-i you can add the host enters, Kindly refer the link point number 3 & 4

AS/400 TCP/IP Configuration

2. Could you refer the SAP Note

969660 - 6.40 Patch Collection Installation : Unix

BR

SS

Former Member
0 Kudos

Hi again Manu

Here is the softlinks and where they are pointing:

I've also copied the files from /sapmnt/NP3/exe to /usr/sap/NP3/DVEBMGS00/exe

Former Member
0 Kudos

Hi again

No links created in profile directory:

Sriram2009
Active Contributor
0 Kudos

Hi Yogresh

Are you able to access the share folder in IBM-i (that on SAP share from your desktop)? If possible do the IPL and then try again.

BR

SS

Former Member
0 Kudos

Hi Manu

Screenshot:

Former Member
0 Kudos

Hi there

I do have the 10.100.0.161 EXPSPRD.PECITY.CO.ZA EXPSPRD entry in the hostfile as per earlier screenshot regarding host file

Sriram2009
Active Contributor
0 Kudos

Hi Yogresh

You can refer the SAP Note for the error message "ERROR CRL Read  QcstListClusterResourceGroups Failed"


2050833 - IBM i: Extra HA Error messages in SAPSTRSRV Joblog and Trace


BR

SS

Former Member
0 Kudos

Hi Yogesh,

the links under /usr/sap/NP2/<instance>/exe should not be there it should be a physical location.

and if the links are under /usr/sap/NP3/SYS/exe those are fine then.

Let us know the details.

regards,

Ram

Former Member
0 Kudos

Hi Yogesh,

Also do keep below parameter in your profile:-

service/protectedwebmethods = NONE

Regards,

Ram

Former Member
0 Kudos

Hi Ramkrishna

I do have that parameter set in the instance profile

former_member185326
Active Participant
0 Kudos

Hi Yogesh,

Have you checked if FQDN is maintained in /hosts file of your system? If not, pls check and maintain that entry. It is crucial for upgrade/ installation in case of Java stacks.

Regards,

Shahid

Former Member
0 Kudos

AS400 does not use /etc/hosts like normal Linux does

I set two parameters for this:

SAPLOCALHOSTFULL = EXPSPRD.PECITY.CO.ZA

icm/host_name_full = EXPSPRD.PECITY.CO.ZA

Host file on OS has this entry for EXPSPRD:

former_member185326
Active Participant
0 Kudos

Hi Yogesh,

You will have to maintain "EXPSPRD.PECITY.CO.ZA" in your hosts file. Check with your AS400 admin to maintain this entry for name resolution.

Regards,

Shahid Ahmed

manumohandas82
Active Contributor
0 Kudos

Hi

From your logs

Trusted http connect via Unix domain socket '/tmp/.sapstream50213' enabled.

Trusted http connect via Unix domain socket '/tmp/.sapstream50213' for <sid>OFR enabled too.

Stop SAP system and sapstartsrv

Please delete the .sapsteram files  in your /tmp location

Also

Can you paste your STD_Server0 logs .

Thanks

Manu

Former Member
0 Kudos

Thank Manu

I've restarted the O/S and SAP, and cleared out the .sapstream files, issue still remains

Former Member
0 Kudos

Hi Manu

STD logs here:

    STD_DISPATCHER


--------------------------------------------------------------------------------

stdout/stderr redirect

--------------------------------------------------------------------------------

node name   : dispatcher

pid         : 4099

system name : NP3

system nr.  : 00

started at  : Mon Feb 23 16:20:25 2015

[Thr 01] MtxInit: 10000 0 2

(CompilerOracle read from file /sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/.hotspot_compiler)

SAP J2EE Engine Version 7.00   PatchLevel 112178.450 is starting...

Loading: LogManager ... 366 ms.

Loading: PoolManager ... 3 ms.

Loading: ThreadManager ... 2.314: [GC2.314: [ParNew: 35712K->2150K(39680K), 0.0088980 secs] 35712K->2150K(258176K), 0.0097750 secs] [Times: user=0.02 sys=0.00, real=0.01 secs]

82 ms.

Loading: IpVerificationManager ... 7 ms.

Loading: ConnectionsManipulator ... 21 ms.

Loading: ClassLoaderManager ... 14 ms.

Loading: ClusterManager ... 191 ms.

Loading: PortsManager ... 16 ms.

Loading: LockingManager ... 59 ms.

Loading: ConfigurationManager ... 3.590: [GC3.590: [ParNew: 37862K->3622K(39680K), 0.0312600 secs] 37862K->5743K(258176K), 0.0316970 secs] [Times: user=0.06 sys=0.01, real=0.03 secs]

2041 ms.

Loading: LicensingManager ... 71 ms.

Loading: CacheManager ... 136 ms.

Loading: ServiceManager ...

Loading services.:

5.376: [GC5.376: [ParNew: 39334K->3967K(39680K), 0.0274010 secs] 41455K->8782K(258176K), 0.0276970 secs] [Times: user=0.02 sys=0.01, real=0.03 secs]

  Service security started. (25 ms).

  Service licensing started. (24 ms).

  Service httptunneling started. (35 ms).

  Service sld started. (30 ms).

Feb 23, 2015 4:20:32 PM  ...tup.impl1.dispatcher.R3StartupService [SAPEngine_System_Thread[impl:6]_17] Fatal: Couldn't complete initial communication!

Exception is : The socket name is not available on this system.

Feb 23, 2015 4:20:32 PM  ...tup.impl1.dispatcher.R3StartupService [SAPEngine_System_Thread[impl:6]_17] Fatal: StackTrace is :

java.net.NoRouteToHostException: The socket name is not available on this system.

    at java.net.PlainSocketImpl.socketConnect(Native Method)

    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:320)

    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:186)

    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:173)

    at java.net.Socket.connect(Socket.java:471)

    at java.net.Socket.connect(Socket.java:417)

    at java.net.Socket.<init>(Socket.java:313)

    at java.net.Socket.<init>(Socket.java:128)

    at com.sap.engine.services.r3startup.impl1.dispatcher.R3StartupService.initializeR3Socket(R3StartupService.java:417)

    at com.sap.engine.services.r3startup.impl1.dispatcher.R3StartupService.start(R3StartupService.java:150)

    at com.sap.engine.services.r3startup.dispatcher.R3StartupService.start(R3StartupService.java:34)

    at com.sap.engine.core.service630.container.ServiceRunner.startCommunicationServiceFrame(ServiceRunner.java:228)

    at com.sap.engine.core.service630.container.ServiceRunner.run(ServiceRunner.java:158)

    at com.sap.engine.frame.core.thread.Task.run(Task.java:64)

    at com.sap.engine.core.thread.impl6.SingleThread.execute(SingleThread.java:82)

    at com.sap.engine.core.thread.impl6.SingleThread.run(SingleThread.java:154)

  Service memory started. (50 ms).

  Service r3startup started. (62 ms).

  Service timeout started. (55 ms).

  Service jmx_notification started. (41 ms).

  Service webservices started. (57 ms).

5.504: [GC5.505: [GC5.505: [ParNew: 14351K->2097K(39680K), 0.0195240 secs] 19166K->9165K(258176K), 0.0197370 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]

[1 CMS-initial-mark: 7068K(218496K)] 9165K(258176K), 0.0222700 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]

5.527: [CMS-concurrent-mark-start]

  Service keystore started. (73 ms).

  Service iiop started. (160 ms).

5.626: [CMS-concurrent-mark: 0.080/0.099 secs] [Times: user=0.13 sys=0.02, real=0.10 secs]

5.627: [CMS-concurrent-preclean-start]

5.637: [CMS-concurrent-preclean: 0.009/0.011 secs] [Times: user=0.01 sys=0.01, real=0.01 secs]

5.638: [CMS-concurrent-abortable-preclean-start]

  Service ssl started. (117 ms).

  Service jmx started. (239 ms).

  Service shell started. (988 ms).

  Service telnet started. (518 ms).

  Service p4 started. (1741 ms).

  Service classload started. (2 ms).

8.855: [GC8.855: [ParNew: 37809K->3038K(39680K), 0.0089340 secs] 44877K->10106K(258176K), 0.0093320 secs] [Times: user=0.02 sys=0.00, real=0.01 secs]

  Service log_configurator started. (4186 ms).

  Service configuration started. (3 ms).

  Service basicadmin started. (227 ms).

9.909: [CMS-concurrent-abortable-preclean: 1.257/4.271 secs] [Times: user=2.84 sys=0.33, real=4.27 secs]

9.910: [GC[YG occupancy: 21817 K (39680 K)]9.911: [Rescan (parallel) , 0.0096720 secs]9.920: [weak refs processing, 0.0012610 secs]9.922: [class unloading, 0.0041280 secs]9.926: [scrub symbol table, 0.0060700 secs]9.932: [scrub string table, 0.0005950 secs] [1 CMS-remark: 7068K(218496K)] 28886K(258176K), 0.0251890 secs] [Times: user=0.03 sys=0.00, real=0.02 secs]

9.936: [CMS-concurrent-sweep-start]

  Service adminadapter started. (26 ms).

9.986: [CMS-concurrent-sweep: 0.033/0.050 secs] [Times: user=0.08 sys=0.01, real=0.05 secs]

9.987: [CMS-concurrent-reset-start]

9.988: [CMS-concurrent-reset: 0.001/0.001 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]

  Service http started. (377 ms).

  Service jms_provider started. (367 ms).

10.140: [GC10.140: [ParNew: 38750K->3185K(39680K), 0.0269230 secs] 45611K->12429K(258176K), 0.0272250 secs] [Times: user=0.02 sys=0.01, real=0.02 secs]

Could not open the ICU common library.

   The following files must be in the path described by

   the environment variable "LIBPATH":

   libicuuc30.a, libicudata30.a, libicui18n30.a [nlsui0_mt.c 1489] pid = 5032

STD_SERVER0


--------------------------------------------------------------------------------

stdout/stderr redirect

--------------------------------------------------------------------------------

node name   : server0

pid         : 4099

system name : NP3

system nr.  : 00

started at  : Mon Feb 23 16:22:35 2015

[Thr 01] MtxInit: 10001 0 2

(CompilerOracle read from file /sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/.hotspot_compiler)

SAP J2EE Engine Version 7.00   PatchLevel 112178.450 is starting...

Loading: LogManager ... 340 ms.

Loading: PoolManager ... 4 ms.

Loading: ApplicationThreadManager ... 3.553: [GC3.554: [ParNew: 409600K->2732K(512000K), 0.0277240 secs] 409600K->2732K(1994752K), 0.0286600 secs] [Times: user=0.06 sys=0.00, real=0.03 secs]

176 ms.

Loading: ThreadManager ... 47 ms.

Loading: IpVerificationManager ... 7 ms.

Loading: ClassLoaderManager ... 11 ms.

Loading: ClusterManager ... 152 ms.

Loading: LockingManager ... 47 ms.

Loading: ConfigurationManager ... 1602 ms.

Loading: LicensingManager ... 78 ms.

Loading: CacheManager ... 85 ms.

Loading: ServiceManager ...

Loading services.:

  Service DQE started. (5 ms).

  Service caf~eu~odi~mnuacc started. (0 ms).

  Service memory started. (48 ms).

  Service cross started. (30 ms).

  Service file started. (83 ms).

  Service runtimeinfo started. (19 ms).

  Service caf~eu~cc~api started. (46 ms).

  Service timeout started. (100 ms).

  Service trex.service started. (55 ms).

  Service userstore started. (11 ms).

  Service jmx_notification started. (18 ms).

  Service p4 started. (151 ms).

  Service classpath_resolver started. (25 ms).

11.391: [GC11.391: [ParNew: 412332K->34848K(512000K), 0.0478220 secs] 412332K->34848K(1994752K), 0.0482350 secs] [Times: user=0.11 sys=0.01, real=0.05 secs]

15.276: [GC15.276: [ParNew: 444448K->51200K(512000K), 0.0457200 secs] 444448K->51200K(1994752K), 0.0461340 secs] [Times: user=0.11 sys=0.01, real=0.04 secs]

  Service deploy started. (9765 ms).

  Service com.sap.portal.runtime.config.kmreadonly started. (5 ms).

  Service MigrationService started. (27 ms).

  Service bi~mmr~deployer started. (5 ms).

  Service log_configurator started. (13079 ms).

  Service locking started. (3 ms).

  Service http started. (137 ms).

  Service naming started. (189 ms).

  Service failover started. (31 ms).

  Service licensing started. (37 ms).

  Service javamail started. (61 ms).

  Service appclient started. (51 ms).

  Service ts started. (59 ms).

  Service jmsconnector started. (72 ms).

  Service connector started. (122 ms).

  Service configuration started. (21 ms).

  Service MobileArchiveContainer started. (24 ms).

  Service iiop started. (158 ms).

  Service MobileSetupGeneration started. (59 ms).

  Service dbpool started. (2702 ms).

  Service UT started. (7 ms).

  Service caf~eu~gp~mail~cf started. (35 ms).

  Service com.sap.security.core.ume.service started. (966 ms).

  Service tc~di~sdic~srv started. (571 ms).

  Service security started. (605 ms).

  Service classload started. (24 ms).

  Service applocking started. (44 ms).

  Service shell started. (67 ms).

  Service tc~eCATTPing~service started. (14 ms).

  Service telnet started. (28 ms).

  Service developmentserver started. (68 ms).

  Service keystore started. (231 ms).

  Service com.sap.portal.fpn.service started. (7 ms).

  Service ssl started. (13 ms).

  Service ejb started. (285 ms).

  Service tc~sec~securestorage~service started. (73 ms).

  Service dsr started. (179 ms).

  Service tc~sec~certrevoc~service started. (155 ms).

  Service caf~um~metadata~imp started. (425 ms).

  Service servlet_jsp started. (486 ms).

  Service caf~eu~gp~model started. (22 ms).

  Service caf~eu~er~service started. (0 ms).

  Service tc~eu~jwf~ui~wizsvc started. (28 ms).

  Service jmx started. (469 ms).

25.854: [GC25.855: [ParNew: 460800K->54338K(512000K), 0.0662950 secs] 460800K->54338K(1994752K), 0.0666930 secs] [Times: user=0.17 sys=0.00, real=0.07 secs]

  Service sld started. (475 ms).

  Service prtbridge started. (318 ms).

  Service com.adobe~FontManagerService started. (312 ms).

  Service tc~lm~ctc~confs~service_sda started. (553 ms).

  Service com.adobe~DocumentServicesLicenseSupportService started. (220 ms).

  Service com.adobe~PDFManipulation started. (228 ms).

  Service com.adobe~DataManagerService started. (331 ms).

  Service com.adobe~XMLFormService started. (204 ms).

  Service com.adobe~DocumentServicesBinaries2 started. (345 ms).

  Service CUL started. (579 ms).

  Service apptracing started. (158 ms).

Feb 23, 2015 4:23:02 PM  ...utilxi.prop.api.PropertySourceFactory [SAPEngine_System_Thread[impl:5]_67] Fatal: XI properties could not be initialized. Check ExchangeProfile (or aii.properties).

Thrown:

java.lang.Throwable: dummy Throwable for stack trace

    at com.sap.aii.utilxi.prop.api.PropertySourceFactory.getPropertySource(PropertySourceFactory.java:55)

    at com.sap.aii.utilxi.misc.api.AIIProperties.sync(AIIProperties.java:607)

    at com.sap.aii.utilxi.misc.api.AIIProperties.<init>(AIIProperties.java:378)

    at com.sap.aii.utilxi.misc.api.AIIProperties.getInstance(AIIProperties.java:405)

    at com.sap.aii.af.service.sld.SLDAccess.syncExchangeProfile(SLDAccess.java:43)

    at com.sap.aii.af.service.cpa.impl.util.SLDReader.run(SLDReader.java:89)

    at com.sap.engine.frame.core.thread.Task.run(Task.java:64)

    at com.sap.engine.core.thread.impl5.SingleThread.execute(SingleThread.java:83)

    at com.sap.engine.core.thread.impl5.SingleThread.run(SingleThread.java:156)

  Service caf~um~relgroups~imp started. (1002 ms).

  Service basicadmin started. (935 ms).

  Service com.adobe~LicenseService started. (50 ms).

  Service com.adobe~DocumentServicesConfiguration started. (134 ms).

  Service adminadapter started. (191 ms).

  Service com.sap.portal.pcd.gl started. (10 ms).

  Service com.sap.nw.kmc.readonly started. (9 ms).

  Service com.sap.nw.kmc.readonlymbean started. (18 ms).

  Service webservices started. (1265 ms).

  Service caf~runtime~connectivity~impl started. (19745 ms).

  Service com.sap.portal.prt.sapj2ee started. (127 ms).

  Service com.adobe~TrustManagerService started. (136 ms).

Could not open the ICU common library.

   The following files must be in the path described by

   the environment variable "LIBPATH":

   libicuuc30.a, libicudata30.a, libicui18n30.a [nlsui0_mt.c 1489] pid = 5850

Former Member
0 Kudos

std_dispatcher shows following entry


Feb 23, 2015 2:39:13 PM  ...tup.impl1.dispatcher.R3StartupService [SAPEngine_System_Thread[impl:6]_28] Fatal: Couldn't complete initial communication!

Exception is : The socket name is not available on this system.

Feb 23, 2015 2:39:13 PM  ...tup.impl1.dispatcher.R3StartupService [SAPEngine_System_Thread[impl:6]_28] Fatal: StackTrace is :

java.net.NoRouteToHostException: The socket name is not available on this system.

    at java.net.PlainSocketImpl.socketConnect(Native Method)

    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:320)

    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:186)

    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:173)

    at java.net.Socket.connect(Socket.java:471)

    at java.net.Socket.connect(Socket.java:417)

    at java.net.Socket.<init>(Socket.java:313)

    at java.net.Socket.<init>(Socket.java:128)

    at com.sap.engine.services.r3startup.impl1.dispatcher.R3StartupService.initializeR3Socket(R3StartupService.java:417)

    at com.sap.engine.services.r3startup.impl1.dispatcher.R3StartupService.start(R3StartupService.java:150)

    at com.sap.engine.services.r3startup.dispatcher.R3StartupService.start(R3StartupService.java:34)

    at com.sap.engine.core.service630.container.ServiceRunner.startCommunicationServiceFrame(ServiceRunner.java:228)

    at com.sap.engine.core.service630.container.ServiceRunner.run(ServiceRunner.java:158)

    at com.sap.engine.frame.core.thread.Task.run(Task.java:64)

    at com.sap.engine.core.thread.impl6.SingleThread.execute(SingleThread.java:82)

    at com.sap.engine.core.thread.impl6.SingleThread.run(SingleThread.java:154)

former_member185239
Active Contributor
0 Kudos

Hi Yogesh,

Follow the below steps:

1. Open the configtool and bring the system to Normal mode. File-->Safe mode select No for each instance.

2. Then try to start the java system.

With Regards

Ashutosh Chaturvedi

Former Member
0 Kudos

I've tried doing this earlier, still didn't startup SAP java

former_member185326
Active Participant
0 Kudos

Yogesh,

You need to add FQDN hostname entry in your /etc/hosts file and try.

Regards,

Shahid Ahmed

Former Member
0 Kudos

Hi Yogesh,

Can you please attach the dev_Server0 & dev_disp logs and .std, *err & *.out log files too.

Also while going thorugh the log seen one jar file missing can you check on it too.

==================================

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

==================================

But this may not be the issue.

From dev_disp & dev_server0 might give us a better picture.

Regards,

Ram

Former Member
0 Kudos

This is an AS400 machine, and gives the following output:

Former Member
0 Kudos

Hi Yogesh,

There is one thing more i have noticed, Do you have the gateway separately installed and did you added the gateway startup parameter in the start profile, if yes.

==============================================

Feb 23, 2015 10:37:59 AM [Error ]: Return code condition success evaluated to false for process sapcontrol for action wait for started.

Feb 23, 2015 10:37:59 AM [Error ]: The following problem has occurred during step execution: com.sap.sdt.util.diag.DiagException: Could not start SAP instance with number 0.

Could not check if the instance number 0 on host EXPSPRD is started.

Detected instance wait for started has an GATEWAY feature. It cannot be handled by SUM and will be ignored.

Return code condition success evaluated to false for process sapcontrol for action wait for started.

==============================================

We had similar incident for solman where in the CCMS agent startup was mentioned in start profile. And while starting the sapcontrol it was always treating as a separate instance and due to which the java and sapcontrol was failing. After removing that parameter we were able to start the system.

Do remove the parameter temporarily from the Startup profile. And try to start the instance again.

Update us on the same.

Regards,

Ram

former_member185326
Active Participant
0 Kudos

Check if you have "EXPSPRD.PECITY.CO.ZA" entry in /hosts file for name resolution. Check with your AS400 Admin how to add and get that entry created against your IP address.

Regards,

Shahid Ahmed

Former Member
0 Kudos

Please see attached

Former Member
0 Kudos

Hi again

What parameter did you disable ? only parameters I have referring to the gateway are located in the shadow instance profile <NP3_DVEBMGS02_EXPSPRD> [02 being shadown instance]

Has the following GW entries:

# Modify instance number during downtime

rdisp/mshost = EXPSPRD

rdisp/msserv = 3602

rdisp/msserv_internal = 3902

rdisp_sna_gw_service = sapgw02

Former Member
0 Kudos

Hi Yogesh,

For gateway, Do check in your start profile if you have separate parameters for the gateway instanc if yes do hash them up.

Also you can hash out this parameters so that the default gateway port of your instance can be used during upgrade.

And you can get the start profiles using below command:-

if your instance is 00 then

sapcontrol -nr 00 -function GetStartProfile | grep protectedwebmethods

Let us know whats the outcome of this.

Regards,

Ram

manumohandas82
Active Contributor
0 Kudos

Hi Yogesh ,

Please share the bootstrap logs + std_server logs .

Thanks ,

Manu

Former Member
0 Kudos

Hi Yogesh,

Do share the sapstartsrv and instance logs from /sum/abap/log or your /sum/java/log.

And SDM log from the same directory.

Let us know the details.

Regards,

Ram

Former Member
0 Kudos

hi Yogesh,

Also do try to start the system manually if you can using startsap exe from /sum/abap/exe & profiles from your /sum/<SID>/profile directory may be this can help too.

Let us know the outcome.

Regards,

Ram

Former Member
0 Kudos

Ok, I've tried adding the files, bu attachments are limited:


DEV_JCONTROL



DEV_JCONTROL

---------------------------------------------------

trc file: "dev_jcontrol", trc level: 1, release: "721"

---------------------------------------------------

node name  : jcontrol

pid        : 102211

system name : NP3

system nr.  : 00

started at  : Mon Feb 23 14:38:04 2015

arguments      :

      arg[00] : /usr/sap/NP3/SYS/exe/run/jcontrol

      arg[01] : pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

      arg[02] : -DSAPSTART=1

**********************************************************************

Used property files

-> files [00] : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> files [01] : /usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties

**********************************************************************

**********************************************************************

Bootstrap nodes

-> [00] bootstrap            : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [01] bootstrap_ID9224300  : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [02] bootstrap_ID9224350  : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

**********************************************************************

**********************************************************************

Worker nodes

-> [00] ID9224300            : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [01] ID9224350            : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [02] sdm                  : /usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties

**********************************************************************

**********************************************************************

JStartupReadInstanceProperties: read instance properties [/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties:/usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties]

-> ms host    : EXPSPRD

-> ms port    : 3901

-> OS libs    : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> Admin URL  :

-> run mode  : normal

-> run action : NONE

-> enabled    : yes

**********************************************************************

[Thr 01] Mon Feb 23 14:38:05 2015

[Thr 01] JStartupIOS4JDKInit: OS is 7.1.

[Thr 01] JStartupIOS4JDKInit: OS4DefaultProperties file /usr/sap/NP3/SYS/j2ee/OS4Default.properties rewritten.

[Thr 01] SigISetDefaultAction : default handling for signal SIGCHLD

[Thr 01] [Node: bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap]

-> node name          : bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID0092243

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] JControlExecuteBootstrap: execute bootstrap process [bootstrap]

[Thr 01] [Node: bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap]

-> node name          : bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID0092243

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] JStartupIGetJ2EELibrary: Default J2EE library computed as: SAPJ2EE00.

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_bootstrap

-> arg[03] = -nodeId=-1

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=bootstrap

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_bootstrap.out

-> arg[08] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_bootstrap.out

-> arg[09] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_bootstrap

-> arg[10] = -mode=BOOTSTRAP

-> arg[11] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[12] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] JStartupICreateProcess: fork process (pid 102215)

[Thr 01] Mon Feb 23 14:38:10 2015

[Thr 01] JControlExecuteBootstrap: read instance values after global bootstrap

**********************************************************************

JStartupReadInstanceProperties: read instance properties [/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties:/usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties]

-> ms host    : EXPSPRD

-> ms port    : 3901

-> OS libs    : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> Admin URL  :

-> run mode  : normal

-> run action : NONE

-> enabled    : yes

**********************************************************************

[Thr 01] JControlExecuteBootstrap: enumerate the nodes after global bootstrap

**********************************************************************

Used property files

-> files [00] : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> files [01] : /usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties

**********************************************************************

**********************************************************************

Bootstrap nodes

-> [00] bootstrap            : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [01] bootstrap_ID9224300  : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [02] bootstrap_ID9224350  : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

**********************************************************************

**********************************************************************

Worker nodes

-> [00] ID9224300            : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [01] ID9224350            : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> [02] sdm                  : /usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties

**********************************************************************

**********************************************************************

JStartupReadInstanceProperties: read instance properties [/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties:/usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties]

-> ms host    : EXPSPRD

-> ms port    : 3901

-> OS libs    : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> Admin URL  :

-> run mode  : normal

-> run action : NONE

-> enabled    : yes

**********************************************************************

[Thr 01] [Node: dispatcher bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap_ID9224300]

-> node name          : dispatcher bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID009224300

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] [Node: server0 bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap_ID9224350]

-> node name          : server0 bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID009224350

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] [Node: bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap]

-> node name          : bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID0092243

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] [Node: dispatcher bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap_ID9224300]

-> node name          : dispatcher bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID009224300

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] JControlExecuteBootstrap: execute bootstrap process [bootstrap_ID9224300]

[Thr 01] [Node: dispatcher bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap_ID9224300]

-> node name          : dispatcher bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID009224300

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_bootstrap

-> arg[03] = -nodeId=-1

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=bootstrap_ID9224300

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_bootstrap_ID9224300.out

-> arg[08] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_bootstrap_ID9224300.out

-> arg[09] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_bootstrap_ID9224300

-> arg[10] = -mode=BOOTSTRAP

-> arg[11] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[12] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] JStartupICreateProcess: fork process (pid 102227)

[Thr 01] Mon Feb 23 14:38:16 2015

[Thr 01] [Node: server0 bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap_ID9224350]

-> node name          : server0 bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID009224350

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] JControlExecuteBootstrap: execute bootstrap process [bootstrap_ID9224350]

[Thr 01] [Node: server0 bootstrap] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap_ID9224350]

-> node name          : server0 bootstrap

-> node type          : bootstrap

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -Djco.jarm=1

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 256M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster

-> class path        : ./bootstrap/launcher.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.offline.OfflineToolStart

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID009224350

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_bootstrap

-> arg[03] = -nodeId=-1

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=bootstrap_ID9224350

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_bootstrap_ID9224350.out

-> arg[08] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_bootstrap_ID9224350.out

-> arg[09] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_bootstrap_ID9224350

-> arg[10] = -mode=BOOTSTRAP

-> arg[11] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[12] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] JStartupICreateProcess: fork process (pid 102239)

[Thr 258] Mon Feb 23 14:38:21 2015

[Thr 258] JControlRequestFunc: Thread 258 started as listener thread for np messages.

[Thr 515] JControlSocketServerFunc: Thread 515 started as handler thread for control socket.

[Thr 01] JControlIBuildProcessList: Maximum error count is set to 4

[Thr 01] [Node: dispatcher] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [ID9224300]

-> node name          : dispatcher

-> node type          : dispatcher

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -verbose:gc -Djava.security.policy=./java.policy -Djco.jarm=1 -XX:+UseConcMarkSweepGC -Dcom.sap.jvm.scenario=j2ee -Djava.security.policy=.java.policy -XX:+PrintGCDetails -XX:+PrintGCTimeStamps

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 512M

-> init heap size    : 256M

-> stack size        : 2M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/dispatcher

-> class path        : ./bin/boot/boot.jar:./bin/system/bytecode.jar:.

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.boot.Start

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        :

-> debuggable        : no

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] [Node: server0] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [ID9224350]

-> node name          : server0

-> node type          : server

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -verbose:gc -Djco.jarm=1 -Djava.security.policy=./java.policy -Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy -Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy -Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy -XX:MaxNewSize=600M -XX:NewSize=600M -XX:MaxPermSize=512M -XX:PermSize=512M -XX:+UseConcMarkSweepGC -XX:+DisableExplicitGC -XX:TargetSurvivorRatio=90 -XX:SurvivorRatio=4 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:SoftRefLRUPolicyMSPerMB=1 -XX:HeapDumpPath=OOM.hprof -XX:+HeapDumpOnOutOfMemoryError -Djava.awt.headless=true -Dsun.io.useCanonCaches=false -Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine.services.ts.jts.ots.PortableInterceptor.JTSInitializer -Dcom.sap.jvm.scenario=j2ee

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 2048M

-> init heap size    : 2048M

-> stack size        : 2M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/server0

-> class path        : ./bin/boot/boot.jar:./bin/boot/jaas.jar:./bin/system/bytecode.jar:.

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.boot.Start

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> parameters        :

-> debuggable        : no

-> debug mode        : no

-> debug port        : 50021

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] INFO: Invalid property value [Debbugable=yes]

[Thr 01] [Node: SDM] java home is set by profile parameter

[Thr 01]    Java Home: /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

[Thr 01] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/NP3/SYS/exe/run/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [sdm]

-> node name          : SDM

-> node type          : sdm

-> node execute      : yes

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    :

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 2048M

-> root path          : /usr/sap/NP3/DVEBMGS00/SDM/program

-> class path        : /usr/sap/NP3/DVEBMGS00/SDM/program/bin/SDM.jar

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : SDMInternal

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> shutdown class    : com.sap.sdm.jstartup.shutdown.InternalShutDown

-> parameters        : server sdmhome=/usr/sap/NP3/DVEBMGS00/SDM/program

-> debuggable        : yes

-> debug mode        : no

-> debug port        : 60000

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] JControlMSConnect: attached to message server (EXPSPRD/3901)

[Thr 772] JControlMSMessageFunc: Thread 772 started as listener thread for ms messages.

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_dispatcher

-> arg[03] = -nodeId=0

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224300

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_dispatcher.out

-> arg[08] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_dispatcher.out

-> arg[09] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_dispatcher

-> arg[10] = -mode=JCONTROL

-> arg[11] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[12] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] JStartupICreateProcess: fork process (pid 102251)

[Thr 01] JControlICheckProcessList: process dispatcher started (PID:102251)

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_server

-> arg[03] = -nodeId=1

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224350

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_server0.out

-> arg[08] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_server0.out

-> arg[09] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_server0

-> arg[10] = -mode=JCONTROL

-> arg[11] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[12] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] JStartupICreateProcess: fork process (pid 102252)

[Thr 01] JControlICheckProcessList: process server0 started (PID:102252)

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_sdm

-> arg[03] = -nodeId=2

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/SDM/program/config/sdm_jstartup.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=sdm

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_sdm.out

-> arg[08] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_sdm.out

-> arg[09] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_sdm

-> arg[10] = -mode=JCONTROL

-> arg[11] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[12] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] JStartupICreateProcess: fork process (pid 102254)

[Thr 01] JControlICheckProcessList: process SDM started (PID:102254)

[Thr 01] JControlMSSendLogon: delete SDM logon information (rc=0)

[Thr 01] Mon Feb 23 14:38:31 2015

[Thr 01] JControlMSSendLogon: insert SDM logon information (port=50018; rc=0)

[Thr 01] Mon Feb 23 14:38:36 2015

[Thr 01] JControlICheckProcessList: process dispatcher (pid:102251) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process dispatcher

[Thr 01] JControlIResetProcess: [dispatcher] not running -> increase error count (1)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224300 1424695101 -bz /usr/sap/NP3/SYS/global

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_dispatcher

-> arg[03] = -nodeId=0

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224300

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_dispatcher.out

-> arg[08] = -jvmOutMode=append

-> arg[09] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_dispatcher.out

-> arg[10] = -stdOutMode=append

-> arg[11] = -traceMode=append

-> arg[12] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_dispatcher

-> arg[13] = -mode=JCONTROL

-> arg[14] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[15] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] Mon Feb 23 14:38:37 2015

[Thr 01] JStartupICreateProcess: fork process (pid 102285)

[Thr 01] JControlICheckProcessList: process dispatcher started (PID:102285)

[Thr 01] Mon Feb 23 14:38:52 2015

[Thr 01] JControlICheckProcessList: process dispatcher (pid:102285) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process dispatcher

[Thr 01] JControlIResetProcess: [dispatcher] not running -> increase error count (2)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224300 1424695117 -bz /usr/sap/NP3/SYS/global

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_dispatcher

-> arg[03] = -nodeId=0

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224300

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_dispatcher.out

-> arg[08] = -jvmOutMode=append

-> arg[09] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_dispatcher.out

-> arg[10] = -stdOutMode=append

-> arg[11] = -traceMode=append

-> arg[12] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_dispatcher

-> arg[13] = -mode=JCONTROL

-> arg[14] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[15] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] Mon Feb 23 14:38:53 2015

[Thr 01] JStartupICreateProcess: fork process (pid 102306)

[Thr 01] JControlICheckProcessList: process dispatcher started (PID:102306)

[Thr 01] JControlICheckProcessList: process server0 (pid:102252) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process server0

[Thr 01] JControlIResetProcess: [server0] not running -> increase error count (1)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224350 1424695101 -bz /usr/sap/NP3/SYS/global

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_server

-> arg[03] = -nodeId=1

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224350

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_server0.out

-> arg[08] = -jvmOutMode=append

-> arg[09] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_server0.out

-> arg[10] = -stdOutMode=append

-> arg[11] = -traceMode=append

-> arg[12] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_server0

-> arg[13] = -mode=JCONTROL

-> arg[14] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[15] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] Mon Feb 23 14:38:54 2015

[Thr 01] JStartupICreateProcess: fork process (pid 102312)

[Thr 01] JControlICheckProcessList: process server0 started (PID:102312)

[Thr 01] Mon Feb 23 14:39:04 2015

[Thr 01] JControlICheckProcessList: process dispatcher (pid:102306) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process dispatcher

[Thr 01] JControlIResetProcess: [dispatcher] not running -> increase error count (3)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224300 1424695133 -bz /usr/sap/NP3/SYS/global

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_dispatcher

-> arg[03] = -nodeId=0

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224300

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_dispatcher.out

-> arg[08] = -jvmOutMode=append

-> arg[09] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_dispatcher.out

-> arg[10] = -stdOutMode=append

-> arg[11] = -traceMode=append

-> arg[12] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_dispatcher

-> arg[13] = -mode=JCONTROL

-> arg[14] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[15] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] Mon Feb 23 14:39:05 2015

[Thr 01] JStartupICreateProcess: fork process (pid 102337)

[Thr 01] JControlICheckProcessList: process dispatcher started (PID:102337)

[Thr 01] Mon Feb 23 14:39:20 2015

[Thr 01] JControlICheckProcessList: process dispatcher (pid:102337) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process dispatcher

[Thr 01] JControlIResetProcess: [dispatcher] not running -> increase error count (4)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224300 1424695145 -bz /usr/sap/NP3/SYS/global

[Thr 01] Mon Feb 23 14:39:26 2015

[Thr 01] JControlICheckProcessList: process server0 (pid:102312) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process server0

[Thr 01] JControlIResetProcess: [server0] not running -> increase error count (2)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224350 1424695134 -bz /usr/sap/NP3/SYS/global

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_server

-> arg[03] = -nodeId=1

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224350

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_server0.out

-> arg[08] = -jvmOutMode=append

-> arg[09] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_server0.out

-> arg[10] = -stdOutMode=append

-> arg[11] = -traceMode=append

-> arg[12] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_server0

-> arg[13] = -mode=JCONTROL

-> arg[14] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[15] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] Mon Feb 23 14:39:27 2015

[Thr 01] JStartupICreateProcess: fork process (pid 102362)

[Thr 01] JControlICheckProcessList: process server0 started (PID:102362)

[Thr 01] Mon Feb 23 14:39:52 2015

[Thr 01] JControlICheckProcessList: process server0 (pid:102362) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process server0

[Thr 01] JControlIResetProcess: [server0] not running -> increase error count (3)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224350 1424695166 -bz /usr/sap/NP3/SYS/global

**********************************************************************

JControlStartJLaunch: program = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[00] = /usr/sap/NP3/SYS/exe/run/jlaunch

-> arg[01] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[02] = -DSAPINFO=NP3_00_server

-> arg[03] = -nodeId=1

-> arg[04] = -file=/usr/sap/NP3/DVEBMGS00/j2ee/cluster/instance.properties

-> arg[05] = -syncSem=107472

-> arg[06] = -nodeName=ID9224350

-> arg[07] = -jvmOutFile=/usr/sap/NP3/DVEBMGS00/work/jvm_server0.out

-> arg[08] = -jvmOutMode=append

-> arg[09] = -stdOutFile=/usr/sap/NP3/DVEBMGS00/work/std_server0.out

-> arg[10] = -stdOutMode=append

-> arg[11] = -traceMode=append

-> arg[12] = -locOutFile=/usr/sap/NP3/DVEBMGS00/work/dev_server0

-> arg[13] = -mode=JCONTROL

-> arg[14] = pf=/usr/sap/NP3/SYS/profile/NP3_DVEBMGS00_EXPSPRD

-> arg[15] = -DSAPSTART=1

-> lib path = LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> exe path = PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

**********************************************************************

[Thr 01] JStartupICreateProcess: fork process (pid 102383)

[Thr 01] JControlICheckProcessList: process server0 started (PID:102383)

[Thr 01] Mon Feb 23 14:40:18 2015

[Thr 01] JControlICheckProcessList: process server0 (pid:102383) died (RUN-FLAG)

[Thr 01] JControlIResetProcess: reset process server0

[Thr 01] JControlIResetProcess: [server0] not running -> increase error count (4)

[Thr 01] JControlICheckProcessList: running flight recorder:

[Thr 01]    /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin/java -classpath ../j2ee/cluster/bootstrap/sap.com~tc~bl~offline_launcher~impl.jar com.sap.engine.offline.OfflineToolStart com.sap.engine.flightrecorder.core.Collector ../j2ee/cluster/bootstrap -node ID9224350 1424695192 -bz /usr/sap/NP3/SYS/global

DEV_SERVER0



**********************************************************************

JStartupIReadSection: read node properties [ID9224350]

-> node name          : server0

-> node type          : server

-> node execute      : yes

-> jlaunch parameters :

-> java path          : /usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4

-> java parameters    : -verbose:gc -Djco.jarm=1 -Djava.security.policy=./java.policy -Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy -Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy -Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy -XX:MaxNewSize=600M -XX:NewSize=600M -XX:MaxPermSize=512M -XX:PermSize=512M -XX:+UseConcMarkSweepGC -XX:+DisableExplicitGC -XX:TargetSurvivorRatio=90 -XX:SurvivorRatio=4 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:SoftRefLRUPolicyMSPerMB=1 -XX:HeapDumpPath=OOM.hprof -XX:+HeapDumpOnOutOfMemoryError -Djava.awt.headless=true -Dsun.io.useCanonCaches=false -Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine.services.ts.jts.ots.PortableInterceptor.JTSInitializer -Dcom.sap.jvm.scenario=j2ee

-> java vm version    : 1.4.2_34

-> java vm vendor    : SAP Java Server VM (SAP AG)

-> java vm type      : server

-> java vm cpu        : ppc64

-> heap size          : 2048M

-> init heap size    : 2048M

-> stack size        : 2M

-> root path          : /usr/sap/NP3/DVEBMGS00/j2ee/cluster/server0

-> class path        : ./bin/boot/boot.jar:./bin/boot/jaas.jar:./bin/system/bytecode.jar:.

-> OS libs path      : /usr/sap/NP3/DVEBMGS00/j2ee/os_libs

-> main class        : com.sap.engine.boot.Start

-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class    : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path    : /usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar

-> shutdown class    : com.sap.engine.boot.Start

-> parameters        :

-> debuggable        : no

-> debug mode        : no

-> debug port        : 50021

-> shutdown timeout  : 120 sec.

**********************************************************************

[Thr 01] JLaunchISetDebugMode: set debug mode [no]

[Thr 01] JStartupIGetILEExecutablePath: original setting of environment variable PATH in ILE: /usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/bin:.

[Thr 01] JHVM_SetExecutablePath: Successfully set in ILE: PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/bin:.

[Thr 01] JHVM_SetExecutablePath: Successfully set in ILE: ILE_PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/bin:.

[Thr 01] JHVM_SetExecutablePath: Successfully set in PASE: ILE_PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/bin:.

[Thr 01] JHVM_SetExecutablePath: Successfully set in PASE: PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

[Thr 01] JHVM_SetExecutablePath: Successfully set in ILE: PASE_PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

[Thr 01] JHVM_SetExecutablePath: Successfully set in PASE: PASE_PATH=/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/SYS/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:/usr/bin:.:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/bin

[Thr 01] JStartupIGetJ2EELibrary: Default J2EE library computed as: SAPJ2EE00.

[Thr 01] JStartupIGetILELibraryPath: original setting of environment variable LIBPATH in ILE: /usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

[Thr 01] JHVM_SetLibraryPath: Successfully set in ILE: LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

[Thr 01] JHVM_SetLibraryPath: Successfully set in ILE: ILE_LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

[Thr 01] JHVM_SetLibraryPath: Successfully set in PASE: ILE_LIBPATH=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

[Thr 772] JLaunchIStartFunc: Thread 772 started as Java VM thread.

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_USE_DESCRIPTOR_STDIO=N already set

[Thr 772] JStartupIPaseInitJavaEnvironment: Setting QIBM_USE_DESCRIPTOR_STDIO=Y

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_USE_DESCRIPTOR_STDIO=N already set in ILE

[Thr 772] JStartupIPaseInitJavaEnvironment: Setting QIBM_USE_DESCRIPTOR_STDIO=Y in ILE

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_PASE_DESCRIPTOR_STDIO=T already set

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_PASE_DESCRIPTOR_STDIO=T already set in ILE

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_JAVA_PASE_ALLOW_PREV=1 already set

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_JAVA_PASE_ALLOW_PREV=1 already set in ILE

[Thr 772] JStartupIPaseInitJavaEnvironment: PASE_THREAD_ATTACH=Y already set

[Thr 772] JStartupIPaseInitJavaEnvironment: PASE_THREAD_ATTACH=Y already set in ILE

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_JAVA_PASE_STARTUP=/usr/lib/start64 already set

[Thr 772] JStartupIPaseInitJavaEnvironment: QIBM_JAVA_PASE_STARTUP=/usr/lib/start64 already set in ILE

**********************************************************************

JHVM_LoadJavaVM: VM Arguments of node [server0]

-> stack  : 1048576 Bytes

-> arg[  0]: exit

-> arg[  1]: abort

-> arg[  2]: vfprintf

-> arg[  3]: -verbose:gc

-> arg[  4]: -Djco.jarm=1

-> arg[  5]: -Djava.security.policy=./java.policy

-> arg[  6]: -Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy

-> arg[  7]: -Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy

-> arg[  8]: -Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy

-> arg[  9]: -XX:MaxNewSize=600M

-> arg[ 10]: -XX:NewSize=600M

-> arg[ 11]: -XX:MaxPermSize=512M

-> arg[ 12]: -XX:PermSize=512M

-> arg[ 13]: -XX:+UseConcMarkSweepGC

-> arg[ 14]: -XX:+DisableExplicitGC

-> arg[ 15]: -XX:TargetSurvivorRatio=90

-> arg[ 16]: -XX:SurvivorRatio=4

-> arg[ 17]: -XX:+PrintGCDetails

-> arg[ 18]: -XX:+PrintGCTimeStamps

-> arg[ 19]: -XX:SoftRefLRUPolicyMSPerMB=1

-> arg[ 20]: -XX:HeapDumpPath=OOM.hprof

-> arg[ 21]: -XX:+HeapDumpOnOutOfMemoryError

-> arg[ 22]: -Djava.awt.headless=true

-> arg[ 23]: -Dsun.io.useCanonCaches=false

-> arg[ 24]: -Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine.services.ts.jts.ots.PortableInterceptor.JTSInitializer

-> arg[ 25]: -Dcom.sap.jvm.scenario=j2ee

-> arg[ 26]: -Dsys.global.dir=/usr/sap/NP3/SYS/global

-> arg[ 27]: -Dapplication.home=/sapmnt/NP3/exe

-> arg[ 28]: -Djava.class.path=/usr/sap/NP3/SYS/exe/run/jstartup.jar:/usr/sap/NP3/SYS/exe/run/jvmx.jar:./bin/boot/boot.jar:./bin/boot/jaas.jar:./bin/system/bytecode.jar:.

-> arg[ 29]: -Djava.library.path=/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64/server:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/lib/ppc64:/sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/../lib/ppc64:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib:/lib:/usr/lib:/QSYS.LIB/SAPJ2EE00.LIB:/usr/sap/NP3/DVEBMGS00/j2ee/os_libs:/usr/sap/NP3/SYS/exe/run:/usr/sap/NP3/hdbclient:/sapdb/clients/NP3/lib/lib64:/sapdb/clients/NP3/lib

-> arg[ 30]: -Dmemory.manager=2048M

-> arg[ 31]: -Xmx2048M

-> arg[ 32]: -Xms2048M

-> arg[ 33]: -Xss2M

-> arg[ 34]: -DLoadBalanceRestricted=no

-> arg[ 35]: -Djstartup.mode=JCONTROL

-> arg[ 36]: -Djstartup.ownProcessId=102383

-> arg[ 37]: -Djstartup.ownHardwareId=M0555766080

-> arg[ 38]: -Djstartup.whoami=server

-> arg[ 39]: -Djstartup.debuggable=no

-> arg[ 40]: -Xjvmx

-> arg[ 41]: -XsapSystem=00

-> arg[ 42]: -XmonGcCallback

-> arg[ 43]: -DSAPINFO=NP3_00_server

-> arg[ 44]: -DSAPSTART=1

-> arg[ 45]: -DSAPSTARTUP=1

-> arg[ 46]: -DSAPSYSTEM=00

-> arg[ 47]: -DSAPSYSTEMNAME=NP3

-> arg[ 48]: -DSAPMYNAME=EXPSPRD_NP3_00

-> arg[ 49]: -DSAPDBHOST=EXPSPRD

-> arg[ 50]: -Dj2ee.dbhost=EXPSPRD

**********************************************************************

Mon Feb 23 14:39:54 2015

(CompilerOracle read from file /sapmnt/NP3/exe/jvm/as400_pase_64/sapjvm_4.1.042/sapjvm_4/jre/.hotspot_compiler)

[Thr 772] JHVM_LoadJavaVM: Java VM created OK.

**********************************************************************

JHVM_BuildArgumentList: main method arguments of node [server0]

**********************************************************************

[Thr 11310] Mon Feb 23 14:39:55 2015

[Thr 11310] JHVM_RegisterNatives: com.sap.bc.krn.perf.PerfTimes

[Thr 11310] JHVM_RegisterNatives: 8 of 9 methods in com.sap.bc.krn.perf.PerfTimes registered.

[Thr 11310] JHVM_RegisterNatives: all methods unregistered.

[Thr 11310] JHVM_RegisterNatives: 8 of 9 methods in com.sap.bc.krn.perf.PerfTimes registered.

Mon Feb 23 14:39:56 2015

2.218: [GC2.218: [ParNew: 409600K->2721K(512000K), 0.0164230 secs] 409600K->2721K(1994752K), 0.0174120 secs] [Times: user=0.03 sys=0.01, real=0.02 secs]

[Thr 11310] JHVM_RegisterNatives: com.sap.bc.proj.jstartup.JStartupFramework

[Thr 11310] JLaunchISetClusterId: set cluster id 9224350

[Thr 11310] JLaunchISetState: change state from [Initial (0)] to [Waiting for start (1)]

[Thr 11310] JLaunchISetState: change state from [Waiting for start (1)] to [Starting (2)]

Mon Feb 23 14:40:03 2015

9.097: [GC9.097: [ParNew: 412321K->31786K(512000K), 0.0537340 secs] 412321K->31786K(1994752K), 0.0541300 secs] [Times: user=0.17 sys=0.01, real=0.06 secs]

Mon Feb 23 14:40:06 2015

12.469: [GC12.469: [ParNew: 441386K->48064K(512000K), 0.0427110 secs] 441386K->48064K(1994752K), 0.0431400 secs] [Times: user=0.13 sys=0.00, real=0.05 secs]

Mon Feb 23 14:40:14 2015

20.270: [GC20.270: [ParNew: 457664K->54732K(512000K), 0.0618140 secs] 457664K->54732K(1994752K), 0.0623030 secs] [Times: user=0.16 sys=0.00, real=0.07 secs]

[Thr 40612] Mon Feb 23 14:40:16 2015

[Thr 40612] JHVM_RegisterNatives: com.sap.security.core.server.vsi.service.jni.VirusScanInterface

[Thr 40612] JHVM_RegisterNatives: 23 of 24 methods in com.sap.security.core.server.vsi.service.jni.VirusScanInterface registered.

[Thr 40612] JHVM_RegisterNatives: all methods unregistered.

[Thr 40612] JHVM_RegisterNatives: 23 of 24 methods in com.sap.security.core.server.vsi.service.jni.VirusScanInterface registered.

SAPSTARTSRV



---------------------------------------------------

trc file: "sapstartsrv.log", trc level: 0, release: "721"

---------------------------------------------------

pid        90733

[Thr 01] Mon Feb 23 12:55:13 2015

SAP HA Trace: INFO:  SAP_HA_InitEx: Using USRSPACE Library: R3NP3400

SAP HA Trace: INFO:  SAP_HA_InitEx: SAP IBM i Cluster library 'sapos4ha' initialized

CCMS agent initialization for instance type ABAP: return code 0.

CCMS agent start: return code 0.

Initializing SAPControl Webservice

Starting AutoRestart thread

[Thr 515] Mon Feb 23 12:55:14 2015

AutoRestart thread started

SapSSLInit failed => https support disabled

Starting WebService thread

Webservice thread started, listening on port 50213

Trusted http connect via Unix domain socket '/tmp/.sapstream50213' enabled.

Trusted http connect via Unix domain socket '/tmp/.sapstream50213' for <sid>OFR enabled too.

trusted unix domain socket user is starting SAP System at 2015/02/23 12:55:23

SAP HA Trace: ERROR: CRL::Read: QcstListClusterResourceGroups Failed: CPFBB02

SAP HA Trace:        0000    00000100 0000001A C3D7C6C2 C2F0F2F0 EBCDIC ........CPFBB020

SAP HA Trace:        0010    5CD5D6D5 C5404040 4040                    *NONE

SAP HA Trace: ERROR: SAP_HA_FindSAPInstance: CRGList.Read() Failed

trusted unix domain socket user is starting SAP System at 2015/02/23 13:13:41

SAP HA Trace: ERROR: CRL::Read: QcstListClusterResourceGroups Failed: CPFBB02

SAP HA Trace:        0000    00000100 0000001A C3D7C6C2 C2F0F2F0 EBCDIC ........CPFBB020

SAP HA Trace:        0010    5CD5D6D5 C5404040 4040                    *NONE

SAP HA Trace: ERROR: SAP_HA_FindSAPInstance: CRGList.Read() Failed

trusted unix domain socket user is stopping SAP System at 2015/02/23 13:53:00

SAP HA Trace: ERROR: CRL::Read: QcstListClusterResourceGroups Failed: CPFBB02

SAP HA Trace:        0000    00000100 0000001A C3D7C6C2 C2F0F2F0 EBCDIC ........CPFBB020

SAP HA Trace:        0010    5CD5D6D5 C5404040 4040                    *NONE   

SAP HA Trace: ERROR: SAP_HA_FindSAPInstance: CRGList.Read() Failed

former_member185239
Active Contributor
0 Kudos

Hi Yogesh,

Delete the sapstream file from /tmp folder.

With Regards

Ashutosh Chaturvedi

manumohandas82
Active Contributor
0 Kudos

Hi

Can you provide the output for the following

sapcontrol.exe -prot PIPE -host <hostname> -nr <instance number> -function GetSystemInstanceList from your central system


Do you have multiple application servers ?



Thanks ,

Manu

Former Member
0 Kudos

Hi Manu

Just have the 1 application server:

Former Member
0 Kudos

Hi Yogesh,

Just try to restart your JAVA system once using SAPCONTROL commands line and check for the SDM process in it whether started.

I have seen similar issue which are resolved after restarting the JAVA.

sapcontrol -nr <instance no> -function RestartSystem

due verify if your sapstartsrv service is running before doing this. and do a service restart for the sapcontrol too.

sapcontrol -nr <instance no> -function RestartService

Let us know the outcome of same.

Regards,

Ram

Former Member
0 Kudos

Good day Ramkrishna

I've tried doing that, it does not make a difference, i also looked and applied information from this note

1563660 - sapcontrol, user authorization issues (SUM)