J2EE does not start after changing the instance number

[MichaelTe]

Hello all,

we have a ABAP+JAVA AddIn.

Release NW2004s SR2.

After changing the instance number of our ci from 00 to 15 the j2ee Server will not come up.

The ABAP is starting fine.

We changed the filenames and the directory names to the new value (DVEBMGS00 to DVEBMGS15 and so on).

We changed the profiles of the ci and the scs (Default, Start and Instance).

Also we changed some paths in the config tool.

We are not sure if we caught every entry.

It seems that we missed some entries. But we can't find them in the config tool.

Every help would be appreciated,

Michael

Here's the output of the dev_jcontrol:

cisol:soladm 198> cat dev_jcontrol

---------------------------------------------------
trc file: "/usr/sap/SOL/DVEBMGS15/work/dev_jcontrol", trc level: 1, release: "700"
---------------------------------------------------
node name   : jcontrol
pid         : 17411
system name : SOL
system nr.  : 15
started at  : Thu Apr 26 13:57:58 2007
arguments         :
          arg[00] : jcontrol
          arg[01] : pf=/usr/sap/SOL/SYS/profile/SOL_DVEBMGS15_cisol
          arg[02] : -DSAPSTART=1
          arg[03] : -DCONNECT_PORT=15167
          arg[04] : -DSAPSYSTEM=15
          arg[05] : -DSAPSYSTEMNAME=SOL
          arg[06] : -DSAPMYNAME=bxsap2_SOL_15
          arg[07] : -DSAPPROFILE=/usr/sap/SOL/SYS/profile/SOL_DVEBMGS15_cisol
          arg[08] : -DFRFC_FALLBACK=ON
          arg[09] : -DFRFC_FALLBACK_HOST=localhost


[Thr 182906379008] Thu Apr 26 13:57:58 2007
[Thr 182906379008] *** WARNING => 
INFO: Unknown property [instance.box.number=SOLDVEBMGS00bxsap2] 
[jstartxx_mt. 841]
[Thr 182906379008] *** WARNING => 
INFO: Unknown property [instance.en.host=bxsap2] [jstartxx_mt. 841]
[Thr 182906379008] *** WARNING => 
INFO: Unknown property [instance.en.port=3201] [jstartxx_mt. 841]
[Thr 182906379008] *** WARNING => 
INFO: Unknown property [instance.system.id=0] [jstartxx_mt. 841]

**********************************************************************
JStartupReadInstanceProperties: read instance properties
[/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties:
/usr/sap/SOL/DVEBMGS15/SDM/program/config/sdm_jstartup.properties]
-> ms host    : cisol
-> ms port    : 3901
-> OS libs    : /usr/sap/SOL/DVEBMGS00/j2ee/os_libs
-> Admin URL  :
-> run mode   : NORMAL
-> run action : NONE
-> enabled    : yes
**********************************************************************


**********************************************************************
Used property files
-> files [00] : 
/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties
-> files [01] : 
/usr/sap/SOL/DVEBMGS15/SDM/program/config/sdm_jstartup.properties
**********************************************************************

**********************************************************************
Instance properties
-> ms host    : cisol
-> ms port    : 3901
-> os libs    : /usr/sap/SOL/DVEBMGS00/j2ee/os_libs
-> admin URL  :
-> run mode   : NORMAL
-> run action : NONE
-> enabled    : yes
**********************************************************************

**********************************************************************
Bootstrap nodes
-> [00] bootstrap            : 
/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties
-> [01] bootstrap_ID5817600  : 
/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties
-> [02] bootstrap_ID5817650  : 
/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties
**********************************************************************

**********************************************************************
Worker nodes
-> [00] ID5817600            : 
/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties
-> [01] ID5817650            : 
/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties
-> [02] sdm                  : 
/usr/sap/SOL/DVEBMGS15/SDM/program/config/sdm_jstartup.properties
**********************************************************************

[Thr 182906379008] 
SigISetDefaultAction : default handling for signal 17
[Thr 182906379008] 
JControlExecuteBootstrap: execute bootstrap process [bootstrap]
[Thr 1074792800] 
JControlDPMessageFunc: Thread 1074792800 started as handler thread for R/3 dispatcher messages.
[Thr 182906379008] 
[Node: bootstrap] java home is set by profile parameter
        Java Home: /usr/lib/java
[Thr 182906379008] 
JStartupICheckFrameworkPackage: can't find framework package /usr/sap/SOL/SYS/exe/run/jvmx.jar

**********************************************************************
JStartupIReadSection: read node properties [bootstrap]
-> node name          : bootstrap
-> node type          : bootstrap
-> node execute       : yes
-> java path          : /usr/lib/java
-> java parameters    : -Djco.jarm=1
-> java vm version    : J2RE 1.4.2 IBM J9 2.2 Linux amd64-64 j9xa64142-20070317 (JIT enabled)
-> java vm vendor     : IBM J9SE VM (IBM Corporation)
-> java vm type       : <default>
-> java vm cpu        : amd64
-> heap size          : 256M
-> root path          : /usr/sap/SOL/DVEBMGS00/j2ee/cluster
-> class path         : ./bootstrap/launcher.jar
-> OS libs path       : /usr/sap/SOL/DVEBMGS00/j2ee/os_libs
-> main class         : com.sap.engine.offline.OfflineToolStart
-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework
-> registr. class     : com.sap.bc.proj.jstartup.JStartupNatives
-> framework path     : /usr/sap/SOL/SYS/exe/run/jstartup.jar:/usr/sap/SOL/SYS/exe/run/jvmx.jar
-> parameters         : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID0058176
-> debuggable         : yes
-> debug mode         : no
-> debug port         : 60000
-> shutdown timeout   : 120000
**********************************************************************


**********************************************************************
JControlStartJLaunch: program = /usr/sap/SOL/DVEBMGS15/exe/jlaunch
-> arg[00] = /usr/sap/SOL/DVEBMGS15/exe/jlaunch
-> arg[01] = pf=/usr/sap/SOL/SYS/profile/SOL_DVEBMGS15_cisol
-> arg[02] = -DSAPINFO=SOL_15_bootstrap
-> arg[03] = -nodeId=-1
-> arg[04] = -file=/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties
-> arg[05] = -syncSem=203948156
-> arg[06] = -nodeName=bootstrap
-> arg[07] = -jvmOutFile=/usr/sap/SOL/DVEBMGS15/work/jvm_bootstrap.out
-> arg[08] = -stdOutFile=/usr/sap/SOL/DVEBMGS15/work/std_bootstrap.out
-> arg[09] = -locOutFile=/usr/sap/SOL/DVEBMGS15/work/dev_bootstrap
-> arg[10] = -mode=BOOTSTRAP
-> arg[11] = pf=/usr/sap/SOL/SYS/profile/SOL_DVEBMGS15_cisol
-> arg[12] = -DSAPSTART=1
-> arg[13] = -DCONNECT_PORT=15168
-> arg[14] = -DSAPSYSTEM=15
-> arg[15] = -DSAPSYSTEMNAME=SOL
-> arg[16] = -DSAPMYNAME=bxsap2_SOL_15
-> arg[17] = -DSAPPROFILE=/usr/sap/SOL/SYS/profile/SOL_DVEBMGS15_cisol
-> arg[18] = -DFRFC_FALLBACK=ON
-> arg[19] = -DFRFC_FALLBACK_HOST=localhost
-> lib path = LD_LIBRARY_PATH=/opt/IBMJava2-amd64-142/jre/bin:
/opt/IBMJava2-amd64-142/jre/bin:
/opt/IBMJava2-amd64-142/jre/bin/j9vm:
/opt/IBMJava2-amd64-142/jre/bin:
/usr/sap/SOL/DVEBMGS00/exe:
/usr/sap/SOL/DVEBMGS00/exe:
/usr/sap/SOL/DVEBMGS00/exe:
/tmp/sapinst_exe.9724.1176820540:
/usr/sap/SOL/SYS/exe/run:/sapdb/programs/lib:
/usr/lib:/usr/sap/SOL/DVEBMGS00/j2ee/os_libs:
/usr/sap/SOL/DVEBMGS15/exe:/usr/sap/SOL/DVEBMGS15/exe:
/usr/sap/SOL/DVEBMGS15/exe:/usr/sap/SOL/SYS/exe/run:
/sapdb/programs/lib
-> exe path = PATH=/usr/lib/java/bin:
/usr/sap/SOL/DVEBMGS00/j2ee/os_libs:
/sapdb/programs/bin:.:/home_sap/soladm:
/usr/sap/SOL/SYS/exe/run:/opt/kde3/bin:
/opt/gnome/bin:/usr/games:/usr/bin/X11:
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:
/usr/lib/java/bin
**********************************************************************

[Thr 182906379008] JStartupICreateProcess: fork process (pid 17413)

[Thr 182906379008] Thu Apr 26 13:58:01 2007
[Thr 182906379008] *** ERROR => invalid return code of process [bootstrap] (exitcode = 255) [jstartxx_mt. 1642]
[Thr 182906379008] JControlExecuteBootstrap: error executing bootstrap node [bootstrap] (rc = 255)
[Thr 182906379008] JControlCloseProgram: started (exitcode = 255)
[Thr 182906379008] JControlCloseProgram: good bye... (exitcode = 255)

[Former Member]

Hi

Apart from the changes that you made in the directory names and some files there are many files and properties that might me using the instance number.

[Thr 182906379008] *** WARNING => INFO: Unknown property [instance.box.number=SOLDVEBMGS00bxsap2] [jstartxx_mt. 841]

[Thr 182906379008] *** WARNING => INFO: Unknown property [instance.en.host=bxsap2] [jstartxx_mt. 841]

[Thr 182906379008] *** WARNING => INFO: Unknown property [instance.en.port=3201] [jstartxx_mt. 841]

[Thr 182906379008] *** WARNING => INFO: Unknown property [instance.system.id=0] [jstartxx_mt. 841]

It is giving unknown property as the xx there means the instance number.

In the properties file you need to check the values as well.

Regards

Sumit.

[MichaelTe]

Hello Sumit,

in an other thread I saw a post which mentioned that the Warnings about unknown properties aren't the cause.

I also looked in a J2EE Engine which is running and there are also these warnings. And there the instance.box.number is correct. So I think these warnings can be ignored.

> In the properties file you need to check the values as well.

I checked some Property Files and found the wrong instance number. So I changed it.

But in some cases, for example:

/usr/sap/SOLDVEBMGS15/j2ee/cluster/instance.properties

and

/usr/sap/SOL/DVEBMGS15/SDM/program/config/sdm_jstartup.properties

the values are after restart the same as before (the old ones)

So I think this file is rebuild out of the Database. But where can I maintain theese settings? Especially the OS libs path and the ms port:

**********************************************************************
JStartupReadInstanceProperties:
read instance properties
/usr/sap/SOL/DVEBMGS15/j2ee/cluster/instance.properties:
/usr/sap/SOL/DVEBMGS15/SDM/program/config/sdm_jstartup.properties
-> ms host    : cisol
-> ms port    : 3901
-> OS libs    : /usr/sap/SOL/DVEBMGS00/j2ee/os_libs
-> Admin URL  :
-> run mode   : NORMAL
-> run action : NONE
-> enabled    : yes
**********************************************************************

and this ones. Especially the root path and the os libs path?

**********************************************************************
JStartupIReadSection: read node properties [bootstrap]
-> node name          : bootstrap
-> node type          : bootstrap
-> node execute       : yes
-> java path          : /usr/lib/java
-> java parameters    : -Djco.jarm=1
-> java vm version    : J2RE 1.4.2 IBM J9 2.2 Linux amd64-64 ...
-> java vm vendor     : IBM J9SE VM (IBM Corporation)
-> java vm type       : <default>
-> java vm cpu        : amd64
-> heap size          : 256M
-> root path          : /usr/sap/SOL/DVEBMGS00/j2ee/cluster
-> class path         : ./bootstrap/launcher.jar
-> OS libs path       : /usr/sap/SOL/DVEBMGS00/j2ee/os_libs
-> main class         : com.sap.engine.offline.OfflineToolStart
-> framework class    : com.sap.bc.proj.jstartup.JStartupFramework
-> registr. class     : com.sap.bc.proj.jstartup.JStartupNatives
-> framework path     : /usr/sap/SOL/SYS/exe/run/jstartup.jar:
                                 /usr/sap/SOL/SYS/exe/run/jvmx.jar
-> parameters         : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID0058176
-> debuggable         : yes
-> debug mode         : no
-> debug port         : 60000
-> shutdown timeout   : 120000
**********************************************************************

Thanks in advance,

Michael

[Former Member]

Hi Michael,

I would rather recommend to run a system copy routine and install CI with new inst nr 15 for avoiding this complication.

Still, to ensure your system can be repaired now, check the following.

What you see in above log is a block while bootstrapping phase itself. There will be more complication following if startup sequence moves ahead of this phase. Changing instance number for Java stack will be more complicated than ABAP.

What I can propose is, use ConfigTool to change occurence of instance specific details everywhere (including paths, and ports on which WAS engine is supposed to listen, ).

To kick start on what is just mentioned, we start with:

ConfigTool:

1. cluster-data -> Global dispatcher configuration -> managers -> LockingManager

(change to /sapmnt/SOL/profile/SOL_DVEBMGS15_cisol)

Cross check if it has been set locally for each instance node.

2. cluster-data -> Global server configuration -> managers -> LockingManager

(change to /sapmnt/SOL/profile/SOL_DVEBMGS15_cisol)

Cross check if it has been set locally for each instance node.

3. cluster-data -> instance_IDxxxx -> dispatcher -> services -> http -> Ports Property (set http to 51500, ssl to 51501)

Note that above may not be mandatory. Its just the standard SAP WAS engine installations follow.

4. cluster-data -> instance_IDxxxx -> dispatcher -> services -> iiop ->

initialContextPort = 51502

port= 51507

ssl_socket= 51503

Like above, replace port entries using formula (50000 + 100*15 + xx) everywhere (p4, etc.).

Also, do NOT change instanceIDs even if they contain old instance number resemblance.

5. cluster-data -> instance_IDxxxx -> dispatcher -> services -> r3startup -> Properties:

element_0_dir = /usr/sap/SOL/DVEBMGS15/j2ee/cluster/server0 (server1, server2 if you have multiple server nodes)

Continue changing the relevant properties for all nodes.

6. Launch Offline Configurator -> change mode. Goto cluster_data and edit Propertysheet instance.properties.

Adapt RootPath, DebugPort, instance.install.dir, instance.osLibsPath etc)

DO NOT change instance.box.number property.

Make all the above changes and check if this is helpful

Regards,

Praveen