JAVA Stack of our Solution Manager is not comming up

[florin_radulea]

Hello,

I have some problems to start my Java stack on our Solution Manager.

This system worked fine until I installed Wily Introscope agent on it.

So I want to let you know what I did on my system using the following guide:Link: [https://websmp105.sap-ag.de/~sapdownload/011000358700001995842008]

1. I installed the Wily Enterprise Manager on Solution Manager host

2. . I used the option for automated installation using the SMD setup

application. According to Note 797147 I applied ISAGENT02_0-10007435.SCAusing SDM. This was applied successfully

3.Trying bellow option (2) I wasn't able to see the Willy agent and I started to install Wily agent manually. Because we have JDK IBM Java VM 1.4.2 (SR13) x86_64 I used the note 1149214 to create CreateAutoProbeConnector.

4. I added the following JAVA parameters using ./configtol.sh in the following path:

cluster-data>Instance_ID42052>server_ID42052>General (tab)>Java Parameters:

-Xbootclasspath/p:/usr/sap/ccms/wily/connectors/connector.jar:/usr/sap/ccms/wily/Agent.jar

-Dcom.wily.introscope.agentProfile=/usr/sap/ccms/wily/IntroscopeAgent.profile

-Dcom.wily.introscope.agent.agentName=introscope.agent.name

-Dcom.wily.autoprobe.prependToJVMRuntimePath=/usr/lib64/jvm/java/jre/bin/j9vm/jclSC14/classes.zip

(can you tell me if this is the right place(path) to change the Java Parameters?

5.After that I restarted the JAVA engine and this doesn't come up.

I get the following errors in /usr/sap/ZSP/DVEBMGS00/work/dev_bootstrap file:

+JVMJ9VM015W Initialization error for library jclscar_23(14): JVMJ9VM009E J9VMDllMain failed (this error is related in note 1149214 but I don't understand how to handle the sollution provided by this note)

[Thr 1084238144] *** ERROR => JHVM_LoadJavaVM: Cannot create Java VM (rc=-4) [jhvmxx_mt.c 553]

[Thr 1084238144] *** ERROR => Cannot load Java VM (server) (rc=-1) [jlnchxxi_mt. 752]

[Thr 1084238144] **********************************************************************

• ERROR => Java VM initialization failed.

• Please see SAP Note 943602 , section 'Java VM initialization issues'

• for additional information and trouble shooting. +

I took a look in note 943602 but this doesn't apply in my case

In file /usr/sap/ZSP/DVEBMGS00/work/dev_jcontrol I get the errors:

+ [Thr 47433620458384] *** ERROR => invalid return code of process [bootstrap] (exitcode = 255) [jstartxx_mt. 1642]

[Thr 47433620458384] JControlExecuteBootstrap: error executing bootstrap node [bootstrap] (rc = 255)

[Thr 47433620458384] JControlCloseProgram: started (exitcode = 255)

[Thr 47433620458384] JControlCloseProgram: good bye... (exitcode = 255)+

[former_member432219 · Accepted solution]

Hi Florin

The first thing I notice is that since bootstrap fails you must have added the Wily parameters to the list of bootstrap VM parameters. This is incorrect, you don't want to monitor bootstrap with Wily since it is as the name suggests a process that only runs during startup

So the first thing I would do is remove these parameters from the list in InstanceID - Message Server and Bootstrap - java parameters in the configtool. Save your changes and restart the instance

The startup should proceed further then, but it's possible that the dispatcher or server will face the same issue as bootstrap did

If so, then from the description you've given, then you must follow 'Updating an agent that was installed using SMD' from note 1149214

[florin_radulea · Accepted solution]

Hi Patrick,

Thank you for your response.

I deleted the Java Parameters from InstanceID - Message Server and Bootstrap

-Xbootclasspath/p:/usr/sap/ccms/wily/connectors/connector.jar:/usr/sap/ccms/wily/Agent.jar

-Dcom.wily.introscope.agentProfile=/usr/sap/ccms/wily/IntroscopeAgent.profile

-Dcom.wily.introscope.agent.agentName=introscope.agent.name

-Dcom.wily.autoprobe.prependToJVMRuntimePath=/usr/lib64/jvm/java/jre/bin/j9vm/jclSC14/classes.zip

after that I restarted the JAVA stack using TX SMICM>Administration>J2EE Instance local > Send Soft Shutdown>With Restart, but I get exactly the same error. Should I restart the JAVA stack using at OS level stopsap all and startsap all?

--Can you please exactly tell me where I have to set the Wily Parameters in configtool? (in Guide there is not specified in which tab to set the JAVA Parameters)

--Can you please explain me which is the differences between General Parameters and BootStrap Parameters ?

--Can you please let me know how exactly to update the agent that was installed using SMD' from note 1149214, because I don't understand how to run again the SMD (the SMD is unavailable since J2EE is not started)

See bellow the content from note 1149214:

Updating an agent that was installed using SMD

If you used SMD to install the agent, it is sufficient to execute the Introscope agent setup again for the system that is to be monitored. The setup then generates the AutoProbeConnector automatically.

For Solution Manager versions prior to Support Package 19, you must specifiy the path for the file classes.zip explicitly in the Setup UI (section AIX, also applicable to Linux). The path (for example, /opt/IBMJava2-amd64-142/jre/bin/j9vm/jclSC14/classes.zip) depends on the SR of the JDK and can be found in the examples below.

As of Solution ManagerSupport Package 19, the parameter is adjusted automatically. The Introscope agent setup must be executed manually.

Thank you,

Best regards,

Florin Radulea

[former_member432219 · Accepted solution]

Hi Florin

cluster-data>Instance_ID42052>server_ID42052>General (tab)>Java Parameters is the correct path for the server VM/java parameters. It wasn't my intention that you should remove the parameters from there

Instead you should remove them from location

cluster-data-->Instance_ID42052 - Message server and bootstrap (tab) - java parameters

The java parameters here are for the main bootstrap process

Refer to the documentation here

[Startup, Operation and Shutdown of a Java Instance|http://help.sap.com/saphelp_nw70ehp2/helpdata/en/a2/852aca5cd96f4796e51ba0f7a28926/frameset.htm]

Bootstrap is a short lived process so there is no real need to add monitoring parameters to its java parameters

So you should remove the Wily parameters from there

I'm not an expert in Wily configuration, but what you could do is remove all Wily related paramters from the configtool so that the AS Java will start again, and you can run the SMD to install the agent. Before doing so check that JAVA_HOME is set correctly and jstart/vm/home in the central instance profile is also set correctly.

If after installing the agent through SMD the same issue occurs again, I think you'll have to update the agent manually

I'm not 100% sure of the syntax you should use, I'll look for it

[florin_radulea · Accepted solution]

Hi Patrick,

Thank you very much for your response and explications.

I want to inform you that before to remove the Wily related parameters from the configtool I uninstalled and installed the SMD (Solution Monitoring Diagnostics) using the Solution Manager installation master, and after that I removed all Wily parameters from configtool. After that I restarted the JAVA instance but unfortunately exact the same error...?

Do you know if this installation and installation of SMD can cause this error?

Thank you,

Best regards,

Florin Radulea

[former_member432219 · Accepted solution]

Hi Florian, something just came to mind that I overlooked previously.

Normally when one has a problem with java parmaters and they prevent a AS Java process (e.g dispatcher or server) from starting, removing them from the configtool is enough to resolve the problem, since after a restart they will be removed from the instance.properties file on the file system and are no longer used by the instance

However when there is a problem with the bootstrap java parameters it is a special case becuase it is the job of the bootstrap process to update the instance.properties file that is used by the instance. However bootstrap itself uses the instance.properties file and since it contains the problematic parameters bootstrap fails each time. This means although the problematic parameters have been removed from the DB using the configtool, the instance.properties is never updated because bootstrap fails and the problematic parameters remain in the instance.properties file. It's a catch 22 problem

The solution is to open the file usr/sap/<SID>/<instanceID>/j2ee/cluster/instance.properties and remove the Wily parameters from the line bootstrap.JavaParameters. Then assuming you have successfully removed the parameters from the configtool, when you restart the instance bootstrap should succeed and instance.properties will be updated and the instance should start

[florin_radulea · Accepted solution]

Hi Patrick,

Thank you very much for your time and your valuables suggestions, that helped me to solve my strange problem.

This was the problem that you said above. I deleted manually the Boostrap Java parameters from instance.properties file and my Java engine was able to start up. Thank you very much again....I learned a lot from you.

I awarded maxim points to you.

Have a nice day.

Best regatds,

Florin Radulea