solaris cluster for node1 is down

[Former Member]

Hi All,

We have finished an clock change  activity and when we start the production  cluster server , the WebAS resource is getting failed to start and stay online.

System details:EEC6ehp5

OS :sun0S 5.11

Db:oracle 11.2.0.2

Below are the logs and let me know if there are solutions to my issue .

Mar 31 04:20:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:20:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 836985 daemon.error] Probe for dispatcher returned 246
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 833212 daemon.info] Attempting to start the data service under process monitor facility.
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 344614 daemon.info] Started SAP processes under PMF successfully.
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_SID> has a value of <PRD>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Name> has a value of <DVEBMGS00>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Number> has a value of <00>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_startup_script> has a value of </usr/sap/PRD/SYS/exe/run/startsap_webas_server>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_shutdown_script> has a value of </usr/sap/PRD/SYS/exe/run/stopsap_webas_server>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 836985 daemon.error] Probe for dispatcher returned 246
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 702018 daemon.notice] Waiting for the SAP Application server to come online.
Mar 31 04:20:16 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:20:26 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 836985 daemon.error] Probe for dispatcher returned 246
Mar 31 04:20:26 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 702018 daemon.notice] Waiting for the SAP Application server to come online.
Mar 31 04:20:36 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="prd-SAP-rg,prd-webas-res,0.svc", cmd="/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg", Failed to stay up.
Mar 31 04:20:36 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 819736 daemon.notice] PMF is restarting process that died: tag=prd-SAP-rg,prd-webas-res,0.svc, cmd_path=/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg, max_retries=0, num_retries=0
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_SID> has a value of <PRD>
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Name> has a value of <DVEBMGS00>
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Number> has a value of <00>
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_startup_script> has a value of </usr/sap/PRD/SYS/exe/run/startsap_webas_server>
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_shutdown_script> has a value of </usr/sap/PRD/SYS/exe/run/stopsap_webas_server>
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:20:36 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:20:52 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="prd-SAP-rg,prd-webas-res,0.svc", cmd="/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg", Failed to stay up.
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 836985 daemon.error] Probe for dispatcher returned 251
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 702018 daemon.notice] Waiting for the SAP Application server to come online.
Mar 31 04:22:34 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 819736 daemon.notice] PMF is restarting process that died: tag=prd-SAP-rg,prd-webas-res,0.svc, cmd_path=/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg, max_retries=0, num_retries=0
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_SID> has a value of <PRD>
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Name> has a value of <DVEBMGS00>
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Number> has a value of <00>
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_startup_script> has a value of </usr/sap/PRD/SYS/exe/run/startsap_webas_server>
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_shutdown_script> has a value of </usr/sap/PRD/SYS/exe/run/stopsap_webas_server>
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:22:34 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:22:50 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="prd-SAP-rg,prd-webas-res,0.svc", cmd="/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg", Failed to stay up.
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 836985 daemon.error] Probe for dispatcher returned 251
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 702018 daemon.notice] Waiting for the SAP Application server to come online.
Mar 31 04:24:32 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 819736 daemon.notice] PMF is restarting process that died: tag=prd-SAP-rg,prd-webas-res,0.svc, cmd_path=/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg, max_retries=0, num_retries=0
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_SID> has a value of <PRD>
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Name> has a value of <DVEBMGS00>
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Number> has a value of <00>
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_startup_script> has a value of </usr/sap/PRD/SYS/exe/run/startsap_webas_server>
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_shutdown_script> has a value of </usr/sap/PRD/SYS/exe/run/stopsap_webas_server>
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:24:32 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:24:48 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="prd-SAP-rg,prd-webas-res,0.svc", cmd="/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg", Failed to stay up.
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 836985 daemon.error] Probe for dispatcher returned 251
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 702018 daemon.notice] Waiting for the SAP Application server to come online.
Mar 31 04:26:30 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 819736 daemon.notice] PMF is restarting process that died: tag=prd-SAP-rg,prd-webas-res,0.svc, cmd_path=/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg, max_retries=0, num_retries=0
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_SID> has a value of <PRD>
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Name> has a value of <DVEBMGS00>
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Number> has a value of <00>
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_startup_script> has a value of </usr/sap/PRD/SYS/exe/run/startsap_webas_server>
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_shutdown_script> has a value of </usr/sap/PRD/SYS/exe/run/stopsap_webas_server>
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:26:30 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:26:46 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="prd-SAP-rg,prd-webas-res,0.svc", cmd="/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg", Failed to stay up.
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 836985 daemon.error] Probe for dispatcher returned 251
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_start]: [ID 702018 daemon.notice] Waiting for the SAP Application server to come online.
Mar 31 04:28:28 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 819736 daemon.notice] PMF is restarting process that died: tag=prd-SAP-rg,prd-webas-res,0.svc, cmd_path=/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg, max_retries=0, num_retries=0
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_SID> has a value of <PRD>
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Name> has a value of <DVEBMGS00>
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Instance_Number> has a value of <00>
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_startup_script> has a value of </usr/sap/PRD/SYS/exe/run/startsap_webas_server>
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <Webas_shutdown_script> has a value of </usr/sap/PRD/SYS/exe/run/stopsap_webas_server>
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:28:28 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_startR3]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:28:44 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="prd-SAP-rg,prd-webas-res,0.svc", cmd="/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg", Failed to stay up.
Mar 31 04:30:13 xxxxxxxxxxx Cluster.RGM.global.rgmd: [ID 764140 daemon.error] Method <webas_svc_start> on resource <prd-webas-res>, resource group <prd-SAP-rg>, node <xxxxxxxxxxx>: Timeout.
Mar 31 04:30:13 xxxxxxxxxxx Cluster.PMF.pmfd: [ID 541277 daemon.info] PMF did not restart tag <prd-SAP-rg,prd-webas-res,0.svc>, cmd_path="/bin/ksh -c /opt/SUNWscsapwebas/webas/bin/webas_startR3 -R prd-webas-res -T SUNW.sapwebas:6 -G prd-SAP-rg"
Mar 31 04:30:13 xxxxxxxxxxx Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <webas_svc_stop> for resource <prd-webas-res>, resource group <prd-SAP-rg>, node <xxxxxxxxxxx>, timeout <300> seconds
Mar 31 04:30:13 xxxxxxxxxxx Cluster.RGM.global.rgmd: [ID 863274 daemon.debug] 40 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscsapwebas/webas/bin/webas_svc_stop>:tag=<prd-SAP-rg.prd-webas-res.1>: Calling security_clnt_connect(..., host=<xxxxxxxxxxx>, sec_type {0:WEAK, 1:STRONG, 2:DES} =<1>, ...)
Mar 31 04:30:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 452552 daemon.info] Extension property <SAP_SID> has a value of <PRD>
Mar 31 04:30:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 452552 daemon.info] Extension property <SAP_Instance_Name> has a value of <DVEBMGS00>
Mar 31 04:30:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 452552 daemon.info] Extension property <SAP_Instance_Number> has a value of <00>
Mar 31 04:30:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 452552 daemon.info] Extension property <Webas_startup_script> has a value of </usr/sap/PRD/SYS/exe/run/startsap_webas_server>
Mar 31 04:30:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 452552 daemon.info] Extension property <Webas_shutdown_script> has a value of </usr/sap/PRD/SYS/exe/run/stopsap_webas_server>
Mar 31 04:30:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 452552 daemon.info] Extension property <SAP_User> has a value of <prdadm>
Mar 31 04:30:13 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 452552 daemon.info] Extension property <SAP_Logdir> has a value of <>
Mar 31 04:30:14 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 661560 daemon.info] All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.
Mar 31 04:34:29 xxxxxxxxxxx SC[,SUNW.sapwebas:6,prd-SAP-rg,prd-webas-res,webas_svc_stop]: [ID 805627 daemon.error] Stop command /usr/sap/PRD/SYS/exe/run/stopsap_webas_server r3 DVEBMGS00 zzzzzzz-pub timed out.
Mar 31 04:34:29 xxxxxxxxxxx Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <webas_svc_stop> completed successfully for resource <prd-webas-res>, resource group <prd-SAP-rg>, node <xxxxxxxxxxx>, time used: 85% of timeout <300 seconds>

root@yyyyyyyyyyy # clrs status

=== Cluster Resources ===

Resource Name               Node Name    State     Status Message
-------------               ---------    -----     --------------
prd-webas-res               xxxxxxxxxxx   Offline   Offline
                            yyyyyyyyyyy   Online    Online - Service is online.

prd-msg-res                 xxxxxxxxxxx   Offline   Offline
                            yyyyyyyyyyy   Online    Online - Service is online.

prd-enq-res                 xxxxxxxxxxx   Offline   Offline
                            yyyyyyyyyyy   Online    Online - Service is online.

prd-SAP-oralisten-res       xxxxxxxxxxx   Offline   Offline
                            yyyyyyyyyyy   Online    Online

prd-SAP-oracle-res          xxxxxxxxxxx   Offline   Offline
                            yyyyyyyyyyy   Online    Online

prd-SAP-hastorageplus-res   xxxxxxxxxxx   Offline   Offline
                            yyyyyyyyyyy   Online    Online

prd-SAP-hafoip2-res         xxxxxxxxxxx   Offline   Offline - LogicalHostname offline.
                            yyyyyyyyyyy   Online    Online - LogicalHostname online.

prd-SAP-hasp-rs             xxxxxxxxxxx   Online    Online
                            yyyyyyyyyyy   Online    Online

prd-rep-res                 yyyyyyyyyyy   Offline   Offline
                            xxxxxxxxxxx   Online    Online

prd-SAP-rep-hafoip2-res     yyyyyyyyyyy   Offline   Offline - LogicalHostname offline.
                            xxxxxxxxxxx   Online    Online - LogicalHostname online.

clrg status

=== Cluster Resource Groups ===

Group Name          Node Name      Suspended    Status
----------          ---------      ---------    ------
prd-SAP-rg          xxxxxxxxxxx     No           Offline
                    yyyyyyyyyyy     No           Online

prd-SAP-hasp-rg     xxxxxxxxxxx     No           Online
                    yyyyyyyyyyy     No           Online

prd-SAP-rep-rg      yyyyyyyyyyy     No           Offline
                    xxxxxxxxxxx     No           Online

Regards

Bharat

[csaba_goetz]

Hello Bharatram,

pls. attach sapstart.log, sapstartsrv.log, dev_disp and dev_w0. According to them we'll see what is the cause.

Best regards, Adam Goetz