SAP not started on CI node thru Cluster Script

[Former Member]

Hi

We have installed the PI 7.1 on HP Unix 11v3 and oracle conbination. We have done the cluster package failover

of CI to DB system and done the complete installation on DB with high availability option. Let on we have splited

the CI to CI and keep the DB to DB note. The installation happend successfully and system started successfully on

both node.

Now we are doing the HP-UNIX cluster configuration and integration. But we have found that the DB services are

starting up successfully on DB node but the CI services are not starting up on CI node. The cluster error file

are attached for your reference. Since the project golive is left just 20 day . Request you to provide solution

on the same. Please also let us know if any further log is required.

Regards

Vimal Pathak

[former_member189546]

Hello,

Check the work directory of this instance DVEBMGS01 especially disp_dev.log and the stderr files.

regards,

John Feely

[Former Member]

Dear All

I have checked and did not found any error. Please let us know any setting need to be done for cluster.

Regards

Vimal

[Former Member]

Hi Vimal,

First thing check the SAP profile those are created at DB and CI and the file names of the profiles. As you have done the installation by doing the failover the profile file names may created with the DB name.

one more thing check whether you are able to connect to database from the CI

use R3trans -d by login to the OS user SIDADM

if it fails check trans.log it will give you the clear information.

As this has n number of possibilities to fail . If possible send us the work directory logs at my maild which is there at my busyness card.

thanks,

Chaitanya.

[Former Member]

x

Edited by: Hamid Boutiar on Aug 15, 2011 7:06 PM

[markus_doehr2]

> The cluster error file

> are attached for your reference. Since the project golive is left just 20 day . Request you to provide solution

> on the same. Please also let us know if any further log is required.

Note: This here is not a support channel but forums on a volunteer basis.

You can't attach logfiles here, seems you pasted the text of your OSS call, right?

Markus

[Former Member]

Paste the error log file.

Thanks,

Naveed

[Former Member]

Oct 6 16:16:13 root@htlxici: (start_app ciPP9 01 pp9adm): P: Start

Oct 6 16:16:13 root@htlxici: (start_app ciPP9 01 pp9adm): P: FAIL: NIECONN_REFUSED (Connection refused), NiRawConnect failed in plugin_fopen()

Oct 6 16:16:13 root@htlxici sapinstance.sh[29446]: (watchdog): wait for PID 356 returns 0

Oct 6 16:16:13 root@htlxici sapinstance.sh[29446]: (start_app ciPP9 01 pp9adm): Finishing startup attempt

Oct 6 16:16:13 root@htlxici sapinstance.sh[29446]: (wait_pidlist): Killed the wait_pidlist background shell. PID: 29953

Oct 6 16:16:13 root@htlxici sapinstance.sh[29446]: (start_d): Leaving parallel dialog instance startup delay

Oct 6 16:16:13 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/sapcontrol" owner: "pp9adm"

Oct 6 16:16:14 root@htlxici sapinstance.sh[29446]: (is_ip_local): ciPP9 is local

Oct 6 16:16:14 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect local dispatcher of instance DVEBMGS01 ...

Oct 6 16:16:14 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/dpmon" owner: "pp9adm"

Oct 6 16:16:16 root@htlxici sapinstance.sh[29446]: (crit_test_app): dpmon: /usr/sap/PP9/DVEBMGS01/exe/dpmon=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

Oct 6 16:16:16 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect via RFC to host ciPP9 instance DVEBMGS01 ...

Oct 6 16:16:16 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/startrfc" owner: "pp9adm"

Oct 6 16:16:16 root@htlxici sapinstance.sh[29446]: (watchdog): Watchdog initiated (PIDs: 1016 for 1015 - Timeout: 100 secs)

Oct 6 16:16:17 root@htlxici sapinstance.sh[29446]: (watchdog): wait for PID 1015 returns 1

Oct 6 16:16:17 root@htlxici sapinstance.sh[29446]: (crit_test_app): No connection to instance DVEBMGS01

Oct 6 16:16:17 root@htlxici sapinstance.sh[29446]: (crit_test_app): Delaying 5 secs to allow instance DVEBMGS01 startup/recover

Oct 6 16:16:22 root@htlxici sapinstance.sh[29446]: (is_ip_local): ciPP9 is local

Oct 6 16:16:22 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect local dispatcher of instance DVEBMGS01 ...

Oct 6 16:16:22 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/dpmon" owner: "pp9adm"

Oct 6 16:16:23 root@htlxici sapinstance.sh[29446]: (crit_test_app): dpmon: /usr/sap/PP9/DVEBMGS01/exe/dpmon=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

Oct 6 16:16:24 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect via RFC to host ciPP9 instance DVEBMGS01 ...

Oct 6 16:16:24 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/startrfc" owner: "pp9adm"

Oct 6 16:16:24 root@htlxici sapinstance.sh[29446]: (watchdog): Watchdog initiated (PIDs: 1466 for 1464 - Timeout: 100 secs)

Oct 6 16:16:25 root@htlxici sapinstance.sh[29446]: (watchdog): wait for PID 1464 returns 1

Oct 6 16:16:25 root@htlxici sapinstance.sh[29446]: (crit_test_app): No connection to instance DVEBMGS01

Oct 6 16:16:25 root@htlxici sapinstance.sh[29446]: (crit_test_app): Delaying 10 secs to allow instance DVEBMGS01 startup/recover

Oct 6 16:16:35 root@htlxici sapinstance.sh[29446]: (is_ip_local): ciPP9 is local

Oct 6 16:16:35 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect local dispatcher of instance DVEBMGS01 ...

Oct 6 16:16:35 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/dpmon" owner: "pp9adm"

Oct 6 16:16:36 root@htlxici sapinstance.sh[29446]: (crit_test_app): dpmon: /usr/sap/PP9/DVEBMGS01/exe/dpmon=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

Oct 6 16:16:36 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect via RFC to host ciPP9 instance DVEBMGS01 ...

Oct 6 16:16:36 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/startrfc" owner: "pp9adm"

Oct 6 16:16:36 root@htlxici sapinstance.sh[29446]: (watchdog): Watchdog initiated (PIDs: 1919 for 1917 - Timeout: 100 secs)

Oct 6 16:16:37 root@htlxici sapinstance.sh[29446]: (watchdog): wait for PID 1917 returns 1

Oct 6 16:16:37 root@htlxici sapinstance.sh[29446]: (crit_test_app): No connection to instance DVEBMGS01

Oct 6 16:16:37 root@htlxici sapinstance.sh[29446]: (crit_test_app): Delaying 15 secs to allow instance DVEBMGS01 startup/recover

Oct 6 16:16:52 root@htlxici sapinstance.sh[29446]: (is_ip_local): ciPP9 is local

Oct 6 16:16:52 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect local dispatcher of instance DVEBMGS01 ...

Oct 6 16:16:53 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/dpmon" owner: "pp9adm"

Oct 6 16:16:54 root@htlxici sapinstance.sh[29446]: (crit_test_app): dpmon: /usr/sap/PP9/DVEBMGS01/exe/dpmon=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

Oct 6 16:16:54 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect via RFC to host ciPP9 instance DVEBMGS01 ...

Oct 6 16:16:54 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/startrfc" owner: "pp9adm"

Oct 6 16:16:54 root@htlxici sapinstance.sh[29446]: (watchdog): Watchdog initiated (PIDs: 2412 for 2410 - Timeout: 100 secs)

Oct 6 16:16:55 root@htlxici sapinstance.sh[29446]: (watchdog): wait for PID 2410 returns 1

Oct 6 16:16:55 root@htlxici sapinstance.sh[29446]: (crit_test_app): No connection to instance DVEBMGS01

Oct 6 16:16:55 root@htlxici sapinstance.sh[29446]: (crit_test_app): Delaying 20 secs to allow instance DVEBMGS01 startup/recover

Oct 6 16:17:15 root@htlxici sapinstance.sh[29446]: (is_ip_local): ciPP9 is local

Oct 6 16:17:15 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect local dispatcher of instance DVEBMGS01 ...

Oct 6 16:17:15 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/dpmon" owner: "pp9adm"

Oct 6 16:17:16 root@htlxici sapinstance.sh[29446]: (crit_test_app): dpmon: /usr/sap/PP9/DVEBMGS01/exe/dpmon=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

Oct 6 16:17:16 root@htlxici sapinstance.sh[29446]: (crit_test_app): Trying to connect via RFC to host ciPP9 instance DVEBMGS01 ...

Oct 6 16:17:16 root@htlxici sapinstance.sh[29446]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/startrfc" owner: "pp9adm"

Oct 6 16:17:16 root@htlxici sapinstance.sh[29446]: (watchdog): Watchdog initiated (PIDs: 2878 for 2875 - Timeout: 100 secs)

Oct 6 16:17:17 root@htlxici sapinstance.sh[29446]: (watchdog): wait for PID 2875 returns 1

Oct 6 16:17:17 root@htlxici sapinstance.sh[29446]: (crit_test_app): No connection to instance DVEBMGS01

Oct 6 16:17:17 root@htlxici sapinstance.sh[29446]: (crit_test_app): Instance DVEBMGS01 not responding

Oct 6 16:17:17 root@htlxici sapinstance.sh[29446]: (start_d): ERROR: Instance DVEBMGS01 - fatal startup problem detected

Oct 6 16:17:17 root@htlxici master_control_script.sh[28963]: ##### Failed to start package ciPP9, rollback steps #####

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: Entering SGeSAP sapinstance.sh 'stop' runtime steps ...

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: SGeSAP - B.05.00.00 - Last modified 02/11/08

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: Found: /etc/cmcluster/scripts/sgesap/sap_functions.sh

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: Found: /etc/cmcluster/scripts/sgesap/customer_functions.sh

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: Not found: /etc/cmcluster/scripts/ext/ciPP9_customer_functions.sh

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (check_versions): Version check passed

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (check_versions): Log Level is 4

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (set_environment DVEBMGS01): Instance administrator : 'pp9adm'

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (set_environment DVEBMGS01): Instance home directory : '/home/pp9adm'

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (set_environment DVEBMGS01): Startprofile : '/usr/sap/PP9/SYS/profile/PP9_DVEBMGS01_ciPP9'

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (set_environment DVEBMGS01): Instance profile : '/usr/sap/PP9/SYS/profile/PP9_DVEBMGS01_ciPP9'

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (set_environment DVEBMGS01): Instance work directory : '/usr/sap/PP9/DVEBMGS01/work'

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/sappfpar" owner: "pp9adm"

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (init_params_sap_inst): Trying to determine instance type of DVEBMGS01...

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (init_params_sap_inst): Package will handle Application Server Instance DVEBMGS01

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (check_access): Testing directory /sapmnt/PP9/global...

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (watchdog): Watchdog initiated (PIDs: 3216 for 3215 - Timeout: 100 secs)

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (watchdog): wait for PID 3215 returns 0

Oct 6 16:17:18 root@htlxici sapinstance.sh[3089]: (check_access): /sapmnt/PP9/global accessible

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (check_access): Testing directory /usr/sap/PP9...

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (watchdog): Watchdog initiated (PIDs: 3244 for 3243 - Timeout: 100 secs)

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (watchdog): wait for PID 3243 returns 0

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (check_access): /usr/sap/PP9 accessible

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (init_params_sap_inst): System is not using Instance Agents

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (stop_d): Initiate stopping of dialog instance DVEBMGS01 on ciPP9

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (is_ip_local): ciPP9 is local

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (stop_app ciPP9 01 pp9adm): Attempting local stop of instance...

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (stop_d): Waiting for parallel shutdown sequence to finish

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/sapcontrol" owner: "pp9adm"

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (stop_app ciPP9 01 pp9adm): Running /usr/sap/PP9/DVEBMGS01/exe/sapcontrol -nr 01 -function StopWait 100 1

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (watchdog): Watchdog initiated (PIDs: 3325 for 3322 - Timeout: 100 secs)

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (wait_pidlist): Forked the wait_pidlist background shell. PID: 3319

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (wait_pidlist): PIDLIST: 3271 -1

Oct 6 16:17:19 root@htlxici sapinstance.sh[3089]: (wait_pidlist): PIDlist watchdog initiated (PIDlist: 3271)

Oct 6 16:17:20 root@htlxici sapinstance.sh[3089]: (watchdog): wait for PID 3322 returns 0

Oct 6 16:17:20 root@htlxici sapinstance.sh[3089]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/cleanipc" owner: "pp9adm"

Oct 6 16:17:22 root@htlxici sapinstance.sh[3089]: (clean_ipc): Searching shmem of DVEBMGS01 on htlxici using normal cleanup policy

Oct 6 16:17:23 root@htlxici: (clean_ipc): P: OsKey: 10110 0x0000277e Shared Memory Key: 10 removed

Oct 6 16:17:23 root@htlxici sapinstance.sh[3089]: (wait_pidlist): Killed the wait_pidlist background shell. PID: 3319

Oct 6 16:17:23 root@htlxici sapinstance.sh[3089]: (stop_d): Parallel dialog instance shutdown finished

Oct 6 16:17:23 root@htlxici sapinstance.sh[3089]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/sapcontrol" owner: "pp9adm"

Oct 6 16:17:25 root@htlxici sapinstance.sh[3089]: (is_ip_local): ciPP9 is local

Oct 6 16:17:25 root@htlxici sapinstance.sh[3089]: (crit_test_app): Trying to connect local dispatcher of instance DVEBMGS01 ...

Oct 6 16:17:25 root@htlxici sapinstance.sh[3089]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/dpmon" owner: "pp9adm"

Oct 6 16:17:26 root@htlxici sapinstance.sh[3089]: (crit_test_app): dpmon: /usr/sap/PP9/DVEBMGS01/exe/dpmon=>sapparam: SAPSYSTEMNAME neither in Profile nor in Commandline

Oct 6 16:17:26 root@htlxici sapinstance.sh[3089]: (crit_test_app): Trying to connect via RFC to host ciPP9 instance DVEBMGS01 ...

Oct 6 16:17:26 root@htlxici sapinstance.sh[3089]: (find_sap_binary): found location: "/usr/sap/PP9/DVEBMGS01/exe/startrfc" owner: "pp9adm"

Oct 6 16:17:26 root@htlxici sapinstance.sh[3089]: (watchdog): Watchdog initiated (PIDs: 4369 for 4368 - Timeout: 100 secs)

Oct 6 16:17:27 root@htlxici sapinstance.sh[3089]: (watchdog): wait for PID 4368 returns 1

Oct 6 16:17:27 root@htlxici sapinstance.sh[3089]: (crit_test_app): No connection to instance DVEBMGS01

Oct 6 16:17:27 root@htlxici sapinstance.sh[3089]: (crit_test_app): Instance DVEBMGS01 not responding

Oct 6 16:17:27 root@htlxici sapinstance.sh[3089]: Leaving SGeSAP sapinstance.sh 'stop' runtime steps ...

Oct 6 16:17:27 root@htlxici package_ip.sh[4578]: Removing 172.31.112.131

Oct 6 16:17:27 root@htlxici package_ip.sh[4578]: Remove IP address 172.31.112.131 from subnet 172.31.112.0

Oct 6 16:17:27 root@htlxici filesystem.sh[4610]: Number of components: 5 6

Oct 6 16:17:27 root@htlxici filesystem.sh[4610]: sorted components in array: 5 6

Oct 6 16:17:27 root@htlxici filesystem.sh[4610]: sorted indexes: 0 1

Oct 6 16:17:27 root@htlxici filesystem.sh[4610]: processing 1 /usr/sap/PP9/DVEBMGS01 1

Oct 6 16:17:27 root@htlxici filesystem.sh[4610]: processing 0 /usr/sap/PP9 1

Oct 6 16:17:27 root@htlxici filesystem.sh[4610]: Mount order: 1 2

Oct 6 16:17:27 root@htlxici filesystem.sh[4610]: max mnt order: 2

Oct 6 16:17:28 root@htlxici filesystem.sh[4610]: Unmounting filesystem on /dev/vx/dsk/pp9cidg/DVEBMGS61

Oct 6 16:17:29 root@htlxici filesystem.sh[4610]: Unmounting filesystem on /dev/vx/dsk/pp9cidg/sap_PP9

Oct 6 16:17:29 root@htlxici volume_group.sh[4678]: Deporting disk group pp9cidg

Oct 6 16:17:30 root@htlxici master_control_script.sh[28963]: ###### Failed to start package for ciPP9 ######

[Former Member]

Review the sapinstance.sh script also check the work directory, see if you can find any information in the log files.

FAIL: NIECONN_REFUSED (Connection refused), NiRawConnect failed in plugin_fopen() 

You can get this error for different reasons.

Thanks,

Naveed