MAIN_SHDIMP/SUBOD_SHD2_RUN/STOP_SHDI_SHD2 stuck

[former_member205758]

HI Exports,

Need Very urgent help, upgrade is in process and we are in Preprocessing Phase - MAIN_SHDIMP/SUBOD_SHD2_RUN/STOP_SHDI_SHD2 step.

normally this step should take more then 1 mins to shutdown Shadow instance but from past 12 hours the step is not completed.

When checked in SUM/Abap/log last log was generated 9 hours but there is no error reported in log of SUM tool.

Logs of  STOPSHD1.LOG

tail -100 STOPSHDI.LOG

1 ETQ201 Entering upgrade-phase "MAIN_SHDIMP/SUBMOD_SHD2_RUN/STOP_SHDI_SHD2" ("20150721213615")

1 ETQ399 Removing function description 'SUBST_START_BATCHJOB' from RFC cache.

1 ETQ399 Removing function description 'SUBST_CHECK_BATCHJOB' from RFC cache.

1 ETQ399 Removing function description 'SUBST_START_REPORT_IN_BATCH' from RFC cache.

2 ETQ367 Connect variables are set for standard instance access

4 ETQ399 System-nr = '00', GwService = 'sapgw00' Client = '000'

4 ETQ399 Environment variables:

4 ETQ399   dbs_ora_schema=SAPR3

1 ETQ200 Executing actual phase 'MAIN_SHDIMP/SUBMOD_SHD2_RUN/STOP_SHDI_SHD2'.

1 ETQ399 Phase arguments:

2 ETQ399 Arg[0] = 'STOP_SERVICE'

1 ETQ399 SYSTEM MANAGER: Initializing.

1 ETQ399 SYSTEM MANAGER: STOPSAP on UNIX.

1 ETQ399 SYSTEM MANAGER: STOPSAP with SAPCONTROL.

1 ETQ399 SYSTEM MANAGER: SAPControl tries to control all instances.

1 ETQ399 SYSTEM MANAGER: Calling getInstances.

1 ETQ399 SYSTEM MANAGER: found instance for action STOP : Instance Number: 02

1 ETQ399 SYSTEM MANAGER: ControlInstance with SAPCONTOL action STOP for instance 02.

1 ETQ399 SYSTEM HEALTH MANAGER: running preCheck for instance 02 on host sapehx01.

2 ETQ399 SYSTEM HEALTH MANAGER: Checking instance number 2 for used ports on host 'sapehx01'.

3 ETQ399 SYSTEM HEALTH MANAGER: Port 3202 appears to be used.

1 ETQ399 SYSTEM HEALTH MANAGER: call test rfc.

4 ETQ399 Set RFC variables for shadow connect:

4 ETQ399 System-nr = '02', GwService = 'sapgw02' Client = '000'

1 ETQ359 RFC Login to: System="EHX", AsHost="sapehx01" Nr="02", GwHost="sapehx01", GwService="sapgw02"

2 ETQ232 RFC Login succeeded

4 ETQ010 Date & Time: 20150721213615

1 ETQ233 Calling function module "UPG_IS_SHADOW_SYSTEM" by RFC

2 ETQ373 parameter "IV_R3UP" = "X"

1 ETQ234 Call of function module "UPG_IS_SHADOW_SYSTEM" by RFC succeeded

4 ETQ010 Date & Time: 20150721213615

2 ETQ373 parameter "EV_SHADOW" = "X"

1 ETQ399 SYSTEM HEALTH MANAGER: check for instance processlist.

1 ETQ399 SAPCONTROL MANAGER: getProcessList with host: sapehx01 and instance: 02

3 ETQ120 20150721213616: PID 10813590 execute '/usr/sap/EHP7/SUM/abap/exe/sapcontrol -format script -prot NI_HTTP -host sapehx01 -nr 02 -function GetProcessList', output written to '/usr/sap/EHP7/SUM/abap/log/SAPup.ECO'.

3 ETQ122 20150721213616: PID 10813590 exited with status 3 (time: 0.000/0.060/0.112 real/usr/sys)

1 ETQ399 SYSTEM HEALTH MANAGER: System up and running, go on with stop action

1 ETQ399 SAPCONTROL MANAGER: stopWait with host: sapehx01 and instance: 02

3 ETQ120 20150721213616: PID 10813592 execute '/usr/sap/EHP7/SUM/abap/exe/sapcontrol -prot NI_HTTP -host sapehx01 -nr 02 -function StopWait 300 10', output written to '/usr/sap/EHP7/SUM/abap/log/SAPup.ECO'.

LOG of SAPup.ECO:

EXECUTING /usr/sap/EHP7/SUM/abap/exe/sapcontrol -format script -prot NI_HTTP -host sapeht01 -nr 02 -function GetProcessList

21.07.2015 21:36:16

GetProcessList

OK

0 name: msg_server

0 description: MessageServer

0 dispstatus: GREEN

0 textstatus: Running

0 starttime: 2015 07 21 15:16:43

0 elapsedtime: 6:19:33

0 pid: 7143444

1 name: enserver

1 description: EnqueueServer

1 dispstatus: GREEN

1 textstatus: Running

1 starttime: 2015 07 21 15:16:43

1 elapsedtime: 6:19:33

1 pid: 10748044

2 name: disp+work

2 description: Dispatcher

2 dispstatus: GREEN

2 textstatus: Running

2 starttime: 2015 07 21 15:16:43

2 elapsedtime: 6:19:33

2 pid: 15335606

3 name: gwrd

3 description: Gateway

3 dispstatus: GREEN

3 textstatus: Running

3 starttime: 2015 07 21 15:16:44

3 elapsedtime: 6:19:32

3 pid: 14287024

SAPup> Process with PID 10813590 terminated with status 3 at 20150721213616!

SAPup> Starting subprocess with PID 10813592 in phase 'STOP_SHDI_SHD2' at 20150721213616

    ENV: DIR_LIBRARY=/usr/sap/EHP7/SUM/abap/exe

    ENV: JAVA_HOME=/usr/java14_64

    ENV: LIBPATH=/usr/sap/EHP7/SUM/abap/exe:/usr/sap/EHP7/SUM/jvm/jre/lib/ppc64/server:/usr/sap/EHP7/SUM/jvm/jre/lib/ppc64:/usr/sap/EHP7/SUM/jvm/jre/../lib/ppc64:/usr/sap/EHP7/SUM/jvm/jre/lib/ppc64/jli:/usr/sap/EHP7/SUM/jvm/jre/../lib/ppc64/jli:/usr/lib:/lib:/usr/sap/EHT/SYS/exe/run:/oracle/client/11x_64/instantclient

    ENV: NLS_LANG=AMERICAN_AMERICA.WE8DEC

    ENV: ORACLE_BASE=/oracle

    ENV: ORACLE_SID=EHT

    ENV: PATH=/usr/sap/EHP7/SUM/abap/exe:/usr/java14_64/bin:.:/export/home/ehtadm:/usr/sap/EHT/SYS/exe/run:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java5/jre/bin:/usr/java5/bin

    ENV: SAPDATA_HOME=/oracle/EHT

    ENV: SAPSYSTEMNAME=EHT

    ENV: dbms_type=ORA

    ENV: dbs_ora_schema=SAPR3

    ENV: dbs_ora_tnsname=EHT

    PWD: /usr/sap/EHP7/SUM/abap/tmp

EXECUTING /usr/sap/EHP7/SUM/abap/exe/sapcontrol -prot NI_HTTP -host sapeht01 -nr 02 -function StopWait 300 10

....................................................................................................................................................................................

So that where its hanging and not going forward and it also not erroring in out.

Any valuable suggession would be great as we need reach deadline.

Thanks

[former_member205758 · Accepted solution]

The issue was solved by killing the SUM processes and restarting the Upgrade and it went well.