SHUTDWN_BY NO_MORE_CHILD_PROCESSES

[former_member196171]

Hi Experts ,

I have just upgraded my Netweaver system from NW 7.00 EHP2 to NW 7.31  on Oracle 11G / AIX 6.1,

Upgrade went fine and system was running fine also , but system does not come up after restarted,

ASCS instance is running , but  DVEBMGS00 does not start up, please help

Here are dev_sapstart and INSTSTAT FILEs entries as below

/usr/sap/KXS/DVEBMGS00/work

INSTSTAT FILE

START_TIME 2013/02/28 22:28:01
START_PROF /usr/sap/K9S/SYS/profile/KXS_DVEBMGS00_sapkxsci
SYSTEMNAME KXS
INST_NAME  DVEBMGS00
CMD_LINE   pf=/usr/sap/KXS/SYS/profile/K9S_DVEBMGS00_sapkxsci
SHUTDWN_BY NO_MORE_CHILD_PROCESSES
SHUTDWN_AT 2013/02/28 22:28:01
EXIT_TIME  2013/02/28 22:28:01
EXIT_WITH  3 No more child processes

In dev_sapstart
---------------------------------------------------
trc file: "dev_sapstart", trc level: 1, release: "720"
---------------------------------------------------

Thu Feb 28 22:28:01 2013
SigISetDefaultAction : default handling for signal SIGCHLD

[Former Member]

The issue has resolved which has occured on Linux 3.0.101-0.46-default. The work directory was not giving any info except the INSTSTAT.log was giving info saying "Exit with 3 No more child processes."

The root cause was, the sapstart process, was hanging.

Resolutioin was, the kill -9 command worked to kill the process and the SAP instance has started.

Clean the processes, that might help in this situation.

Regards,

Durga PK Saitana

[lalit_rana2]

Thanks for your info i have followed your instruction and issue has been resolved.

[former_member196171]

Hi Everybody,

R3trans -d is already attached , startsap is not even upto the point to generate traces like dev_ms dev_w0 ..

I guess problem  sapstartsrvc can not recognize the instance type as ABAP as sapstartsrvc.log shows,

CCMS agent initialization: WARNING: cannot recognize an instance type,

continuing in the non-ABAP mode.

[Former Member]

Hello Abu Zahir Al-Mamun

The solution for your problema is described on the note 1031799 - Instance cannot be started

I hope it may help.

Best Regards,

Andres Amaya

[former_member185239]

Hi Al,

Please paste the logs for r3trans -d , dev_disp , dev_ms , dev_w0.

With Regards

Ashutosh Chaturvedi

[Former Member]

Hi,

Please execute R3trans -d and paste the trans.log for more information.

Please make sure that you have updated the kernel level in the following places .

1./sapmnt/SID/exe

2./usr/sap/SID/DV*/exe

3./usr/sap/SID/SC*/exe

Stop all the SAP & Oracle processes.. .Kill any SAP services running in k9sadm and try to start it again.

If it is not coming, please paste the dev_w0 log available in the work directory.

Thanks and Regards,

Vimal

[former_member196171]

Hi Maria Joseoh,

Thanks for getting back to me on this , based on your reply , I have bounced the AIX servers ( CI and DB nodes as distributed installation ) ,  and tried to start SAP again , I have the same problem , Kernel versions are same with above directories

it does not even go to the  generate  trace file dev_w0 , only logs I found are as below

sapkxsci:kxsadm 14> cd /home/kxsadm
sapkxsci:kxsadm 15> R3trans -d
This is R3trans version 6.23 (release 720 - 31.10.12 - 20:12:06).
unicode enabled version
R3trans finished (0000).
sapkxsci:kxsadm 16> startsap
Checking KXS Database
Database is running
-------------------------------------------
Starting Startup Agent sapstartsrv
OK
Instance Service on host sapkxsci started
-------------------------------------------
starting SAP Instance ASCS02
Startup-Log is written to /home/kxsadm/startsap_ASCS02.log
-------------------------------------------
/usr/sap/KXS/ASCS02/exe/sapcontrol -prot NI_HTTP -nr 02 -function Start
Instance on host sapkxsci started
Starting Startup Agent sapstartsrv
OK
Instance Service on host sapkxsci started
-------------------------------------------
starting SAP Instance DVEBMGS00
Startup-Log is written to /home/kxsadm/startsap_DVEBMGS00.log
-------------------------------------------
/usr/sap/KXS/DVEBMGS00/exe/sapcontrol -prot NI_HTTP -nr 00 -function Start
Instance on host sapkxsci started
sapkxsci:kxsadm 17> ps -ef | grep kxsadm
  kxsadm  9240792        1   0 09:59:21      -  0:00 /usr/sap/KXS/ASCS02/exe/sapstartsrv pf=/usr/sap/KXS/SYS/profile/START_ASCS02_sapkxsci -D -u kxsadm
  kxsadm  9633806        1   0 10:37:47      -  0:00 /usr/sap/KXS/DVEBMGS00/exe/sapstartsrv pf=/usr/sap/KXS/SYS/profile/KXS_DVEBMGS00_sapkxsci -D
  kxsadm  9830548        1   0 10:37:30      -  0:00 sapstart pf=/usr/sap/KXS/SYS/profile/START_ASCS02_sapkxsci
  kxsadm 10223784 11206684   0 10:38:21  pts/1  0:00 grep kxsadm
  kxsadm 11010176  9830548   0 10:37:31      -  0:00 en.sapKXS_ASCS02 pf=/usr/sap/KXS/SYS/profile/KXS_ASCS02_sapkxsci
  kxsadm 11206684 11272220   0 10:34:29  pts/1  0:00 -csh
  kxsadm 11993122 11206684   9 10:38:21  pts/1  0:00 ps -ef
  kxsadm 12058762  9633806   0 10:37:59      -  0:00 sapstart pf=/usr/sap/KXS/SYS/profile/KXS_DVEBMGS00_sapkxsci
  kxsadm 12517390  9830548   0 10:37:31      -  0:00 ms.sapKXS_ASCS02 pf=/usr/sap/KXS/SYS/profile/KXS_ASCS02_sapkxsci

sapstartsrv.log

---------------------------------------------------
trc file: "sapstartsrv.log", trc level: 0, release: "720"
---------------------------------------------------
pid        9633806

[Thr 01] Fri Mar  1 10:37:47 2013
No halib defined => HA support disabled
CCMS agent initialization: WARNING: cannot recognize an instance type, continuing in the non-ABAP mode.
CCMS agent initialization for instance type UNKNOWN: return code 0.
CCMS agent start: return code 0.
Initializing SAPControl Webservice
Starting AutoRestart thread
[Thr 515] Fri Mar  1 10:37:48 2013
AutoRestart thread started
SapSSLInit failed => https support disabled
Starting WebService thread
Webservice thread started, listening on port 50013
Trusted http connect via Unix domain socket '/tmp/.sapstream50013' enabled.

trusted unix domain socket user is starting SAP System at 2013/03/01 10:37:58
~

sapstart.log :

SAP-R/3-Startup Program Rel 720 V2.0 (2008/11/15)
-------------------------------------------------

Starting at 2013/03/01 10:37:59
Startup Profile: "/usr/sap/KXS/SYS/profile/K9S_DVEBMGS00_sapkxsci"

Starting Programs
-----------------
(10813688) Waiting for Child Processes to terminate.
(10813688) **** No more Child Processes to wait for.
(10813688) Parent Shutdown at 2013/03/01 10:37:59

Execute Post-Shutdown Commands
------------------------------
(10813688) Exiting with Return-Code 3. (No more child processes)
~

dev_sapstart trace file:

---------------------------------------------------
trc file: "dev_sapstart", trc level: 1, release: "720"
---------------------------------------------------

Fri Mar  1 10:37:59 2013
SigISetDefaultAction : default handling for signal SIGCHLD
~

INSTSTAT file:

START_TIME 2013/03/01 10:37:59
START_PROF /usr/sap/KXS/SYS/profile/K9S_DVEBMGS00_sapkxsci
SYSTEMNAME KXS
INST_NAME  DVEBMGS00
CMD_LINE   pf=/usr/sap/KXS/SYS/profile/KXS_DVEBMGS00_sapk9sci
SHUTDWN_BY NO_MORE_CHILD_PROCESSES
SHUTDWN_AT 2013/03/01 10:37:59
EXIT_TIME  2013/03/01 10:37:59
EXIT_WITH  3 No more child processes

[Former Member]

Check for logs like dev_w* in /usr/sap/KXS/DVEBMGS00/work directory. They must be there. If exists attach dev_w0 file to message.

Regards,

Roman