Shadow instance couldn't be started, check 'STARTSFI.LOG' and 'DEVTRACE.LOG': Process G:\usr\sap\EPD\SUM\abap\exe/sapcontrol.exe exited with 2, see 'G:\usr\sap\EPD\SUM\abap\log\SAPup.ECO' for details

[Former Member]

We're facing the error below below during the Prepocessing phase of  EHP6 upgrade on our production server.

This trouble ticket was created by SAPup on 20131009100402
-------------------------------------------------------------
SAPup broke during phase START_SHDI_FIRST in module MAIN_SHDRUN / Shadow System Operations: SPDD and Activation
Error  Message: Shadow instance couldn't be started, check 'STARTSFI.LOG' and  'DEVTRACE.LOG': Process G:\usr\sap\EPD\SUM\abap\exe/sapcontrol.exe  exited with 2, see 'G:\usr\sap\EPD\SUM\abap\log\SAPup.ECO' for details
-------------------------------------------------------------
-------------------------------------------------------------
Summary of SAPup:
-------------------------------------------------------------
SAPehpi Release:     lmt_004
SAPehpi Version:     SAPup release lmt_004 version 53.000
Start Release:          731
Target Release:          731
-------------------------------------------------------------
Summary of host system details:
-------------------------------------------------------------
SID:               EPD
Host:               HFDT1SE01
MS Host:          SAPHFDT1SEGRP
GW Host:          HFDT1SE01
Start Path:          D:\usr\sap\EPD\DVEBMGS07\exe     
Kernel Path:          D:\usr\sap\EPD\DVEBMGS07\exe
-------------------------------------------------------------
Summary of operating system details:
-------------------------------------------------------------
OS Type:          Windows NT X86_64
OS Version:          6.1
-------------------------------------------------------------
Summary of database details:
-------------------------------------------------------------
Database Type:          syb
Database Version:     15.7.0.101
-------------------------------------------------------------
Summary of RFC details:
-------------------------------------------------------------
Host:               HFDT1SE01
GW Host:          HFDT1SE01
Client:               000
Destination:          EPD
Language:          E
System No.:          08
-------------------------------------------------------------
-------------------------------------------------------------

We  have attached the dev_disp log file from the SUM directory.In addation  we have attached the following log files for your further analysis:

SAPup.ECO
SAPup_troubleticket.log
STARTSFI.LOG

Please assist

Regards

SAPup_troubleticket.log.txt.zip

STARTSFI.LOG.txt.zip

SAPup.ECO.txt.zip

[Reagan · Accepted solution]

<ES> Info: em/initial_size_MB( 32750MB) not multiple of em/blocksize_KB( 4096KB)
<ES> Info: em/initial_size_MB rounded up to 32752MB
Using implementation view
<EsNT> Using memory model view.
<EsNT> Memory Reset disabled as NT default
*** ERROR => <EsNT> CreateFileMapping name=Global\ES_SEG_08_012 ,size=2044 MB, error=1455 [esnti.c      1276]

I see a memory allocation problem

Reduce the em/initial_size_MB to a 4096 and restart the system

Regards

RB

[Former Member · Accepted solution]

Thank you All for your contributions, Reagan's recommendation  has worked for us  we have changed the em/initial_size_MB parameter in the instance profile of the shadow system and the upgrade has continued,it  was a memory allocation problem.

Regards 

[Former Member · Accepted solution]

Wonderful, I'm glad it works now.

I just want to recommend you take a look at note 88416, which explains the PHYS_MEMSIZE parameter I mentioned, which you should tweak for systems running on Windows instead of em/initial_size_MB.

[Former Member · Accepted solution]

Thanks Reagan the error logs doesn't really show up the em/initial_size_MB as a problem but when updated the value to 4096, I could able to start the shadow instance.

Thank you very much.

[former_member646396]

Dear Sourav Brahma,

Thanks for your info. I had solved my problems while SPS upgrading in solman 7.2 After maintaining the parameter em/initial_size_MB to 8190MB in the instance profile.

It works fine and moved on next ohase.

Thanks

Vijay Sharma

[Former Member]

Hi RB,

Good morning!

I am also getting same issue,I did start and stop the  main instance and shadow instance , cleared the shared memory.but no luck.

shadow instance dev_disp file error info

I  *** ERROR => e=13 semget(22338,1,992) (13: Permission denied) [semux.c

499]

M  *** ERROR => ThrRegisterSem: SemInit SEM_CCMS_AS_MONI_KEY [thxxrun1.c   782]

M  *** ERROR => ThCallHooks: event handler ThrRegisterSem for event BEFORE_DB_CONNECT failed [thxxtool3.c  302]

M  in_ThErrHandle: 1

M  *** ERROR => ThCallHooks: hook failed (step 1, th_errno 5, action 3, level 1) [thxxhead.c   11698]M  Info for wp 5

Please advise me on this.

[maay]

changed the em/initial_size_MB parameter in the instance profile of the shadow system and then continue the upgrade.

it looks like a memory allocation problem.

[maay]

Special thanks to Reagan.

Had the same issue , resolved by updating the parameter.

Thanks for the good work.

[Former Member]

Hi Reagan,

I also had the same issue ,, it was resolved by updating the parameter . Thanks man

@Roy Thanks for highlighting the error .

[Former Member]

Hi,

Check services entries at OS level and reboot the server it works.

BR,

Prabhakar

[Former Member]

Hi,

Check for Space left on device any file system filled up?

Check sapnote 525677

BR,

Prabhakar.

[Former Member]

I don't see the dev_disp log attached, but my first thought was memory problem. 

I have run quite few EHP installations in the last few weeks, and SUM has always set up the shadow instance with way too much memory (PHYS_MEMSIZE = 50% on a host that is already running 2 systems and their databases).  I usually changed it to something like 20%, which was still almost 20 GB, and the shadow instance came up just fine.  Of course, you can always increase your swap space instead.

[Former Member]

Susan

On which profile do we set PHYS_MEMSIZE ,is it the shadow instance profile or source system instance profile.?

Regards

[ashish_vikas]

Please can you also go through SAP Note 525677 - Problems when starting shadow instance

best regards

ashish

[Former Member]

We have gone through SAPnote 525677 and tried to start the shadow manually but we got the error below

G:\usr\sap\EPD\SUM\abap\bin>SAPup startshd
Enter the path of your root directory [G:\usr\sap\EPD\SUM\abap]:
Root directory =
Starting shadow system ...
Restarting service ...

FATAL ERROR: Process G:\usr\sap\EPD\SUM\abap\exe/sapcontrol.exe exited with 2, see 'G:\usr\sap\EPD\SUM\abap\log\SAPup.ECO' for details

Regards

[Reagan]

Check the suggestion to reduce em/initial_size_MB I have mentioned below

Regards

RB

[Former Member]

Hi,

Please run the below command to check if all the services are green.

sapcontrol -nr 08 -function GetProcessList

If yes then this mean that SUm tool is not able to login with user DDIC.

1888639 - DDIC RFC connection error to shadow instance during Upgrades: Password logon no longer pos...

If some of the services are stopped please run this.

sapcontrol -nr 08 -function GetVersionInfo

Please run these command in the SUM/abap/exe directory where the new kernel is in shadow system.

Thanks

Rishi abrol

[Former Member]

The Enqueue and msg_server.exe are green  but the disp+work.exe is gray and stopped .I have pasted the output of the dev_disp above.

Regards 

[Reagan]

Hello

I don't see the dev_disp or dev_w0 trace files attached.

If possible stop the main system and restart the server.

Then start the shadow instance manually.

Additionally make sure that you have a supported OS and DB for EHP6

Regards

RB

[Former Member]

Our database is Sybase ASE 15.7 and OS is Windows 2008 R2.Find below the dev_disp output f from the SUM directory

---------------------------------------------------

trc file: "dev_disp", trc level: 1, release: "720"

---------------------------------------------------

sysno      08

sid        EPD

systemid   562 (PC with Windows NT)

relno      7200

patchlevel 0

patchno    401

intno      20020600

make       multithreaded, Unicode, 64 bit, optimized

profile    G:\usr\sap\EPD\SUM\abap\EPD\SYS\profile\EPD_DVEBMGS08_HFDT1SE01

pid        29876

kernel runs with dp version 138000(ext=119000) (@(#) DPLIB-INT-VERSION-138000-UC)

length of sys_adm_ext is 588 bytes

*** SWITCH TRC-HIDE on ***

***LOG Q00=> DpSapEnvInit, DPStart (08 29876) [dpxxdisp.c   1320]

    shared lib "dw_xml.dll" version 401 successfully loaded

    shared lib "dw_xtc.dll" version 401 successfully loaded

    shared lib "dw_stl.dll" version 401 successfully loaded

    shared lib "dw_gui.dll" version 401 successfully loaded

    shared lib "dw_mdm.dll" version 401 successfully loaded

    shared lib "dw_rndrt.dll" version 401 successfully loaded

    shared lib "dw_abp.dll" version 401 successfully loaded

    shared lib "dw_sym.dll" version 401 successfully loaded

    shared lib "dw_aci.dll" version 401 successfully loaded

rdisp/softcancel_sequence :  -> 0,5,-1

use internal message server connection to port 3908

rdisp/dynamic_wp_check : 0

rdisp/calculateLoadAverage : 1

Thu Oct 10 16:25:25 2013

*** WARNING => DpNetCheck: NiAddrToHost(1.0.0.0) took 4 seconds

***LOG GZZ=> 1 possible network problems detected - check tracefile and adjust the DNS settings [dpxxtool2.c  6438]

MtxInit: 30000 0 0

DpSysAdmExtInit: ABAP is active

DpSysAdmExtInit: VMC (JAVA VM in WP) is not active

DpIPCInit2: write dp-profile-values into sys_adm_ext

DpIPCInit2: start server >HFDT1SE01_EPD_08                        <

DpShMCreate: sizeof(wp_adm)        29328    (2256)

DpShMCreate: sizeof(tm_adm)        5517056    (27448)

DpShMCreate: sizeof(wp_ca_adm)        64000    (64)

DpShMCreate: sizeof(appc_ca_adm)    64000    (64)

DpCommTableSize: max/headSize/ftSize/tableSize=500/16/1384064/1384080

DpShMCreate: sizeof(comm_adm)        1384080    (2744)

DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0

DpShMCreate: sizeof(slock_adm)        0    (296)

DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0

DpShMCreate: sizeof(file_adm)        0    (80)

DpShMCreate: sizeof(vmc_adm)        0    (2160)

DpShMCreate: sizeof(wall_adm)        (41664/42896/64/192)

DpShMCreate: sizeof(gw_adm)    48

DpShMCreate: sizeof(j2ee_adm)    3952

DpShMCreate: SHM_DP_ADM_KEY        (addr: 0000000007CE0050, size: 7161296)

DpShMCreate: allocated sys_adm at 0000000007CE0060

DpShMCreate: allocated wp_adm_list at 0000000007CE30B0

DpShMCreate: allocated wp_adm at 0000000007CE32A0

DpShMCreate: allocated tm_adm_list at 0000000007CEA540

DpShMCreate: allocated tm_adm at 0000000007CEA590

DpShMCreate: allocated wp_ca_adm at 000000000822D4A0

DpShMCreate: allocated appc_ca_adm at 000000000823CEB0

DpShMCreate: allocated comm_adm at 000000000824C8C0

DpShMCreate: system runs without slock table

DpShMCreate: system runs without file table

DpShMCreate: allocated vmc_adm_list at 000000000839E760

DpShMCreate: system runs without vmc_adm

DpShMCreate: allocated gw_adm at 000000000839E810

DpShMCreate: allocated j2ee_adm at 000000000839E850

DpShMCreate: allocated ca_info at 000000000839F7D0

DpShMCreate: allocated wall_adm at 000000000839F860

Thu Oct 10 16:25:26 2013

DpCommAttachTable: attached comm table (header=000000000824C8C0/ft=000000000824C8D0)

DpSysAdmIntInit: initialize sys_adm

rdisp/test_roll : roll strategy is DP_NORMAL_ROLL

dia token check not active (5 token)

MBUF state OFF

DpCommInitTable: init table for 500 entries

DpRqQInit: keep protect_queue / slots_per_queue 0 / 2001 in sys_adm

rdisp/queue_size_check_value :  -> on,50,30,40,500,50,500,80

EmInit: MmSetImplementation( 2 ).

MM global diagnostic options set: 0

<ES> client 0 initializing ....

<ES> EsILock: use spinlock for locking

<ES> InitFreeList

<ES> block size is 4096 kByte.

<ES> Info: em/initial_size_MB( 32750MB) not multiple of em/blocksize_KB( 4096KB)

<ES> Info: em/initial_size_MB rounded up to 32752MB

Using implementation view

<EsNT> Using memory model view.

<EsNT> Memory Reset disabled as NT default

*** ERROR => <EsNT> CreateFileMapping name=Global\ES_SEG_08_012 ,size=2044 MB, error=1455 [esnti.c      1276]

*****************************************************

* Memory diagnostic                                 *

*****************************************************

Systeminformation

------------------------------------------------

Processor-Typ             : Processor-Count           : 32

Operating System          : NT 6.1, Build 7601

Service Pack              : Service Pack 1

------------------------------------------------

NT Pagefile Informations

------------------------------------------------

Config. minimum size      : 41943040 K

Config. maximum size      : 41943040 K

Avail.  maximum size      : 41943040 K

| Num|Pagefile                      |  Min.Size|  Max.Size| Avail.Max| Curr.Size|

|----|------------------------------|----------|----------|----------|----------|

|   1|c:\pagefile.sys               |41943040 K|41943040 K|41943040 K|41943040 K|

------------------------------------------------

NT Task Manager Informations

------------------------------------------------

Total Handles             :     4096

Total Threads             :        0

Total Processes           :       10

Commit Charge Total       :        0 K

Commit Charge Limit       : 1961633360 K

Commit Charge Peak        :        0 K

Phys.Memory Total         : 1961088692 K

Phys.Memory Available     :        0 K

File Cache                :       54 K

Kernel Memory Total       :        0 K

Kernel Memory Paged       :        0 K

Kernel Memory Nonpaged    :        0 K

------------------------------------------------

Memory usage of current process

------------------------------------------------

Total virt.address space  : 0000008589934464 K

Avail.virt.address space  : 0000008588863404 K

Private Pages             :        0 K

Total heap size           :    10159 K

Virtual memory regions    :       36 K

Uncommitted heap memory   :     4716 K

Allocated heap memory     :     5180 K

Moveable heap memory      :        0 K

DDE shared heap memory    :     3125 K

------------------------------------------------

Memory usage of all processes

-----------------------------

| PID|Image               |Instance   |  Work.Set|   WS Peak|Priv.Pages|   PP Peak|Pg Fault|

|----|--------------------|-----------|----------|----------|----------|----------|--------|

|4368|conhost.exe         |           |    5736 K|    5736 K|    3904 K|   37528 K|       1|

|4776|conhost.exe         |           |    5732 K|    5732 K|    3900 K|   37528 K|       1|

|1660|msg_server.EXE      |[MS] EPD_06|   19864 K|   19868 K|   20524 K|   20676 K|       4|

|5568|conhost.exe         |           |    5728 K|    5728 K|    3896 K|    3896 K|       1|

|4356|enserver.EXE        |[**] EPD_06|  123984 K|  123984 K|  843128 K|  843140 K|      30|

|5412|conhost.exe         |           |    5732 K|    5732 K|    3900 K|    3900 K|       1|

|2928|conhost.exe         |           |    5732 K|    5732 K|    3904 K|   37528 K|       1|

|5964|disp+work.EXE       |[DP] EPD_07|  124192 K|  124232 K|   93688 K|   95476 K|      41|

|5576|conhost.exe         |           |    5880 K|    5904 K|    3912 K|    3912 K|       4|

|5652|igswd.EXE           |[**] EPD_07|   12460 K|   12960 K|    9500 K|   10964 K|       3|

|5816|conhost.exe         |           |    5756 K|    5756 K|    3904 K|    3904 K|       1|

|1396|igsmux.exe          |           |   33576 K|   33616 K|   34132 K|   34324 K|       8|

|4756|igspw.exe           |           |   42780 K|   43448 K|   42736 K|   42924 K|      44|

|5596|igspw.exe           |           |   42612 K|   43248 K|   42884 K|   43068 K|      45|

|6232|gwrd.EXE            |[GW] EPD_07|   25660 K|   25700 K|   20356 K|   20552 K|       9|

|6240|icman.EXE           |[**] EPD_07|  121216 K|  121256 K|  120912 K|  120980 K|      33|

|6248|disp+work.EXE       |[WP] EPD_07|  408740 K| 1393332 K|  229060 K|  654708 K|   86545|

|6256|disp+work.EXE       |[WP] EPD_07|   62608 K|   62608 K|  101708 K|  102480 K|      15|

|6264|disp+work.EXE       |[WP] EPD_07|  142336 K|  143548 K|  111568 K|  111836 K|      42|

|6272|disp+work.EXE       |[WP] EPD_07|  246360 K| 1271972 K|  159384 K|  591880 K|    1706|

|6280|disp+work.EXE       |[WP] EPD_07|  145956 K|  149524 K|  120772 K|  122020 K|     840|

|6288|disp+work.EXE       |[WP] EPD_07|   56004 K|   56004 K|  101020 K|  101824 K|      13|

|6296|disp+work.EXE       |[WP] EPD_07|   57520 K|   57520 K|  101636 K|  102488 K|      14|

|6304|disp+work.EXE       |[WP] EPD_07|  268264 K|  866652 K|  180352 K|  180544 K|    4931|

|6312|disp+work.EXE       |[WP] EPD_07|  102684 K|  103924 K|  116096 K|  116276 K|      38|

|6320|disp+work.EXE       |[WP] EPD_07|   56496 K|   56496 K|  101456 K|  101820 K|      13|

|6328|disp+work.EXE       |[WP] EPD_07|   82940 K|   88368 K|  109952 K|  110040 K|      21|

|6336|disp+work.EXE       |[WP] EPD_07|  249752 K|  261160 K|  165912 K|  172092 K|   10160|

|6344|disp+work.EXE       |[WP] EPD_07|  196176 K| 1202248 K|  169336 K|  582432 K|    2336|

|6352|disp+work.EXE       |[WP] EPD_07|  244764 K|  257296 K|  142724 K|  143804 K|     142|

|6360|disp+work.EXE       |[WP] EPD_07|   69104 K|   69640 K|  106412 K|  106684 K|     477|

|6368|disp+work.EXE       |[WP] EPD_07|   56720 K|   56720 K|  101612 K|  102588 K|      14|

|5084|enrepserver.EXE     |[**] EPD_06|   40300 K|   40340 K|  179864 K|  180052 K|      30|

|7136|conhost.exe         |           |    5696 K|    5696 K|    3900 K|    3900 K|       1|

|29760|sapstartsrv.exe     |           |  103204 K|  103224 K|  130120 K|  187972 K|      39|

|32564|conhost.exe         |           |    5732 K|    5732 K|    3900 K|   37908 K|       1|

|31156|msg_server.EXE      |[MS] EPD_08|   19648 K|   19648 K|   20556 K|   20560 K|       4|

|32620|conhost.exe         |           |    5736 K|    5736 K|    3896 K|    3896 K|       1|

|32488|enserver.EXE        |[**] EPD_08|   53588 K|   53588 K|   70008 K|   70008 K|      13|

|31092|conhost.exe         |           |    5728 K|    5728 K|    3896 K|    3896 K|       1|

|29876|disp+work.EXE       |           |   50180 K|   50180 K|   81456 K|   89360 K|      12|

|31648|conhost.exe         |           |    5752 K|    5752 K|    3900 K|    3900 K|       1|

|----|--------------------|-----------|----------|----------|----------|----------|--------|

|    |Sum                 |           | 3328628 K|**********| 3875676 K|**********|********|

*** Error 11 while initializing OS dependent part.

*** ERROR => DpEmInit: EmInit (1) [dpxxdisp.c   10757]

*** ERROR => DpMemInit: DpEmInit (-1) [dpxxdisp.c   10671]

*** DP_FATAL_ERROR => DpSapEnvInit: DpMemInit

*** DISPATCHER EMERGENCY SHUTDOWN ***

increase tracelevel of WPs

NiWait: sleep (10000ms) ...

NiISelect: timeout 10000ms

NiISelect: maximum fd=1

NiISelect: read-mask is NULL

NiISelect: write-mask is NULL

Thu Oct 10 16:25:36 2013

NiISelect: TIMEOUT occured (10000ms)

dump system status

Workprocess Table (long)            Thu Oct 10 13:25:36 2013

========================

No Type  Pid    Status  Cause Start Rstr  Err Sem Time Program          Cl  User         Action                    Table

-------------------------------------------------------------------------------------------------------------------------------

Dispatcher Queue Statistics            Thu Oct 10 13:25:36 2013

===========================

+------+--------+--------+--------+------------+------------+

|  Typ |    now |   high |    max | writes     |  reads     |

+------+--------+--------+--------+------------+------------+

| NOWP |      0 |      0 |   2000 |          0 |          0 |

+------+--------+--------+--------+------------+------------+

|  DIA |      0 |      0 |   2000 |          0 |          0 |

+------+--------+--------+--------+------------+------------+

|  UPD |      0 |      0 |   2000 |          0 |          0 |

+------+--------+--------+--------+------------+------------+

|  ENQ |      0 |      0 |   2000 |          0 |          0 |

+------+--------+--------+--------+------------+------------+

|  BTC |      0 |      0 |   2000 |          0 |          0 |

+------+--------+--------+--------+------------+------------+

|  SPO |      0 |      0 |   2000 |          0 |          0 |

+------+--------+--------+--------+------------+------------+

|  UP2 |      0 |      0 |   2000 |          0 |          0 |

+------+--------+--------+--------+------------+------------+

max_rq_id        0

wake_evt_udp_now    0

wake events          total     0,  udp     0 (  0%),  shm     0 (  0%)

since last update    total     0,  udp     0 (  0%),  shm     0 (  0%)

Dump of tm_adm structure:            Thu Oct 10 13:25:36 2013

=========================

Term    uid  man user    term   lastop  mod wp  ta   a/i (modes)

Workprocess Comm. Area Blocks            Thu Oct 10 13:25:36 2013

=============================

Slots: 1000, Used: 0, Max: 0

+------+--------------+----------+-------------+

|   id | owner        |   pid    | eyecatcher  |

+------+--------------+----------+-------------+

NiWait: sleep (5000ms) ...

NiISelect: timeout 5000ms

NiISelect: maximum fd=1

NiISelect: read-mask is NULL

NiISelect: write-mask is NULL

Thu Oct 10 16:25:41 2013

NiISelect: TIMEOUT occured (5000ms)

DpHalt: shutdown server >HFDT1SE01_EPD_08                        < (normal)

DpJ2eeDisableRestart

DpHalt: switch off Shared memory profiling

ShmProtect( 57, 3 )

ShmProtect(SHM_PROFILE, SHM_PROT_RW

ShmProtect( 57, 1 )

ShmProtect(SHM_PROFILE, SHM_PROT_RD

DpWakeUpWps: wake up all wp's

DpHalt: stop work processes

DpHalt: terminate gui connections

DpHalt: wait for end of work processes

DpHalt: not attached to the message server

DpHalt: cleanup EM

EsCleanup( )

EsCleanup ....

EmCleanup() -> 0

Es2Cleanup: Cleanup ES2

***LOG Q05=> DpHalt, DPStop ( 29876) [dpxxdisp.c   11581]

DpHalt: Good Bye .....

[Former Member]

Hello Reagan,

I have faced the exactly similar issue while performing SOLMAN upgrade in low capacity Windows Box. I found em/initial_size_MB is taking too much memory in Shadow Instance hence reduced it to a value multiple of 4096 i.e 8192 MB and restarted Shadow Instance.

It worked.

Many thanks for providing the solution.

Best Regards,

Sourav Brahma