Dispatcher not coming UP

[Former Member]

Hi Team,

One of our application server dispatcher is not coming up after i have changed the login/no_automatic_user_sapstar as we need to login in one client. Enclosed the dispathcher log below.

-

---

trc file: "dev_disp", trc level: 1, release: "640"

-

---

Mon May 04 14:45:35 2009

kernel runs with dp version 138000(ext=102000) (@(#) DPLIB-INT-VERSION-138000-UC)

length of sys_adm_ext is 524 bytes

sysno 01

sid QEC

systemid 560 (PC with Windows NT)

relno 6400

patchlevel 0

patchno 247

intno 20020600

make: multithreaded, Unicode

pid 4068

***LOG Q00=> DpSapEnvInit, DPStart (01 4068) [dpxxdisp.c 1110]

shared lib "dw_xml.dll" version 247 successfully loaded

shared lib "dw_xtc.dll" version 247 successfully loaded

shared lib "dw_stl.dll" version 247 successfully loaded

shared lib "dw_gui.dll" version 247 successfully loaded

shared lib "dw_mdm.dll" version 247 successfully loaded

Mon May 04 14:45:49 2009

• WARNING => DpNetCheck: NiAddrToHost(1.0.0.0) took 14 seconds

***LOG GZZ=> 1 possible network problems detected - check tracefile and adjust the DNS

settings [dpxxtool2.c 3882]

MtxInit: -2 0 0

DpSysAdmExtInit: ABAP is active

DpIPCInit2: start server >cambsvr13_QEC_01 <

DpShMCreate: sizeof(wp_adm) 18880 (1452)

DpShMCreate: sizeof(tm_adm) 29558776 (14772)

DpShMCreate: sizeof(wp_ca_adm) 24000 (80)

DpShMCreate: sizeof(appc_ca_adm) 8000 (80)

DpShMCreate: sizeof(comm_adm) 1160000 (580)

DpShMCreate: sizeof(vmc_adm) 0 (424)

DpShMCreate: sizeof(wall_adm) (384056/329560/64/184)

• ERROR => DpShMCreate: ShmCreate SHM_DP_ADM_KEY [dpxxtool2.c 1586]

• ERROR => DpIPCInit2: DpShMCreate [dpxxtool2.c 435]

• DP_FATAL_ERROR => DpSapEnvInit: DpIPCInit2

• DISPATCHER EMERGENCY SHUTDOWN ***

NiWait: sleep (5000 msecs) ...

NiISelect: timeout 5000 ms

NiISelect: maximum fd=1

NiISelect: read-mask is NULL

NiISelect: write-mask is NULL

Mon May 04 14:45:54 2009

NiISelect: TIMEOUT occured (5000 ms)

DpHalt: shutdown server >cambsvr13_QEC_01 < (normal)

DpJ2eeDisableRestart

Switch off Shared memory profiling

ShmProtect( 57, 3 )

ShmProtect(SHM_PROFILE, SHM_PROT_RW

ShmProtect( 57, 1 )

ShmProtect(SHM_PROFILE, SHM_PROT_RD

not attached to the message server

***LOG Q05=> DpHalt, DPStop ( 4068) [dpxxdisp.c 9094]

Good Bye ....

ANy help would be appriciated.

Thanks & Regards,

Vinod

[Former Member · Accepted solution]

The shared memory is still locked by a process on your system. Have you tried restarting the server?

- J. Haynes

[Former Member]

HI All,

Thankq Very much for your time.

This problem has been fixed after we have changed the parameter EM/ADRESS_SAPCE_MB from 128 MB to 512 MB. But i dont know how it got changed to 128 MB.

Thanks again for all your support.

Best Regards,

Vinod

[Former Member]

Hi Friends,

i have got same error in my system i.e Dispatcher was stopped, although it gets resolved by rebooting the server & restrting the application.But is there any other way to resolve it so that such problem doesn't come in future.

We haven't changed any parmeters & value of parameter em/address_space_MB =256.

Thanks in advance.

Joy Garg

Edited by: Joy Garg on Jun 18, 2009 12:17 PM

[Former Member]

Hey Vinod

Please check the page file assigned size and used size, one or our 2008 servers has 24 gb assigned but was only allocating 8

Also ensure you have available disk space on the drives where swap is asigned

Regards

[Former Member]

Hi Vinod,

As told by all above, your system occupied the shared memory and unable to release.

For this best solution is to reboot the server and then start your SAP application.

Regards

Nick Loy

[Former Member]

Hi All,

I have restarted the server as you all recommended however still dispacther is not coming up. Dispatcher going end as soon as i start the instance.

Now i see below error in Dispacther trace. I haven't changed any memory parameters

ERROR => <EsNT> no free local page max= 16384 requested Blocks=170 [esnti.c 2347]

• ERROR => EgInit: EsCreateShared() failed. info='EG_GLOBAL_AREA', blocks=170, rc=11 [egxx.c 299]

• ERROR => DpEmInit: EmInit (1) [dpxxdisp.c 8474]

• ERROR => DpMemInit: DpEmInit (-1) [dpxxdisp.c 8413]

Please suggest....

Many Thanks,

Vinod

[Former Member]

You might want to delete and re-create the page file setup for this system. Are there any other applications running on this server that might use up the swap partition?

J. Haynes

[Former Member]

Vinod

Please send the dev_w0 log from that application server

but the best thing is to revert back the parameter and then start SAP;though I am sure the issue is not due to the parameter specified but better try it

but please send the dev_w0 logs

that will be useful

[Former Member]

Hi Rohit,

Thanks for your reply.

I have tried revert the parameter but i dont know why still dispatcher wasn't coming up. Please see enclosed work process trace (dev_w0). I have posted first and last 50 lines as its huge.

Please help us in indentifying the problem and let us know if you find any thing strange on developer trace.

-

---

trc file: "dev_w0", trc level: 1, release: "640"

-

---

*

*

I Mon May 04 12:44:47 2009

I MtxInit: 0 0 0

M SHM_PRES_BUF (addr: 0CAD0040, size: 6448128)

M SHM_ROLL_AREA (addr: 29440040, size: 268435456)

M SHM_PAGING_AREA (addr: 10560040, size: 134217728)

M SHM_ROLL_ADM (addr: 0D100040, size: 2678942)

M SHM_PAGING_ADM (addr: 0D390040, size: 525344)

M ThCreateNoBuffer allocated 544152 bytes for 1000 entries at 0D420040

M ThCreateNoBuffer index size: 3000 elems

M ThCreateVBAdm allocated 12160 bytes (50 server) at 0D4B0040

X EmInit: MmSetImplementation( 2 ).

X <ES> client 0 initializing ....

X Using implementation flat

X ES initialized.

B

B Mon May 04 12:44:48 2009

B dbntab: NTAB buffers attached

B dbntab: Buffer FTAB(hash header) (addr: 0D550088, size: 536)

B dbntab: Buffer FTAB(anchor array) (addr: 0D5502A0, size: 167036)

B dbntab: Buffer FTAB(item array) (addr: 0D578F20, size: 751464)

B dbntab: Buffer FTAB(data area) (addr: 0D630688, size: 32079872)

B dbntab: Buffer IREC(hash header) (addr: 0F4D0088, size: 536)

B dbntab: Buffer IREC(anchor array) (addr: 0F4D02A0, size: 167036)

B dbntab: Buffer IREC(item array) (addr: 0F4F8F20, size: 187848)

B dbntab: Buffer IREC(data area) (addr: 0F526CE8, size: 6416384)

B dbntab: Buffer STAB(hash header) (addr: 0FB50088, size: 536)

B dbntab: Buffer STAB(anchor array) (addr: 0FB502A0, size: 167036)

B dbntab: Buffer STAB(item array) (addr: 0FB78F20, size: 187848)

B dbntab: Buffer STAB(data area) (addr: 0FBA6CE8, size: 3208192)

B dbntab: Buffer TTAB(hash header) (addr: 18570088, size: 1888)

B dbntab: Buffer TTAB(anchor array) (addr: 185707E8, size: 167036)

B dbntab: Buffer TTAB(item array) (addr: 18599468, size: 500976)

B dbntab: Buffer TTAB(data area) (addr: 18613958, size: 6095208)

B db_con_shm_ini: WP_ID = 0, WP_CNT = 13, CON_ID = -1

B dbstat: TABSTAT buffer attached (addr: 18BF9C50)

B dbtbxbuf: Buffer TABL (addr: 1A5300C8, size: 31024128, end: 1C2C64C8)

B dbtbxbuf: Buffer TABLP (addr: 1C2D00C8, size: 14336000, end: 1D07C0C8)

B dbexpbuf: Buffer EIBUF (addr: 1D0800D0, size: 4194304, end: 1D4800D0)

B dbexpbuf: Buffer ESM (addr: 1D4900D0, size: 4194304, end: 1D8900D0)

B dbexpbuf: Buffer CUA (addr: 1D8A00D0, size: 4096000, end: 1DC880D0)

B dbexpbuf: Buffer OTR (addr: 1DC900D0, size: 4194304, end: 1E0900D0)

B dbcalbuf: Buffer CALE (addr: 0FED0040, size: 500000, end: 0FF4A160)

M rdisp/reinitialize_code_page -> 0

M icm/accept_remote_trace_level -> 0

M rdisp/no_hooks_for_sqlbreak -> 0

M CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.

S *** init spool environment

S initialize debug system

T Stack direction is downwards.

T debug control: prepare exclude for printer trace

T new memory block 05BE0590

S spool kernel/ddic check: Ok

S using table TSP02FX for frontend printing

S 1 spool work process(es) found

S frontend print via spool service enabled

S printer list size is 150

S printer type list size is 50

S queue size (profile) = 300

S hostspool list size = 3000

S option list size is 30

S found processing queue enabled

S found spool memory service RSPO-RCLOCKS at 272D00A8

S doing lock recovery

S setting server cache root

S found spool memory service RSPO-SERVERCACHE at 272D0430

S using messages for server info

S size of spec char cache entry: 297028 bytes (timeout 100 sec)

S size of open spool request entry: 2236 bytes

S immediate print option for implicitely closed spool requests is disabled

A

A -PXA--

---

A PXA INITIALIZATION

A System page size: 4kb, total admin_size: 23136kb, dir_size: 23100kb.

A Attached to PXA (address 39470040, size 400000K)

A abap/pxa = shared protect gen_remote

A PXA INITIALIZATION FINISHED

A -PXA--

---

A

A ABAP ShmAdm attached (addr=68824000 leng=20955136 end=69C20000)

A >> Shm MMADM area (addr=68C844A8 leng=126848 end=68CA3428)

A >> Shm MMDAT area (addr=68CA4000 leng=16236544 end=69C20000)

A RFC Destination> destination cambsvr13_QEC_01 host cambsvr13 system QEC systnr 1 (cambsvr13_QEC_01)

A RFC Options> H=cambsvr13,S=01,d=2

C The IRow interface is supported by this OLEDB provider

A RFC FRFC> fallback activ but this is not a central instance.

A

A RFC rfc/signon_error_log = -1

A RFC rfc/dump_connection_info = 0

A RFC rfc/dump_client_info = 0

A RFC rfc/cp_convert/ignore_error = 1

A RFC rfc/cp_convert/conversion_char = 23

A RFC rfc/wan_compress/threshold = 251

A RFC rfc/recorder_pcs not set, use defaule value: 2

A RFC rfc/delta_trc_level not set, use default value: 0

A RFC rfc/no_uuid_check not set, use default value: 0

A RFC abap/SIGCHILD_default not set, use default value: 0

A RFC rfc/bc_ignore_thcmaccp_retcode not set, use default value: 0

A RFC Method> initialize RemObjDriver for ABAP Objects

M ThrCreateShObjects allocated 27042 bytes at 0FF90040

N SsfSapSecin: putenv(SECUDIR=E:\usr\sap\QEC\DVEBMGS01\sec): ok

N

M Mon May 04 12:44:59 2009

M hostaddrlist return 0

M hostaddrlist return 0

M hostaddrlist return 0

M hostaddrlist return 0

M

M Mon May 04 12:45:02 2009

M hostaddrlist return 0

M hostaddrlist return 0

X

X Mon May 04 12:45:03 2009

X **** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c 2220]*

X

X Mon May 04 12:45:09 2009

X *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c 2220]

X

X Mon May 04 12:45:17 2009

X *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c 2220]

X

X Mon May 04 12:45:41 2009

X *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c 2220]

X

X Mon May 04 12:45:58 2009

X *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c 2220]

X

X Mon May 04 12:46:08 2009

X *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c 2220]

X

X Mon May 04 12:46:18 2009

X *** ERROR => EmActiveData: Invalid Context Handle -1 [emxx.c 2220]

B

B Mon May 04 12:50:38 2009

B dbmyclu : info : my major identification is 3232235714, minor one 1.

B dbmyclu : info : Time Reference is 1.12.2001 01:00:00h GMT.

B dbmyclu : info : my uuid is 79A3C24B9E90C841A5C155EF934051E3.

B dbmyclu : info : current optimistic cluster level: 3

B dbmyclu : info : pessimistic reads set to 2.

B

B Mon May 04 13:04:59 2009

B create_con (con_name=+DBO+0000)

B New connection 1 created

B Connect to +DBO+0000 as with MSSQL_SERVER=cambsvr13 MSSQL_DBNAME=QEC OBJECT_SOURCE=qec

C Thread ID:5848

C dbmssslib.dll patch info

C patchlevel 0

C patchno 241

C patchcomment MSSQL: memory leak in DSQL access to (x)strings (2) (1225411)

C np:(local) connection used on CAMBSVR13

C Connected to db server : [CAMBSVR13] server_used : [np:(local)], dbname: QEC, dbuser: dbo

C pn_id:CAMBSVR13_QEC_QEC

B Connection 1 opened (DBSL handle 1)

B Wp Hdl ConName ConId ConState TX PRM RCT TIM MAX OPT Date Time DBHost

B 000 000 R/3 000000000 ACTIVE NO YES NO 000 255 255 20090504 124440 CAMBSVR13

B 000 001 +DBO+0000 000000001 ACTIVE NO NO NO 004 255 255 20090504 130459 CAMBSVR13

S

S Mon May 04 13:09:58 2009

S found spool memory service RSPO-ACTIONS at 272D81F8

B

B Mon May 04 13:50:00 2009

B create_con (con_name=+DBO+0000)

B New connection 2 created

B Connect to +DBO+0000 as with MSSQL_SERVER=cambsvr13 MSSQL_DBNAME=QEC OBJECT_SOURCE=qec

C Thread ID:5848

C dbmssslib.dll patch info

C patchlevel 0

C patchno 241

C patchcomment MSSQL: memory leak in DSQL access to (x)strings (2) (1225411)

C np:(local) connection used on CAMBSVR13

C Connected to db server : [CAMBSVR13] server_used : [np:(local)], dbname: QEC, dbuser: dbo

C pn_id:CAMBSVR13_QEC_QEC

B Connection 2 opened (DBSL handle 2)

B Wp Hdl ConName ConId ConState TX PRM RCT TIM MAX OPT Date Time DBHost

B 000 000 R/3 000000000 ACTIVE NO YES NO 000 255 255 20090504 124440 CAMBSVR13

B 000 001 +DBO+0000 000000013 ACTIVE YES NO NO 004 255 255 20090504 130459 CAMBSVR13

B 000 002 +DBO+0000 000000014 ACTIVE NO NO NO 004 255 255 20090504 135000 CAMBSVR13

M

M Mon May 04 13:52:37 2009

M ThIUsrDel: th_rollback_usrdelentry = 1

M

M Mon May 04 13:59:17 2009

M in_ThErrHandle: 1

M ThIErrHandle: new stat of W0 is WP_SHUTDOWN

M ThIErrHandle: I'm during shutdown

M PfStatDisconnect: disconnect statistics

M Entering ThSetStatError

C SQLBREAK: DBSL_CMD_SQLBREAK: CbOnCancel was not set. rc: 1

M ThIErrHandle: don't try rollback again

M ThShutDownServer: shutdown server

M PfStatIndInit: Initializing Index-Record

M PfWriteIntoFile: copied shared buf (0 bytes) to local buf

M PfTimeFactor: new value: 3600

M Entering ThReadDetachMode

M call ThrShutDown (1)...

B Disconnecting from ALL connections:

B Wp Hdl ConName ConId ConState TX PRM RCT TIM MAX OPT Date Time DBHost

B 000 000 R/3 000000000 INACTIVE NO YES NO 000 255 255 20090504 124440 CAMBSVR13

B 000 001 +DBO+0000 000000013 INACTIVE NO NO NO 003 255 255 20090504 130459 CAMBSVR13

B 000 002 +DBO+0000 000000014 INACTIVE NO NO NO 003 255 255 20090504 135000 CAMBSVR13

B Disconnected from connection 0

B Disconnected from connection 1

B Disconnected from connection 2

B statistics db_con_commit (com_total=1292, com_tx=159)

B statistics db_con_rollback (roll_total=20, roll_tx=0)

M ***LOG Q02=> wp_halt, WPStop (Workproc 0 5352) [dpnttool.c 357]

Many Thanks,

Vinod

Edited by: vinod kumar on May 4, 2009 4:36 PM

Edited by: vinod kumar on May 4, 2009 4:39 PM

[Former Member]

Hello

did you try restart of that application server,I guess that will resolve the issue

Rohit

[brian_walker]

• ERROR => DpShMCreate: ShmCreate SHM_DP_ADM_KEY http://dpxxtool2.c 1586

• ERROR => DpIPCInit2: DpShMCreate http://dpxxtool2.c 435

• DP_FATAL_ERROR => DpSapEnvInit: DpIPCInit2

• DISPATCHER EMERGENCY SHUTDOWN ***

This is a repercussion of SAP bringing SAP to the stability of the Windows platform. This happens to us periodically and it looks like SAP doesn't clean up a shared memory area when stopping SAP, so it cannot start back up again.

Reflect, repent, reboot and order will return.

Brian