Dispatcher Stopped - SAP Stops

[joo_migueldimas]

Hello gurus,

SAP System: ECC 5.0

DB: SQL Server 2000

Operation System: Windows Server 2003 X32

We had a problem a few days ago that resulted in stopping the SAP system... We still do not understand why this happens, and the biggest problem that was not the first time ocorrur!!

I checked the log files dev_disp and dev_w0, you can see them below:

DEV_DISP:

...
Wed Jul 14 18:11:59 2010
NiPWrite: ENOBUFS: buffers reduced to 4/212 (errNo=10055; cnt=1; len=212; hdl 15 / socket 1248)
NiPWrite: ENOBUFS: buffers reduced to 4/212 (errNo=10055; cnt=2; len=212; hdl 15 / socket 1248)
NiPWrite: ENOBUFS: buffers reduced to 4/212 (errNo=10055; cnt=3; len=212; hdl 15 / socket 1248)
NiPWrite: ENOBUFS: buffers reduced to 4/128 (errNo=10055; cnt=4; len=212; hdl 15 / socket 1248)
NiPWrite: ENOBUFS: buffers reduced to 4/32 (errNo=10055; cnt=5; len=212; hdl 15 / socket 1248)
*** ERROR => DpTmSend: NiBufSend failed(rc=-6)->disconnect tm: 27 [dpxxdisp.c   10981]
RM-T27, U4735, 300     FORMACAO, LUX-DanielAbreu, 18:11:59, M0, W0, SESS, 2/0
*** ERROR => DpRqCheck: T27 in stat TM_SLOT_FREE [dpxxdisp.c   5805]
***LOG Q0G=> DpRqBadHandle, bad_req ( DIA) [dpxxdisp.c   4621]
*** ERROR => BAD REQUEST - Reason: DpRqCheck failed (line 5195): [dpxxdisp.c   4623]
-IN-- sender_id DISPATCHER        tid  27    wp_ca_blk   -1      wp_id -1
-IN-- action    SEND_TO_WP        uid  4735   appc_ca_blk -1      type  DIA  
-IN-- new_stat  NO_CHANGE         mode 0     len         -1      rq_id 39366
-IN-- req_info  LOGOFF CANCELMODE 

Wed Jul 14 18:12:14 2010
*** ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c      647]

Wed Jul 14 18:12:32 2010
NiPWrite: ENOBUFS: buffers reduced to 4/610 (errNo=10055; cnt=1; len=610; hdl 10 / socket 1308)
NiPWrite: ENOBUFS: buffers reduced to 4/610 (errNo=10055; cnt=2; len=610; hdl 10 / socket 1308)
NiPWrite: ENOBUFS: buffers reduced to 4/512 (errNo=10055; cnt=3; len=610; hdl 10 / socket 1308)
NiPWrite: ENOBUFS: buffers reduced to 4/128 (errNo=10055; cnt=4; len=610; hdl 10 / socket 1308)
NiPWrite: ENOBUFS: buffers reduced to 4/32 (errNo=10055; cnt=5; len=610; hdl 10 / socket 1308)
*** ERROR => DpTmSend: NiBufSend failed(rc=-6)->disconnect tm: 30 [dpxxdisp.c   10981]
RM-T30, U3708, 300      SMENDES, PRT-SILVIA, 18:12:32, M1, W1, ZFI0, 2/2

Wed Jul 14 18:12:33 2010
*** ERROR => DpRqCheck: T30 in stat TM_SLOT_FREE [dpxxdisp.c   5805]
***LOG Q0G=> DpRqBadHandle, bad_req ( DIA) [dpxxdisp.c   4621]
*** ERROR => BAD REQUEST - Reason: DpRqCheck failed (line 5195): [dpxxdisp.c   4623]
-IN-- sender_id DISPATCHER        tid  30    wp_ca_blk   -1      wp_id -1
-IN-- action    SEND_TO_WP        uid  3708   appc_ca_blk -1      type  DIA  
-IN-- new_stat  NO_CHANGE         mode 1     len         -1      rq_id 39386
-IN-- req_info  LOGOFF CANCELMODE 

Wed Jul 14 18:12:34 2010
*** ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c      647]

Wed Jul 14 18:12:35 2010
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
***LOG Q0N=> DpRqNoWpHandle, MsSndName () [dpxxdisp.c   4161]
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
***LOG Q0N=> DpRqNoWpHandle, MsSndName () [dpxxdisp.c   4161]

Wed Jul 14 18:13:14 2010
*** ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c      647]

Wed Jul 14 18:13:34 2010
*** ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c      647]

Wed Jul 14 18:13:35 2010
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
***LOG Q0N=> DpRqNoWpHandle, MsSndName () [dpxxdisp.c   4161]
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
***LOG Q0N=> DpRqNoWpHandle, MsSndName () [dpxxdisp.c   4161]

Wed Jul 14 18:13:54 2010
*** ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c      647]


Wed Jul 14 18:15:14 2010
*** ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c      647]

Wed Jul 14 18:15:54 2010
*** ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c      647]

Wed Jul 14 18:16:09 2010
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
***LOG Q0N=> DpRqNoWpHandle, MsSndName () [dpxxdisp.c   4161]
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
***LOG Q0N=> DpRqNoWpHandle, MsSndAdmin () [dpxxdisp.c   4245]
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
***LOG Q0N=> DpRqNoWpHandle, MsSndName () [dpxxdisp.c   4161]
*** ERROR => MsISnd2: not_attached [msxxi.c      1334]
...

[joo_migueldimas]

Este post é referente à situação da Bicafé, em que o sistema produtivo (PRD) pára abruptamente sem se perceber a razão. Vou encerrar este post...

Thank you for your help!

[former_member4224]

The dispatcher lost connection with the message server.

Wed Jul 14 18:15:54 2010

• ERROR => MsIAttachEx: NiBufConnect to sapprodt/sapmsPRT failed (rc=NIECONN_REFUSED) [msxxi.c 647]

Wed Jul 14 18:16:09 2010

• ERROR => MsISnd2: not_attached [msxxi.c 1334]

***LOG Q0N=> DpRqNoWpHandle, MsSndName () [dpxxdisp.c 4161]

Usually this is a netowrk issue. You should check the connection between this instance and the server sapprodt.

If hadn't network issue you should check the message server trace file (dev_ms) at the same timestamp.

Maybe tehre some thing happens with the message server.

cheers, Clebio

[joo_migueldimas]

Hello Clébio Dossa,

Ok Clébio!... We will analize our own network... ! But regarding that log dev_ms I check it now but unfortunately the data content in there (dev_ms and dev_ms old) show only lines/events from 18 July 2010 and not from the day when this problem (14 July 2010)

Even so I will put here this two log files (dev_ms & dev_ms.old):

dev_ms :

---------------------------------------------------
trc file: "dev_ms", trc level: 1, release: "640"
---------------------------------------------------

[Thr 2084] Sun Jul 18 10:15:31 2010
[Thr 2084] MsSSetTrcLog: trc logging not active
[Thr 2084] systemid   560 (PC with Windows NT)
[Thr 2084] relno      6400
[Thr 2084] patchlevel 0
[Thr 2084] patchno    210
[Thr 2084] intno      20020600
[Thr 2084] pid        4664
[Thr 2084] ***LOG Q01=> MsSInit, MSStart (Msg Server 1 4664) [msxxserv.c   1689]
[Thr 2084] MsHttpInit: full qualified hostname = sapprodt
[Thr 2084] HTTP logging is switch off
[Thr 2084] set HTTP state to LISTEN
[Thr 2084] *** I listen to port sapmsPRT (3600) ***
[Thr 2084] *** HTTP port 8100 state LISTEN ***
[Thr 2084] CUSTOMER KEY: >M1271086647<

dev_ms.old :

---------------------------------------------------
trc file: "dev_ms", trc level: 1, release: "640"
---------------------------------------------------

[Thr 4112] Sun Jul 18 10:00:36 2010
[Thr 4112] MsSSetTrcLog: trc logging not active
[Thr 4112] systemid   560 (PC with Windows NT)
[Thr 4112] relno      6400
[Thr 4112] patchlevel 0
[Thr 4112] patchno    210
[Thr 4112] intno      20020600
[Thr 4112] pid        4916
[Thr 4112] ***LOG Q01=> MsSInit, MSStart (Msg Server 1 4916) [msxxserv.c   1689]
[Thr 4112] MsHttpInit: full qualified hostname = sapprodt
[Thr 4112] HTTP logging is switch off
[Thr 4112] set HTTP state to LISTEN
[Thr 4112] *** I listen to port sapmsPRT (3600) ***
[Thr 4112] *** HTTP port 8100 state LISTEN ***
[Thr 4112] CUSTOMER KEY: >M1271086647<

[Thr 544] Sun Jul 18 10:07:24 2010
[Thr 544] MsSExit: received SIGINT (2)
[Thr 544] ***LOG Q02=> MsSHalt, MSStop (Msg Server 4916) [msxxserv.c   5420]

Any thing seems to you strange in above log files?!

Kind regards,

João Dimas - Portugal

[former_member4224]

Hi João,

Everything is normal with the logs that you have posted..

But the timestamp from the logs of dispatcher (showed before) as from on Wed Jul 14 18:15:54 2010.

And the actual logs from message server are only after Sun Jul 18 10:00:36 2010.

If you are facing this issue constantly, the better option is configure the NIPING tool to analyse the network behavior during the issue.

Review the example 2 in note 500235. You shouls start NIPING as server in the message server host, and as client in the instance that is facing the problem. If the problem occurs again during the niping test, it will broken and you do find a error in the log. This means that you are facing some problem of connectivity between the message server and the instance.

NIPING is only to test the network layer between the server, there is no relation with SAP system.

If you face the same error message in the dispatcher, but the NIPING still working, the problem is other, and in this case you should analyse the SAP system.

cheers, Clebio

[joo_migueldimas]

DEV_W0:

 M  *****************************************************************************
M  *  LOCATION    SAP-Server sapprodt_PRT_00 on host sapprodt (wp 0)
M  *  ERROR       ThRqOutCheck: bad request
M  *
M  *  TIME        Wed Jul 14 18:12:01 2010
M  *  RELEASE     640
M  *  COMPONENT   Taskhandler
M  *  VERSION     1
M  *  RC          4
M  *  MODULE      thxxhead.c
M  *  LINE        9881
M  *  COUNTER     17
M  *****************************************************************************
M  Entering ThSetStatError
M  ThCallHooks: call hook >ASTAT-collect commit handling< for event BEFORE_ROLLBACK
M  ThCallHooks: call hook >dyKeyTableReset< for event BEFORE_ROLLBACK
M  ThCallHooks: call hook >SpoolHandleHook< for event BEFORE_ROLLBACK
M  ThCallHooks: call hook >ThVBICmRbHook< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >ThNoClearPrevErr< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >rsts_after_rollback< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >ThNoResetMutex< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >SpoolHandleHook< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >ab_SosShmBeforeCommit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ASTAT-collect commit handling< for event BEFORE_COMMIT
M  ThCallHooks: call hook >rsts_before_commit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ThVBICmRbHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >dyKeyTableReset< for event AFTER_COMMIT
M  ThCallHooks: call hook >rsts_after_commit< for event AFTER_COMMIT
M  ThCallHooks: call hook >ThNoClearPrevErr< for event AFTER_COMMIT
M  ThCallHooks: call hook >SpoolHandleHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >rsts_storage_free< for event AFTER_STORAGE_FREE
M  ThCallHooks: call hook >db_x_short_mem_sync_point< for event AFTER_STORAGE_FREE
M  ThIErrHandle: Entering ThReplyToMsg
M  ThIErrHandle: Entering ThErrHdlUser
M  ThCallHooks: call hook >ASTAT-collect commit handling< for event BEFORE_ROLLBACK
M  ThCallHooks: call hook >dyKeyTableReset< for event BEFORE_ROLLBACK
M  ThCallHooks: call hook >SpoolHandleHook< for event BEFORE_ROLLBACK
M  ThCallHooks: call hook >ThVBICmRbHook< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >ThNoClearPrevErr< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >rsts_after_rollback< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >ThNoResetMutex< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >SpoolHandleHook< for event AFTER_ROLLBACK
M  ThCallHooks: call hook >ab_SosShmBeforeCommit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ASTAT-collect commit handling< for event BEFORE_COMMIT
M  ThCallHooks: call hook >rsts_before_commit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ThVBICmRbHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >dyKeyTableReset< for event AFTER_COMMIT
M  ThCallHooks: call hook >rsts_after_commit< for event AFTER_COMMIT
M  ThCallHooks: call hook >ThNoClearPrevErr< for event AFTER_COMMIT
M  ThCallHooks: call hook >SpoolHandleHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >rsts_storage_free< for event AFTER_STORAGE_FREE
M  ThCallHooks: call hook >db_x_short_mem_sync_point< for event AFTER_STORAGE_FREE
M  ThCallHooks: call hook >ab_SosShmBeforeCommit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ASTAT-collect commit handling< for event BEFORE_COMMIT
M  ThCallHooks: call hook >rsts_before_commit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ThVBICmRbHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >dyKeyTableReset< for event AFTER_COMMIT
M  ThCallHooks: call hook >rsts_after_commit< for event AFTER_COMMIT
M  ThCallHooks: call hook >ThNoClearPrevErr< for event AFTER_COMMIT
M  ThCallHooks: call hook >SpoolHandleHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >ab_SosShmBeforeCommit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ASTAT-collect commit handling< for event BEFORE_COMMIT
M  ThCallHooks: call hook >rsts_before_commit< for event BEFORE_COMMIT
M  ThCallHooks: call hook >ThVBICmRbHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >dyKeyTableReset< for event AFTER_COMMIT
M  ThCallHooks: call hook >rsts_after_commit< for event AFTER_COMMIT
M  ThCallHooks: call hook >ThNoClearPrevErr< for event AFTER_COMMIT
M  ThCallHooks: call hook >SpoolHandleHook< for event AFTER_COMMIT
M  ThCallHooks: call hook >ITSP: EndOfSession< for event BEFORE_SESSION_CANCEL
M  ThCallHooks: call hook >RtmClearSession< for event BEFORE_SESSION_CANCEL
M  ThCallHooks: call hook >HttpClearSession< for event BEFORE_SESSION_CANCEL
M  ThCallHooks: call hook >SpoolHandleHook< for event BEFORE_SESSION_CANCEL
M  ***LOG R47=> ThResFree, delete (001004) [thxxmode.c   1310]
M  ThCallHooks: call hook >SAP-Trace buffer write< for event BEFORE_TASK_SWITCH
M  ThCallHooks: call hook >ImcResetPtr< for event BEFORE_TASK_SWITCH
M  ThCallHooks: call hook >DbgBeforeTaskSwitch< for event BEFORE_TASK_SWITCH
M  ThCallHooks: call hook >HttpResetPtr< for event BEFORE_TASK_SWITCH