Connection to sap not possible, all dialog work processes in running status

[Former Member]

Dear SAP gurus,

We have problem with two of our productive systems (different servers) . This morning there was two times problem with connection to the system, via dpmon i saw there are all dialog work proccesses in running status, no connection was possible. After restart is was ok, but the problem

appeares again and again after some time. Please help me with this issue, it is very critical.

here is the dev_disp log :

r4201:ispadm 60> more dev_disp

---------------------------------------------------
trc file: "dev_disp", trc level: 1, release: "46D"
---------------------------------------------------

Tue Jul 1 10:56:32 2008
relno 4640
patchlevel 0
patchno 2318
intno 0
pid 3351042

***LOG Q00=> DpSapEnvInit, DPStart (00 3351042) [dpxxdisp.c 921]
MtxInit: -2 0 0
DpIPCInit: start server >r4201i1_ISP_00 <
MBUF state OFF
MBUF opmode USE
EmInit: MmSetImplementation( 2 ).
EM/TOTAL_SIZE_MB = 28000

Tue Jul 1 10:56:34 2008
***LOG Q0K=> DpMsAttach, mscon ( r4201i1) [dpxxdisp.c 8384]
use SAPLOCALHOST=<r4201i1> as internal hostname
MBUF set hwid_state to HWID_PENDING
DpStartStopMsg: send start message (myname is >r4201i1_ISP_00 <)
DpStartStopMsg: start msg sent
CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.
CCMS: start to initalize 3.X shared alert area (first segment).
DpMsgAdmin: Set release to 4640, patchlevel 0
MBUF state PREPARED
MBUF component UP
MBUF set hwid_state to SAP_O_K
(F1785294464 )
DpMsgAdmin: Set patchno for this platform to 2318
Release check o.K.

Tue Jul 1 10:56:37 2008
MBUF state ACTIVE

Tue Jul 1 10:58:51 2008
SoftCancel request for T188 M0 received from REMOTE_TERMINAL

Tue Jul 1 11:10:02 2008
Network error of client T142, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

Tue Jul 1 11:10:57 2008
*** Warning => NiAddrToHost took 55 seconds
Client address of T142 is 10.42.197.30(10.42.197.30)
***LOG Q04=> DpRTmPrep, NiBufReceive (159 GAJDOSECHOVA142 P10063 )
[dpxxdisp.c 8106]
RM-T142, U159, 400 GAJDOSECHOVA, P10063, 11:09:34, M1, W3, ZCSR, 3/2
*** ERROR => DpRqNoWpHandle: req for DISCONNECTED tm [dpxxdisp.c 3077]SoftCancel request for T368 M0 received from REMOTE_TERMINAL
SoftCancel request for T244 M0 received from REMOTE_TERMINAL
SoftCancel request for T113 M0 received from REMOTE_TERMINAL
Network error of client T109, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

Tue Jul 1 11:11:52 2008
*** Warning => NiAddrToHost took 55 seconds
Client address of T109 is 10.42.197.31(10.42.197.31)
***LOG Q04=> DpRTmPrep, NiBufReceive (120 VELICKA 109 P10240 )
[dpxxdisp.c 8106]
RM-T109, U120, 400 VELICKA, P10240, 11:08:35, M0, W2, CIC0, 4/1
Network error of client T271, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

Tue Jul 1 11:12:47 2008
*** Warning => NiAddrToHost took 55 seconds
Client address of T271 is 10.42.197.28(10.42.197.28)
***LOG Q04=> DpRTmPrep, NiBufReceive (540 FOJTIKOVA 271 P10260 )
[dpxxdisp.c 8106]
RM-T271, U540, 400 FOJTIKOVA, P10260, 11:11:52, M0, W4, CIC0, 2/1
Network error of client T150, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

Tue Jul 1 11:13:42 2008
*** Warning => NiAddrToHost took 55 seconds
Client address of T150 is 10.42.197.155(10.42.197.155)
***LOG Q04=> DpRTmPrep, NiBufReceive (162 INCEDI 150 P1855 )
[dpxxdisp.c 8106]
RM-T150, U162, 400 INCEDI, P1855, 11:08:40, M0, W15, CIC0, 5/2
*** ERROR => DpTmSend: NiBufSend failed(rc=-6)->disconnect tm: 332
[dpxxdisp.c 9048]
RM-T332, U1395, 400 6KVAH, p2501, 11:10:57, M0, W44, CIC0, 2/2
*** ERROR => DpTmSend: NiBufSend failed(rc=-6)->disconnect tm: 269
[dpxxdisp.c 9048]
RM-T269, U537, 400 5SIMS, p6453, 11:11:52, M1, W5, EL27, 4/2
*** ERROR => DpTmSend: NiBufSend failed(rc=-6)->disconnect tm: 252
[dpxxdisp.c 9048]
RM-T252, U429, 400 3ZAMT, N10184, 11:11:52, M0, W27, EEDM, 2/1
*** ERROR => DpRqNoWpHandle: req for DISCONNECTED tm [dpxxdisp.c 3077]*** ERROR => DpRqNoWpHandle: req for DISCONNECTED tm [dpxxdisp.c 3077]*** ERROR => DpRqCheck: mode 0 in status CANCEL/HAND_SHAKE
[dpxxdisp.c 4841]
***LOG Q0G=> DpRqBadHandle, bad_req ( DIA) [dpxxdisp.c 3880]
*** ERROR => BAD REQUEST - Reason: DpRqCheck failed (line 4311):
[dpxxdisp.c 3882]
-IN-- sender_id REMOTE_TERMINAL tid 150 wp_ca_blk 26 wp_id -1
-IN-- action SEND_TO_WP uid 162 appc_ca_blk -1 type
DIA
-IN-- new_stat NO_CHANGE mode 0 len 91 rq_id
4115
*** ERROR => DpRqCheck: mode 0 in status CANCEL/HAND_SHAKE
[dpxxdisp.c 4841]
***LOG Q0G=> DpRqBadHandle, bad_req ( DIA) [dpxxdisp.c 3880]
*** ERROR => BAD REQUEST - Reason: DpRqCheck failed (line 4311):
[dpxxdisp.c 3882]
-IN-- sender_id REMOTE_TERMINAL tid 109 wp_ca_blk 5 wp_id -1
-IN-- action SEND_TO_WP uid 120 appc_ca_blk -1 type
DIA
-IN-- new_stat NO_CHANGE mode 0 len 376 rq_id
4133
*** ERROR => DpRqCheck: mode 0 in status CANCEL/HAND_SHAKE
[dpxxdisp.c 4841]
***LOG Q0G=> DpRqBadHandle, bad_req ( DIA) [dpxxdisp.c 3880]
*** ERROR => BAD REQUEST - Reason: DpRqCheck failed (line 4311):
[dpxxdisp.c 3882]
-IN-- sender_id REMOTE_TERMINAL tid 269 wp_ca_blk 159 wp_id -1
-IN-- action SEND_TO_WP uid 537 appc_ca_blk -1 type
DIA
-IN-- new_stat NO_CHANGE mode 0 len 219 rq_id
4138
*** ERROR => DpHdlSoftCancel: terminal has token [dpxxdisp.c 11982]
RM-T368, U1988, 400 JJANISOV, P1761, 11:10:00, M0, W26, , 2/0
Network error of client T329, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

Tue Jul 1 11:14:02 2008
*** Warning => NiAddrToHost took 20 seconds
Client address of T329 is 10.42.101.49(10.42.101.49)
***LOG Q04=> DpRTmPrep, NiBufReceive (1418 6BLUV 329 n2153 )
[dpxxdisp.c 8106]
RM-T329, U1418, 400 6BLUV, n2153, 11:09:17, M0, W10, ZPPL, 2/1
Network error of client T310, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

Tue Jul 1 11:14:57 2008
*** Warning => NiAddrToHost took 55 seconds
Client address of T310 is 10.42.95.65(10.42.95.65)
***LOG Q04=> DpRTmPrep, NiBufReceive (1154 5RICI 310 P6342 )
[dpxxdisp.c 8106]
RM-T310, U1154, 400 5RICI, P6342, 11:10:57, M0, W50, EL28, 2/1
Network error of client T287, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

Tue Jul 1 11:15:17 2008
*** Warning => NiAddrToHost took 20 seconds
Client address of T287 is 10.42.53.89(p10172.smp.rwegroup.cz)
***LOG Q04=> DpRTmPrep, NiBufReceive (974 SFOLTYNO 287 P10172 )
[dpxxdisp.c 8106]
RM-T287, U974, 400 SFOLTYNO, P10172, 11:08:43, M0, W1, CIC0, 2/1
Network error of client T267, NiBufReceive (-6: NIECONN_BROKEN),
dp_tm_status=3

[Former Member]

There was problem with DNS, dont know exactly what happened but our system admins solved the problem and it is ok now.

JM

[former_member829550]

hi josef,

these have been observed in ur logs:::

Tue Jul 1 11:10:02 2008

Network error of client T142, NiBufReceive (-6: NIECONN_BROKEN),

dp_tm_status=3

Tue Jul 1 11:12:47 2008

• Warning => NiAddrToHost took 55 seconds*

Client address of T271 is 10.42.197.28(10.42.197.28)

***LOG Q04=> DpRTmPrep, NiBufReceive (540 FOJTIKOVA 271 P10260 )

[dpxxdisp.c 8106]

RM-T271, U540, 400 FOJTIKOVA, P10260, 11:11:52, M0, W4, CIC0, 2/1

*Network error of client T150, NiBufReceive (-6: NIECONNBROKEN),*_

*dptm_status=3*_

Tue Jul 1 11:14:02 2008

• Warning => NiAddrToHost took 20 seconds*

Client address of T329 is 10.42.101.49(10.42.101.49)

***LOG Q04=> DpRTmPrep, NiBufReceive (1418 6BLUV 329 n2153 )

[dpxxdisp.c 8106]

RM-T329, U1418, 400 6BLUV, n2153, 11:09:17, M0, W10, ZPPL, 2/1

*Network error of client T310, NiBufReceive (-6: NIECONNBROKEN),*_

*dptm_status=3*_

please check ur network settings are correctly configured or not.

ask ur network admin to check them.

18 DIA 3007018 Stop GUI yes 0 0 88SAPLSGUI 400

26 DIA 258416 Stop GUI yes 0 0 87SAPLOLEA 400

48 DIA 345276 Stop GUI yes 0 0 30SAPLEFND 400

toooo many dialog processes are in stopped status.

please check dev_w18, dev_26, dev_48 in st11 tcode.

hope u will get some hints about the server....

regards,

bhupesh

[Former Member]

Hi,

I have answer from sap

Dear Mr. Macinka,

thank you for the detailed problem description.
As dicussed via phone, the scenarion, that the same problem occures
on two systems and network related errors
RM-T310, U1154, 400 5RICI, P6342, 11:10:57, M0, W50, EL28, 2/1
Network error of client T287, NiBufReceive (-6: NIECONN_BROKEN),
are recorded in the dev_disp trace are signs of a problem with the net
and a malfunction on the network area. There is no problem with the SAP
system. Restarting the server usually solve this problem. Please make a
NIPING test on the server in case this situation occurs again. NIPING
tool is described in the following note:

500235 Network Diagnosis with NIPING

and the second answer

Dear Mr. Macinka,

I have checked the traces again and I can see the warnings:
*** Warning => NiAddrToHost took 55 seconds

If the network is not stable, there are many disconnects.

This means that the dispatcher blocks about 55 seconds when it tries to
get the hostname of a client (SAPGui) which was disconnected.
You find some further information about that in the
note 674630 *** Warning => NiAddrToHost took 21 seconds

Please check the DNS configuration, in the meantime you can switch off
the reverse name lookup with the parameter
rdisp/reverse_name_lookup = 0

Futher issue is described in the note
1131092 Enqueue Work Prozesse in State "run"

This scenario is also applicable for other type of WPs, then enqueue,
in this case for dia wps.
For kernel 46D there is no solution, only manually cancelation of the
wps.

In your case, if you solve the unstable network connection and the reverse name lookup problem, the hang situation will not happen.

Please use niping to check the stable network connection.

So our network admins are working on it now, but sollution is not known yet, seems like big load on DNS server...

[Former Member]

here is the dpmon output when the problems occured ->

Workprocess Table (long) Tue Jul 1 10:43:01 2008========================

No Ty. Pid Status Cause Start Err Sem CPU Time Program Cl
User Action Table
-----------------------------------------------------------------------------------------------------------------------
0 DIA 3416750 Run yes 0 0 86SAPLSNR3 400
IMIKULAS
1 DIA 4538818 Run yes 0 0 0
2 DIA 3474660 Run yes 0 0 0
3 DIA 3929066 Run yes 0 0 132SAPLEVDB 400
6PETK Sequential Read ERCH
4 DIA 3268678 Run yes 0 0 0
5 DIA 3404518 Run yes 0 0 0
6 DIA 139764 Run yes 0 0 30SAPLE10G 400
OPENMIND
7 DIA 3449954 Run yes 0 0 0
8 DIA 3904336 Run yes 0 0 0
9 DIA 3248272 Run yes 0 0 0
10 DIA 3383962 Run yes 0 0 0
11 DIA 422200 Run yes 0 0 0
12 DIA 357308 Run yes 0 0 0
13 DIA 3227708 Run yes 0 0 0
14 DIA 332038 Run yes 0 0 4011SAPLERCH 400
DPATROVSKA1 Sequential Read ERCH
15 DIA 3044598 Run yes 0 0 0
16 DIA 3912454 Run yes 0 0 0
17 DIA 3223644 Run yes 0 0 0
18 DIA 3007018 Stop GUI yes 0 0 88SAPLSGUI 400
HEINDL
19 DIA 311766 Run yes 0 0 30SAPLBUPA 400
VSOKOLIK
20 DIA 3429582 Run yes 0 0 0
21 DIA 3908454 Run yes 0 0 10 400
5STEL
22 DIA 2977948 Run yes 0 0 0
23 DIA 291262 Run yes 0 0 0
24 DIA 516332 Run yes 0 0 0
25 DIA 3875678 Run yes 0 0 0
26 DIA 258416 Stop GUI yes 0 0 87SAPLOLEA 400
SICHANOVA
27 DIA 382010 Run yes 0 0 0
28 DIA 454862 Run yes 0 0 0
29 DIA 3855260 Run yes 0 0 30SAPLSNR3 400
6KRAN
30 DIA 184820 Run yes 0 0 0
31 DIA 402100 Run yes 0 0 0
32 DIA 450758 Run yes 0 0 0
33 DIA 389748 Run yes 0 0 10 400
3SOBB
34 DIA 357580 Run yes 0 0 0
35 DIA 3355606 Run yes 0 0 0
36 DIA 438464 Run yes 0 0 10 400
2KREU
37 DIA 4510046 Run yes 0 0 0
38 DIA 381516 Run yes 0 0 0
39 DIA 353468 Run yes 0 0 0
40 DIA 426162 Run yes 0 0 0
41 DIA 4501866 Run yes 0 0 30SAPLFKLO 400
4MENJ
42 DIA 401560 Run yes 0 0 0
43 DIA 349368 Run yes 0 0 10 400
INCEDI
44 DIA 324448 Run yes 0 0 0
45 DIA 369188 Run yes 0 0 0
46 DIA 299778 Run yes 0 0 0
47 DIA 365090 Stop GUI yes 0 0 30SAPLCOIH 400
3STRA
48 DIA 345276 Stop GUI yes 0 0 30SAPLEFND 400
5TETD
49 DIA 291798 Run yes 0 0 0
50 DIA 360990 Run yes 0 0 30SAPLE10E 400
JKONECNA
51 DIA 4493568 Run yes 0 0 0
52 DIA 271268 Run yes 0 0 0
53 DIA 348684 Run yes 0 0 0
54 DIA 267166 Run yes 0 0 0
55 DIA 340538 Run yes 0 0 0
56 DIA 4485546 Run yes 0 0 0
57 DIA 304238 Run yes 0 0 0
58 DIA 263068 Run yes 0 0 0
59 DIA 258960 Run yes 0 0 0
60 DIA 4477390 Run yes 0 0 0
61 DIA 332540 Run yes 0 0 0
62 DIA 397462 Run yes 0 0 0
63 DIA 254720 Run yes 0 0 0
64 DIA 4464982 Run yes 0 0 0
65 UPD 328348 Wait yes 0 0 0
66 UPD 242544 Wait yes 0 0 0
67 UPD 300168 Wait yes 0 0 0
68 UPD 426416 Wait yes 0 0 0
69 UPD 193486 Wait yes 0 0 0
70 ENQ 414102 Run yes 0 0 0
71 BTC 283638 Run yes 0 8 2540SAPLFKB2 400
0KRAK
72 BTC 381302 Run yes 0 0 1419SAPLES21 400
MSTRUHOVA1
73 BTC 275432 Run yes 0 0 10 400
SYSCUA
74 BTC 377200 Run yes 0 0 4012SAPLERCH 400
DPATROVSKA1 Sequential Read ERCH
75 BTC 283728 Run yes 0 0 3515SAPLSNR3 400
VYLETALM
76 BTC 373102 Wait yes 0 0 0
77 BTC 279630 Wait yes 0 0 0
78 BTC 381052 Wait yes 0 0 0
79 BTC 369002 Wait yes 0 0 0
80 BTC 364892 Wait yes 0 0 0
81 BTC 271464 Wait yes 0 0 0
82 BTC 360796 Wait yes 0 0 0
83 BTC 376916 Wait yes 0 0 0
84 BTC 372814 Wait yes 0 0 0
85 BTC 364604 Wait yes 0 0 0
86 BTC 263210 Wait yes 0 0 0
87 BTC 356404 Wait yes 0 0 0
88 BTC 348192 Wait yes 0 0 0
89 BTC 255012 Wait yes 0 0 0
90 BTC 331994 Wait yes 0 0 0
91 BTC 311936 Wait yes 0 0 0
92 BTC 299638 Wait yes 0 0 0
93 SPO 230612 Wait yes 0 0 0
94 SPO 295534 Wait yes 0 0 0
95 SPO 217800 Wait yes 0 0 0
96 SPO 222402 Wait yes 0 0 0
97 UP2 193112 Run yes 0 0 10 400
ANDRESIKOVA
98 UP2 201878 Wait yes 0 0 0
99 UP2 172842 Wait yes 0 0 0
100 UP2 328568 Wait yes 0 0 0


s - stop workprocess
k - kill workprocess (with core)
r - enable restart flag (only possible in wp-status "ended")
q - quit
m - menue