All DIA work processes occupied by SAPLSENA

[yakcinar]

Hello All,

We have an ERP System with ERP 6.0 SAP_APPL 600 SP18 and SAP_BASIS 7.00 SP 22.

OS is Windows 2008 R2 and DB is SQL Server 2008.10.50.600.

Kernel version is 7.22 EXT UC SPL 126 (latest)

We have HA system infrastructure, ERS on both cluster nodes and extra 8 Dialog Instances. Totally 10 Instances.

Our kernel was 7.21 512 and last weekend we changed it to 7.21 SPL 712.

After this change we encountered occupation of SAPLSENA reports of DIA wps.

System got stucked after these SAPLSENA reports invasion.

We encountered this several times during week. And today unfortunately we had to restart system 3 times for this problem.

For solution we checked below notes and also applied 7.22 EXT UC SPL 126 kernel to our system.

1384191 - SM50: Many processes in status "holds ENQ"

1998968 - Update task hangs in program SAPLSENA

We searched through scn and sap pages but unfortunately could not find a solution.

Did anyboy have this kind of problem.

Can you help about the issue, please?

I attached a screenshot also.

Thanks and Regards,

Yuksel AKCINAR

[JPReyes · Accepted solution]

Hi Yuksel,

This is a typical symptom of enque overload, SAPLSENA is just the program that enque uses. For some reason the new kernel has caused enque performance issues.

I would go back to the previous kernel 7.21 512 to stabilize the system again. Then use a sandbox to test before introducing a new kernel into the system.

There is a very nice article in the wiki about enque overload troobleshooting that might be handy

Enqueue overload - collecting and analyzing the enqueue log information - Application Server Infrast...

To finish up, a wise man told me in a session on Teched that if your kernel is not giving you any trouble do NOT change it.

Regards, JP

[yakcinar · Accepted solution]

Hello Juan,

Sorry for the late answer.

We opened an OSS message in parallel but could not get a solution for the problem.

I could not apply your solution for this message, either.

Unfortunately we restored old 7.21 512 kernel back and now we don't get the error.

Thank you for your answer.

Regards,

Yuksel AKCINAR

[isaias_freitas · Accepted solution]

Hello Yuksel,

Does this system have an ASCS instance, where the Standalone Enqueue would be running?

Or does it still have a "classic central instance" with a work process of type enqueue?

Regards,

Isaías

[isaias_freitas · Accepted solution]

We have HA system infrastructure, ERS on both cluster nodes and extra 8 Dialog Instances. Totally 10 Instances.

OK, it seems that the answer is "you have an ASCS"...

Please confirm this, and confirm that no ABAP instance has a work process of type enqueue.

[yakcinar · Accepted solution]

Hello Isaias,

Yes we have ASCS built on cluster system.

And there is no enqueue wp on any instances.

Thanks and Regards,

Yuksel AKCINAR

[isaias_freitas · Accepted solution]

Hello Yuksel,

Thanks for the confirmation.

In this case, proceed as suggested by Juan.

Regards,

Isaías

[yakcinar · Accepted solution]

Hello Juan and Isaias,

Today we had the same issue with old kernel again on our system. Problem seems not about Kernel.

I could not collect logs using SM12 test functionality. Switching log does not bring any log.

I collected below logs.

Can you check the logs and give your valuable recommendations, please?

Remind:

We have MSCS cluster on ARPCLS1 and ARPCLS2 nodes. And additional 8 more DIs. (ARPCLS03 to ARPCLS11)

There are ERS10 instances on both node. (I am not sure whether thes ERS servers are used? Seems as if SM12 diagnosis tells that standalone ES is used.)

ASCS00 is on ARPCLS2 now.

SM12 --> Extras --> Diagnosis

Lock management operation mode 

Lock management in a standalone server 

HOST: ARPSAPGRP 

INST: 00 

Diagnosis 

Test lock operations do not contain errors.

Optimistic locks are implemented in the kernel

Test lock operations do not contain errors.

Test lock operations do not contain errors.

SM21

Error calling the central lock handler

> Dequeue All call failed

SM66

There are some processes got stuck on SAPLSENA report. These number increases to hundreds.

ARPCLS05_ARP_01 28  DIA      8404 Running ENQ        Yes    1            37 9933DBI      SAPLSENA

ARPCLS06_ARP_01 66  DIA   1053456 Running ENQ        Yes               1813 9918MSEF     SAPLSENA

ARPCLS11_ARP_01 **  DIA     13156 Running ENQ        Yes                861 9901MCARI    SAPLSENA

ARPCLS1_ARP_01  96  DIA      7320 Running ENQ        Yes               4927 9913BSA      SAPLSENA

ARPCLS1_ARP_01   0  DIA     16228 Running ENQ        Yes    1           479 9921MSEF     SAPLSENT

dev_enq* trace files on ASCS Instance

dev_enqio_1

[Thr 6468] Fri Aug 05 11:06:02 2016

[Thr 6468] ***LOG Q0I=> NiIRead: P=10.7.1.125:60837; L=10.7.1.115:3200: recv (10054: WSAECONNRESET: Connection reset by peer) [nixxi.cpp 5082]

[Thr 6468] *** ERROR => NiIRead: SiRecv failed for hdl 436/sock 1776

    (SI_ECONN_BROKEN/10054; I4; ST; P=10.7.1.125:60837; L=10.7.1.115:3200) [nixxi.cpp    5082]

[Thr 6468] Fri Aug 05 11:06:34 2016

[Thr 6468] ***LOG Q0I=> NiIRead: P=10.7.1.125:65159; L=10.7.1.115:3200: recv (10054: WSAECONNRESET: Connection reset by peer) [nixxi.cpp 5082]

[Thr 6468] *** ERROR => NiIRead: SiRecv failed for hdl 8090/sock 5692

    (SI_ECONN_BROKEN/10054; I4; ST; P=10.7.1.125:65159; L=10.7.1.115:3200) [nixxi.cpp    5082]

dev_enqlisten

[Thr 7864] Fri Aug 05 11:23:30 2016

[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

[Thr 7864] Fri Aug 05 11:23:31 2016

[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

[Thr 7864] Fri Aug 05 11:23:32 2016

[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

dev_enqwork

[Thr 4612] Mon Aug 01 05:27:34 2016

[Thr 4612] EnqAllocSnapshotPck info: out of snapshot packages

[Thr 4612] *** ERROR => EnqSendEntry: EnqAllocSnapshotPck() rtc = 8 [enxxhead.c   8899]

[Thr 4612] Mon Aug 01 05:32:36 2016

[Thr 4612] EnqAllocSnapshotPck info: out of snapshot packages

[Thr 4612] *** ERROR => EnqSendEntry: EnqAllocSnapshotPck() rtc = 8 [enxxhead.c   8899]

[Thr 4612] Mon Aug 01 05:42:35 2016

[Thr 4612] EnqAllocSnapshotPck info: out of snapshot packages

[Thr 4612] *** ERROR => EnqSendEntry: EnqAllocSnapshotPck() rtc = 8 [enxxhead.c   8899]

Dev_w15 on ARPCLS1 instance

M Fri Aug 05 11:35:03 2016

M  ***LOG Q0I=> NiIRead: P=10.7.1.115:3200; L=10.7.1.111:63107: recv (10054: WSAECONNRESET: Connection reset by peer) [nixxi.cpp 508

M  *** ERROR => NiIRead: SiRecv failed for hdl 13/sock 1436

    (SI_ECONN_BROKEN/10054; I4; ST; P=10.7.1.115:3200; L=10.7.1.111:63107) [nixxi.cpp    5082]

M  {root-id=A0369F2D42E21ED696DB3B3DE6BE5F1C}_{conn-id=00000000000000000000000000000000}_0

M

M Fri Aug 05 11:35:08 2016

M  ***LOG Q0I=> NiIRead: P=10.7.1.115:3200; L=10.7.1.111:63113: recv (10054: WSAECONNRESET: Connection reset by peer) [nixxi.cpp 508

M  *** ERROR => NiIRead: SiRecv failed for hdl 14/sock 1436

    (SI_ECONN_BROKEN/10054; I4; ST; P=10.7.1.115:3200; L=10.7.1.111:63113) [nixxi.cpp    5082]

M  {root-id=A0369F2D42E21ED696DB3B3DE6BE5F1C}_{conn-id=00000000000000000000000000000000}_0

Thanks and Regards,

Yuksel AKCINAR

[isaias_freitas · Accepted solution]

Hello Yuksel,

The issue seems to be related to:

dev_enqlisten
[Thr 7864] Fri Aug 05 11:23:30 2016
[Thr 7864] *** WARNING => NiICreateHandle: no more free handles (1014) [nixxi.cpp    466]

This is addressed at the SAP KBA 2196513.

Regards,

Isaías