SystemAck sent back could bring the system down?

[suwandi_cahyadi]

Dear Experts,

Once we were overloaded with many article master interface.

The scenario of the article master interface is from BS_A to BS_B, Service Interface: SI_Artmas

Therefore we have hundred of thousands Artmas messages with 'to be delivered' status in the adapter engine and this holding back our daily transaction interfaces.

In that case, to save our daily transaction interface, we set the status of the CC to inactive (in Integration Builder) and also stop the communication channel (in RWB). After the CC is set to inactive and stopped, we cancel the remaining article master interfaces that are still in scheduled status.

After all the article master interface data have been cancelled, we re-start the CC and change the status to active.

Suddenly there are some messages that I see at the RWB Integration Engine message monitoring:

from BS_B to BS_A, Service Interface: SI_Artmas.

When I try to view the detail of the message, the message content only has the SOAP content (it doesn't have the payload content). As I see there is this xml data in the SOAP content:

<SAP:MessageClass>SystemAck</SAP:MessageClass>

<SAP:ProcessingMode>asynchronous</SAP:ProcessingMode>

The next morning, we have a PI down (when I try to open the RWB, we have a 503 - Service unavailable response). For this, we asked the Basis to restart the PI production server.

After that I check the SM58 in our PI production server we have a lot of stuck messages with this description:

Caller = PIAFUSER

Function Module = IDOC_INBOUND_ASYNCHRONOUS

Status Text =  EDI: Table passed to EDI_SEGMENTS_ADD_BLOCK is empty.

Double clicking the message I see a text saying: A L E A U D

Actually after checking the IDX_NOALE report, we have configured to turn off all of the Acknowledgement for all message type.

What I want to ask is:

- Why is there a SystemAck sent back when we have configured no ack needed in IDX_NOALE report? Is there something that I've done wrong? Could this be the reason why the system is down the next morning? (recorded in RWB there were seven hundred thousands of messages sent back as SystemAck)

- What should I do to the stuck messages from PIAFUSER in SM58? I'm afraid they could hold other newer messages, if SAP use Queue Mode for TRFC as in SMQ2?

Thank you,

Suwandi C.

[suwandi_cahyadi]

Hi Experts,

After more investigation, I found the messages that cause the flooding are from June 2013 (it was a long time ago). Maybe those message were sent when the IDX_NOALE has not been turned off.

as I see there are around 700k of messages are in the system error state in adapter engine for June 2013. Whenever I re-start the output channel, those 700k messages are re-processed again, and since the error is in the data (that causes the variable substitution not found). They turn to be error again, then the ack messages are sent.

What I want to ask now are:

- Looking at the audit log of the message, I see an entry Retry: 5. It seems that the message has retried for 5 times. While actually seeing at the Communication Channel > Settings. The maximum retry is configured to 3 and the interval is 5 minutes.

- How to delete those 700k messages all at once without causing those messages to send CANCEL ack message? Because right now, I click the details of those 700k messages and the maximum select in the detail is 10k messages and it takes time to cancel 700k messages. I'm aware there's a cancel button at the bottom of the Adapter engine database overview screen. But when press it, there's a java error (it seems that the function is not implemented in my implementation).

Thank you,
Suwandi C.

[suwandi_cahyadi]

Dear All,

I've found a new finding about this issue.

This disaster occur again yesterday. Actually this could happen every time I start and stop the related channel.

Once the channel is stopped then started. PI re-try to send some messages back then in June 27th 2013.

But the message has a DynamicConfiguration error because the variable substitution is not found. When this happen, PI gives information "Acknowledgement creation triggered for type: SystemErrorAck" and also "Acknowledgement creation triggered for type: AckNotSupported".

Somehow the Ack messages is flooded into PI even until 1 million messages and bring the system down because it runs out of memory.

What to do with these messages? Because when I enter search the message id in SXI_MONITOR. It says the message is not found even when I have put the start date to blank in the selection screen.

Will changing the configuration back to 1 will stop this error? Since those are old error messages that PI try to re-send.

Thank you,

Suwandi C.

[Former Member]

Hi,

Are you sure no entries for ack has been enabled in idx_noale for your idoc message types to get suppress the acknowledgments.

Do more investigation regarding the no of messages got stuck in inbound/outbound queues for your idoc message

Regards

Rajesh