Messages in " TO BE DELIVERED STATUS " in XI - Adapter Engine

[rubanprasanth_s]

Dear Experts,

In our Production environment, at least once in every 15 days we face issue in adapter engine , where messages are in " TO BE DELIVERED STATUS ".

This is not for one scenario , but for all scenarios. I also do not see any blocking message.

So whenever it happens, we are going for a restart option, and once after Java Stack restart all messages starts flowing automatically.

I want to know why this is happening,

Could you let me know the root cause of this issue and how to prevent this ?

Regards,
Ruban.S

[engswee · Accepted solution]

Hi Ruban

Did you try checking my recommendations listed in the blog below?

Rgds

Eng Swee

[rubanprasanth_s · Accepted solution]

Dear Eng Swee Yeoh,

Thanks for your response.

I read your block. It has given me many information.

But in my case there is no messages with status ""Delivering"" ,  All messages are in status ""To Be Delivered"". So in that case just finding the oldest message and STOP/STARTing the channel will release the hanging threads ?

Kindly share your thoughts.

Regards,
Ruban.S

[engswee · Accepted solution]

Hi Ruban

Can you also check the Adapter Engine Status to make sure there are no queues where the threads have reached the maximum and remains there?

What is your PI version and SP level?

Rgds

Eng Swee

[rubanprasanth_s · Accepted solution]

Hello Eng Swee Yeoh,

Thanks a lot for helping me with lot of information on this.

1. i will check the same next time when this issue repeats ( Now we have already restarted the system).

But could you give me some more information on below points Please with an example:

For example if "Maximum Number of Threads" = 10.

1. what is the limit of "Number of Entries in Queue"  ? at least approximate number what you think ? Does it have direct proportion link with "Maximum Number of Threads"

2. whenever i noticed i see "Threads Assigned/Working" = 0/0. What are the values it can take and what it should not.

Regards,
Ruban.S

[engswee · Accepted solution]

Hi Ruban

I've already mentioned it in my blog:-

Over here, a large value in "Number of Entries in Queue" will indicate the blockage. You will also notice that the Assigned/Working threads are equal to the Max Threads. If you click into the queue, you would be able to see the messages in the queue.

1. There is no exact number. If you see the number keeps on increasing and does not decrease over time, that will indicate messages are blocked which will correlate to the To Be Delivered messages.

2. If max is 10, and this number becomes 10/10 and does not decrease, it indicates blockage.

If your system is getting this issue periodically, I'd suggest you implement the preventive measures I listed in my blog. Otherwise you will most likely hit this again in the near future.

Rgds

Eng Swee

[Former Member]

Hello Ruban,

Did you check if the threads getting hung up during this duration?

Basically, messages goes to " TO BE DELIVERED STATUS " because of unavailability of threads.

You can check for the below probable reasons:

1. There is some messages already failed and are retrying as a result utilising all the threads.

2. There are some EOIO messages which have failed and rest of them are holding and have utilised the threads.

Please check if one of these options are affecting your system.

Also, plan if the threads of the system can be increased depending on your hardware capacity of processing messages.

Thanks and Regards,

Satanik

[Former Member]

Hi Ruban,

Could you please confirm if the messages which are going in " TO BE DELIVERED STATUS " are Adapter Specific?

Also, did you try stopping and starting the channels prior to Java restart?

Thanks and Regards,

Satanik

[rubanprasanth_s]

Hello Satanik,

Thanks for your response.

Messages belongs to different Adapters.

Since there were no error in channels, I did not do Stop/Start.

Regards,
Ruban.S

[former_member186851]

Hello Ruban,

We also face the same issue sometimes.

It is because of Memory and consumer threads.

And there is no solution except restarting the Java server.

Permanent solutions could be increasing the memory and processing threads.

[engswee]

It is not necessarily due to memory limitations. You might want to read the blog I wrote about this (mentioned in my reply below) to get a better understanding on why this happens.

[former_member186851]

Thanks Eng..

Nice blog.