SAP PI/PO 7.31 - Single Stack - Best way of handling 'TBDL' messages

[abranjan]

Hi All,

I would like to know what is the best way of processing 'TBDL' messages that are shown in 7.31 message monitoring.

One way I know is to resend the messages but if the volume is huge (100k+), is there a strategy to reprocess them automatically by creating a Job? AFAIK, a job only picks NDLV messages for reprocessing.

Please share some information on handling such scenarios.

(PS: I've been through many links with TBDL issue but strategy wise, could not find any concrete responses. One of them suggested cancelling all using SQL command. I would like to know if there are some other ways.)

Thanks,

Abhishek

[marksmyth · Accepted solution]

Hi Abhishek,

I think we could do with some more details to answer this query. Are these Production messages or Dev/QA messages where it doesn't really matter if they are processed or not?

What is the Quality of Service of this scenario? If EOIO, then the messages in TBDL are not the issue here. See the blog How to deal with stuck EOIO messages in the XI 3.0 Adapter Framework

If it is not EO, then there are 2 main causes of this:

1. a lack of resources to process the volume of messages (see note 1662269 How to Monitor the PI Adapter Framework queues). And Messaging System queue properties after XI 3.0 SP19 / XI 7.0SP11, if you need to increase the queues/threads available.
2. there is some connection issue with whatever it is you are connecting to. This would require deeper analysis.

Regards

Mark

[abranjan · Accepted solution]

Hi Mark,

Thank you for your reply.

Here are some details:

1. QoS: EO

2. Box: Quality

3. System: 7.31, Java Only Stack

4. Queue Analysis: I remember seeing 150k+ messages in Dispatcher queue

5. Messaging Connections: these are maintained/tuned properly (Send, Recv threads are set appropriately. maxReceivers is also set to avoid thread shortage for other interfaces)

To be precise, these will be my questions:

1. TBDL message are handled automatically by the system. Is there a specific time frame(or any setting) under which they start getting delivered automatically? Or it waits for resource/thread recovery?

2. If messages remain in TBDL for a longer time than expected(like 3-4 days), is there a way to resend them? Any job configuration that can do this?

3. I would like to understand what factors on sender/receiver side can cause this to happen? Any supporting/clarifying documents? (1808554 is already analyzed and we are on a higher patch level)

4. Changing message status at DB level is advisable or not? If yes, then setting them to NDLV and reprocessing using a job is suggested?

[marksmyth · Accepted solution]

Hi Abhishek,

1. TBDL message are handled automatically by the system. Is there a specific time frame(or any setting) under which they start getting delivered automatically? Or it waits for resource/thread recovery?

- TBDL is an intermediate message status. Depending on the number of queues/threads for this adapter type, you will also have 5/10 messages in Delivering status. Something is preventing these messages from being delivered and this results in the backlog of messages in TBDL status. Only when the messages in Delivering status are processed will the threads be released and then become available for the messages which are backed up in TBDL.

2. If messages remain in TBDL for a longer time than expected(like 3-4 days), is there a way to resend them? Any job configuration that can do this?
- There is no way to resend TBDL messages. As above, this is an intermediate status. The focus should be on the messages in Delivering status and why these are not reaching their target and thus freeing up the threads for the backlog.

3. I would like to understand what factors on sender/receiver side can cause this to happen? Any supporting/clarifying documents? (1808554 is already analyzed and we are on a higher patch level)

- It is very difficult to answer this. There could be many different factors which could cause this to happen. Receiver system may not be available, network issues, bugs on SAP side etc. It really depends on the individual case. Best practice here would be to ensure any timeout settings are configured for the relevant adapter types. This can help prevent issues.

4. Changing message status at DB level is advisable or not? If yes, then setting them to NDLV and reprocessing using a job is suggested?
We do not advise to make any changes to message status at DB level. This option should only ever be used as an absolute last resort.

Ultimately the best way to get to the bottom of this is capture Java Thread dumps and create a support ticket with SAP.

Regards
Mark

[abranjan · Accepted solution]

Hi Mark,

Thanks for your elaborate and to-the-point answers, pretty much clears out a lot of doubts.

Regarding TBDL resend, I've done it by selecting the messages and doing a resend. Only issue, if the message count is huge, doing a 'resend' becomes cumbersome. Although, from your answer, I am assuming it is not possible to do a mass resend using standard jobs.

Regarding timeout settings, can you please let me know which adapter setting you are referring to?

Also, it would be great if you can shed some light on what decides the value of parameter messaging.queue.capacityWait?

Thanks,

Abhishek

[marksmyth · Accepted solution]

Hi Abhishek,

I don't think it is possible to resend a message in TBDL status. The standard jobs are designed to pick up messages which have a restartable status i.e. restartable error status. Messages in TBDL, Delivering, Holding status etc cannot be restarted by standard jobs or manual actions (I think).

Regarding the timeouts, each adapter type, has a corresponding FAQ note. In these notes, the timeouts are discussed in more detail. For example:
821267 FAQ: XI 3.0 / PI 7.0 / PI 7.1 / PI 7.3 File Adapter - Q.8 and Q.48

831162 FAQ: XI 3.0 / PI 7.0 / PI 7.1 / PI 7.3 JDBC Adapter - Q.28
1078420 XI/PI JDBC Adapter: Setting JDBC driver properties for DB - for JDBC driver timeouts

I really don't know much about the messaging.queue.capacityWait parameter. It is a new parameter and based on the info in the note the value of this is quite specific to the scenario used.

Regards

Mark

[aashish_sinha · Accepted solution]

Hi,

I have a very simple suggestion. This issue arises when your system threads are not available. To free your system thread, Cancle all failed messages, means look out for messages with SYSTEM ERROR or WAITING and cancle these messages. Don't do anything for TBDL messaes. Once you are done with cancelling above 2 state messages, TBDL will start processing again automatically.

Cancel message for a longer time frame to get more message in one go.

Regards

Aashish Sinha

[abranjan · Accepted solution]

Hi Mark,

Thanks a lot for your help. Really appreciate for taking out time for this.

Regards,

Abhishek

[abranjan · Accepted solution]

Hi Aashish,

A very helpful suggestion indeed.

Only issue I see here is if the WAITING messages have dependence or impending business relevance, we might have to rethink before cancelling it and get along with the successors.

SYSTER ERROR one can be cancelled though to ensure a smooth flow.

Thanks for your insight.

Regards,

Abhishek