PI 7.1 MessageExpiredException in SOAP to RFC scenario

[Former Member]

I have a synchronous SOAP to RFC scenario like below.

Request message: Webservice (SOAP Adapter) ---> XI ----------> RFC Adapter ---> ECC Function modules
Response message: Webservice (SOAP Adapter) <--- XI <---------- RFC Adapter <--- ECC Function modules

The external system calls XI web service via SOAP Adapter. The input data is then sent from sender SOAP Adapter to receiver RFC adapter to call RFC function module in the receiving ECC side.

My SOAP sender channel is failed with below error.

com.sap.engine.interfaces.messaging.api.exception.MessageExpiredException: Message ID (OUTBOUND) expired.

In SXMB_MONI same message is failed with system failure state having below error details.

<SAP:Code area="MESSAGE">GENERAL</SAP:Code>

  <SAP:P1 />

  <SAP:P2 />

  <SAP:P3 />

  <SAP:P4 />

  <SAP:AdditionalText>com.sap.engine.interfaces.messaging.api.exception.MessageExpiredException: Message ID(INBOUND) expired</SAP:AdditionalText>

  <SAP:Stack />

I have browsed many threads on SDN with mention of similar issues.

like http://scn.sap.com/message/1001054#1001054 and http://scn.sap.com/community/pi-and-soa-middleware/blog/2006/06/08/xi-timeouts-timeouts-timeouts and many other.

Below are the list of check I have done till now as per suggestions available on SDN.

• a)      In my receiver RFC channel max connection is already set to 20.
• b)      As per SAP Note 791379, I have increased the value of timeout parameter “xiadapter.inbound.timeout.default” in NWA to 5 minutes from default 3 minutes.
• c)      As per SAP Note 730870, I have checked the value for the RFC Adapter parameter “syncMessageDeliveryTimeoutMsec” and it is already set to default value 5 minutes.
• d)      I have set my ABAP team to debug on FM but the FM is a standard BAPI which is being used. And no problem is found from ABAP perspective.

I am observing this problem in my production system only with a random pattern. I am getting below mentioned error in average 15-20 calls out of 100 for the same interface. Also I was not able to simulate the issue in any of the DEV and QA systems.

Any inputs to troubleshoot this problem will be highly appreciated.

[Former Member]

Hi Bharat,

I am facing exactly the same issue, and we are not ale to replicate the issue in QA/DEV. How did your solve this issue. can you please let me know??

Regards,

Sachi

[Former Member]

take the payload from prod and test in Dev/QA

[Former Member]

Hi All,

I am not sure if it helps but in our one odd case we found that there was an Infinite Loop enabled in ECC RFC FM.

Thanks,

Shaibayan

[former_member212325]

Hello All,

What is the solution of this MessageExpiredException? I am facing the same issue.

Regards,

Sunitha

[fanglin_ye]

Hello Bharat,

i have met the same problem and all the notes which you have mentioned i have also read. Do you resolve the problem?

i have one more question: how to find out the reason about the expired exception. I mean, maybe the connection between java stack and abap stack is not good. Maybe RFC-Connection pool is too small. Maybe the remote function in backend system takes long time. Maybe ...

Best regards,

Fanglin Ye

[fanglin_ye]

Hello,

the answer from Rajesh ist right. My problem was resolved. thx.

Best regards,

Fanglin Ye

[Former Member]

Also i have set value of XI.Timeout parameter to 10 minutes (600000 milisecondds) in module parameter table for the soap sender adapter following the note 856597 for Soap Adapter.

But i am still getting the same timeout error after 5 minutes only and it is not waiting for 10 minutes before throwing the below exception.

XIAdapterFramework:GENERAL:com.sap.engine.interfaces.messaging.api.exc eption.MessageExpiredException: Message a44bead0-cb4b-11e1-c81e-aaa91d f6a70b(INBOUND) expired

Not sure why  XI.Timeout parameter in soap adapter is not effective?

[Former Member]

Hi

The message is timed out in messaging system. So you would need to check the below values in XPI Adapter : XI.

xiadapter.outbound.persistDuration.default

xiadapter.inbound.timeout.default

[Former Member]

Rajesh,

xiadapter.outbound.persistDuration.default is set to its default value

and i have increased the  value of xiadapter.inbound.timeout.default parameter to 5 minutes from default 3 minutes.

[Former Member]

As per Bharat, I would perhaps look at the random nature of the occurrences and the fact that this is happening only in Production as possible indicators that the calls inbound might not be consistently routed.  Is your PROD environment HA-enabled and clustered?  Are the RFC settings and configuration the same across this tier in the environment?  What are the java trace/default logs saying?  You my find some trends there that point to the source of the trouble...

[hemant_chahal]

Hi Bharat,

You can try increasing the SOAP receiver adapter timeout. please check this link http://scn.sap.com/thread/1955524

Regards,

Hemant Chahal

[Former Member]

Hi Hemant,

Thanks for the reply.

My scenario is having SOAP Sender channel and I have already increased the value of xi.timeout parameter as mentioned in the link you provided.

[Former Member]

Did you restart the related Java services after making your value changes?

What does the Adapter Engine status show from the RWB?

Another document worthwhile having a reference to or copy of is here for Synch Timeout troubleshooting.....

[Former Member]

Hi Bradley,

I did not restarted related Java services as it is not mentioned in the note, however i see messages getting timed out only after 5 minutes(increased value).

RWB shows sender communication channel in error state with the below mentioned error.

com.sap.engine.interfaces.messaging.api.exception.MessageExpiredException: Message ID (OUTBOUND) expired.

[Former Member]

Hi Bharat,

I guess your RFC takes more than 5 mins here.

Please excute your RFC in ECC and check how much time that takes to execute with the inputs what you would pass from PI.

And then change the paramter xiadapter.inbound.timeout.default accordingly.

[Former Member]

Hi Bharat,

I think the issue might be on the ECC side more than anything. Is your response time quite quick in Dev & QA?

If the RFC that you care calling is a custom (Z) one rather than an SAP standard one then you might want to setup a code performance / optmisation exercise. Maybe also run the question past teh Abap forum here on the best way to 'latency' test the function module.

If it's an SAP Standard function module, then the chances are that there are other performance issues on the ECC production box which Basis will need to check. Also, if you are using a load balancer in production, configure you RFC receiver channel accordingly (i.e. to use the load balancer instead of a direct call the backend app server).

While you're doing that you can probably do a load test using SoapUI or other load test tool in Dev and QA to ascertain average response times. It might even be that Basis need to perform stats.

Regards, Trevor

[Former Member]

Hi Rajesh,

I too feel the same but i am not able to validate this because i was not able to simulate this in DEV and QA. I tried to execute the RFC in my DEV and QA box but there it take max 5-10 seconds to execute.

[Former Member]

Hi Trevor,

Thanks for reply.

Not sure if i should move this post to ABAP forum over here. Also, is it necessary to restart java services after changing parameter values in NWA, as i see changes being reflected at runtime. ?

I think the issue might be on the ECC side more than anything. Is your response time quite quick in Dev & QA?

Yes, response time is quite quick in Dev & QA.

If the RFC that you care calling is a custom (Z) one rather than an SAP standard one then you might want to setup a code performance / optmisation exercise. Maybe also run the question past teh Abap forum here on the best way to 'latency' test the function module.

BAPI which we are using is custom (Z) one but it is actually a wrapper for the standard BAPI  BAPI_CONTRACT_CREATEFROMDATA and BAPI_TRANSACTION_COMMIT. And there is no custom code involved.

If it's an SAP Standard function module, then the chances are that there are other performance issues on the ECC production box which Basis will need to check. Also, if you are using a load balancer in production, configure you RFC receiver channel accordingly (i.e. to use the load balancer instead of a direct call the backend app server).

Any suggestion to check from Basis end ? Yes we are using load balancer in production and RFC receive channel is configured accordingly to use the load balancer.

While you're doing that you can probably do a load test using SoapUI or other load test tool in Dev and QA to ascertain average response times. It might even be that Basis need to perform stats.

Knowing that response time is extremely low (in milliseconds ) as compared to production, do you think this stats may help?

[Former Member]

Ok. Why don't you increase the xiadapter.inbound.timeout to 15-20 mins and see?

[Former Member]

If it's an SAP Standard function module, then the chances are that there are other performance issues on the ECC production box which Basis will need to check. Also, if you are using a load balancer in production, configure you RFC receiver channel accordingly (i.e. to use the load balancer instead of a direct call the backend app server).
Any suggestion to check from Basis end ? Yes we are using load balancer in production and RFC receive channel is configured accordingly to use the load balancer.

Yes, you could check which tables are being impacted in the BAPI and Basis can run some stats on them as a start. Basis could also check & fix the memory & CPU utilization on the ECC side.

While you're doing that you can probably do a load test using SoapUI or other load test tool in Dev and QA to ascertain average response times. It might even be that Basis need to perform stats.
Knowing that response time is extremely low (in milliseconds ) as compared to production, do you think this stats may help?

In this case it won't help.

In my opinion, increasing the timeout parameter exponentially is just a band-aid, the root cause of the issue is not being addressed. You also haven't provided an indication of how long the external system calling this service is prepared to wait for the response, thats something else that should be considered. It looks like a resource issue (or lack therof) on the ECC side so it's something that Basis should work on because it could also impact other areas on ECC.

Regards, Trevor