cancel
Showing results for 
Search instead for 
Did you mean: 

Sync messages logging / archiving

Former Member
0 Kudos

Hi all,

My client needs to keep the log (message timestamp, status and body) for 10+ sync interfaces all within the same namescape for at least 1 year. The approximate messages exchanged in a year will be about 20-25 million. Currently, all sync messages are logged using LOGGING_SYNC = 1 parameter for the integration server because of this requirement and since I can't choose to log only selected sycn interfaces, all sync interfaces are being logged. This is our first limitation - is there a way around this - just to log what you want to keep?

I thought of using archiving first but realized standart PI SARA archiving using object BC_XMB does not support sync messages archiving (SAP Note 872388).

I then though about increasing the default retention period for sync messages to 1 year, but I've read about people having SXMB_MONI, messaging, DB and other performance issues when logs start to grow above 1 million. We're using Oracle DB and normally even 250 million records for a table in Oracle is not anything to be worried about when reorganization & indexing is OK, but I'm not sure if it would be still be OK when used with PI. This is the second question, would this really be a problem, when Persistance Layer Analysis shows that the default SXMSPMAST size should be around 800k?

The hopeless and third option that comes to mind is to take regular database backups and just copy them to a dummy PI system used only to read message logs when and if necessary. And fourth is obviously to code a custom report to read a custom created archive by a custom regular job which will log these messages to custom tables/files.

We're unfortunately still using PI 7.0. All interfaces use proxy-http(s) communication (HTTP adapter, not SOAP) from SAP to an external server so keeping track of the Java DB and adapter engine is not necessary. And my client is planning to move to PI 7.3 within the next 6 months. These interfaces will probably be moved to the Java stack completely for better performance when the new system is ready, so if I can find a solution that will work in both cases, even the better.

All helpful responses will be rewarded, thank you.

Kind Regards,

S. Gökhan Topçu

Accepted Solutions (1)

Accepted Solutions (1)

sunil_singh13
Active Contributor
0 Kudos

As you mentioned you have Proxy to Http(s) scenario. In Case of Proxy, messages gets stored in Application System as well. So what you can you do is keep the message in your ECC system and disable the Sync logging in PI. In case of debugging you can always refer to messages stored in ECC through transaction SXI_MONITOR.

Considering various pipeline services to be executed throughout the life cycle of a Message, Having over million messages in DB will slow down the system and eventually you will run into many issues. 

Now Coming to your Individual quations:

1. Interface Based Sync Logging:

http://help.sap.com/saphelp_nw04/helpdata/en/d6/49543b1e49bc1fe10000000a114084/frameset.htm

·        Set tracing in the message header. This means that this message is traced even if tracing is deactivated explicitly in the configuration. The runtime uses the highest of the values from the locally configured trace level and the trace level in the message.

You have to do that in Proxy code (in your case)

<soapenv:Envelope

    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">

  <b> <soapenv:Header>

      <sap:Diagnostic soapenv:mustUnderstand="1"

          xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"

          xmlns:sap="http://sap.com/xi/XI/Message/30"

          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

          xmlns:xsd="http://www.w3.org/2001/XMLSchema">

         <sap:TraceLevel>Fatal</sap:TraceLevel>

         <sap:Logging>On</sap:Logging>

      </sap:Diagnostic>

   </soapenv:Header></b> 

<soapenv:Body>

     ...........................................................

   </soapenv:Body>

</soapenv:Envelope>

Other way is event based logging. Search for Blog by Michal.

2. 800K Message in a Year is still Ok with good hardware. We are managing with 900K Messages in a year

Former Member
0 Kudos

Hi Sunil,

Thanks for the information. I've gone through the SAP library link. According to it, there are two ways to activate logging for an individual message even if logging is turned off in the IE configuration:

1. SXMB_ADM -> Error Analysis Settings: There's no option for interface name or namespace for the condition. Only triggers are USERNAME, CLIENT, MESSAGEID, QoS and QueueID and none will active logging for the individual sync messages we need.

2. Adding the logging header node inside the proxy: Not sure how this can be done within the proxy code? If I was sending a message directly to IE pipeline, this would be easy. Even if done though, it won't prevent my database size from overgrowing since other sync messages volume is low.

I'm also aware that I can activate logging on the sender SAP system and use its internal IE to monitor those messages, but again, same message volume, and eventually same issues will happen, just on a different system. We're trying to log 20-25 million messages, I wouldn't be asking these questions if we could stay within the same limits, lets say 3-4 million max would even work ok.

I couldn't find the blog by Michal you mentioned. I'm now seriously considering coding a custom archiving job if none else has a better idea.

Appreciating all the input, thanks.

sunil_singh13
Active Contributor
0 Kudos

Hi,

We can build SOAP header body in proxy class generated through Service Interface and set the parameters accordingly.

Have a look at : http://scn.sap.com/community/pi-and-soa-middleware/blog/2012/10/19/abap-proxy-protocols-use-cases-an...

Also if you store it ECC system at least your PI system will have better performance.

Thanks,

Sunil

Answers (0)