cancel
Showing results for 
Search instead for 
Did you mean: 

Performance issue with a PI System

Former Member
0 Kudos

Hello

I am currently doing my Master Thesis, which is about a Software Experiment comparing performance between two "architectures" in SAP.

However, I am getting some unexplained pauses in the PI system, which I don't understand. I am not sure, whether this is a problem with the settings or a problem with the PI system itself (there might be a reason for the pauses), but the pauses are skewing the results and therefore the survey.

The scenario is this:

A number of XML files is placed on one system, and we want to extrapolate data from these files and insert the data in a database on another system.

The servers are all 3 GHz servers (4 cores), with 16 GB ram. The "file system" and the "database system" are both running SAP 6.0, while the PI system is running 7.0.

Architecture 1 - Two servers:

We use an ABAP program and a XSLT transformation to extrapolate and transfer data from the system with the files to the system with database. An experiment with 10.000 files took 191,879 seconds.

Architecture 2: - Three servers:

The same servers are used as in Architecture 1, however a PI System is used to extrapolate and transfer the data from the system with the files to the system with database. I am using a File Adapter. An experiment with 10.000 files took 728,53 seconds.

This is quite a bit longer and the reason for this can be found in the timestamps used to calculate the time. Each piece of data gets a timestamp from each server, an a example can be seen below:

...

...42.4440000

...42.4440000

...45.8040000

...45.8190000

...

As you can see there is about a 3 sec pause, and this pause occurs once every 20-30 files. Can you point at any obvious reasons for this small pause?

As I am learning PI is a complex system, so I am pretty sure you need more information about how the system was set up, unless the answer is simple. I am of course, more than willing to supply more information, so that I might get a correct evaluation of the performance of the PI system, however I am not sure what kind information you want. Because of the complexity of the system, a full description of the set up and the settings would transform this post into a novel.

Best Regards and thank you in advance!

Andreas

Accepted Solutions (1)

Accepted Solutions (1)

Former Member
0 Kudos

Hi,

Architecture 1 - Two servers:

We use an ABAP program and a XSLT transformation to extrapolate and transfer data from the system with the files to the system with database. An experiment with 10.000 files took 191,879 seconds.

Point-to-Point communication is always faster, but when we talk about complex IT architecture, then the idea is to streamline the flow of communication through a central hub. So, performance will definitely won't be the same.

Architecture 2: - Three servers:

The same servers are used as in Architecture 1, however a PI System is used to extrapolate and transfer the data from the system with the files to the system with database. I am using a File Adapter. An experiment with 10.000 files took 728,53 seconds.

What is the polling interval of File Adapter? What is the size of each files? What is the PI hardware configuration? What mapping you are using? All these issues can affect / slower down the performance.

Hope this helps.

Regards,

Neetesh

Former Member
0 Kudos

Dear Neetesh

Thank you for your answer!

Point-to-Point communication is always faster, but when we talk about complex IT architecture, then the idea is to streamline the flow of communication through a central hub. So, performance will definitely won't be the same.

This is true, I did not expect the PI system to perform nearly as good as the Point-to-Point system, I am more worried that the results is not representative, because of all the pauses.

What is the polling interval of File Adapter? What is the size of each files? What is the PI hardware configuration? What mapping you are using? All these issues can affect / slower down the performance.

What is the polling interval of File Adapter?

When you say Polling Interval of the File Adapter, do you mean how often it checks for Files? Because that is the only option I can find:

"Number of seconds that the adapter is to wait if no files are found for processing" (from SAP Guide)

If so, this is set to 0. In order to start the test, I deactivate the File Adapter, copy the files and activate it again. It starts almost immediately with getting the files.

What is the size of each files?

The size of the files varies between 2KB and 128KB, depending of the amount of data. The processing time does go up, but not as much as you would expect. For example 10.000 files with the size of 2KB takes around 12 min and 10.000 files with the size of 128KB takes around 15 min. The problem is that in both cases, there is these 3 sec pauses.

What is the PI hardware configuration?

I am not sure what you mean. The server is a 3 GHz server (4 cores), with 16 GB ram.

What mapping you are using?

All data elements from the XML files are mapped directly to the IDOC. There is no processing (or call to any functions) or any changes made to the data, so the mapping is direct. Except I have made a custom function, which simply calls a timestamp function, in order to get the timing.

On Tuesday (two days time from now), I get more servertime, I will try to see if I can find an actual Polling Interval of the File Adapter, if this is what you meant.

Best,

Andreas

Answers (1)

Answers (1)

abhay_rajhans2
Contributor
0 Kudos

Hi

Along with reason mentioned in above post below mentioned reason can also affect performance.

When you are processing multiple messages in XI then those messages will be passed in Queues.

So unless first message get processed in the queue next message will not get processed. This may affect the performance.

Former Member
0 Kudos

Dear Rajhans

Thank you for your answer!

When you are processing multiple messages in XI then those messages will be passed in Queues.

So unless first message get processed in the queue next message will not get processed. This may affect the performance.

This makes good sense... I am still a bit unfamiliar with the inner workings of the PI system (still learning), so I am not sure how to check it, or if this is the problem, how to fix it. I will get more server time on Tuesday (two days time from now), and I will see if I can discover anything.

Best,

Andreas

Former Member
0 Kudos

Hi,

Also check the How-To document How to Implement a High Volume Process Integration Scenario located in Articles - Service Bus-Based Integration to see which points you need to consider for a high-volume scenario like yours.

As PI is composed of double-stack, Adapter Engine and Integration Engine has their own settings and logic for processing the messages. Adapter Engine utilizes Java threads, whereas Integration Engine uses queues which is managed by the queue scheduler and work processes to process messages.

Every adapter on the adapter engine can be configured (not per the communication channel, but generically for the whole adapter), as well as the processing queues can be configured on the IE in ABAP side. So you may assign the number of threads per adapter in the adapter engine for example from the service PI AF Core by using the parameter messaging.connectionDefinition, whereas integration engine configuration (SXMB_ADM) and queue(SMQR) & system settings(RZ10) will be affecting the processing times in ABAP.

If you can further check if the delay happens in Java stack or ABAP stack by using the monitoring tools, it will help you look in the right place. Does the Adapter Engine pauses for 3 seconds sometimes while sending the messages to the adapter engine, or does the Integration Engine in PI pauses 3 seconds to send it to the backend SAP system?

You can check the Adapter Engine configuration parameters from this link:

http://help.sap.com/saphelp_nwpi711/helpdata/en/48/cbb490cea80783e10000000a42189d/content.htm

Also kindly refer to the following documents (apart from the note, you can find them by sarching in SDN):

Note 894509 - XI Performance Check

PI Best Practices - Sizing and Performance Tuning

Process Integration Performance Check(XI 7.0)

SAP NetWeaver Process Integration Tuning Guide

SOA Middleware u2013 High Volume Scenarios with SAP NetWeaver Process Integration

Regards,

S. Gökhan TOPÇ

Former Member
0 Kudos

Dear S. Gökhan

Thank you very much your reply. It was very helpful!

By following your tips and the links you gave me, I was able to narrow down the problem to the Integration Engine. Also I insert a timestamp in the mapping, which takes place in the Integration Engine and the pauses are already visible there, so the cause must be found before the mapping.

In the Integration Engine, the problem is that DB_ENTRY_QUEUING and DB_SPLITTER_QUEUING takes to long (found by using transaction SXMB_IFR > Runtime Workbench > Performance Monitoring > Detailed Data Aggregate > 5 min intervals). For example in one interval:

DB_ENTRY_QUEUING: 0,014

PLSRV_XML_VALIDATION_RQ_INB: 3,76

DB_SPLITTER_QUEUING: 4,395

PLSRV_MAPPING_REQUEST: 8,18

This means that all messages processed in the 5 min interval has spent around 3 sec in the first queue and 4 sec in the last queue, so both queues takes to long.

The fix to this is the same, so I followed all the advice in the Performance Check, checking for blocking LUW, increasing the number of queues and the number of DIA's and checking that MAXCONN is set to 10. However none of this did anything to the processing time (no increase or decrease) and the pauses are still there. For me this means that there can be two explanations:

1: The problem is not in the queues, but I have no other explanation or idea what it might be then.

2: This is simply the performance of the PI System, as intended, when using this scenario.

It was very interesting to read the links you gave to the High-Volume scenarios. It is not possible for me to avoid the IDoc adapter, which means that using the Advanced Adapter Engine (to bypass the Integration Engine) is not an option. However the IDoc packaging sounds very promising, although I am still puzzled by the 3 sec pauses.

Best regards

Andreas

Former Member
0 Kudos

Hello Andre,

I'm happy that these helped. It is hard to pinpoint the exact problem from the info you provided, but a 3 sec pause is definitely not a general operating behavior of PI in high loads. You should find the exact cause to fix the problem and prove that PI is instead a powerful tool I might be able to help you further if you could post the whole message trace from SXMB_MONI after activating tracing from IE configuration in SXMB_ADM. For your scenario, you should definitely consider your options for packaging, and you can also use ABAP mappings to avoid calls from AS ABAP to AS Java. Or, even better, you can collect/package the messages before mapping and do a bulk mapping either in ABAP or Java to boost your performance further. As we are focusing on finding the cause 3sec delay, these options are available for you to play with. But the delay can be caused of many factors - hardware, database, system configurations, etc. So kindly post your trace and let's see what really happens at that stage.

Regards,

S. Gökhan TOPÇ

Former Member
0 Kudos

Dear S. Gökhan

Thanks again for your help.

You can the SOAP header below. It is perhaps not the best example, as the first queue does not show any particular delay, however the second one does.

Sorry for the link, however I could not get this forum to show the data in human readable format.

http://dl.dropbox.com/u/322194/soap_retursvar.xml

I am sure it is not a hardware problem. I tried using the PI system as the file system in the point to point scenario, and there was no problem (no pauses) in this and the system functions well in all other aspects.

However I was told today that time was running out (because sudden and external factors affecting the company at which I am doing my Thesis) and it is possible that I will not be able to get more server time, so I would hate for you to spend to much time on this, without knowing that I might not be able to try out any new solutions. Of course, if you can find any wrong settings in the trace, I would be extremely grateful, as it would either be part of my report, or if I can "sneak back in", be tested.

Based on the previous we discussed, I have dedicated a section of my Thesis, that explains alternative strategies (Advanced Adapter and MSG packing), but that part will, for now, remain untested.

Best,

Andreas