Portal Performance Benchmarking

[former_member245181]

Hi All,

We have a 4 CPU box with 16GB of RAM, running a SAP J2EE Cluster (DB, SCS, Dispatcher, and 4 x server processes). Each server process has been locked down to only 40 Application threads (Min and Max) and System Threads also locked down to 40.

When we have about ~250 users logged in and running ESS employee lookups with a think time of ~30sec, we become CPU bottlenecked. There seems to be no real single issue, the system is just busy. The Garbage collector doesn't appear to be the cause either. "vmstat" shows that the CPUs are 0% idle and the majority is User%.

I've tried all sorts of things: Reducing the Logging, Reducing the threads, disabling the HTTP compression - but it is still CPU bottlecked. Of course, when CPU becomes bottlecked, the performance degrades considerably.

If I use VA -> Cluster -> Server -> Services -> Monitoring -> Performance -> Requests Performance -> Requests per second, I see that each server process has a max 'Number of JARM Requests per second' of ~18.

Is it possible to get some performance benchmarking stats from you? How many ' JARM Requests per second' does your server support before CPU is bottnecked? (also what is the spec of your box)?

We use Solaris 64, EP7 SPS8, with Sun JDK 1.4.2_09 (We'll upgrade this JDK soon).

Thanks in advance,

Regards

Anthony

[former_member197561 · Accepted solution]

Dear Anthony,

With same hardware and Java Server installed on it there would be different number of requests per second possible to process, becaseu this depends on the functionality of the application.

For Portal , according to this benchmark

http://www.sap.com/campaigns/benchmark/appbm_epess.epx

with results:

http://www.sap.com/solutions/benchmark/ep-ess_results.htm

If we take one of the results 792619 navigations per hour, this would mean 220 navigations per second on the given hardware.

This information may not be very helpful: most reliable is to check the hardware which is required for 250 portal users in Quick Sizer

http://service.sap.com/quicksizer -> SAP NetWeaver -> Portal & KMC

and then compare it to your test system.

Best regards,

Sylvia

[former_member245181]

Thanks for your reply Frank. I'm still interested to find a benchmark for ' JARM Requests per second'. What's a normal figure? During our load tests, I've seen this go to 22 per server process.

If anyone can supply me with their figures (you'll need JARM activated), I will reward points.

Regards

Anthony

[Former Member]

Anthony:

Is there some previous benchmark that tells you, you can run more than this quantity of workload?

Generally speaking, if you can reach 100% CPU or attain healthy CPU queueing you are at a satisfactory limit. I am not saying there isn't some more (minor) tuning you can do, but to be able to reach and saturate the CPU is actually a good achievement. Many times other blocks or queue issues are inhibiting the goal of CPU saturation in Benchmarking work.

In the cluster we run with low cost Intel 2-CPU systems our systems tend to max out at about 80 Virtual Users. Our think time and many other factors vary which makes a direct comparison invalid, but you may just be hitting a legitimate maximum for that configuration and workload.

As far as generalities, the Portal tends to get CPU constrained first. The NetWeaver Portal tends to be CPU intensive, especially when you are using Portal Services, specially built Java Components, KM or other things that are Portal resident. These "active" services will benchmark much differently then a 5k static page which is what you might find in some of the white papers floating around.

One thing you should not do is correlate Load Runner (or Silk or whatever tool) Virtual Users with real world users. It is totally incorrect to try to match Virtual Users to Real World users, it's a losing proposition. Try to find the transaction per hour (page views per hour), and target that for scalability goals, and not specifically the user count. The User Count can help you with total session load per box, Memory requirements per Sessioned user, etc, but it's better to target the Transaction volume.

Virtual users are simply that, and you can never simulate think time correctly with the chaos and randomness of the real world. In fact most virtual users are far more aggressive than real world users. For the following reasons...

SAP Applications take time to work in, it's not like a news page. Think times can often take minutes in the real world.

Virtual users aren't dealing with Desktop OS and rendering requirements of the real world.

Virtual Users are probably sitting on the same LAN or are just 1 IP Switch away from the Target server environment, this is totally not the case in the real world.

Nobody knows the exact factor of virtual users to real world users but the discrepency can be very high. Most tests totally overestimate the real world load which is probably a good thing, that's why capacity planning and performance monitoring of Productive systems is another key step, and something you want to circle back to your benchmarking environment with.

I hope that helped.

Frank Ober

[former_member245181]

Thanks for your responses Markus and Martin.

Markus:

I don't think the CPU load is caused by the garbage collector. I've monitored the GC logs and watched the GC durations and frequency for each server node during the high load times, and have found that the GC's do not run constantly.

Markin:

I found that note 746961 was not really suitable for our situation as the XI services mentioned are not available on our Portal.

For everyone's reference, I therefore did the following to attempt to disable JARM:

- VA -> Global Configuration -> Server -> Services -> Performance Tracing

- jarm/switch: off (old: on)

- Restart Cluster

See this:

http://help.sap.com/saphelp_nw2004s/helpdata/en/18/c246cf15be4daebb0df034bf5d6ba8/frameset.htm

Regards

Anthony

[martin_moser]

Hi Anthony,

I think it is a good idea to disable JARM. Please check Note 746971 on how to disable JARM for several applications.

Cheers,

Martin

[kohlerm]

Hi,

Please check the sizing guide at http://service.sap.com/sizing.

You should also check whether your scenario is memory limited by taking a look at the garbage collector log.

You will find some hints of how to do this in this Forum.

If it's memory limited, the garbage collector will run frequently, which at the end will make it cpu limited.

Regards,

Markus