Bechmarking i5 OS with ERP

[Former Member]

Hi

We are presently running SAP ECC 6.0 with DBM on i5 OS with Power6 machine. Over these years we were facing lots of performance issues and investment towards i5 OS is also on the higher side. Presently we are at a stage of upgrading our existing machine to Power7. In this process we have searched for the SAP ERP 6.0 benchmark results on i5 OS with power 7 processor. Surprisingly it was found that all power 7 machines are benchmarked with AIX / Linux as operating system and DB2 for Unix as database. Is it that SAP gives better performance on Unix platforms than i5 OS. In addition we have observed that IBM also put lots of efforts in unix to add lot of features like compression, Memory management to better the ERP performance.

Experts pls provide your comments. Your comments are valuable for us to take a decision whether to migrate to Unix platform.

[RSchmerbauch · Accepted solution]

Hi Harish,

You asked for help from the experts. In this case, that's me (and my team here in Rochester). I have worked on the IBM development team for the past 17 years and have been doing the benchmarks for a majority of that time, including work with the AIX benchmarking team. Lab measurements between AIX and i led to us deciding that the performance was close to the point that we are sizing the same between AIX and i in the sizing tables for Power. Input a number of SAPs to the sizing for a system and you get the same HW recommendation from IBM for either OS. If you want to believe you understand how the SAP source code works better than the team in Walldorf and I do, then I don't know how we can help you.

Benchmarking requires a significant investment to build skills to compete. AIX must make this investment for SD since they are directly competing with HP and Sun for the unix customer set all the time - performance is one of few differentiators, so it is important to AIX. For similar reasons, AIX is the one carrying the IBM flag in other benchmark wars - simply because it is a high priority for them in fighting for the generic Unix market.

From an IBM business investment perspective, it made more sense for us to do a second SAP benchmark on i than to duplicate what AIX is doing with SD. Why spend the money twice when IBM Power will only be #1 once on SD? Since we have only one DB option on i, and BI is doing the harder queries, this is a good benchmark for IBM to do on i because we don't have to repeat BI on Oracle like AIX needs to do with SD.

Finally, in general, since system stability and ease of use are priorities for the i customer set, those areas get the IBM i investment priority rather than running lots of benchmarks. Benchmarks are even a secondary priority within my team compared to delivering the function SAP needs from future OS releases.

I talked today with Volker. We think the HW environment you describe should be able to handle your 1000 users with no difficulty. This makes us think there are some serious tuning problems or you are using a lot of z reports that need optimizing. Both of those would fall under consulting and Volker is one that is very good at that.

You might also start by verifying some basic settings yourself as found at

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101697

It won't address problems with z reports, but it at least tells you some of the basic transactions like ST02 for tuning.

I believe you also may have some further contact with the Germany IBM/SAP team.

[Former Member · Accepted solution]

We have carried out analysis of some of the programs through SAP. We havnt seen any solid recommendations from SAP which could drastically improve the performance of the system. Ron I am putting forward this queries as a step to rectify the issues if it are any in the existing system. While I am asking this question me and my team is putting full effort to improve the system. We are already in the path of upgrading our system to V6R1. I have already seen the note which you had mentioned to me and some of the points we are already following and some we are implementing.

As per you the hardware can handle 1000 users. But as per us we have provided huge amount of hardware. If we have to deploy SAP to rest of our user department we have to invest in more amount of hardware which management is not agreeing. As I told earlier if you can put some expert to look into our system and help us in identifying the issue that would be really helpful. Pls guide me how we have to contact the SAP/IBM Germany team to sort the issue. All basis related verification we do check regularly. If it is Z programs pls if you can help us to identify. If Mr.Volker or any of the expert can look into our system it would be very helpful. Pls look into the entire system rather than looking at some of the programs which we are suggesting. SAP MENA has conducted a similar kind of session for 2 weeks and the outcome is we have to approach CDP team to make our Z programs standard which is not a solution for the problem. We have invested in SAP and now to solve the problems which are written by the SAP suggested consultants we have to again invest. This is where we are looking for different options to run our system.

Hope you have understood.

Regrds

Harish

[Former Member · Accepted solution]

Hi Harish,

I sent you a mail offline.

We would be willing to help you, but I will now stop conversion on this threat via this forum, because it is not possible to discuss on stuff, that I can not really see.

I cannot do anything against problems, that occured in the past - if there is anything pretty wrong - which I still strongly assume - this needs to be fixed in order to proceed in a useful manner. This has nothing to do with any platform ...

You can contact us at any time.

Regards

Volker Gueldenpfennig, consolut international ag

http://www.consolut.com http://www.4soi.de http://www.easymarketplace.de

[RSchmerbauch]

SAP only allows published benchmarks to be made public.

We've already stated why AIX publishes SD and not BI, and why IBM i publishes BI and not SD,

mainly to meet the general requirements of the AIX and i markets.

And so we're back to the statements that the sizing tables (that are published to business partners)

are the same for Power regardless of AIX and i, and they were based on SD internal measurements

(exactly the unpublished ones I can't tell you about). That's because we use the same source code,

compiler, runtime(PASE), processor, dimms, bus,.... the executables are very close to each other.

So for the same SAP workload (like SD) the performance is quite similar. Yes there are little differences

but so much of the time is in the SAP app server code, the little things are hardly measurable.

I'm repeating myself so I'll stop here.

[RSchmerbauch]

HI Harish,

If I understand you correctly SAP MENA has already told you that your Z programs are a concern for them and they suggested changes were necessary, but you don't want to do that. I wish you had mentioned that earlier, but I'm glad that is now known. Volker can either confirm this finding or he may find that something else is holding the performance back.

I agree with Volker, this discussion is at the end. Right now, the type of help you seem to need most is technical consulting assistance directly on your system. That should be handled outside this forum. Volker is very experienced and this is work he does often so I hope you will listen to him.

If you want to discuss SAP on i in general further, I know you have an advocate in Rochester, you have several contacts in IBM and SAP Germany now, or you can email me directly at any time. At some point, you may want to come here or to Germany for a full briefing - I believe those have been scheduled but canceled for various reasons in the past. I also would recommend the SAP on i Basis class that my colleagues did - I think you'll find it useful. At the moment though - a technical consultant should be the priority to get the performance where it should be.

Regards, Ron

[Former Member]

Hi Ron

Pls note as mentioned by me earlier SAP MENA has analyzed the system and has not provided any report in which they suggested specific changes in the Z programs. If any such report is there about our system we are open to look into that and correct the issues. But for your kind information we havnt got any feedback about Z programs.

I think we have diverted from the topic for which the query was posted for. It was related to Benchmarking results of iSeries. Any results are there which has been tried internally which is not published.

Harish

[RSchmerbauch]

Harish, your assertions are misleading.

My team runs the SAP on i benchmarks.

Here are the facts:

i does BI benchmarks (harder SQL).

aix does SD because of the need to show direct unix perf vs Sun/HP.

aix uses BOTH db2 and Oracle DB in the SD benchmarks. IBM likes db2, but many aix cust still use Oracle, so aix does both.

There is little difference between db2 and Oracle results with SD because the SD spends almost all it's time in ABAP.

This is where you aren't making sense. Your system is on intel with an i DB back end. If it's slow, and you think SD is the right comparison point, then you are really saying that INTEL is the slow part because SD (ERP) is mostly ABAP. I understand you are talking with IBM and SAP about all this later today, so hopefully the team can help your understanding.

Since i and aix share the same kernel source code(as all other platforms), same compiler and same processor, the same performance can be expected for ABAP.

Any difference in DB server performance DB2 on i or AIX would probably not be noticed much in your 3-tier environment where your Intel servers are taking the majority of the time doing ABAP processing.

[Former Member]

Ron

Thanks for your time to reply to my query. I really dont know how it really does not make sense if as an System i customer looking for a benchmark of their operating system and hardware before upgrade. We need to see results. Not verbal commitments. Why SAP has invested on benchmarking. this is to make the customers aware on the optimal platform and operating system and database which they have to go for. I still dont agree on the your statement that AIX and System i works in similar manner. Both operating system has different features different way of handling memory. Neither in TOP TPC benchmarks nor in SAP benchmarks i5 OS is benchmarked. Your point on BI benchmarking in system i is not acceptable when we are looking for an erp system. BI is an Analytical system and ERP is a transactional base system. Expectations from these systems are different and the way the benchmarking is carried out is also different. I cant agree the point that in both cases DB is stretched. Even though it is stretched it is carried out for different expectations.

As you mentioned we had long discussion with SAP and IBM on this topic and no one could give a reason why we have to stay with iSeries. Unix and Windows has large customer base and we could solve most of our issues through forums. SAP Teched focuses on Windows and Unix. So in our experience the System i is not given the equal preference as like windows and Unix.

[Former Member]

Hello , this is Walter Lang , in the IBM SAP Competence Center / Walldorf , and i would like to provide my comments to the questions raised in this thread. Please feel free to discuss them openly. Regards , W:L.

Q1:

We are presently running SAP ECC 6.0 with DBM on i5 OS with Power6 machine. Over these years we were facing lots of performance issues and investment towards i5 OS is also on the higher side.

A1:

I am sorry to hear about performance problems on your IBM system, and i do hope they could be resolved to your satisfaction..

Our experience from more than 3500 live SAP systems worldwide running at IBM i customers, is that the great majority of them runs smoothly with no major problems.

Occasional problems do occur , like on other DB/OS platforms, and are usually resolved quickly and competently by the joint IBM SAP support team , or by local BPs with professional SAP-IBM i skills.

When reporting problems, it is important the provide precise OS/DB data information ( e.g. DB/400) to ensure fast and correct problem assignment to the right SAP expert support team.

SAP is a powerful , but also a challenging application, and requires specific skills and knowledge on any OS/DB platform to ensure best performance and operation.

Q2

Presently we are at a stage of upgrading our existing machine to Power7. In this process we have searched for the SAP ERP 6.0 benchmark results on i5 OS with power 7 processor.

Surprisingly it was found that all power 7 machines are benchmarked with AIX / Linux as operating system and DB2 for Unix as database.

Is it that SAP gives better performance on Unix platforms than i5 OS.

A2:

IBM assumes equal SAP performance on AIX and IBM i , as both OSs are running on identical POWER hardware platform. Consequently, the IBM SAP sizing and configuration guidelines are identical for AIX and IBM i , and SAP solution landscapes are designed identically for AIX and IBM i. Also, IBM does run SAP benchmarks on both POWER AIX ( for SAP SD ), and on POWER IBM i ( for SAP BW ). But for economic reasons IBM does not run both benchmarks on both OSs.

Q3:

In addition we have observed that IBM also put lots of efforts in unix to add lot of features like compression, Memory management to better the ERP performance.

A3:

As commented earlier by IBM Rochester , both functions, DB compression, as well as AME, have their pros and cons. They do provide advantages in specific usage scenarios , but also bring along their drawbacks, like e.g. additional system management , higher demand for hardware, or slower response times, in certain situations. Please see SAP Note 980067 on the effects of DB compression , just as an example.

The IBM i DB2 development team is working to provide alternative enhancements, other than compression, e.g. to better manage variable length and LOB data, which is where most of the compression would take place.

While IBM i does not provide DB compression or AME, it does offer advantages in other important areas , like native integration of DB2 into the IBM i operating system, its unique "Single Level Storage" concept for low-effort DB management, or its unique "Subsystem" feature , which allows perfect self managing CPU and memory virtualization for SAP workload - just to name some key benefits. Our existing SAP-IBM i customer base widely confirms these benefits in the feedback we receive in regular user group meetings and in numerous SAP-IBM i customer references across the world.

[Former Member]

Thank you very much for the detailed reply.

-

---

-

---

A1:

I am sorry to hear about performance problems on your IBM system, and i do hope they could be resolved to your satisfaction..

Our experience from more than 3500 live SAP systems worldwide running at IBM i customers, is that the great majority of them runs smoothly with no major problems.

Occasional problems do occur , like on other DB/OS platforms, and are usually resolved quickly and competently by the joint IBM SAP support team , or by local BPs with professional SAP-IBM i skills.

When reporting problems, it is important the provide precise OS/DB data information ( e.g. DB/400) to ensure fast and correct problem assignment to the right SAP expert support team.

SAP is a powerful , but also a challenging application, and requires specific skills and knowledge on any OS/DB platform to ensure best performance and operation.

-

---

R1:

We presently have around 1000 users using SAP and the configuration of our landscape is Central instance 7 core with 93GB RAM and 2 TB HDD. Which is very huge according to all the experts who has seen the system. Administration cost if you compare . Windows and Unix platform has less administration cost. Simple example is the Online reorg. We are presently shutting down the system for carrying out online reorg as recommended by IBM. AIX has the feature of Online automatic reorg. Every month we carry out an entire system backup by shutting down the system. In AIX and Windows we have the facility to take OS partition also online backup. We are using BRMS for backing up iSeries. Network card teaming in iSeries is not like other OS, where in if one network card goes down other takes over. It is just configured with Virtual IP. There are more customer base for AIX than iSeries because of which IBM as well as SAP is more concentrating on this platform. In our experience there are lots of issues which SAP also has seen for the first time in our environment. I feel it would be better to be in open platform rather than a proprietary OS. Any advantage are we getting by saying the DB is integrated with OS? Lots of page faults appear in the system even after providing 93 GB RAM. Getting IBM i Skilled person is also difficult today compared to Unix. IN SAPTeched's all sessions talks about Windows and Unix not iSeries. If we ask queries no one is aware. If SAP gives equal importance to i5 OS then the person who is delivering the session should be aware. Any services like near zero downtime service which was launched last year was first delivered for Windows and Unix customers. iSeries road map was project as not clear.

-

---

A2:

IBM assumes equal SAP performance on AIX and IBM i , as both OSs are running on identical POWER hardware platform. Consequently, the IBM SAP sizing and configuration guidelines are identical for AIX and IBM i , and SAP solution landscapes are designed identically for AIX and IBM i. Also, IBM does run SAP benchmarks on both POWER AIX ( for SAP SD ), and on POWER IBM i ( for SAP BW ). But for economic reasons IBM does not run both benchmarks on both OSs.

-

---

R2:

I do not agree with your statement that SAP performance is same on both AIX and IBM i. Both OS architecture is entirely different. Given a normal PC if we install Windows 2000 or Windows XP or Windows 2003 all will have different performance matrix. Neither in TPC nor in SAP benchmarks I could see i5 OS.

-

---

A3:

As commented earlier by IBM Rochester , both functions, DB compression, as well as AME, have their pros and cons. They do provide advantages in specific usage scenarios , but also bring along their drawbacks, like e.g. additional system management , higher demand for hardware, or slower response times, in certain situations. Please see SAP Note 980067 on the effects of DB compression , just as an example.

The IBM i DB2 development team is working to provide alternative enhancements, other than compression, e.g. to better manage variable length and LOB data, which is where most of the compression would take place.

While IBM i does not provide DB compression or AME, it does offer advantages in other important areas , like native integration of DB2 into the IBM i operating system, its unique "Single Level Storage" concept for low-effort DB management, or its unique "Subsystem" feature , which allows perfect self managing CPU and memory virtualization for SAP workload - just to name some key benefits. Our existing SAP-IBM i customer base widely confirms these benefits in the feedback we receive in regular user group meetings and in numerous SAP-IBM i customer references across the world.

-

---

R3:

I have read through the document that if you enable compression there will be increase in CPU and memory usage by 10 %. But if the overall CPU and memory usage is less by migrating to AIX there itself there will be an initial advantage. After that we will slowly introduce compression.

Apart from all these if an expert from IBM competency center can analyze our existing system and help us to identify this huge usage of resources that would be really helpful. We still haven't got any concrete answer from any of the experts . That is the reason why we are looking for a change.

Edited by: Harish Madhavadas on Aug 4, 2011 6:14 PM

Edited by: Harish Madhavadas on Aug 4, 2011 6:15 PM

Edited by: Harish Madhavadas on Aug 4, 2011 6:56 PM

[Former Member]

Experts, Awaiting for your reply pls

[Former Member]

Hi Harish,

first I think, it was not easy to see, that you replied anything in this long threat as it was not well formatted by the forum software because of its size. I read the reply just yesterday, as I didn't detect anything "changed in there" earlier and just read your first line.

As I'm not from IBM or SAP, I can only support you with consulting on this.

The first questions that I would have for your landscape would be as follows:

You are telling on 1000 users.

You are having 7 Power 7 CPUs and 93GB RAM ?

How much diskspace are you using ?

Are you using other applicication servers like windows ? If yes, how many and how is is the load there ?

How high is your average CPU in the peak hour typically ?

How many page faults (DB & Non-DB) do you have then ?

What is the diskarm utilization in that hour ?

How large is your R3sidDATA ?

What are typical figures of your system for the "peak 15 minutes" or "peak hour" of the wrksyssts in left upper corner ?

% CPU used ... . . . . . : 41,5

% DB-Capacity . . . . . : 18,1

What is the average responsetime in ST03 for that hour ?

You are on ECC 6 ?

Unicode ?

Regards

Volker Gueldenpfennig, consolut international ag

http://www.consolut.com http://www.4soi.de http://www.easymarketplace.de

[Former Member]

Thanks for your Reply

We have 1000 users using the system.

Yes we are having 7 Power 6 CPUs and 93GB RAM.

Average CPU in peak hours is 85 %.

DB fault is around 1000 and non DB fault is around 1500.

Disk arm utilization will be around 60 to 80 %.

R3SIDdata is 2TB.

Typical figures of our system for the "peak

% CPU used . . . . . . . : 94.3

% DB capability . . . . : 184.5

average response time in ST03 for that hour is 2078

Yes we are using ECC with Unicode.

Regards

Harish

[Former Member]

Hi Harish,

these numbers sound like horror ;-((

I could understand all numbers except the following ones:

% CPU used . . . . . . . : 94.3

% DB capability . . . . : 184.5

Might this be a peak of "2s" or so ? I was asking for a period of e.g. 5 -15 minutes during your peak time - that result would be very interesting ...

So, you are using no windows application servers ?

Regards

Volker Gueldenpfennig, consolut international ag

http://www.consolut.com http://www.4soi.de http://www.easymarketplace.de

[Former Member]

Hi

We are using 6 Windows 2003 ENT Edth Application servers with configuration Dual Quad core processor with 32GB RAM.

% CPU Utilization for 15 Mins

89

84

85

% Database capacity for 15 mins I cant get from the performance monitor. The number what I had shown is that particulat instance.

Harish

[Former Member]

Hi Harish,

I'm sorry, but I was asking for the following numbers on a 15-min level and cannot see any result in yout posting here:

% CPU used . . . . . . . : 94.3

% DB capability . . . . : 184.5

As you are using 6 Wintel-Servers in your environment, it would be interesting how high the CPU load would be there - my guess would be: very small ...

Regards

Volker Gueldenpfennig, consolut international ag

http://www.consolut.com http://www.4soi.de http://www.easymarketplace.de

[Former Member]

Hi Volker

% CPU Utilization for 15 mins in 5 mins interval time 89 , 94, 90 , 92

% DB Capability for 15 mins in 5 min interval time 75, 85, 82, 85

Windows application servers utilization is below 30% always.

Regards

Harish

[Former Member]

Hi Harish,

ok, that sounds really like a horrible application ...

You are using 6 dual quad cores with let's say 25% CPU load. So, you seem to use about 6 * 2 * 4 * 0,25 CPUs = 12 CPUs Windows. This is just the application, that is running in ABAP on Windows.

Additionally, you are having 7 P6 CPUs, that are more or less exhaused - and more or less from DB effort only. So, you cannot expect, that the load changes on the wintel site, as this would be the same with each DB. On the iSeries this could be a bit high, but might be normal as well.

So, I would see most likely the root cause in the high application load on your system.

If you were interested, we could discuss this offline.

Regards

Volker Gueldenpfennig, consolut international ag

http://www.consolut.com http://www.4soi.de http://www.easymarketplace.de

[Former Member]

Hi Harish,

Even the disk arm utilization looks very high for your system.

What type of Cache Controller card are you using on your system? How many disks and how many arms writing to it??

When you say 1000 users, do you mean total users or concurrent users?? I would assume total as 1000 concurrent users will be too much....

Also when you say "Power 6 machine with 7 cores and 93GB RAM ", do you mean 7 physical processors associated with this LPAR or are you referring to virtual Processors, which you see in SAP?

Is this the only SAP instance running on this Frame or are their other partitons??? How is the CPU shared on the HMC?

Dedicated or Shared??

We run our SAP for almost 800 users and we use ECC6.0 end to end ( for everything almost) and we run it in 2.1 Physical CPU and 23 GB's of RAM with a response time of less than 400 ms...

How is your machine pool like on the system???

There are a lot of variables on this open ended question, you have put through...

I handle both AIX and system i and have not seen any difference on any of them in terms of performance...

Infact at times I love system i at times, when I do not have to worry about file system as in AIX...

Let us know what you configurations are and maybe someone can shed some light...

Have you tried one of the SAP early watch sesions??

Thanks

Abhi

[Former Member]

Thanks for your reply

Our disk subsystem is on SAN. DS8700.

Hi Abhi

Yes I mean 7 physical processor with 93GB BAM.

Total users are 1000 not concurrent.

Machine pool was 10% of the Base tool till we updated the latest PTF. But after the PTF I had found that the machine pool is not getting used. So I had reduced the machine pool by 5%.

This is the only SAP instance which is running on the system.

We have already done Early watch session and we could not get any major factors which could improve our system.

The memory features and compression features available in AIX are not available with System i. So how can both operating systems work similarly if both has different architecture. I feel this is impossible.

What are the configuration you require.

Regards

Harish

[Former Member]

Hi Harish

Now this is where it gets interesting. you are running V5R4 and are using SAN attached Disk,

I am not sure how many customer run V5R4 and use SAN. I would think majority use local disk on V5R4.

With V6R1, I am sure the numbers have gone up, but would be interesting to see that comparison.

Wonder why your disk arm utlization is so high....The storage is in the SAN and that means you need to get the SAN sizing as well. What else is attached to the SAN??? Do other applications write to the SAN, which also have a performance issue?

If you are IPLing from SAN, what IOP/IOA do you have? FC # 2847/FC # 5760 ??

I would like to see how the SAN sizing results. That disk arm % seems high in your case and could be the possible cause of an issue.

Thanks

Abhi

[Former Member]

Hi Abhi

It has been recommended by IBM to use SAN to achieve DR while we are running V5R4. We have followed all the guidelines by IBM to size the SAN Storage. We are using FC # 2847 for booting from SAN.

By adding disk arms we had brought down the disk arm utilization, but still the problem persist. We have other SAP and Non SAP iSeries applications accessing SAN. We have got the SAN configured from IBM experts and the configuration was vetted many times.

We are in the process of testing V6R1.

Regards

Harish

[Former Member]

Awaiting reply from Experts

[Former Member]

In my experience, I5 OS offers very much performance that UNIX. The I5 OS can install in POWER 7 machine and SAP Support this installation.

Please explain more in detail your performance issue. Maybe is only adjust the instance.

Regards,

William Neira

[brian_walker]

You won't find any standard SD benchmarks for iSeries or zSeries after the year 1998 or so. I suspect the reason for that is because the price/performance ratio is not favorable for comparison to other platforms that SAP can run on. It is certainly not the case that IBM finds the benchmarks meaningless or not worthwhile since they continue to get xSeries and pSeries SAP systems certified with the standard SD benchmark. I don't think the issue is performance, but rather the cost of performance parity vs alternative platforms.

Brian

[Former Member]

Our ECC 6.0 runs on i5 OS V5R4 Power 6 machine with 7 cores and 93GB RAM serving 1000 users. We have 6 application servers with 8 core Windows 2003 servers with 32 GB RAM. Which ever consultant who has seen our landscape was surprised about the amount of Hardware resources which we have provided to serve 1000 users. We have gone live with only 30 % of our business on SAP. Management is not willing to go live with other modules due to investment. So the objective in front of us was price Vs performance. Studying all the benchmarking results, the number of customers using the unix platform and the way IBM and SAP is supporting DB2 for Unix with new features we are of the opinion that SAP works better on AIX platform with DB2 as Database. There are case studies where SAP themselves have moved their servers to DB2 for Unix /Windows.

Harish