Considerations for configuration of Virtual Machines (VMware/Xen/...)?

[Former Member]

Hello all,

since virtualization is pushed more and more nowadays, the SAP Linuxlab will face discussions and questions regarding technologies, sizing, system parameters, etc.

I'd like to ask all people here, regardless if you use Linux or not:

<b>How would you implement virtualization technologies at your company?</b>

Would you plan to configure your environment in a way, that all virtual machines are configured equally to save money and work time having only one image to maintain, e.g. 2 virtual CPU's and 8GB of memory? Or would you size every virtual machine in another way? How would you attach the storage at your site, FC or maybe NFS? What kind of system would you virtualize? Only DEV or QA systems, or even PROD systems as well?

The advantages of virtualization are obvious. Having only virtual machines, a hardware upgrade can be done without downtime. You simple do a live migration on the new hardware and shutdown the old one afterwards. Having pre-configured systems at hand, you can provide DEV/QA systems in a matter of seconds.

The reason why I'm so interested and excited about this topic is, that we (the Linuxlab) want to provide a guideline or whitepaper for the upcoming Xen virtualization technique, thus your valuable input is needed. Writing papers without feedback from the real world is senseless

So, now I'm waiting for your answers ,

Cheers, Hannes

[nelis]

BUMP

I'd thought I'd make this thread active again. Is there any "white paper" or documentation available now since this thread was first created ?

We're looking to move our SAP development and training systems into a virtualized environment and I am interested in what the best route to take would be, VMWare, XEN or other ?

From reading note 962334 XEN is not fully supported yet. Also, if running SAP in a virtual machine requires you to have a "fixed memory allocation" for the database, "memory reservation" and "cpu reservation"(refer note 1056052) or require resources which seem to have little benefit of it running in a virtual environment, then what exactly are the benefits ? I can't see any costs savings to be honest but perhaps I'm missing something.

I'm not exactly fond of the idea of running SAP in a virtual environment but I'm tasked to investigate whether it would be a viable option. I'm looking for ammunition basically and hoping someone can convince me it's the "way to go".

Thanks and regards,

Nelis

[hannes_kuehnemund]

Hey Nelis,

I almost forgot about this thread, thanks for reactivating. And of course there are some news coming soon and I also have some remarks on the notes you mentioned. But let us first answer some of your questions and concerns

> Is there any "white paper" or documentation

> available now since this thread was first created ?

Not yet, but we are in the middle of a certification process. When the certification process is finished, there will be a white paper available. At least for Xen (I'll probably the person who's going to write a paper like this).

> From reading note 962334 XEN is not fully supported yet.

Right. Nevertheless, when starting your project now, we may be able to give you a individual release statement for SAP on Xen. If so, please open a customer message on component BC-OP-LNX and ask directly for Hannes Kuehnemund / Dev Support. We'll handle the things left in the official call then.

> Also, if running SAP in a virtual machine requires

> ou to have a "fixed memory allocation" for the

> database, "memory reservation" and "cpu reservation"

> (refer note 1056052) or require resources which ...

Please take into account, that this information is for VMware only. They do not apply for Xen. We are currently conducting tons of test cases with Xen to determine the borders which shouldn't be crossed. For what I can say now, the limitations for Xen are far beyond these for VMware.

> I'm not exactly fond of the idea of running SAP in a virtual

> environment but I'm tasked to investigate whether it would

> be a viable option. I'm looking for ammunition basically and

> hoping someone can convince me it's the "way to go".

We may had the same intention some years ago, but Linux on Power (running in LPAR's) or Linux on System z (z/VM) is already running virtualized and many customers use it productively already. Xen and VMware are just two more players, mainly on the mainstream hardware platform. As Xen will not have such restrictive limitation as VMware, you can definitely count on Xen to be a platform for SAP in the future.

If you have more questions, you can contact me directly, my E-Mail address is listed in my Business Card.

Thanks,

Hannes

[nelis]

Hi Hannes,

Thank you for your reply. Please can I ask you to reply in this thread, which I will be watching, when you have documentation available for Xen.

In the mean time I think I must setup a test system in a Xen environment just to familiarize myself with it and see how the whole thing fits together.

Regards,

Nelis

[markus_doehr2]

We are using virtualization in our (also production) environments - unfortunately not (yet) on Linux but on Solaris X86_64 using Solaris zones.

We have several big boxes (8-way Opteron (4 x DualCore), 24/32 GB RAM) and share various systems (Test, QA, Sandboxes, ABAB, Java, ABAP+Java) on those virtualize environments using resource control techniques of the Solaris OS (shared memory sizes, Fair Share Scheduler/cpu-shares etc.)

We used that approach in contrast to the current "Blade-Hype" since it's much easier to provide additional resources if needs arise (e. g. initial downloads on a CRM system or big BI-rebuild tasks over night). It's a matter of one command to provide more CPU to a system instead of "moving" it over to a second box - which will finally produce a disruption.

We discussed internally a lot whether it would be better to have a number of different size blades and use e. g. Adaptive Controller technologies to get to the point but finally we decided to use a virtualized big box instead of many small ones.

All around we provide about 20 instances on three of those boxes, including non-SAP components such as LDAP or RemoteAccess solutions for the internal environment.

Solaris Zones are different than Xen, VMWare or Virtuozzo because there is only one kernel running and only one "instance" of all programs for the whole system.

Setting up another zone for a SAP system is a matter of some minutes because you provide the necessary environment ONE TIME (C/C++-Runtime, Java) and this is just "used" instead of installing a whole operating system again. This makes that environment very flexible, however, with the drawback, that you will patch "everything at once" if you install an OS patch.

Storage is shared on several FC LUNs (/usr/sap, DB-DATA, DB-LOG, ZONE-root) with all the advantages of ZFS (clones, snapshots, detach/re-attach etc.) without having the need of other expensive filesystem solutions (e. g. VxFS). The performance drawback of using a "shared LUN" is less than 5 %.

NFS did NEVER come to my mind, I will never understand, why people run databases over NFS. Not blaming those who do it - I'm sure it works, but I wouldn't want to neither setup nor run such an environment (personally).

I will certainly love to evaluate Xen - with all the differences, pros and cons and if it's stable and running I wouldn't hesitate to put it into production.

I am a Linux addict, although the statement above doesn't give that impression. If Linux would have been that far at the time of installing those instances, we would have choosen that. VMware was never under discussion.

The next thing we will evaluate (beside Xen) is the possibility of installing a "Brandz"-Zone - a Linux OS (Redhat, CentOS) under the control of a Solaris Zone.

--

Markus

[hannes_kuehnemund]

Hi Markus,

first of all, thank you very much for your detailed answer so far. As the Xen certification for productive use is upcoming soon, I still have some question left

When reflecting the layout of your system landscape, 3 big AMD boxes having 8 cores each, running 20 system overall, what is the normal system "size"? I guess, for small systems you configure one CPU and 4GB of memory, and for the bigger systems up to 4 CPU's with 16GB RAM?

As being in the phase of evaluating certification rules for pre-defined virtual systems we need the "size" of these systems. From your posting I would guess, that there are no system with more then 6 or 8 virtual CPU's.

At the moment two different VM setups are on the certification road: 2CPU and 8GB of memory and 4CPU and 16GB of memory.

We will also recommend the use of an FC storage (maybe iSCSI) or a external SCSI shelf with a decend RAID level, because testing showed a bad performance when using an internal disk for several virtual machines.

Would you use the Xen feature of live migration in your productive environment (this would allow hardware replacements without shutting down the running virtual systems), or is the dynamic resource allocation benefit enough?

Thanks,

Hannes

[markus_doehr2]

> Hi Markus,

>

> first of all, thank you very much for your detailed

> answer so far. As the Xen certification for

> productive use is upcoming soon, I still have some

> question left

you're very welcome

>

> When reflecting the layout of your system landscape,

> 3 big AMD boxes having 8 cores each, running 20

> system overall, what is the normal system "size"? I

> guess, for small systems you configure one CPU and

> 4GB of memory, and for the bigger systems up to 4

> CPU's with 16GB RAM?

we don't "divide" the systems into core and memory.

Each of the virtual systems sees the full system (CPU, memory) so e. g. Java and the MaxDB kernel can be configured to use all 8 CPUs/cores. Resource management is then done using "give instance A 20 cpu shares and instance B 30". If a system is running too slow we just add some more CPU shares.

For memory, we don't configure the virtual boxes themselves but the applications running on them (ABAP buffers, Java heap sizes, CACHE_SIZES of the database etc.) This will enable the system to perform well and systems, that are rarely used are still running but consuming less memory (ABAP buffers not filled etc.) With that we can make sure, we don't have any virtual systems consuming memory and doing nothing, which would be basically the same as having physical systems running and doing nothing. All zones share everything and you control them via resource controls.

[...]

>

> We will also recommend the use of an FC storage

> (maybe iSCSI) or a external SCSI shelf with a decend

> RAID level, because testing showed a bad performance

> when using an internal disk for several virtual

> machines.

absolutely. We consider internal disks as operating system disks or even just as swap disks.

>

> Would you use the Xen feature of live migration in

> your productive environment (this would allow

> hardware replacements without shutting down the

> running virtual systems), or is the dynamic resource

> allocation benefit enough?

For our environment it would be enough to use resource controls dynamically, however, this can be a nice feature for future cases.

Get back to me if you want more info.

--

Markus