CI on Power/AIX, DI on Intel/Linux RHEL

[Former Member]

Hi,

We are in the middle of an ERP 6.0 migration from R/3 4.6C. We're currently running 7 SAP landscape mostly on IBM Power 6 under AIX / Oracle.

We have resized the DB/CIs to handle the ERP6/Unicode overhead however we still haven't changed DI servers, some of them are still old RS64-IV 750 Mhz machines (about 20).

In order to reduce the TCO and free some pSeries partitions, we're planning to move all our DIs to Linux RHEL on Intel X86-64 boxes (IBM x3650 M2 or HP DL580 G5)

Does somebody have already done such a move ? What are the risks ? Are there compatibility, performance, other issues, ... ?

Your experience will be greatly appreciated

Thanks

[stefan_koehler · Accepted solution]

Hello Serge,

> Does somebody have already done such a move ?

We tried, but stopped it after some major problems (SAN implementation on Linux)

> Are there compatibility, performance, other issues, ... ?

The SAP system and the oracle database itself works fine (please take care of sapnote #914177).

But in my experience it depends so much on the hardware combination, that you use.

If you just work with local disks .. everything is fine. But if you have a SAN environment it is getting worse. You have nearly no management tools (really needed for big systems with many SAN disks) and the SAN implementation with MPIO in the kernel is not very trustable and you have some limitations (load balancing via FC-adapter, etc.).

Also please keep in mind that you don't have such a nice LVM implementation on linux like you have it on AIX. Of course you can mirror your LVs, etc. but you can not extend/shrink them on-the-fly.

Just a few points that you should think of.

Regards

Stefan

[markus_doehr2 · Accepted solution]

> Also please keep in mind that you don't have such a nice LVM implementation on linux like you have it on AIX. Of course you can mirror your LVs, etc. but you can not extend/shrink them on-the-fly.

DIs on SAN with MPxIO?

Markus

[stefan_koehler · Accepted solution]

Hi Markus,

> DIs on SAN with MPxIO?

Yes, of course - why not?

For example in our environment nearly all servers are running in blade centers (except the big database instances). The blades have no local disks .. so you have SAN boot with 2 fc-adapters .. MPIO )

As i said in my opinion the whole linux stuff depends on the hardware components and the infrastructure.

Regards

Stefan

[markus_doehr2 · Accepted solution]

> > DIs on SAN with MPxIO?

> Yes, of course - why not?

> For example in our environment nearly all servers are running in blade centers (except the big database instances). The blades have no local disks .. so you have SAN boot with 2 fc-adapters .. MPIO )

I see

We have MPxIO activated and running on the main database servers without any issue. The setup was a bit... well - not-that-intuitive but it works. We don't use, however, blade systems but rack mount machines because in those servers I can stuck whatever FC-HBA

I have also made the experience that the vendor-branded-builtin-HBAs can sometimes a bit prissy about getting them to do what they should; one reason why we don't put SAP systems on blades (amongst other reasons).

> As i said in my opinion the whole linux stuff depends on the hardware components and the infrastructure.

Absolutely true - although the support for Linux improved a lot in the last two years.

Markus

[Former Member · Accepted solution]

Hi Markus

We won't use blades but rack servers, probably the new IBM x3650 M2 and storage on internal disks. We might have access to a 24 cores HP DL 580 G5.The IBM seems better to me for an app server. What you recommand ?

Could you please help me to see more clearly what might be the advantages and drawbacks of the solution ?

The management is of course seduced by the "lower cost" argument (Intel servers are 10 times cheaper than Power6 servers and probably faster too) but I've got to make sure we won't get into trouble with a heterogeneous solution.

Thank you

[markus_doehr2 · Accepted solution]

> We won't use blades but rack servers, probably the new IBM x3650 M2 and storage on internal disks. We might have access to a 24 cores HP DL 580 G5.The IBM seems better to me for an app server. What you recommand ?

We use HP hardware all around (no IBM here) - and we're pretty happy with it.

> Could you please help me to see more clearly what might be the advantages and drawbacks of the solution ?

A thing that comes immediately to my mind is "sort order" - which comes into place when your system is not Unicode. The locale implementations on AIX and Linux are different so your users may become slightly confused. Apart from that I'd switch the kernel directories to the new layout (/<sapmnt>/SYS/exe/nuc|uc/<platform>/) to make kernel patching easier.

AIX is big endian, Linux little endian, so if you run (binary) exports on both architectures you'd have to make sure, the target/receiving system is aware of the difference. This comes, however, only into place when your transfer data to non-SAP systems and those rely on the correct endianess.

If you run Java instances on AIX you might be familiar with the IBM JDK culprits. Linux x64 (x86_64) uses the IBM JDK too so that stays the same.

> The management is of course seduced by the "lower cost" argument (Intel servers are 10 times cheaper than Power6 servers and probably faster too) but I've got to make sure we won't get into trouble with a heterogeneous solution.

If you don't "trust" Linux that much you can also take Solaris x64 into regard (http://www.saponsolaris.com), has MPxIO builtin and is supported on the HP boxes.

Generally heterogeneous environments are fine, It's a bit more to pay attention to (e. g. kernel patching).

Markus

[Former Member · Accepted solution]

>

> > We won't use blades but rack servers, probably the new IBM x3650 M2 and storage on internal disks. We might have access to a 24 cores HP DL 580 G5.The IBM seems better to me for an app server.  What you recommand ?
> 
> We use HP hardware all around (no IBM here) - and we're pretty happy with it. 
> 
> > Could you please help me to see more clearly what might be the advantages and drawbacks of the solution ?
> 
> A thing that comes immediately to my mind is "sort order" - which comes into place when your system is not Unicode. The locale implementations on AIX and Linux are different so your users may become slightly confused. Apart from that I'd switch the kernel directories to the new layout (/<sapmnt>/SYS/exe/nuc|uc/<platform>/) to make kernel patching easier.
> 
> AIX is big endian, Linux little endian, so if you run (binary) exports on both architectures you'd have to make sure, the target/receiving system is aware of the difference. This comes, however, only into place when your transfer data to non-SAP systems and those rely on the correct endianess.
> 
You'll use Unicode so that shouldn't be an issue
> If you run Java instances on AIX you might be familiar with the IBM JDK culprits. Linux x64 (x86_64) uses the IBM JDK too so that stays the same.
> 
We'll have a portal 7.X (SRM7) and a PI 7.1 by next so I'd look at the JVM issues carefully.
> > The management is of course seduced by the "lower cost" argument (Intel servers are 10 times cheaper than Power6 servers and probably faster too) but I've got to make sure we won't get into trouble with a heterogeneous solution.
> 
> If you don't "trust" Linux that much you can also take Solaris x64 into regard (http://www.saponsolaris.com), has MPxIO builtin and is supported on the HP boxes.
> 
> Generally heterogeneous environments are fine, It's a bit more to pay attention to (e. g. kernel patching).
> 
> 
It's not that I don't "trust" Linux, it's just that when I say it may be the best time to consider a move from RISC to X86 or partly (AS only) with new architectures from Intel in particular, people look at me and say : "if what you say is true, all SAP customers would have already done it !"
> 
> Markus

Thank you

[hannes_kuehnemund · Accepted solution]

Stefan wrote:

... Of course you can mirror your LVs, etc. but you can not extend/shrink them on-the-fly.

Stefan, this is not true, using lvm2 on Linux you can online resize your logical volumes and use e2resize afterwards to adapt the filesystem size online too.

Also, you arguments regarding SAN management tools, you can still use them from the CI which runs on AIX. There is no need to do the SAN management from the DI Linux boxes.

Regards,

Hannes

[Former Member · Accepted solution]

Sorry I misused the quotes.

Your comments are welcomed

Thank you

[markus_doehr2 · Accepted solution]

> We'll have a portal 7.X (SRM7) and a PI 7.1 by next so I'd look at the JVM issues carefully.

Your Portal then uses an IBM JDK, the PI 7.1 comes with a Java 5.0 from SAP - this is no more an issue since SAP takes care of that

> It's not that I don't "trust" Linux, it's just that when I say it may be the best time to consider a move from RISC to X86 or partly (AS only) with new architectures from Intel in particular, people look at me and say : "if what you say is true, all SAP customers would have already done it !"

AFAIK more than 50 % of the customers doing "Wintel" + Oracle (I wouldn't do that but that's another story). So - x86 - be it Intel Xeon or AMD - can't be that wrong if you just count the numbers.

Markus

[stefan_koehler · Accepted solution]

Hello Hannes,

> this is not true, using lvm2 on Linux you can online resize your logical volumes and use e2resize afterwards to adapt the filesystem size online too.

i am sorry .. . please correct me what i have done wrong by extending mirrored LVs, that's what i was talking about.

shell> vgcreate vgtest /dev/sdb /dev/sdc
shell> vgdisplay
  --- Volume group ---
VG Name               vgtest
Format                lvm2
...
shell> lvcreate -L 4G -m1 -n mirrorlv --corelog /dev/sdb /dev/sdc

shell> lvdisplay -a
  --- Logical volume ---
  LV Name                /dev/vgtest/mirrorlv
  VG Name                vgtest
  LV Status              available
  # open                 2
...
...
 --- Logical volume ---
  LV Name                /dev/vgtest/mirrorlv_mimage_0
  VG Name                vgtest
 LV Status              available
  # open                 1
...
...
 --- Logical volume ---
  LV Name                /dev/vgtest/mirrorlv_mimage_1
  VG Name                vgtest
  LV Status              available
  # open                 1
...
...

shell> df
Filesystem           1K-blocks      Used Available Use% Mounted on
....
/dev/mapper/vgtest-mirrorlv
                       4706156     32840   4673316   1% /mnt
....

shell> lvextend -L+100M /dev/vgtest/mirrorlv
Extending 2 mirror images.
Mirrors cannot be resized while active yet.

I want to extend the logical volume "mirrorlv" (and the corresponding file system "/mnt") on-the-fly while the file system is available all the time. But this seems to be not possible.

> There is no need to do the SAN management from the DI Linux boxes.

You maybe misunderstood. I don't mean to manage the SAN itself from the linux box. I mean the SAN management on the linux box (scan for new disks while system is up, get serial numbers of the LUNs for mirroring the LVs, activating or deactivating pathes, etc.).

I am looking forward to your answers.

Regards

Stefan

[hannes_kuehnemund · Accepted solution]

Hi Stefan,

i am sorry .. . please correct me what i have done wrong by extending mirrored LVs, that's what i was talking about.

Ah okay, you meant to resize the mirror. I misunderstood you, looked for me like you mentioned that extend in general does not work.

Regarding SAN, I'm using an SGITP9300 (IBM DS4300? like) storage and I can create LUNs on the fly and assign them to the connected boxes. Of course, the tricky part is that you have booted your Linux with the correct HBA driver settings and additionally, the storage and the driver should work together.

So, as you have no vendor lock on commodity hardware with an commodity OS like Linux, you should take special care of what you are doing, especially compatibility wise. The advantage of such a vendor lock is the good integration of the OS with the proprietary hardware.

Thanks,

Hannes