[Leaplist] [2/2] CentOS install -- Fake RAID (FRAID) and DeviceMapper (dmraid)

[Bryan J. Smith]

[ Breaking this up into two (2), separate responses ]

Hank Lambert wrote:  
> I tried to install CentOS 3 times yesterday before getting
> a good install.  The machine was an XP machine, with 2 IDE
> hard drives in RAID 0.

This is most likely "Fake RAID" (FRAID).

I.e., the mainboard firmware has 16-bit BIOS Extended Int13h Disk
Services for boot-time support.  But once the 32/64-bit OS kernel
loads, they are useless, and you need a 100% software driver to take
over.

E.g., when you installed Windows XP, you pressing F6 at install-time
(unless the FRAID driver was included by Microsoft).

What a FRAID driver does is read the independent disk organization
and allow the OS to do various operations if it was a real, hardware
RAID controller (which it is _not_ -- despite being on-mainboard). 
The driver controls 100% of the operations.

Under latter Linux 2.6 kernel releases, DeviceMapper RAID (dmraid)
controls detection of FRAID organization.  Once dmraid "maps" the bit
layout, the Linux software RAID routines are then used.  RHEL 4
(2.6.9) does not contain this support, but RHEL 5 (2.6.18) should. 
The Red Hat Anaconda installer should detect the FRAID organization
once dmraid loads, and it will setup the initial RAM/root disk
(initrd) image file with the dmraid and Linux software RAID support.

nVidia MCP RAID-0/1 are supported perfectly, and most Intel ICH units
are as well.  RAID-5 on either may be iffy.  I've never tried ViA,
but I generally dislike ViA's ATA support in Linux.

Instead of /dev/hda (or /dev/sda) and /dev/hdb (or /dev/sdb), you'll
see /dev/mapper or /dev/dm* or possibly a vendor-specific /dev/nvmap*
or other device (actually, you may see variations of all three). 
That's the dmraid setting up remapped devices so the kernel can use
the organization.

> I did not select RAID on this install,

What do you mean "I did not select RAID on this install"?
Did you mean you disabled it in the BIOS?

In any case, if the disks have had a previous FRAID organization on
them, the Red Hat Anaconda installer (RHEL 5) will detect that and
load dmraid.  Once dmraid is loaded, it will stripe (RAID-0), mirror
(RAID-1), stripe+mirror (RAID-10), etc... as Anaconda detected.

The problem that will result when you reboot, the BIOS and installed
images will differ.  In the case of RAID-0, the BIOS will choke on
the hand-off and fail to load anything (GRUB won't reach mid-stage
1), because the data is striped over two disks (when the BIOS assumes
only disk 1).

> but for some reason, the CentOS install was identifying the
> drives wrong.

Yes, because the Anaconda installer saw the previous FRAID-0
organization and used it.  To prevent that, in addition to disabling
the RAID setting in the BIOS, you need to wipe the disks and "break
the stripes."  Some FRAID firmwares can do that at the BIOS/POST
screen.  Some rely on their own software in Windows to do it.

Understand the "detection" is actually a feature.  E.g., I can take
my disks off of my 3Ware hardware RAID controller and put them in a
system with a non-3Ware device and the Linux kernel can still read
the RAID volumes and get at my data (boot from them is a different
story, obviously).  That's the power of DeviceMapper, which is an
evolving, flexible set of capabilities in kernel 2.6.  DeviceMapper
RAID (dmraid) is just one of them.

> So I unplugged the slave hard drive, and performed another
> installation.  This time, during the installation, it displayed
> the hard drive correctly.

Yep.  Because Anaconda still noted it was a FRAID-0 volume, but the
other disk in the stripe was missing.  It assumed it was broken and
unusable (if you lose any disk in RAID-0, it's a toasted volume), so
I assume it prompted you to destroy the disk at some point.

> The install finished, and I rebooted into the server.

Lesson:  

If you use FRAID, understand what you're getting into.

Under Windows XP/2003, this requires you to have the "driver disk"
and hit "F6" at boot.  Otherwise its installer just panics with "no
hard drive."

Under newer Linux 2.6 kernels with dmraid, it will detect RAID
organization.  You either have to have all disks _and_ the BIOS
enabled so it can boot, or you have to disable the setting in the
BIOS and get rid of the organization so Linux won't see it.

Again, unlike Windows (which _destroys_ or at least "looses" FRAID
volumes on the slightest case of "lack of driver" -- I've seen that
too many times), Linux's DeviceMapper RAID (dmraid) is extremely
intelligent and can "detect" FRAID volumes, even when they are no
longer on the original controller with that FRAID driver's
assumptions!

It's an outstanding feature, but you have to know what you're doing. 
If you don't want to deal with it, get a $120, 2-channel SATA 3Ware
9650SE-2LP (PCIe x1 true hardware RAID).


-- 
Bryan J. Smith       Professional, Technical Annoyance
b.j.smith at ieee.org  http://www.linkedin.com/in/bjsmith
------------------------------------------------------
       Fission Power:  An Inconvenient Solution