[Leaplist] 3Ware RAID Management -- WAS: Lots of 5-10 3Ware
(hardware RAID)
Bryan J. Smith
b.j.smith at ieee.org
Thu Feb 28 19:44:00 EST 2008
"Bryan J. Smith" <b.j.smith at ieee.org> wrote:
> 3Ware continues to support the 7000/8000 series, including their
> new 3DM2 "GUI" (web) management tool designed for the 9000 series
> (9.x) releases for kernel 2.6 supporting older 7000/8000 cards.
The
> older 3DM software (7.x) for kernel 2.4 also works.
BTW, the 3DM2 9.5 release may work on kernel 2.4 as well, I've just
never tried it. The 3DM2 9.5 release is required for 2.6 and its new
IOCTL support. You do _not_ have to run 3DM2, it's an intelligent
controller that uses its driver for everything (including rebuilds).
It's just very nice to have, with e-mail and other notification,
etc... It uses dmesg/syslog/etc... facilities like an intelligent
controller normally would.
Here's more on 3Ware's own management and 3DM2:
http://www.3ware.com/products/raid_management.asp
http://www.3ware.com/products/pdf/3DM2_DS_041904.pdf
It's web-based with a dead-simple installer. Installs the service,
starts up on port 888 (HTTP-SSL) by default -- two user types, user
(view) and admin (control). You can bring drives and arrays up/down
_without_ panicing the damn kernel, which is a Godsend (unless you
use software RAID atop of 3Ware, that's where 99.9% of the complaints
come from which have nothing to do with 3Ware, long tangent).
Also, there is standard, Linux SMART support (smartctl) outside of
3Ware's own tools as well. The nice thing about an intelligent
controller is that it has "persistent memory" of events (between
reboots), unlike software-based SMART. I can look at any "event"
going back years (yes, I've done this on my home server with an
8-channel 7000 series ;).
>From the smartctl man page:
-d TYPE, --device=TYPE
Specifies the type of the device. The valid
arguments to this option are ata, scsi, mar-
vell, cciss,N and 3ware,N. If this option is
not used then smartctl will attempt to guess
the device type from the device name.
Under Linux, to look at SATA disks behind
Marvell SATA controllers (using Marvell’s
´linuxIAL´ driver rather than libata driver)
use ´-d marvell´. Such controllers show up as
Marvell Technology Group Ltd. SATA I or II
controllers using lspci, or using lspci -n
show a vendor ID 0x11ab and a device ID of
either 0x5040, 0x5041, 0x5080, 0x5081, 0x6041
or 0x6081. The ´linuxIAL´ driver seems not
(yet?) available in the Linux kernel source
tree, but should be available from system
vendors (ftp://ftp.aslab.com/ is known to
provide a patch with the driver).
To look at ATA disks behind 3ware SCSI RAID
controllers, use syntax such as:
smartctl -a -d 3ware,2 /dev/sda
smartctl -a -d 3ware,0 /dev/twe0
smartctl -a -d 3ware,1 /dev/twa0
where in the argument 3ware,N, the integer N
is the disk number (3ware ´port´) within the
3ware ATA RAID controller. The allowed val-
ues of N are from 0 to 15 inclusive. The
first two forms, which refer to devices
/dev/sda-z and /dev/twe0-15, may be used with
3ware series 6000, 7000, and 8000 series con-
trollers that use the 3x-xxxx driver. Note
that the /dev/sda-z form is deprecated start-
ing with the Linux 2.6 kernel series and may
not be supported by the Linux kernel in the
near future. The final form, which refers to
devices /dev/twa0-15, must be used with 3ware
9000 series controllers, which use the
3w-9xxx driver.
Note that if the special character device
nodes /dev/twa? and /dev/twe? do not exist,
or exist with the incorrect major or minor
numbers, smartctl will recreate them on the
fly. Typically /dev/twa0 refers to the first
9000-series controller, /dev/twa1 refers to
the second 9000 series controller, and so on.
Likewise /dev/twe0 refers to the first
6/7/8000-series controller, /dev/twa1 refers
to the second 6/7/8000 series controller, and
so on.
Note that for the 6/7/8000 controllers, any
of the physical disks can be queried or exam-
ined using any of the 3ware’s SCSI logical
device /dev/sd? entries. Thus, if logical
device /dev/sda is made up of two physical
disks (3ware ports zero and one) and logical
device /dev/sdb is made up of two other phys-
ical disks (3ware ports two and three) then
you can examine the SMART data on any of the
four physical disks using either SCSI device
/dev/sda or /dev/sdb. If you need to know
which logical SCSI device a particular physi-
cal disk (3ware port) is associated with, use
the dmesg or SYSLOG output to show which SCSI
ID corresponds to a particular 3ware unit,
and then use the 3ware CLI or 3dm tool to
determine which ports (physical disks) corre-
spond to particular 3ware units.
If the value of N corresponds to a port that
does not exist on the 3ware controller, or to
a port that does not physically have a disk
attached to it, the behavior of smartctl
depends upon the specific controller model,
firmware, Linux kernel and platform. In some
cases you will get a warning message that the
device does not exist. In other cases you
will be presented with ´void´ data for a
non-existent device.
Note that if the /dev/sd? addressing form is
used, then older 3w-xxxx drivers do not pass
the "Enable Autosave" (´-S on´) and "Enable
Automatic Offline" (´-o on´) commands to the
disk, and produce these types of harmless
syslog error messages instead: "3w-xxxx:
tw_ioctl(): Passthru size (123392) too big".
This can be fixed by upgrading to version
1.02.00.037 or later of the 3w-xxxx driver,
or by applying a patch to older versions. See
http://smartmontools.sourceforge.net/ for
instructions. Alternatively, use the charac-
ter device /dev/twe0-15 interface.
The selective self-test functions (´-t
select,A-B´) are only supported using the
character device interface /dev/twa0-15 and
/dev/twe0-15. The necessary WRITE LOG com-
mands can not be passed through the SCSI
interface.
3ware controllers are currently ONLY sup-
ported under Linux and FreeBSD.
cciss controllers are currently ONLY sup-
ported under Linux.
--
Bryan J. Smith Professional, Technical Annoyance
b.j.smith at ieee.org http://www.linkedin.com/in/bjsmith
------------------------------------------------------
Fission Power: An Inconvenient Solution
More information about the Leaplist
mailing list