1. examples
  2. booting off raid1
  3. updating mdadm.conf
  4. hot swap scsi
  5. when device names change
  6. an example recovery scenario

This page only deals with using the tool mdadm, not the raidtab stuff.

examples

Creating RAID devices:
mdadm --create /dev/md4 --level=1 --raid-devices=2 /dev/sd[ab]8
mdadm --create /dev/md4 --level=5 --raid-devices=3 /dev/sd[abc]8

Stopping RAID:
mdadm --stop /dev/md1

Removing a physical device:
mdadm --fail /dev/md4 /dev/sdc8
mdadm --remove /dev/md4 /dev/sdc8

Adding it back in:
madm --add /dev/md4 /dev/sdc1

booting off raid1

Grub works with software RAID1, so long as grub is installed on each disk.

But there is a problem. Lets say you have sda and sdb. If sdb fails, then if grub is installed on sda then it should boot fine. However, if sba fails, then sdb typically becomes sba. If grub was installed on sdb while it was mapped to device hd1, it will not be able to boot correctly (since there is no longer a hd1).

The trick is to make grub think that it is installing on (hd0) when it is installing on sdb.

# grub
grub> device (hd0) /dev/sdb
grub> root (hd0,0)
grub> setup (hd0)

see also: lists.us.dell.com/pipermail/linux-poweredge/2003-July/014331.html

updating mdadm.conf

If you create a new raid device debian will not notice it on boot unless you also update the file /etc/mdadm/mdadm.conf.

cd /etc/mdadm
cp mdadm.conf mdadm.conf.`date +%y%m%d`
echo "DEVICE partitions" > mdadm.conf
mdadm --detail --scan >> mdadm.conf

Then look at the file to see if it look right to make sure there isn't extra stuff in there.

Here is an example:
DEVICE partitions
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=6b8b4567:327b23c6:643c9869:66334873
   devices=/dev/sda1,/dev/sdb1
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=6b8b4567:327b23c6:643c9869:66334873
   devices=/dev/sda5,/dev/sdb5
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=6b8b4567:327b23c6:643c9869:66334873
   devices=/dev/sda6,/dev/sdb6
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=6b8b4567:327b23c6:643c9869:66334873
   devices=/dev/sda7,/dev/sdb7
ARRAY /dev/md4 level=raid5 num-devices=3 UUID=b1bfcde9:088dc404:2b4bed20:2f1c5da5
   devices=/dev/sda8,/dev/sdb8,/dev/sdc8

hot swap scsi

Hot swapping SCSI drives is not directly related to software RAID, but it is a common task when working with RAID.

To remove a hot swap drive:
# echo "scsi remove-single-device 0 0 X 0" > /proc/scsi/scsi

To add a hot swap drive:
# echo "scsi add-single-device 0 0 X 0" > /proc/scsi/scsi

Where X, starting at 0, is the drive number. The parameters may differ (the channel number might not be zero, for example). cat /proc/scsi/scsi first to see what makes sense.

when device names change

Sometimes when you upgrade a kernel, the names of the devices which compose the array will change. This makes it so that the array can't be constructed at boot.

You can tell this is the case by examining /proc/partitions and comparing it to /etc/mdadm/mdadm.conf.

The re-assemble the array, simply run this command for each array:

/sbin/mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1

Hopefully, root will still mount even if all the other partitions don't. If you have a root, you will probably have the command /sbin/mdadm.

In this case, we are re-assembling md0 from sda1 and sdb1. Once you have assembled the raid device, mount it to make sure it is right. When you are satified, create a new mdadm.conf using the method listed above.

an example recovery scenario

Two RAID1 devices:

md0  sda1 sdb1  /
md1  sda2 sdb2  /var

Lets supposed disk sda dies!

  1. You take out dead sda and reboot. Now all the raid devices are running in degrated mode and sdb has become sda. SCSI likes to name the disks in the order they are encountered, which is weird. So, even though we lost sda, we reboot and it will show only sda in the raid devices.
  2. Then you hotadd a new disk in the first slot. Since there is already an sda this becomes sdb. So we have the weird situation where sda and sdb are switched. If we reboot, they will return to their normal order (ie the first scsi slot will be sda).
  3. Partition the new disk:
    cat /var/backups/partitions.sda.txt | sfdisk /dev/sdb
    You read that right! We use old sda's partition table to partition the new sda, which is currently sdb.
  4. Rebuild the raid:
    mdadm --manage /dev/md0 --add /dev/sdb1
    mdadm --manage /dev/md1 --add /dev/sdb2
    So now our RAID devices have sda added as sdb, and sdb added as sda. It turns out, it doesn't matter.
  5. Reboot. The devices revert to their normal order, and the RAID devices work fine.