On 6/15/24 09:05, Marco van Hulten wrote:
Hello,
I got a new amd64 system with 3 NVMe disks of each 2 TB, with the idea
to put them in RAID-5. I did not realise until now that one cannot
boot from RAID-5.
Would a good approach be to create a root device on one disk (and maybe
altroots on one or both of the others) and use the rest of all disks as
RAID-5 device? Or is there a good reason to boot from a disk separate
from the envisioned RAID-5 configuration?
I just set something up like this, myself. Four 4T disks. I wanted
redundancy but also recoverability.
My solution:
each drive has a 25G disklabel partition and a "almost rest of drive"
disklabel partition ("almost rest" because I'm paranoid about having
to someday replace the drive, and finding the new drive is a thousand
sectors smaller than the old drives. This hasn't been much of a problem
in my observation lately, but I'm old, I remember when Seagate shipped
two drives with the exact same model number, but the replacement drive
had one less cylinder than the original drive...not fun!).
The 25G partitions are in a four drive RAID1, and the "rest of drive"
partitions are in a RAID5 config. The base OS and all standard
partitions is in that 25G array, the "rest of drive" is all data storage.
So..if I lose a drive (or several), I should be able to boot at least
the core OS and get some idea what went wrong. If you need a larger
core OS system, go for it. I do NOT recommend putting just the root
partition on this drive. Make it stand-alone useful.
At this point, some of the kids start screaming, "you can't do a
four drive RAID1!". Yes you can. The fact that your HW RAID card
can't, doesn't mean it's an invalid concept. softraid (and at least
some other software RAID systems) handles >2 drives in a RAID1
config seemingly just fine. It's four copies of the same data.
Stunningly inefficient, not very fast for writes but very robust.
And, what else am I supposed to do with the 25G empty space on the
other drives, anyway? :) (a further benefit -- if I have to swap the
drives to another physical machine, ANY of the drives will able to be
booted, I don't have to make sure I get the right drive in the "drive 0"
position).
One big word of warning: when you have to replace a drive on a system
like this...rebuild one array than the other. You probably don't want
to have the system thrashing between the two partitions on the same disk;
that's a great way to turn a slow process into a glacial process (though
probably not so big a deal with SSDs as it is with spinning drives). So
when I test the drive replacement process, I plan to rebuild the OS
partition first (anticipated time: minutes), then the data partition
later (anticipated time: days).
And yes, I'm testing the behaviors of this thing and the drive replacement
process before I commit it to production.
Nick.
