Re: [GLLUG] Link two RAIDs in one LVM?
Hi, On Tue, Apr 28, 2020 at 01:18:08PM +0200, Dr. Axel Stammler via GLLUG wrote: > I have a 4 TB RAID system (two identical hard disks combined in a > RAID-1, created using mdadm). Now, after a few years, this has > reached 90% capacity, and I am thinking about first adding another > similar 8 TB RAID system and then combining them into one 12 GB > RAID 1+0 filesystem. > Which hardware parameters should I look at? You already received tips to avoid SMR. This cannot be stressed enough. Worse still, Seagate and WD are selling SMR drives without marking them as such. https://www.ixsystems.com/community/resources/list-of-known-smr-drives.141/ https://rml527.blogspot.com/2010/09/hdd-platter-database-seagate-25.html Do not try to use an SMR drive in a RAID array of any kind. Next up, if your drives don't support SCTERC timeout facility then this is not ideal for a Linux RAID system but can be worked around (and should be). Here is an article I wrote many years ago about this but it is still the case. http://strugglers.net/~andy/blog/2015/11/09/linux-software-raid-and-drive-timeouts/ On the linux-raid list many of the requests for help from people whose arrays won't automatically assemble after a failure are because their drives don't support SCTERC and they didn't work around it. > Which method should I use to combine both RAID systems into one? > > - linear RAID Doable but not great because: - Complexity of having arrays be part of an array, e.g. you'll have md0 and md1 be your two RAID-1s then md2 will be a linear of those. - Not ideal performance since all IO will go to one pair and then once capacity is reached all IO will go other. - Not sure if you can continue to grow this one later by adding more devices. > - RAID-0 Doable but not great because: - Complexity of having arrays be part of an array. - Uneven performance because one "half" is actually twice the size of the other "half". - Cannot grow this setup when you add more drives without rebuilding it all again. If all your devices were the same size there would also be the option of reshaping RAID-1 to RAID-10, which is possible with recent mdadm. It turns the RAID-1 into a RAID-0 and then turns that into a single RAID-10. No further grows/reshapes would be possible after that though. > - Large Volume Management (using pvcreate, vgcreate, lvcreate) (LVM stands for Logical Volume Manager btw ) For ease I most likely would go this way. You'd make your existing md device be one Physical Volume, make the new md device be another PV, then make a Volume Group that is both of those PVs with an allocation mode of stripe. Pros: - Can keep adding RAID-1 pairs like this as PVs forever without having to move your data about again. - Pretty simple to manage and understand what is going on. Cons: - Will still be a bit uneven performance since the smaller half will fill up first and then LVM will only allocate from the larger PV. - If you've never used LVM before then it's a whole set of new concepts. In all of the above options though, you are going to have to destroy your filesystem(s) that are on the RAID-1 and put them back onto whatever system you come up with. If you're starting over you could consider ZFS-on-Linux. I've been burnt by btrfs and still see showstopping data loss and availability problems on the btrfs list on a weekly basis so would not recommend it at this time. There is likely to be someone who will say they have been using it for years without issue; if you aren't convinced then subscribe to linux-btrfs for a month and see what other people are still dealing with! Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting -- GLLUG mailing list GLLUG@mailman.lug.org.uk https://mailman.lug.org.uk/mailman/listinfo/gllug
Re: [GLLUG] Link two RAIDs in one LVM?
On Tue, 28 Apr 2020 at 12:18, Dr. Axel Stammler via GLLUG wrote: > > Hi, > > I have a 4 TB RAID system (two identical hard disks combined in a RAID-1, > created using mdadm). Now, after a few years, this has reached 90% capacity, > and I am thinking about first adding another similar 8 TB RAID system and > then combining them into one 12 GB RAID 1+0 filesystem. I should be grateful > for any tips, especially about buying two 8 TB harddisks. > > Which hardware parameters should I look at? > > Which method should I use to combine both RAID systems into one? > > - linear RAID > - RAID-0 > - Large Volume Management (using pvcreate, vgcreate, lvcreate) > Hi, First for RAID, avoid SMR HDDs. (Shingled magnetic recording) I would probably RAID 5 them. 4+4 = 8 for the first disk, against the two other 8 disks. So, say disks are A(4TB), B(4TB), C(8TB), D(8TB) Partitions the 8TB in half. A(4TB), B(4TB), C1(4TB), C2(4TB), D1(4TB), D2(4TB) RAID 5: A,C1,D1 RAID 5: B,C2,D2 Then put the two RAID arrays in the same LVM VG, so that they look like one big disk for the OS. Another alternative is using XFS or BTRFS and configure them with replicas. That is where the filesystem does the replication, thus not needing RAID at all. Or, you could take the approach I take. I remove the old 4TB disks and only copy the few files I need on to the 8TB disks going forward. I can always plug the old 4TB disks in if I need an old file. I have written my own indexer for this. It scans the whole disk, creating an index and thumbnails and then only store the index and thumbnails on the 8TB disks. The index is stored in Elastic Search, so makes it easy to find the files again, and also which disk they are on. So, files I hardly ever need are stored on powered off disks. Kind Regards James -- GLLUG mailing list GLLUG@mailman.lug.org.uk https://mailman.lug.org.uk/mailman/listinfo/gllug
[GLLUG] Link two RAIDs in one LVM?
Hi, I have a 4 TB RAID system (two identical hard disks combined in a RAID-1, created using mdadm). Now, after a few years, this has reached 90% capacity, and I am thinking about first adding another similar 8 TB RAID system and then combining them into one 12 GB RAID 1+0 filesystem. I should be grateful for any tips, especially about buying two 8 TB harddisks. Which hardware parameters should I look at? Which method should I use to combine both RAID systems into one? - linear RAID - RAID-0 - Large Volume Management (using pvcreate, vgcreate, lvcreate) Axel -- GLLUG mailing list GLLUG@mailman.lug.org.uk https://mailman.lug.org.uk/mailman/listinfo/gllug
