On Mon, Jul 12, 2010 at 22:28, Stan Hoeppner <s...@hardwarefreak.com> wrote: > I'm curious as to why you're (apparently) wasting 2/3 of your storage for > redundancy. Have you considered a straight RAID 10 across those 30 > disks/LUNs?
This is a very good question. And the answer is: because Linux's MD does not implement RAID10 the way we expected (as you have found out for yourself). We started out thinking exactly that we'd have a RAID10 stripe with cardinality of 3, instead of the multi-layered MD design. But for us it's important to have full control over what physical disks form the triplets (see below for discussion); instead, MD's so-called RAID10 only guarantees that there will be exactly N copies of each chunk on N different drives, but makes on promise as to on *which* drives. The reason the drive assignment is important to us is that we can achieve more data redundancy if we form each triplet from an iSCSI disk that lives on a different iSCSI target (host). Suppose that you have six iSCSI target hosts h0 through h5, and each of them has five disks d0 through d4. Then if you form the first triplet as (h0:d0, h1:d0, h2:d0), and so forth until (h3:d4, h4:d4, h5:d4), then if any iSCSI host goes down for whatever reason, then all triplets still stay up and are still redundant, only running on two copies instead of three. Linux's RAID10 implementation did not allow us to do this. So we had to layer by first creating RAID1 (or RAID10 with n=3) triplets, and striping them in a higher layer. > I'm also curious as to why you're running software RAID at all given the fact > than pretty much every iSCSI target is itself an array controller with built > in hardware RAID. Can you tell us a little bit about your iSCSI target > devices? Our boss wanted us to only use commodity hardware to build this solution, so we don't employ any fancy RAID controllers - all drives are connected to on-board SATA ports. Staying away from the "black box" implementations as much as possible was also part of the wish list. After dealing with all the idiosyncrasies of iSCSI and software RAID under Linux I am a bit skeptical whether what we are building is going to actually be better than a black-box fiber-attached RAID solution, but it surely is cheaper and more expandable. -- Arcady Genkin -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktinwfqgcutlg6rm0c5pmgtgzwnvfxc-kbzsdq...@mail.gmail.com