RAID10 setup
We got there! Thanks for the encouragement. It took a few more steps, and I couldn't get geom_mirror or geom_stripe to kldload from the Fixit environment and so did it all from a clean install instead. All that remains is to move my /usr onto the new file system. It worked. Thanks again Phil -- Ram Ram ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: RAID10 setup
Make yourself a bootable usb stick with freebsd on i find mine very useful for installs like yours On 8/28/09, Phil Lewis dharm...@gmail.com wrote: We got there! Thanks for the encouragement. It took a few more steps, and I couldn't get geom_mirror or geom_stripe to kldload from the Fixit environment and so did it all from a clean install instead. All that remains is to move my /usr onto the new file system. It worked. Thanks again Phil -- Ram Ram ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org -- Sent from my mobile device ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: RAID10 setup
2009/8/24 John Nielsen li...@jnielsen.net You're on the right track, additional comments inline. On Saturday 22 August 2009 06:49:06 am Phil Lewis wrote: This question was asked a few weeks ago, but the original poster must have had their questions amswered. As follow-ups offered further assistance given more detail, I wonder if I could be so bold as to provide that detail for my own circumstances. I have six disks: ad4 - 500MB ad5 - 500MB ad6 - 500MB ad7 - 400MB ad8 - 500MB ad10 - 500MB These are SATA drives, with ad8 and ad10 on a PCIe SATA controller. ad7 was my first disk and currently contains FreeBSD7.2-RELEASE. I've been using that to gain some familiarity with FreeBSD, but it need not be preserved (in fact, I'd rather not preserve it!). When I built the machine, I just plugged the 400GB drive in any old slot, so it can move if that makes sense. When I got the new drives I tried to get identical to the 400GB drive, but couldn't. The 400GB drive currently has a single slice using the full drive. Just make sure you have the disk(s) you plan to boot from on a controller that will boot in your machine. If the controllers have different performance characteristics then you probably want to share the wealth of the better one between multiple mirrors. What I'd like to end up with is a three-way stripe across three two-way mirrors, containing as much of the system as possible. This is certainly do-able. If it were me I'd put the whole OS on the spare change partitions and leave the whole stripe for your serious data consumer(s): /home, /data, possibly /usr/local or some or all of /var, etc. Depends on your intended use of the storage naturally. I understand that you can't boot from a stripe, so some part of some disk will have to be outside the stripe. However, as the stripe will also be limited to the smallest disk, I'm going to have 5 x 100 GB bits left over anyway, so I guess /boot can go on one of these..? Absolutely. I'd make a gmirror of two or three of them and put / on it. If you really want to be minimal w/ your use of the extra space then you could do /boot as you propose. If possible, I'd like set this up pre-install. If it has to be done post-install, or is easier to describe how to do post-install, then that's fine. Either will work. Exactly how you do it depends on how much of the base system you want to end up on the stripe. From here on in, this email becomes speculative. All of the examples I've seen for setting up GEOM stripes and mirrors have used the raw disk as the base-level provider. On the other hand, I've seen nothing that says that the bottom level cannot be a slice, rather than a raw disk, and given the way GEOM works, I suspect this is true. Yes, you can use partitions, slices or any other GEOM providers as members of gstripe, gmirror and friends. My current plan, based on this assumption, is as follows: With my current FreeBSD installation, create 2 slices on each 500GB disk, 1 x ~400GB, 1 x ~100GB (the same size as the slice of my 400GB disk, and the rest of the disk). Boot from the FreeBSD 7.2-RELEASE dvd, and enter fixit mode. I'm not sure which would be best, or even if both are feasible for what I want to do. (I was at this point in my researchwhen I found this post!). From here, kldload geom_stripe and kldload geom_mirror. Then, create the three mirrors: gmirror label -v main0 /dev/ad4s1 /dev/ad5s1 gmirror label -v main1 /dev/ad6s1 /dev/ad571 gmirror label -v main2 /dev/ad8s1 /dev/ad10s1 This should give me /mirror/main0|main1|main2, right? Right. Next create the stripe: gstripe label -v -s 131072 raid10 /dev/mirror/main0 /dev/mirror/main1 /dev/mirror/main2 (that's all one line) If I'm right so far, then hopefully I should be able to boot to the install dvd again (or just rerun sysnstall?), and from there I should be able to choose a slice from outside 'raid10' to mount /boot, and use 'raid10' for everything else. Do I need anything else on a non-striped slice? /boot or equivalent is the only thing required to smell like a normal disk (which gmirror is capable of but gstripe isn't). You may want to use some of the space for swap. The virtual memory system should do its own version of stripe or interleave if you feed it multiple swap devices. Maybe I could even create another mirror: gmirror label -v boot /dev/ad4s2 /dev/ad5s2 and use that to mount /boot, leaving me with s2 on ad6,8 and 10 as 3 spare 100GB slices? Or am I just way off track? You seem to be pretty well on track. It seems you've already parsed the gstripe and gmirror man pages. You should probably look at fdisk(8) and bsdlabel(8) as well in case sysinstall doesn't tie up all your loose ends. Additionally you could just reinstall to a plain disk (or use
Re: RAID10 setup
Thanks to both of you for the encouraging words. I'm going to do little more reading and schedule some time probably on Wednesday to give this a go. I'll feel a lot more conformable for your responses! Cheers Phil Date: Sun, 23 Aug 2009 23:30:52 -0400 From: John Nielsen Subject: Re: RAID10 setup To: freebsd-questions@freebsd.org Cc: Phil Lewis dharm...@gmail.com Message-ID: 200908232330.53118.li...@jnielsen.net Content-Type: text/plain; charset=iso-8859-1 You're on the right track, additional comments inline. ... You seem to be pretty well on track. It seems you've already parsed the gstripe and gmirror man pages. You should probably look at fdisk(8) and bsdlabel(8) as well in case sysinstall doesn't tie up all your loose ... Date: Mon, 24 Aug 2009 10:53:45 +0100 From: chris scott Subject: Re: RAID10 setup To: John Nielsen Cc: Phil Lewis dharm...@gmail.com, freebsd-questions@freebsd.org Message-ID: d36406630908240253h3d6dd048n21468b6b18b41...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 ... When you create the file systems you should also consider laying gjournal on top of the stripe as well. In most cases it will remove the need for having to fsck the file systems when there's a system crash. Quite useful if the filesystem is large. I also like the label the filesystems with glabel so they appear in the fstab as /dev/ufs/root /dev/ufs/usr /dev/ufs/var etc makes life a little easier ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: RAID10 setup
You're on the right track, additional comments inline. On Saturday 22 August 2009 06:49:06 am Phil Lewis wrote: This question was asked a few weeks ago, but the original poster must have had their questions amswered. As follow-ups offered further assistance given more detail, I wonder if I could be so bold as to provide that detail for my own circumstances. I have six disks: ad4 - 500MB ad5 - 500MB ad6 - 500MB ad7 - 400MB ad8 - 500MB ad10 - 500MB These are SATA drives, with ad8 and ad10 on a PCIe SATA controller. ad7 was my first disk and currently contains FreeBSD7.2-RELEASE. I've been using that to gain some familiarity with FreeBSD, but it need not be preserved (in fact, I'd rather not preserve it!). When I built the machine, I just plugged the 400GB drive in any old slot, so it can move if that makes sense. When I got the new drives I tried to get identical to the 400GB drive, but couldn't. The 400GB drive currently has a single slice using the full drive. Just make sure you have the disk(s) you plan to boot from on a controller that will boot in your machine. If the controllers have different performance characteristics then you probably want to share the wealth of the better one between multiple mirrors. What I'd like to end up with is a three-way stripe across three two-way mirrors, containing as much of the system as possible. This is certainly do-able. If it were me I'd put the whole OS on the spare change partitions and leave the whole stripe for your serious data consumer(s): /home, /data, possibly /usr/local or some or all of /var, etc. Depends on your intended use of the storage naturally. I understand that you can't boot from a stripe, so some part of some disk will have to be outside the stripe. However, as the stripe will also be limited to the smallest disk, I'm going to have 5 x 100 GB bits left over anyway, so I guess /boot can go on one of these..? Absolutely. I'd make a gmirror of two or three of them and put / on it. If you really want to be minimal w/ your use of the extra space then you could do /boot as you propose. If possible, I'd like set this up pre-install. If it has to be done post-install, or is easier to describe how to do post-install, then that's fine. Either will work. Exactly how you do it depends on how much of the base system you want to end up on the stripe. From here on in, this email becomes speculative. All of the examples I've seen for setting up GEOM stripes and mirrors have used the raw disk as the base-level provider. On the other hand, I've seen nothing that says that the bottom level cannot be a slice, rather than a raw disk, and given the way GEOM works, I suspect this is true. Yes, you can use partitions, slices or any other GEOM providers as members of gstripe, gmirror and friends. My current plan, based on this assumption, is as follows: With my current FreeBSD installation, create 2 slices on each 500GB disk, 1 x ~400GB, 1 x ~100GB (the same size as the slice of my 400GB disk, and the rest of the disk). Boot from the FreeBSD 7.2-RELEASE dvd, and enter fixit mode. I'm not sure which would be best, or even if both are feasible for what I want to do. (I was at this point in my researchwhen I found this post!). From here, kldload geom_stripe and kldload geom_mirror. Then, create the three mirrors: gmirror label -v main0 /dev/ad4s1 /dev/ad5s1 gmirror label -v main1 /dev/ad6s1 /dev/ad571 gmirror label -v main2 /dev/ad8s1 /dev/ad10s1 This should give me /mirror/main0|main1|main2, right? Right. Next create the stripe: gstripe label -v -s 131072 raid10 /dev/mirror/main0 /dev/mirror/main1 /dev/mirror/main2 (that's all one line) If I'm right so far, then hopefully I should be able to boot to the install dvd again (or just rerun sysnstall?), and from there I should be able to choose a slice from outside 'raid10' to mount /boot, and use 'raid10' for everything else. Do I need anything else on a non-striped slice? /boot or equivalent is the only thing required to smell like a normal disk (which gmirror is capable of but gstripe isn't). You may want to use some of the space for swap. The virtual memory system should do its own version of stripe or interleave if you feed it multiple swap devices. Maybe I could even create another mirror: gmirror label -v boot /dev/ad4s2 /dev/ad5s2 and use that to mount /boot, leaving me with s2 on ad6,8 and 10 as 3 spare 100GB slices? Or am I just way off track? You seem to be pretty well on track. It seems you've already parsed the gstripe and gmirror man pages. You should probably look at fdisk(8) and bsdlabel(8) as well in case sysinstall doesn't tie up all your loose ends. Additionally you could just reinstall to a plain disk (or use your existing installation) and use dump/restore (and/or rsync) to move your filesystems to their multi-disk destinations. PS. I
RAID10 setup
This question was asked a few weeks ago, but the original poster must have had their questions amswered. As follow-ups offered further assistance given more detail, I wonder if I could be so bold as to provide that detail for my own circumstances. I have six disks: ad4 - 500MB ad5 - 500MB ad6 - 500MB ad7 - 400MB ad8 - 500MB ad10 - 500MB These are SATA drives, with ad8 and ad10 on a PCIe SATA controller. ad7 was my first disk and currently contains FreeBSD7.2-RELEASE. I've been using that to gain some familiarity with FreeBSD, but it need not be preserved (in fact, I'd rather not preserve it!). When I built the machine, I just plugged the 400GB drive in any old slot, so it can move if that makes sense. When I got the new drives I tried to get identical to the 400GB drive, but couldn't. The 400GB drive currently has a single slice using the full drive. What I'd like to end up with is a three-way stripe across three two-way mirrors, containing as much of the system as possible. I understand that you can't boot from a stripe, so some part of some disk will have to be outside the stripe. However, as the stripe will also be limited to the smallest disk, I'm going to have 5 x 100 GB bits left over anyway, so I guess /boot can go on one of these..? If possible, I'd like set this up pre-install. If it has to be done post-install, or is easier to describe how to do post-install, then that's fine. From here on in, this email becomes speculative. All of the examples I've seen for setting up GEOM stripes and mirrors have used the raw disk as the base-level provider. On the other hand, I've seen nothing that says that the bottom level cannot be a slice, rather than a raw disk, and given the way GEOM works, I suspect this is true. My current plan, based on this assumption, is as follows: With my current FreeBSD installation, create 2 slices on each 500GB disk, 1 x ~400GB, 1 x ~100GB (the same size as the slice of my 400GB disk, and the rest of the disk). Boot from the FreeBSD 7.2-RELEASE dvd, and enter fixit mode. I'm not sure which would be best, or even if both are feasible for what I want to do. (I was at this point in my researchwhen I found this post!). From here, kldload geom_stripe and kldload geom_mirror. Then, create the three mirrors: gmirror label -v main0 /dev/ad4s1 /dev/ad5s1 gmirror label -v main1 /dev/ad6s1 /dev/ad571 gmirror label -v main2 /dev/ad8s1 /dev/ad10s1 This should give me /mirror/main0|main1|main2, right? Next create the stripe: gstripe label -v -s 131072 raid10 /dev/mirror/main0 /dev/mirror/main1 /dev/mirror/main2 (that's all one line) If I'm right so far, then hopefully I should be able to boot to the install dvd again (or just rerun sysnstall?), and from there I should be able to choose a slice from outside 'raid10' to mount /boot, and use 'raid10' for everything else. Do I need anything else on a non-striped slice? Maybe I could even create another mirror: gmirror label -v boot /dev/ad4s2 /dev/ad5s2 and use that to mount /boot, leaving me with s2 on ad6,8 and 10 as 3 spare 100GB slices? Or am I just way off track? PS. I can't believe I'm talking about 300 'spare' GB! My first disk was 20MB! And I never filed that! All guidance much appreciated. Phil On 30 July, 15:03, John Nielsen li...@jnielsen.net wrote: On Wednesday 29 July 2009 15:54:42 Richard Fairbanks wrote: OK, so this is what I want to do. I have 4 big fast drives that I want to run inRAID10 (1+0). So, I'll need to mirror two sets of two disks, then ... Of course, if there is a way to create the stripedsetoff mirrors before installation then installing onto that stripe, that'd be perfect. I don't know if that can be done. I'm sure someone has configured aRAID10 standalone system before. (Oh, I'm using 7.2). I'm just stuck at this point! You need to consider where/how you are going to boot the system. It's straightforward to boot from a gmirror'ed UFS filesystem (the BIOS just uses one disk and thinks everything is normal), but you can't do the same from a stripe. You will either need a separate disk/device for your / or /boot partition or you will need to use slices/partitions on your disks. I frequently have the root filesystem on a small gmirror (partitions on 2 disks) then use the equivalent extra space on the remaining disk(s) for swap. Youi should be able to do this pre-installfrom the Fixit shell. Boot to the live CD, enter the shell, kldload geom_mirror and geom_stripe, create the mirrors, create the stripe, exit the shell, start theinstall, and tell sysinstall to use the device node under /dev/stripe for your filesystem. Alternatively you could just do a regularinstallto one of the disks and do everything post-install. In this case you'd still create two mirrors but one of them would only contain a single disk at first. Then create your stripe, dump/restore your files, update fstab (in
