Re: booting off a ZFS pool consisting of multiple striped mirror vdevs
On Tue, Feb 16, 2010 at 12:38 AM, Dan Naumov dan.nau...@gmail.com wrote: I don't know, but I plan to test that scenario in a few days. Matt Please share the results when you're done, I am really curious :) Booting from a stripe of two raidz vdevs works: FreeBSD/i386 boot Default: doom:/boot/zfsloader boot: status pool: doom config: NAME STATE doom ONLINE raidz1 ONLINE label/doom-0 ONLINE label/doom-1 ONLINE label/doom-2 ONLINE raidz1 ONLINE label/doom-3 ONLINE label/doom-4 ONLINE label/doom-5 ONLINE I'd guess a stripe of mirrors would work fine too. If I get a chance I'll test that combo. If booting of a stripe of 3 mirrors should work assuming no BIOS bugs, can you explain why is booting off simple stripes (of any number of disks) currently unsupported? I haven't tested that myself, but everywhere I look seems to indicate that booting off a simple stripe doesn't work or is that everywhere also out of date after your changes? :) It's probably unsupported in Solaris/OpenSolaris because of their bootloader. Our bootloader is completely different from theirs and so is not subject to those restrictions in the ZFS docs. The bottom line is that I think FreeBSD can boot from pretty much any configuration, except possibly from systems with huge numbers of disks. Matt ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: booting off a ZFS pool consisting of multiple striped mirror vdevs
On Thu, Feb 18, 2010 at 10:57 AM, Matt Reimer mattjrei...@gmail.com wrote: On Tue, Feb 16, 2010 at 12:38 AM, Dan Naumov dan.nau...@gmail.com wrote: I don't know, but I plan to test that scenario in a few days. Matt Please share the results when you're done, I am really curious :) Booting from a stripe of two raidz vdevs works: FreeBSD/i386 boot Default: doom:/boot/zfsloader boot: status pool: doom config: NAME STATE doom ONLINE raidz1 ONLINE label/doom-0 ONLINE label/doom-1 ONLINE label/doom-2 ONLINE raidz1 ONLINE label/doom-3 ONLINE label/doom-4 ONLINE label/doom-5 ONLINE I'd guess a stripe of mirrors would work fine too. If I get a chance I'll test that combo. A stripe of three-way mirrors works: FreeBSD/i386 boot Default: mithril:/boot/zfsloader boot: status pool: mithril config: NAME STATE mithril ONLINE mirror ONLINE label/mithril-0 ONLINE label/mithril-1 ONLINE label/mithril-2 ONLINE mirror ONLINE label/mithril-3 ONLINE label/mithril-4 ONLINE label/mithril-5 ONLINE Matt ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: booting off a ZFS pool consisting of multiple striped mirror vdevs
On Fri, Feb 19, 2010 at 1:03 AM, Matt Reimer mattjrei...@gmail.com wrote: On Thu, Feb 18, 2010 at 10:57 AM, Matt Reimer mattjrei...@gmail.com wrote: On Tue, Feb 16, 2010 at 12:38 AM, Dan Naumov dan.nau...@gmail.com wrote: I don't know, but I plan to test that scenario in a few days. Matt Please share the results when you're done, I am really curious :) Booting from a stripe of two raidz vdevs works: FreeBSD/i386 boot Default: doom:/boot/zfsloader boot: status pool: doom config: NAME STATE doom ONLINE raidz1 ONLINE label/doom-0 ONLINE label/doom-1 ONLINE label/doom-2 ONLINE raidz1 ONLINE label/doom-3 ONLINE label/doom-4 ONLINE label/doom-5 ONLINE I'd guess a stripe of mirrors would work fine too. If I get a chance I'll test that combo. A stripe of three-way mirrors works: FreeBSD/i386 boot Default: mithril:/boot/zfsloader boot: status pool: mithril config: NAME STATE mithril ONLINE mirror ONLINE label/mithril-0 ONLINE label/mithril-1 ONLINE label/mithril-2 ONLINE mirror ONLINE label/mithril-3 ONLINE label/mithril-4 ONLINE label/mithril-5 ONLINE Matt A stripe of 3-way mirrors, whoa. Out of curiosity, what is the system used for? I am not doubting that there exist some uses/workloads for a system that uses 6 disks with 2 disks worth of usable space, but that's a bit of an unusual configuration. What are your system/disc specs and what kind of performance are you seeing from the pool? - Sincerely, Dan Naumov ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: booting off a ZFS pool consisting of multiple striped mirror vdevs
On Thu, Feb 18, 2010 at 4:36 PM, Dan Naumov dan.nau...@gmail.com wrote: A stripe of 3-way mirrors, whoa. Out of curiosity, what is the system used for? I am not doubting that there exist some uses/workloads for a system that uses 6 disks with 2 disks worth of usable space, but that's a bit of an unusual configuration. What are your system/disc specs and what kind of performance are you seeing from the pool? It's for a reasonably busy webserver hosting a few hundred domains, which tends to be somewhat seek-intensive. For this pool we had two main criteria: speed and double-disk redundancy. A stripe of three two-way mirrors would only give us single-disk redundancy in the worst case (i.e. losing both disks in one of the mirrors), so we went with two three-way mirrors instead. Even though we're only getting the capacity of two disks worth of space, we'll still have 6x the capacity of the array it's replacing. A simple-minded dd test gives me ~180MB/s writing a single long file, and 400-500MB/s reading. Matt ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
RE: booting off a ZFS pool consisting of multiple striped mirror vdevs
I don't know, but I plan to test that scenario in a few days. Matt Please share the results when you're done, I am really curious :) It *should* work... I made changes a while back that allow for multiple vdevs to attach to the root. In this case you should have 3 mirror vdevs attached to the root, so as long as the BIOS can enumerate all of the drives, we should find all of the vdevs and build the tree correctly. It should be simple enough to test in qemu, except that the BIOS in qemu is a little broken and might not id all of the drives. robert. If booting of a stripe of 3 mirrors should work assuming no BIOS bugs, can you explain why is booting off simple stripes (of any number of disks) currently unsupported? I haven't tested that myself, but everywhere I look seems to indicate that booting off a simple stripe doesn't work or is that everywhere also out of date after your changes? :) - Sincerely, Dan Naumov ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: booting off a ZFS pool consisting of multiple striped mirror vdevs
On Sat, Feb 13, 2010 at 12:04 PM, Dan Naumov dan.nau...@gmail.com wrote: Hello I have succesfully tested and used a full ZFS install of FreeBSD 8.0 on both single disk and mirror disk configurations using both MBR and GPT partitioning. AFAIK, with the more recent -CURRENT and -STABLE it is also possible to boot off a root filesystem located on raidz/raidz2 pools. But what about booting off pools consisting of multiple striped mirror or raidz vdevs? Like this: Assume each disk looks like a half of a traditional ZFS mirror root configuration using GPT 1: freebsd-boot 2: freebsd-swap 3: freebsd-zfs |disk1+disk2| + |disk3+disk4| + |disk5+disk6| My logic tells me that while booting off any of the 6 disks, boot0 and boot1 stage should obviously work fine, but what about the boot2 stage? Can it properly handle booting off a root filesystem thats striped across 3 mirror vdevs or is booting off a single mirror vdev the best that one can do right now? I don't know, but I plan to test that scenario in a few days. Matt ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: booting off a ZFS pool consisting of multiple striped mirror vdevs
On Mon, 2010-02-15 at 10:07 -0800, Matt Reimer wrote: On Sat, Feb 13, 2010 at 12:04 PM, Dan Naumov dan.nau...@gmail.com wrote: Hello I have succesfully tested and used a full ZFS install of FreeBSD 8.0 on both single disk and mirror disk configurations using both MBR and GPT partitioning. AFAIK, with the more recent -CURRENT and -STABLE it is also possible to boot off a root filesystem located on raidz/raidz2 pools. But what about booting off pools consisting of multiple striped mirror or raidz vdevs? Like this: Assume each disk looks like a half of a traditional ZFS mirror root configuration using GPT 1: freebsd-boot 2: freebsd-swap 3: freebsd-zfs |disk1+disk2| + |disk3+disk4| + |disk5+disk6| My logic tells me that while booting off any of the 6 disks, boot0 and boot1 stage should obviously work fine, but what about the boot2 stage? Can it properly handle booting off a root filesystem thats striped across 3 mirror vdevs or is booting off a single mirror vdev the best that one can do right now? I don't know, but I plan to test that scenario in a few days. It *should* work... I made changes a while back that allow for multiple vdevs to attach to the root. In this case you should have 3 mirror vdevs attached to the root, so as long as the BIOS can enumerate all of the drives, we should find all of the vdevs and build the tree correctly. It should be simple enough to test in qemu, except that the BIOS in qemu is a little broken and might not id all of the drives. robert. Matt ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Robert Noland rnol...@freebsd.org FreeBSD ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
booting off a ZFS pool consisting of multiple striped mirror vdevs
Hello I have succesfully tested and used a full ZFS install of FreeBSD 8.0 on both single disk and mirror disk configurations using both MBR and GPT partitioning. AFAIK, with the more recent -CURRENT and -STABLE it is also possible to boot off a root filesystem located on raidz/raidz2 pools. But what about booting off pools consisting of multiple striped mirror or raidz vdevs? Like this: Assume each disk looks like a half of a traditional ZFS mirror root configuration using GPT 1: freebsd-boot 2: freebsd-swap 3: freebsd-zfs |disk1+disk2| + |disk3+disk4| + |disk5+disk6| My logic tells me that while booting off any of the 6 disks, boot0 and boot1 stage should obviously work fine, but what about the boot2 stage? Can it properly handle booting off a root filesystem thats striped across 3 mirror vdevs or is booting off a single mirror vdev the best that one can do right now? - Sincerely, Dan Naumov ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
