Re: [zfs-discuss] ZFS Scalability/performance
On Wed, Jun 20, 2007 at 12:03:02PM -0400, Will Murnane wrote: > Yes. 2 disks means when one fails, you've still got an extra. In > raid 5 boxes, it's not uncommon with large arrays for one disk to die, > and when it's replaced, the stress on the other disks causes another > failure. Then the array is toast. I don't know if this is a problem > on ZFS... but they took the time to implement raidz2, so I'd suggest > it. If you buy all the disks at once and add them to a pool all at once, they should all theoretically have appoximately the same lifespan. When one dies, you can almost count on others following soon after. Nothing sucks more than your "redundant" disk array losing more disks than it can support and you lose all your data anyway. You'd be better off doing a giant non-parity stripe and dumping to tape on a regular basis. ;) -brian -- "Perl can be fast and elegant as much as J2EE can be fast and elegant. In the hands of a skilled artisan, it can and does happen; it's just that most of the shit out there is built by people who'd be better suited to making sure that my burger is cooked thoroughly." -- Jonathan Patschke ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Oliver Schinagl wrote: zo basically, what you are saying is that on FBSD there's no performane issue, whereas on solaris there (can be if write caches aren't enabled) Solaris plays it safe by default. You can, of course, override that safety. Whether it is a performance win seems to be the subject of some debate, but intuitively it seems like it should help for most cases. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
mike wrote: I would be interested in hearing if there are any other configuration options to squeeze the most space out of the drives. I have no issue with powering down to replace a bad drive, and I expect that I'll only have one at the most fail at a time. This is what is known as "famous last words." Sorta like, "we have triple redundant navigation computers on this fancy, new International Space Station." If you want to know what keeps us RAS guys up at night, it is famous last words. [sidebar] What is the last thing a redneck says before he dies? "Hey y'all, watch this!" :-) -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On Wed, Jun 20, 2007 at 09:48:08AM -0700, Eric Schrock wrote: > On Wed, Jun 20, 2007 at 12:45:52PM +0200, Pawel Jakub Dawidek wrote: > > > > Will be nice to not EFI label disks, though:) Currently there is a > > problem with this - zpool created on Solaris is not recognized by > > FreeBSD, because FreeBSD claims GPT label is corrupted. On the other > > hand, creating ZFS on FreeBSD (on a raw disk) can be used under Solaris. > > > > FYI, the primary reason for using EFI labels is that they are > endian-neutral, unlike Solaris VTOC. The secondary reason is that they > are simpler and easier to use (at least on Solaris). > > I'm curious why FreeBSD claims the GPT label is corrupted. Is this > because FreeBSD doesn't understand EFI labels, our EFI label is bad, or > is there a bug in the FreeBSD EFI implementation? I haven't investigated this yet. FreeBSD should understand EFI, so either the last two or a bug in Solaris EFI implementation:) I seem to recall similar problems on Linux with ZFS/FUSE... -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpd81Zg8xdCo.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On Wed, 2007-06-20 at 12:45 +0200, Pawel Jakub Dawidek wrote: > Will be nice to not EFI label disks, though:) Currently there is a > problem with this - zpool created on Solaris is not recognized by > FreeBSD, because FreeBSD claims GPT label is corrupted. Hmm. I'd think the right answer here is to understand why FreeBSD and solaris disagree about EFI/GPT labels. Could be a solaris bug, could be a freebsd bug, but the intent of the label format is to permit interchange between different platforms.. - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
mike wrote: > On 6/20/07, Constantin Gonzalez <[EMAIL PROTECTED]> wrote: > >> One disk can be one vdev. >> A 1+1 mirror can be a vdev, too. >> A n+1 or n+2 RAID-Z (RAID-Z2) set can be a vdev too. >> >> - Then you concatenate vdevs to create a pool. Pools can be extended by >> adding more vdevs. >> >> - Then you create ZFS file systems that draw their block usage from the >> resources supplied by the pool. Very flexible. > > This actually brings up something I was wondering about last night: > > If I was to plan for a 16 disk ZFS-based system, you would probably > suggest me to configure it as something like 5+1, 4+1, 4+1 all raid-z > (I don't need the double parity concept) > > I would prefer something like 15+1 :) I want ZFS to be able to detect > and correct errors, but I do not need to squeeze all the performance > out of it (I'll be using it as a home storage server for my DVDs and > other audio/video stuff. So only a few clients at the most streaming > off of it) > > I would be interested in hearing if there are any other configuration > options to squeeze the most space out of the drives. I have no issue > with powering down to replace a bad drive, and I expect that I'll only Just know that, if your server/disks are up all the time, shutting down your server whilst you wait for replacement drives actually might kill your array. Especially with consumer IDE/SATA drives. Those pesky consumer drivers aren't made for 24/7 usage, i think they spec em at 8hrs a day? Eitherway, that's me being sidetracked, the problem is, you'll have a disk up spinning normally, some access, same temperature! all the time. All of a sudden you change the envirment, you let it cool down and what not. Harddisks don't like that at all! I've even heard of harddisk (cases) cracking because of the temperature differences and such. My requirements are the same, and i want space, but the thought of having more disks die on me while i replace the broken one doesn't really make me happy either. (I personally use only the WD Raid editions of HDD's; wether it's worth it or not, i dunno, but they have better warranty and supposedly should be able to do 24/7 a day) > have one at the most fail at a time. If I really do need room for two > to fail then I suppose I can look for a 14 drive space usable setup > and use raidz-2. > > Thanks, > mike > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On Wed, Jun 20, 2007 at 12:45:52PM +0200, Pawel Jakub Dawidek wrote: > > Will be nice to not EFI label disks, though:) Currently there is a > problem with this - zpool created on Solaris is not recognized by > FreeBSD, because FreeBSD claims GPT label is corrupted. On the other > hand, creating ZFS on FreeBSD (on a raw disk) can be used under Solaris. > FYI, the primary reason for using EFI labels is that they are endian-neutral, unlike Solaris VTOC. The secondary reason is that they are simpler and easier to use (at least on Solaris). I'm curious why FreeBSD claims the GPT label is corrupted. Is this because FreeBSD doesn't understand EFI labels, our EFI label is bad, or is there a bug in the FreeBSD EFI implementation? Thanks, - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Pawel Jakub Dawidek wrote: > On Wed, Jun 20, 2007 at 01:45:29PM +0200, Oliver Schinagl wrote: > >> Pawel Jakub Dawidek wrote: >> >>> On Tue, Jun 19, 2007 at 07:52:28PM -0700, Richard Elling wrote: >>> >>> > On that note, i have a different first question to start with. I > personally am a Linux fanboy, and would love to see/use ZFS on linux. I > assume that I can use those ZFS disks later with any os that can > work/recognizes ZFS correct? e.g. I can install/setup ZFS in FBSD, and > later use it in OpenSolaris/Linux Fuse(native) later? > > The on-disk format is an available specification and is designed to be platform neutral. We certainly hope you will be able to access the zpools from different OSes (one at a time). >>> Will be nice to not EFI label disks, though:) Currently there is a >>> problem with this - zpool created on Solaris is not recognized by >>> FreeBSD, because FreeBSD claims GPT label is corrupted. On the other >>> hand, creating ZFS on FreeBSD (on a raw disk) can be used under Solaris. >>> >>> >>> >> I read this earlier, that it's recommended to use a whole disk instead >> of a partition with zfs, the thing that's holding me back however is the >> mixture of different sized disks I have. I suppose if I had a 300gb per >> disk raid-z going on 3 300 disk and one 320gb disk, but only have a >> partition of 300gb on it (still with me), i could later expand that >> partition with fdisk and the entire raid-z would then expand to 320gb >> per disk (assuming the other disks magically gain 20gb, so this is a bad >> example in that sense :) ) >> >> Also what about full disk vs full partition, e.g. make 1 partition to >> span the entire disk vs using the entire disk. >> Is there any significant performance penalty? (So not having a disk >> split into 2 partitions, but 1 disk, 1 partition) I read that with a >> full raw disk zfs will be beter to utilize the disks write cache, but I >> don't see how. >> > > On FreeBSD (thanks to GEOM) there is no difference what do you have > under ZFS. On Solaris, ZFS turns on write cache on disk when whole disk > is used. On FreeBSD write cache is enabled by default and GEOM consumers > can send write-cache-flush (BIO_FLUSH) request to any GEOM providers. > > zo basically, what you are saying is that on FBSD there's no performane issue, whereas on solaris there (can be if write caches aren't enabled) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On Wed, Jun 20, 2007 at 01:45:29PM +0200, Oliver Schinagl wrote: > > > Pawel Jakub Dawidek wrote: > > On Tue, Jun 19, 2007 at 07:52:28PM -0700, Richard Elling wrote: > > > >>> On that note, i have a different first question to start with. I > >>> personally am a Linux fanboy, and would love to see/use ZFS on linux. I > >>> assume that I can use those ZFS disks later with any os that can > >>> work/recognizes ZFS correct? e.g. I can install/setup ZFS in FBSD, and > >>> later use it in OpenSolaris/Linux Fuse(native) later? > >>> > >> The on-disk format is an available specification and is designed to be > >> platform neutral. We certainly hope you will be able to access the > >> zpools from different OSes (one at a time). > >> > > > > Will be nice to not EFI label disks, though:) Currently there is a > > problem with this - zpool created on Solaris is not recognized by > > FreeBSD, because FreeBSD claims GPT label is corrupted. On the other > > hand, creating ZFS on FreeBSD (on a raw disk) can be used under Solaris. > > > > > > I read this earlier, that it's recommended to use a whole disk instead > of a partition with zfs, the thing that's holding me back however is the > mixture of different sized disks I have. I suppose if I had a 300gb per > disk raid-z going on 3 300 disk and one 320gb disk, but only have a > partition of 300gb on it (still with me), i could later expand that > partition with fdisk and the entire raid-z would then expand to 320gb > per disk (assuming the other disks magically gain 20gb, so this is a bad > example in that sense :) ) > > Also what about full disk vs full partition, e.g. make 1 partition to > span the entire disk vs using the entire disk. > Is there any significant performance penalty? (So not having a disk > split into 2 partitions, but 1 disk, 1 partition) I read that with a > full raw disk zfs will be beter to utilize the disks write cache, but I > don't see how. On FreeBSD (thanks to GEOM) there is no difference what do you have under ZFS. On Solaris, ZFS turns on write cache on disk when whole disk is used. On FreeBSD write cache is enabled by default and GEOM consumers can send write-cache-flush (BIO_FLUSH) request to any GEOM providers. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpZkCuJUZmIl.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On 6/20/07, mike <[EMAIL PROTECTED]> wrote: On 6/20/07, Paul Fisher <[EMAIL PROTECTED]> wrote: > I would not risk raidz on that many disks. A nice compromise may be 14+2 > raidz2, which should perform nicely for your workload and be pretty reliable > when the disks start to fail. Would anyone on the list not recommend this setup? I could live with 2 drives being used for parity (or the "parity" concept) Yes. 2 disks means when one fails, you've still got an extra. In raid 5 boxes, it's not uncommon with large arrays for one disk to die, and when it's replaced, the stress on the other disks causes another failure. Then the array is toast. I don't know if this is a problem on ZFS... but they took the time to implement raidz2, so I'd suggest it. I would be able to reap the benefits of ZFS - self-healing, corrupted file reconstruction (since it has some parity to read from) and should have decent performance (obviously not smokin' since I am not configuring this to try for the fastest possible) And since you'll generally be doing full-stripe reads and writes, you get good bandwidth anyways. Will ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On 20 June, 2007 - Oliver Schinagl sent me these 1,9K bytes:
> Also what about full disk vs full partition, e.g. make 1 partition to
> span the entire disk vs using the entire disk.
> Is there any significant performance penalty? (So not having a disk
> split into 2 partitions, but 1 disk, 1 partition) I read that with a
> full raw disk zfs will be beter to utilize the disks write cache, but I
> don't see how.
Because when given a whole disk, ZFS can safely play with the write
cache in disks without jeopardizing any UFS or so that might be on some
other slice.. Helps when ZFS is batch-writing in a transaction group.
/Tomas
--
Tomas Ögren, [EMAIL PROTECTED], http://www.acc.umu.se/~stric/
|- Student at Computing Science, University of Umeå
`- Sysadmin at {cs,acc}.umu.se
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On 6/20/07, Paul Fisher <[EMAIL PROTECTED]> wrote: I would not risk raidz on that many disks. A nice compromise may be 14+2 raidz2, which should perform nicely for your workload and be pretty reliable when the disks start to fail. Would anyone on the list not recommend this setup? I could live with 2 drives being used for parity (or the "parity" concept) I would be able to reap the benefits of ZFS - self-healing, corrupted file reconstruction (since it has some parity to read from) and should have decent performance (obviously not smokin' since I am not configuring this to try for the fastest possible) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
RE: [zfs-discuss] ZFS Scalability/performance
> From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of mike > Sent: Wednesday, June 20, 2007 9:30 AM > > I would prefer something like 15+1 :) I want ZFS to be able to detect > and correct errors, but I do not need to squeeze all the performance > out of it (I'll be using it as a home storage server for my DVDs and > other audio/video stuff. So only a few clients at the most streaming > off of it) I would not risk raidz on that many disks. A nice compromise may be 14+2 raidz2, which should perform nicely for your workload and be pretty reliable when the disks start to fail. -- paul ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Hi Mike, > If I was to plan for a 16 disk ZFS-based system, you would probably > suggest me to configure it as something like 5+1, 4+1, 4+1 all raid-z > (I don't need the double parity concept) > > I would prefer something like 15+1 :) I want ZFS to be able to detect > and correct errors, but I do not need to squeeze all the performance > out of it (I'll be using it as a home storage server for my DVDs and > other audio/video stuff. So only a few clients at the most streaming > off of it) this is possibe. ZFS in theory does not significantly limit the n and 15+1 is indeed possible. But for a number of reasons (among them performance) people generally advise to use no more than 10+1. A lot of ZFS configuration wisdom can be found on the Solaris internals ZFS Best Practices Guide Wiki at: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide Richard Elling has done a great job of thoroughly analyzing different reliability concepts for ZFS in his blog. One good introduction is the following entry: http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance That may help you find the right tradeoff between space and reliability. Hope this helps, Constantin -- Constantin GonzalezSun Microsystems GmbH, Germany Platform Technology Group, Global Systems Engineering http://www.sun.de/ Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/ Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer Vorsitzender des Aufsichtsrates: Martin Haering ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On 6/20/07, Constantin Gonzalez <[EMAIL PROTECTED]> wrote: One disk can be one vdev. A 1+1 mirror can be a vdev, too. A n+1 or n+2 RAID-Z (RAID-Z2) set can be a vdev too. - Then you concatenate vdevs to create a pool. Pools can be extended by adding more vdevs. - Then you create ZFS file systems that draw their block usage from the resources supplied by the pool. Very flexible. This actually brings up something I was wondering about last night: If I was to plan for a 16 disk ZFS-based system, you would probably suggest me to configure it as something like 5+1, 4+1, 4+1 all raid-z (I don't need the double parity concept) I would prefer something like 15+1 :) I want ZFS to be able to detect and correct errors, but I do not need to squeeze all the performance out of it (I'll be using it as a home storage server for my DVDs and other audio/video stuff. So only a few clients at the most streaming off of it) I would be interested in hearing if there are any other configuration options to squeeze the most space out of the drives. I have no issue with powering down to replace a bad drive, and I expect that I'll only have one at the most fail at a time. If I really do need room for two to fail then I suppose I can look for a 14 drive space usable setup and use raidz-2. Thanks, mike ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Hi, > How are paired mirrors more flexiable? well, I'm talking of a small home system. If the pool gets full, the way to expand with RAID-Z would be to add 3+ disks (typically 4-5). With mirror only, you just add two. So in my case it's just about the granularity of expansion. The reasoning is that of the three factors reliability, performance and space, I value them in this order. Space comes last since disk space is cheap. If I had a bigger number of disks (12+), I'd be using them in RAID-Z2 sets (4+2 plus 4+2 etc.). Here, the speed is ok and the reliability is ok and so I can use RAID-Z2 instead of mirroring to get some extra space as well. > Right now, i have a 3 disk raid 5 running with the linux DM driver. One > of the most resent additions was raid5 expansion, so i could pop in a > matching disk, and expand my raid5 to 4 disks instead of 3 (which is > always interesting as your cutting on your parity loss). I think though > in raid5 you shouldn't put more then 6 - 8 disks afaik, so I wouldn't be > expanding this enlessly. > > So how would this translate to ZFS? I have learned so far that, ZFS ZFS does not yet support rearranging the disk cofiguration. Right now, you can expand a single disk to a mirror or an n-way mirror to an n+1 way mirror. RAID-Z vdevs can't be changed right now. But you can add more disks to a pool by adding more vdevs (You have a 1+1 mirror, add another 1+1 pair and get more space, have a 3+2 RAID-Z2 and add another 5+2 RAID etc.) > basically is raid + LVM. e.g. the mirrored raid-z pairs go into the > pool, just like one would use LVM to bind all the raid pairs. The > difference being I suppose, that you can't use a zfs mirror/raid-z > without having a pool to use it from? Here's the basic idea: - You first construct vdevs from disks: One disk can be one vdev. A 1+1 mirror can be a vdev, too. A n+1 or n+2 RAID-Z (RAID-Z2) set can be a vdev too. - Then you concatenate vdevs to create a pool. Pools can be extended by adding more vdevs. - Then you create ZFS file systems that draw their block usage from the resources supplied by the pool. Very flexible. > Wondering now is if I can simply add a new disk to my raid-z and have it > 'just work', e.g. the raid-z would be expanded to use the new > disk(partition of matching size) If you have a RAID-Z based pool in ZFS, you can add another group of disks that are organized in a RAID-Z manner (a vdev) to expand the storage capacity of the pool. Hope this clarifies things a bit. And yes, please check out the admin guide and the other collateral available on ZFS. It's full of new concepts and one needs some getting used to to explore all possibilities. Cheers, Constantin -- Constantin GonzalezSun Microsystems GmbH, Germany Platform Technology Group, Global Systems Engineering http://www.sun.de/ Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/ Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer Vorsitzender des Aufsichtsrates: Martin Haering ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Constantin Gonzalez wrote: > Hi, > > >> I'm quite interested in ZFS, like everybody else I suppose, and am about >> to install FBSD with ZFS. >> > > welcome to ZFS! > > >> Anyway, back to business :) >> I have a whole bunch of different sized disks/speeds. E.g. 3 300GB disks >> @ 40mb, a 320GB disk @ 60mb/s, 3 120gb disks @ 50mb/s and so on. >> >> Raid-Z and ZFS claims to be uber scalable and all that, but would it >> 'just work' with a setup like that too? >> > > Yes. If you dump a set of variable-size disks into a mirror or RAID-Z > configuration, you'll get the same result as if you had the smallest of > their sizes. Then, the pool will grow when exchanging smaller disks with > larger. > > I used to run a ZFS pool on 1x250GB, 1x200GB, 1x85 GB and 1x80 GB the > following > way: > > - Set up an 80 GB slice on all 4 disks and make a 4 disk RAID-Z vdev > - Set up a 5 GB slice on the 250, 200 and 85 GB disks and make a 3 disk RAID-Z > - Set up a 115GB slice on the 200 and the 250 GB disk and make a 2 disk > mirror. > - Concatenate all 3 vdevs into one pool. (You need zpool add -f for that). > > Not something to be done on a professional production system, but it worked > for my home setup just fine. The remaining 50GB from the 250GB drive then > went into a scratch pool. > > Kinda like playing Tetris with RAID-Z... > > Later, I decided using just paired disks as mirrors are really more > flexible and easier to expand, since disk space is cheap. > > well i'm about to go read the entire admin manual now that I found it and I hope it will explain all my further questions, but before i start doing so, How are paired mirrors more flexiable? Right now, i have a 3 disk raid 5 running with the linux DM driver. One of the most resent additions was raid5 expansion, so i could pop in a matching disk, and expand my raid5 to 4 disks instead of 3 (which is always interesting as your cutting on your parity loss). I think though in raid5 you shouldn't put more then 6 - 8 disks afaik, so I wouldn't be expanding this enlessly. So how would this translate to ZFS? I have learned so far that, ZFS basically is raid + LVM. e.g. the mirrored raid-z pairs go into the pool, just like one would use LVM to bind all the raid pairs. The difference being I suppose, that you can't use a zfs mirror/raid-z without having a pool to use it from? Wondering now is if I can simply add a new disk to my raid-z and have it 'just work', e.g. the raid-z would be expanded to use the new disk(partition of matching size) > Hope this helps, >Constantin > > Thanks, Oliver ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Pawel Jakub Dawidek wrote: > On Tue, Jun 19, 2007 at 07:52:28PM -0700, Richard Elling wrote: > >>> On that note, i have a different first question to start with. I >>> personally am a Linux fanboy, and would love to see/use ZFS on linux. I >>> assume that I can use those ZFS disks later with any os that can >>> work/recognizes ZFS correct? e.g. I can install/setup ZFS in FBSD, and >>> later use it in OpenSolaris/Linux Fuse(native) later? >>> >> The on-disk format is an available specification and is designed to be >> platform neutral. We certainly hope you will be able to access the >> zpools from different OSes (one at a time). >> > > Will be nice to not EFI label disks, though:) Currently there is a > problem with this - zpool created on Solaris is not recognized by > FreeBSD, because FreeBSD claims GPT label is corrupted. On the other > hand, creating ZFS on FreeBSD (on a raw disk) can be used under Solaris. > > I read this earlier, that it's recommended to use a whole disk instead of a partition with zfs, the thing that's holding me back however is the mixture of different sized disks I have. I suppose if I had a 300gb per disk raid-z going on 3 300 disk and one 320gb disk, but only have a partition of 300gb on it (still with me), i could later expand that partition with fdisk and the entire raid-z would then expand to 320gb per disk (assuming the other disks magically gain 20gb, so this is a bad example in that sense :) ) Also what about full disk vs full partition, e.g. make 1 partition to span the entire disk vs using the entire disk. Is there any significant performance penalty? (So not having a disk split into 2 partitions, but 1 disk, 1 partition) I read that with a full raw disk zfs will be beter to utilize the disks write cache, but I don't see how. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
On Tue, Jun 19, 2007 at 07:52:28PM -0700, Richard Elling wrote: > >On that note, i have a different first question to start with. I > >personally am a Linux fanboy, and would love to see/use ZFS on linux. I > >assume that I can use those ZFS disks later with any os that can > >work/recognizes ZFS correct? e.g. I can install/setup ZFS in FBSD, and > >later use it in OpenSolaris/Linux Fuse(native) later? > > The on-disk format is an available specification and is designed to be > platform neutral. We certainly hope you will be able to access the > zpools from different OSes (one at a time). Will be nice to not EFI label disks, though:) Currently there is a problem with this - zpool created on Solaris is not recognized by FreeBSD, because FreeBSD claims GPT label is corrupted. On the other hand, creating ZFS on FreeBSD (on a raw disk) can be used under Solaris. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpogQ3rsG6KV.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Hi, > I'm quite interested in ZFS, like everybody else I suppose, and am about > to install FBSD with ZFS. welcome to ZFS! > Anyway, back to business :) > I have a whole bunch of different sized disks/speeds. E.g. 3 300GB disks > @ 40mb, a 320GB disk @ 60mb/s, 3 120gb disks @ 50mb/s and so on. > > Raid-Z and ZFS claims to be uber scalable and all that, but would it > 'just work' with a setup like that too? Yes. If you dump a set of variable-size disks into a mirror or RAID-Z configuration, you'll get the same result as if you had the smallest of their sizes. Then, the pool will grow when exchanging smaller disks with larger. I used to run a ZFS pool on 1x250GB, 1x200GB, 1x85 GB and 1x80 GB the following way: - Set up an 80 GB slice on all 4 disks and make a 4 disk RAID-Z vdev - Set up a 5 GB slice on the 250, 200 and 85 GB disks and make a 3 disk RAID-Z - Set up a 115GB slice on the 200 and the 250 GB disk and make a 2 disk mirror. - Concatenate all 3 vdevs into one pool. (You need zpool add -f for that). Not something to be done on a professional production system, but it worked for my home setup just fine. The remaining 50GB from the 250GB drive then went into a scratch pool. Kinda like playing Tetris with RAID-Z... Later, I decided using just paired disks as mirrors are really more flexible and easier to expand, since disk space is cheap. Hope this helps, Constantin -- Constantin GonzalezSun Microsystems GmbH, Germany Platform Technology Group, Global Systems Engineering http://www.sun.de/ Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/ Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer Vorsitzender des Aufsichtsrates: Martin Haering ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Scalability/performance
Oliver Schinagl wrote: Hello, I'm quite interested in ZFS, like everybody else I suppose, and am about to install FBSD with ZFS. cool. On that note, i have a different first question to start with. I personally am a Linux fanboy, and would love to see/use ZFS on linux. I assume that I can use those ZFS disks later with any os that can work/recognizes ZFS correct? e.g. I can install/setup ZFS in FBSD, and later use it in OpenSolaris/Linux Fuse(native) later? The on-disk format is an available specification and is designed to be platform neutral. We certainly hope you will be able to access the zpools from different OSes (one at a time). Anyway, back to business :) I have a whole bunch of different sized disks/speeds. E.g. 3 300GB disks @ 40mb, a 320GB disk @ 60mb/s, 3 120gb disks @ 50mb/s and so on. Raid-Z and ZFS claims to be uber scalable and all that, but would it 'just work' with a setup like that too? Yes, for most definitions of 'just work.' I used to match up partition sizes in linux, so make the 320gb disk into 2 partitions of 300 and 20gb, then use the 4 300gb partitions as a raid5, same with the 120 gigs and use the scrap on those aswell, finally stiching everything together with LVM2. I can't easly find how this would work with raid-Z/ZFS, e.g. can I really just put all these disks in 1 big pool and remove/add to it at will? Yes is the simple answer. But we generally recommend planning. To begin your plan, decide your priority: space, performance, data protection. ZFS is very dynamic, which has the property that for redundancy schemes (mirror, raidz[12]) it will use as much a space as possible. For example, if you mirror a 1 GByte drive with a 2 GByte drive, then you will have available space of 1 GByte. If you later replace the 1 GByte drive with a 4 GByte drive, then you will instantly have the available space of 2 GBytes. If you replace the 2 GByte drive with an 8 GByte drive, you will instantly have access to 4 GBytes of mirrored data. And I really don't need to use softwareraid yet still have the same reliablity with raid-z as I had with raid-5? raidz is more reliable than software raid-5. What about hardware raid controllers, just use it as a JBOD device, or would I use it to match up disk sizes in raid0 stripes (e.g. the 3x 120gb to make a 360 raid0). ZFS is dynamic. Or you'd recommend to just stick with raid/lvm/reiserfs and use that. ZFS rocks! -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
