Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-18 Thread Bill Sommerfeld
On Wed, 2009-09-16 at 14:19 -0700, Richard Elling wrote: > Actually, I had a ton of data on resilvering which shows mirrors and > raidz equivalently bottlenecked on the media write bandwidth. However, > there are other cases which are IOPS bound (or CR bound :-) which > cover some of the postings

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-18 Thread Brandon High
On Thu, Sep 17, 2009 at 11:41 AM, Adam Leventhal wrote: >  RAID-3        bit-interleaved parity (basically not used) There was a hardware RAID chipset that used RAID-3. Netcell Revolution I think it was called. It looked interesting and I thought about grabbing one at the time but never got arou

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Adam Leventhal
On Thu, Sep 17, 2009 at 01:32:43PM +0200, Eugen Leitl wrote: > > reasons), you will lose 2 disks worth of storage to parity leaving 12 > > disks worth of data. With raid10 you will lose half, 7 disks to > > parity/redundancy. With two raidz2 sets, you will get (5+2)+(5+2), that > > is 5+5 disks wor

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Marion Hakanson
rswwal...@gmail.com said: > It's not the stripes that make a difference, but the number of controllers > there. > > What's the system config on that puppy? The "zpool status -v" output was from a Thumper (X4500), slightly edited, since in our real-world Thumper, we use c6t0d0 in c5t4d0's place i

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Erik Trimble
Darren J Moffat wrote: Erik Trimble wrote: So SSDs for ZIL/L2ARC don't bring that much when used with raidz2/raidz3, if I write a lot, at least, and don't access the cache very much, according to some recent posts on this list. Not true. Remember: ZIL = write cache ZIL is NOT a write c

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Darren J Moffat
Erik Trimble wrote: So SSDs for ZIL/L2ARC don't bring that much when used with raidz2/raidz3, if I write a lot, at least, and don't access the cache very much, according to some recent posts on this list. Not true. Remember: ZIL = write cache ZIL is NOT a write cache. The ZIL is the In

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Eugen Leitl
On Thu, Sep 17, 2009 at 12:55:35PM +0200, Tomas Ögren wrote: > It's not a fixed value per technology, it depends on the number of disks > per group. RAID5/RAIDZ1 "loses" 1 disk worth to parity per group. > RAID6/RAIDZ" loses 2 disks. RAIDZ3 loses 3 disks. Raid1/mirror loses > half the disks. So in

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Erik Trimble
Eugen Leitl wrote: On Wed, Sep 16, 2009 at 08:02:35PM +0300, Markus Kovero wrote: It's possible to do 3-way (or more) mirrors too, so you may achieve better redundancy than raidz2/3 I understand there's almost no additional performance penalty to raidz3 over raidz2 in terms of CPU l

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Tomas Ögren
On 17 September, 2009 - Eugen Leitl sent me these 2,0K bytes: > On Wed, Sep 16, 2009 at 08:02:35PM +0300, Markus Kovero wrote: > > > It's possible to do 3-way (or more) mirrors too, so you may achieve better > > redundancy than raidz2/3 > > I understand there's almost no additional performance

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Eugen Leitl
On Wed, Sep 16, 2009 at 10:23:01AM -0700, Richard Elling wrote: > This line of reasoning doesn't get you very far. It is much better to > take a look at > the mean time to data loss (MTTDL) for the various configurations. I > wrote a > series of blogs to show how this is done. > http://blogs

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-17 Thread Eugen Leitl
On Wed, Sep 16, 2009 at 08:02:35PM +0300, Markus Kovero wrote: > It's possible to do 3-way (or more) mirrors too, so you may achieve better > redundancy than raidz2/3 I understand there's almost no additional performance penalty to raidz3 over raidz2 in terms of CPU load. Is that correct? So S

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Richard Elling
On Sep 16, 2009, at 7:17 PM, Ross Walker wrote: more resilient to temporary path failures. As another list member pointed out you could also avoid the issue by having a raidz disk per controller. But if I'm buying that kind of big iron I might just opt for a 3par or emc and save myself the

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Ross Walker
On Sep 16, 2009, at 6:43 PM, Bob Friesenhahn > wrote: On Wed, 16 Sep 2009, Ross Walker wrote: There is another type of failure that mirrors help with and that is controller or path failures. If one side of a mirror set is on one controller or path and the other on another then a failure of

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Ross Walker
On Sep 16, 2009, at 6:50 PM, Marion Hakanson wrote: rswwal...@gmail.com said: There is another type of failure that mirrors help with and that is controller or path failures. If one side of a mirror set is on one controller or path and the other on another then a failure of one will not t

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Marion Hakanson
rswwal...@gmail.com said: > There is another type of failure that mirrors help with and that is > controller or path failures. If one side of a mirror set is on one > controller or path and the other on another then a failure of one will not > take down the set. > > You can't get that with RAIDZ

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Bob Friesenhahn
On Wed, 16 Sep 2009, Ross Walker wrote: There is another type of failure that mirrors help with and that is controller or path failures. If one side of a mirror set is on one controller or path and the other on another then a failure of one will not take down the set. You can't get that wit

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Ross Walker
On Sep 16, 2009, at 4:29 PM, "Marty Scholes" wrote: Yes. This is a mathematical way of saying "lose any P+1 of N disks." I am hesitant to beat this dead horse, yet it is a nuance that either I have completely misunderstood or many people I've met have completely missed. Whether a str

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Eric Schrock
On 09/16/09 14:19, Richard Elling wrote: On Sep 16, 2009, at 1:09 PM, Bob Friesenhahn wrote: On Wed, 16 Sep 2009, Thomas Burgess wrote: hrm, i always thought raidz took longerlearn something every day =) And you were probably right, in spite of Richard's lack of knowledge of a study or

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Richard Elling
On Sep 16, 2009, at 1:29 PM, Marty Scholes wrote: Yes. This is a mathematical way of saying "lose any P+1 of N disks." I am hesitant to beat this dead horse, yet it is a nuance that either I have completely misunderstood or many people I've met have completely missed. Whether a stripe of

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Richard Elling
On Sep 16, 2009, at 1:09 PM, Bob Friesenhahn wrote: On Wed, 16 Sep 2009, Thomas Burgess wrote: hrm, i always thought raidz took longerlearn something every day =) And you were probably right, in spite of Richard's lack of knowledge of a study or the feeling in his gut. Just look at t

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Marty Scholes
> Yes. This is a mathematical way of saying > "lose any P+1 of N disks." I am hesitant to beat this dead horse, yet it is a nuance that either I have completely misunderstood or many people I've met have completely missed. Whether a stripe of mirrors or mirror of a stripes, any single failure m

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Bob Friesenhahn
On Wed, 16 Sep 2009, Thomas Burgess wrote: hrm, i always thought raidz took longerlearn something every day =) And you were probably right, in spite of Richard's lack of knowledge of a study or the feeling in his gut. Just look at the many postings here about resilvering and you will se

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Richard Elling
On Sep 16, 2009, at 12:50 PM, Marty Scholes wrote: This line of reasoning doesn't get you very far. It is much better to take a look at the mean time to data loss (MTTDL) for the various configurations. I wrote a series of blogs to show how this is done. http://blogs.sun.com/relling/tags/mttdl"

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Marty Scholes
> This line of reasoning doesn't get you very far. > It is much better to take a look at > the mean time to data loss (MTTDL) for the various > configurations. I wrote a > series of blogs to show how this is done. > http://blogs.sun.com/relling/tags/mttdl"; > target="_blank">http://blogs.sun.com

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Thomas Burgess
hrm, i always thought raidz took longerlearn something every day =) On Wed, Sep 16, 2009 at 2:14 PM, Richard Elling wrote: > On Sep 16, 2009, at 10:42 AM, Thomas Burgess wrote: > > Mirrors are much quicker to replace if one DOES fail though...so i would >> think that bad stuff could happen

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Richard Elling
On Sep 16, 2009, at 10:42 AM, Thomas Burgess wrote: Mirrors are much quicker to replace if one DOES fail though...so i would think that bad stuff could happen with EITHER solutionIf you buy a bunch of hard drives for a raidz and they are all from the same batch they might all fail aroun

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Thomas Burgess
Mirrors are much quicker to replace if one DOES fail though...so i would think that bad stuff could happen with EITHER solutionIf you buy a bunch of hard drives for a raidz and they are all from the same batch they might all fail around the same time...what if you have a raidz2 group and 2 driv

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Richard Elling
On Sep 16, 2009, at 9:38 AM, Marty Scholes wrote: Generally speaking, striping mirrors will be faster than raidz or raidz2, but it will require a higher number of disks and therefore higher cost to The main reason to use raidz or raidz2 instead of striping mirrors would be to keep the cost down,

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Thomas Burgess
At the end of the day, it TOTALLY depends on your needs. raidz may be the best bet for you if you simply do not need the speed of mirrors, and as another user mentioned, it DOES offer better fault tollerence. Figure out what your needs are for your workload THEN ask. These type of loaded questio

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread David Dyer-Bennet
On Wed, September 16, 2009 10:35, cindy.swearin...@sun.com wrote: > Detaching disks from a mirror isn't ideal but if you absolutely have > to reuse a disk temporarily then go with mirrors. See the output below. > You can replace disks in either configuration if you want to switch > smaller disks

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Markus Kovero
38 To: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] RAIDZ versus mirrroed > Generally speaking, striping mirrors will be faster > than raidz or raidz2, > but it will require a higher number of disks and > therefore higher cost to > The main reason to use > raidz or raidz

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Marty Scholes
> Generally speaking, striping mirrors will be faster > than raidz or raidz2, > but it will require a higher number of disks and > therefore higher cost to > The main reason to use > raidz or raidz2 instead > of striping mirrors would be to keep the cost down, > or to get higher usable > space out

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Bob Friesenhahn
On Wed, 16 Sep 2009, en...@businessgrade.com wrote: Hi. If I am using slightly more reliable SAS drives versus SATA, SSDs for both L2Arc and ZIL and lots of RAM, will a mirrored pool of say 24 disks hold any significant advantages over a RAIDZ pool? A mirrored pool will support more IOPs. Th

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Scott Meilicke
I think in theory the ZIL/L2ARC should make things nice and fast if your workload includes sync requests (database, iscsi, nfs, etc.), regardless of the backend disks. But the only sure way to know is test with your work load. -Scott -- This message posted from opensolaris.org _

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Cindy . Swearingen
In addition, if you need the flexibility of moving disks around until the device removal CR integrates, then mirrored pools are more flexible. Detaching disks from a mirror isn't ideal but if you absolutely have to reuse a disk temporarily then go with mirrors. See the output below. You can repla

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread eneal
Quoting David Magda : On Wed, September 16, 2009 10:31, Edward Ned Harvey wrote: Hi. If I am using slightly more reliable SAS drives versus SATA, SSDs for both L2Arc and ZIL and lots of RAM, will a mirrored pool of say 24 disks hold any significant advantages over a RAIDZ pool? Generally spea

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread David Magda
On Wed, September 16, 2009 10:31, Edward Ned Harvey wrote: >> Hi. If I am using slightly more reliable SAS drives versus SATA, SSDs >> for both L2Arc and ZIL and lots of RAM, will a mirrored pool of say 24 >> disks hold any significant advantages over a RAIDZ pool? > > Generally speaking, striping

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Edward Ned Harvey
> Hi. If I am using slightly more reliable SAS drives versus SATA, SSDs > for both L2Arc and ZIL and lots of RAM, will a mirrored pool of say 24 > disks hold any significant advantages over a RAIDZ pool? Generally speaking, striping mirrors will be faster than raidz or raidz2, but it will require

Re: [zfs-discuss] RAIDZ versus mirrroed

2009-09-16 Thread Thomas Burgess
it should be faster. It really depends on what you are using it for though, I've been using raidz for my system and i'm very happy with it. On Wed, Sep 16, 2009 at 8:55 AM, wrote: > Hi. If I am using slightly more reliable SAS drives versus SATA, SSDs for > both L2Arc and ZIL and lots of RAM,