Re: [zfs-discuss] Re: Mirrored Raidz

[Ed Plese]

On Tue, Oct 24, 2006 at 03:31:41PM -0400, Dale Ghent wrote:
> Okay, then if the person can stand to lose even more space, do zfs  
> mirroring on each JBOD. Then we'd have a mirror of mirrors instead of  
> a mirror of raidz's.
> 
> Remember, the OP wanted chassis-level redundancy as well as  
> redundancy within the domain of each chassis. You can't do that now  
> with ZFS unless you combine ZFS with SVM.

If you have the disk space to let you do a mirror of mirrors, you could
create the pool from a series of 4 way mirrors, with each mirror
containing 2 disks from each JBOD enclosure.  This would give you the
ability to sustain the simultaneous failure of an entire enclosure and
at least one (though sometimes multiple) disk failure in the working
enclosure.  This would also be a pure ZFS solution.


Ed Plese
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Jeremy Teo]

On 10/25/06, Jonathan Edwards <[EMAIL PROTECTED]> wrote:


On Oct 24, 2006, at 12:26, Dale Ghent wrote:

> On Oct 24, 2006, at 12:33 PM, Frank Cusack wrote:
>
>> On October 24, 2006 9:19:07 AM -0700 "Anton B. Rang"
>> <[EMAIL PROTECTED]> wrote:
 Our thinking is that if you want more redundancy than RAID-Z,
 you should
 use RAID-Z with double parity, which provides more reliability
 and more
 usable storage than a mirror of RAID-Zs would.
>>>
>>> This is only true if the drives have either independent or identical
>>> failure modes, I think.  Consider two boxes, each containing ten
>>> drives.
>>> Creating RAID-Z within each box protects against single-drive
>>> failures.
>>> Mirroring the boxes together protects against single-box failures.
>>
>> But mirroring also protects against single-drive failures.
>
> Right, but mirrored raidz would in this case protect the admin from:
>
> 1) one entire jbod chassis/comm failure, and
> 2) individual drive failure in the remaining chassis during an
> occurrence of (1)
>
> Since the person is dealing with JBODS and not hardware RAID
> arrays, my suggestion is to combine ZFS and SVM.
>
> 1) Use ZFS and make a raidz-based ZVOL of disks on each of the two
> JBODs
> 2) Use SVM to mirror the two ZVOLs. Newfs that with UFS.
>
> Not at all optimal, but it'll work. It would be nice if you could
> manage a mirror of existing vdevs within ZFS and this mirroring
> would be a special case where it would be dumb and just present the
> volume and pass through most of the stuff to the raidz (or
> whatever) vdev below. It would be silly to double-cksum and
> compress everything, not to mention the possibility of differing
> record sizes.

RFE submitted as 6485689

I'm willing to work on this if no one else is.

:)
--
Regards,
Jeremy
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Torrey McMahon]

Frank Cusack wrote:


I don't think we know what the OP wanted. :-) 



I understand the paranoia around overlapping raid levels - And yes they 
are out to get you - but in the past some of the requirements were 
around performance in a failure mode. Do we have any data concerning the 
performance of a degraded RAIDZ volume?


This might alleviate the fears of someeven though we are out to get 
you. Wait...did I see we? :)


--
Torrey McMahon
Sun Microsystems Inc.

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Frank Cusack]

On October 24, 2006 3:31:41 PM -0400 Dale Ghent <[EMAIL PROTECTED]> wrote:

On Oct 24, 2006, at 3:23 PM, Frank Cusack wrote:


 says a raid-z
vdev has the read throughput of 1 drive for random reads.  Compared
to #drives for a stripe.  That's pretty significant.


Okay, then if the person can stand to lose even more space, do zfs
mirroring on each JBOD. Then we'd have a mirror of mirrors instead of  a
mirror of raidz's.

Remember, the OP wanted chassis-level redundancy as well as  redundancy
within the domain of each chassis. You can't do that now  with ZFS unless
you combine ZFS with SVM.


I don't think we know what the OP wanted. :-)

-frank
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Dale Ghent]

On Oct 24, 2006, at 3:23 PM, Frank Cusack wrote:


 says a raid-z
vdev has the read throughput of 1 drive for random reads.  Compared
to #drives for a stripe.  That's pretty significant.


Okay, then if the person can stand to lose even more space, do zfs  
mirroring on each JBOD. Then we'd have a mirror of mirrors instead of  
a mirror of raidz's.


Remember, the OP wanted chassis-level redundancy as well as  
redundancy within the domain of each chassis. You can't do that now  
with ZFS unless you combine ZFS with SVM.


/dale

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Frank Cusack]

On October 24, 2006 3:15:10 PM -0400 Dale Ghent <[EMAIL PROTECTED]> wrote:

On Oct 24, 2006, at 2:46 PM, Richard Elling - PAE wrote:


Pedantic question, what would this gain us other than better data
retention?
Space and (especially?) performance would be worse with RAID-Z+1
than 2-way mirrors.


You answered your own question, it would gain the user better data
retention :)

The space tradeoff is an obvious side effect and unavoidable. For
situations where this is not an overriding issue, it just isn't an
issue. I don't believe performance would be adversely impacted to a
practical degree, though.


Really?

 says a raid-z
vdev has the read throughput of 1 drive for random reads.  Compared
to #drives for a stripe.  That's pretty significant.

-frank
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Jonathan Edwards]

there's 2 approaches:

1) RAID 1+Z where you mirror the individual drives across trays and  
then RAID-Z the whole thing

2) RAID Z+1 where you RAIDZ each tray and then mirror them

I would argue that you can lose the most drives in configuration 1  
and stay alive:


With a simple mirrored stripe you lose if you lose 1 drives in each  
tray.

With configuration 2 this takes it 2 drives in each tray.
With configuration 1 you have to lose both sides of a 2 mirrored sets  
to fail.


so it's not a space or performance model .. simply an availability  
model with failing disk


Jonathan

On Oct 24, 2006, at 12:46, Richard Elling - PAE wrote:


Pedantic question, what would this gain us other than better data
retention?
Space and (especially?) performance would be worse with RAID-Z+1
than 2-way mirrors.
 -- richard

Frank Cusack wrote:
On October 24, 2006 9:19:07 AM -0700 "Anton B. Rang"  
<[EMAIL PROTECTED]> wrote:
Our thinking is that if you want more redundancy than RAID-Z,  
you should
use RAID-Z with double parity, which provides more reliability  
and more

usable storage than a mirror of RAID-Zs would.


This is only true if the drives have either independent or identical
failure modes, I think.  Consider two boxes, each containing ten  
drives.
Creating RAID-Z within each box protects against single-drive  
failures.

Mirroring the boxes together protects against single-box failures.

But mirroring also protects against single-drive failures.
-frank
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Dale Ghent]

On Oct 24, 2006, at 2:46 PM, Richard Elling - PAE wrote:


Pedantic question, what would this gain us other than better data
retention?
Space and (especially?) performance would be worse with RAID-Z+1
than 2-way mirrors.


You answered your own question, it would gain the user better data  
retention :)


The space tradeoff is an obvious side effect and unavoidable. For  
situations where this is not an overriding issue, it just isn't an  
issue. I don't believe performance would be adversely impacted to a  
practical degree, though.


A "dumb" ZFS mirror strategy in this case would just copy reads and  
writes to and from the vdevs below it, OR pre-package the writes  
itself with compression and checksums and send that data below to the  
raidz's to be stored (which would probably be more problematic to  
implement in the zfs code).


With the latter, checksums and compression would be done only once  
(at the mirror level) and not done by each of the n number of  
underlying vdevs.


So, a little ascii art to summarize:

1) The probably-easiest-to-implement approach:

  [app]
  |
[zfs volume]
  |
   [vdev mirror]  <-- passes thru read/write ops,  
regulates recordsize. It's mainly "dumb"

  |
  [raidz vdev]-- |--[raidz vdev]...  <-- each vdev generates cksums,  
compression per normal

|   |   |   |  |   |   |   |
   [phys devs]  [phys devs]


2) The less-CPU-but-more-convoluted approach:

  [app]
  |
[zfs volume]
  |
   [vdev mirror]  <-- generates cksums, compression,  
regulates recordsize

  |
  [raidz vdev]-- |--[raidz vdev]...  <-- each vdev just stores data  
as it is passed in from above

|   |   |   |  |   |   |   |
   [phys devs]  [phys devs]


Any of those two would be quite handy in the environment where you  
want to mirror data between, say, JBODs and retention is the primary  
goal.


/dale
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Richard Elling - PAE]

Pedantic question, what would this gain us other than better data
retention?
Space and (especially?) performance would be worse with RAID-Z+1
than 2-way mirrors.
 -- richard

Frank Cusack wrote:
On October 24, 2006 9:19:07 AM -0700 "Anton B. Rang" 
<[EMAIL PROTECTED]> wrote:

Our thinking is that if you want more redundancy than RAID-Z, you should
use RAID-Z with double parity, which provides more reliability and more
usable storage than a mirror of RAID-Zs would.


This is only true if the drives have either independent or identical
failure modes, I think.  Consider two boxes, each containing ten drives.
Creating RAID-Z within each box protects against single-drive failures.
Mirroring the boxes together protects against single-box failures.


But mirroring also protects against single-drive failures.

-frank
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Jonathan Edwards]

On Oct 24, 2006, at 12:26, Dale Ghent wrote:


On Oct 24, 2006, at 12:33 PM, Frank Cusack wrote:

On October 24, 2006 9:19:07 AM -0700 "Anton B. Rang"  
<[EMAIL PROTECTED]> wrote:
Our thinking is that if you want more redundancy than RAID-Z,  
you should
use RAID-Z with double parity, which provides more reliability  
and more

usable storage than a mirror of RAID-Zs would.


This is only true if the drives have either independent or identical
failure modes, I think.  Consider two boxes, each containing ten  
drives.
Creating RAID-Z within each box protects against single-drive  
failures.

Mirroring the boxes together protects against single-box failures.


But mirroring also protects against single-drive failures.


Right, but mirrored raidz would in this case protect the admin from:

1) one entire jbod chassis/comm failure, and
2) individual drive failure in the remaining chassis during an  
occurrence of (1)


Since the person is dealing with JBODS and not hardware RAID  
arrays, my suggestion is to combine ZFS and SVM.


1) Use ZFS and make a raidz-based ZVOL of disks on each of the two  
JBODs

2) Use SVM to mirror the two ZVOLs. Newfs that with UFS.

Not at all optimal, but it'll work. It would be nice if you could  
manage a mirror of existing vdevs within ZFS and this mirroring  
would be a special case where it would be dumb and just present the  
volume and pass through most of the stuff to the raidz (or  
whatever) vdev below. It would be silly to double-cksum and  
compress everything, not to mention the possibility of differing  
record sizes.


RFE submitted as 6485689

.je
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Frank Cusack]

On October 24, 2006 2:26:49 PM -0400 Dale Ghent <[EMAIL PROTECTED]> wrote:

Since the person is dealing with JBODS and not hardware RAID arrays,  my
suggestion is to combine ZFS and SVM.

1) Use ZFS and make a raidz-based ZVOL of disks on each of the two JBODs
2) Use SVM to mirror the two ZVOLs. Newfs that with UFS.


Why wouldn't you use a zfs mirror on top of the two zvol's?

-frank
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Dale Ghent]

On Oct 24, 2006, at 12:33 PM, Frank Cusack wrote:

On October 24, 2006 9:19:07 AM -0700 "Anton B. Rang"  
<[EMAIL PROTECTED]> wrote:
Our thinking is that if you want more redundancy than RAID-Z, you  
should
use RAID-Z with double parity, which provides more reliability  
and more

usable storage than a mirror of RAID-Zs would.


This is only true if the drives have either independent or identical
failure modes, I think.  Consider two boxes, each containing ten  
drives.
Creating RAID-Z within each box protects against single-drive  
failures.

Mirroring the boxes together protects against single-box failures.


But mirroring also protects against single-drive failures.


Right, but mirrored raidz would in this case protect the admin from:

1) one entire jbod chassis/comm failure, and
2) individual drive failure in the remaining chassis during an  
occurrence of (1)


Since the person is dealing with JBODS and not hardware RAID arrays,  
my suggestion is to combine ZFS and SVM.


1) Use ZFS and make a raidz-based ZVOL of disks on each of the two JBODs
2) Use SVM to mirror the two ZVOLs. Newfs that with UFS.

Not at all optimal, but it'll work. It would be nice if you could  
manage a mirror of existing vdevs within ZFS and this mirroring would  
be a special case where it would be dumb and just present the volume  
and pass through most of the stuff to the raidz (or whatever) vdev  
below. It would be silly to double-cksum and compress everything, not  
to mention the possibility of differing record sizes.


/dale
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Mirrored Raidz

[Frank Cusack]

On October 24, 2006 9:19:07 AM -0700 "Anton B. Rang" <[EMAIL PROTECTED]> 
wrote:

Our thinking is that if you want more redundancy than RAID-Z, you should
use RAID-Z with double parity, which provides more reliability and more
usable storage than a mirror of RAID-Zs would.


This is only true if the drives have either independent or identical
failure modes, I think.  Consider two boxes, each containing ten drives.
Creating RAID-Z within each box protects against single-drive failures.
Mirroring the boxes together protects against single-box failures.


But mirroring also protects against single-drive failures.

-frank
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[zfs-discuss] Re: Mirrored Raidz

[Anton B. Rang]

>Our thinking is that if you want more redundancy than RAID-Z, you should 
>use RAID-Z with double parity, which provides more reliability and more 
>usable storage than a mirror of RAID-Zs would.

This is only true if the drives have either independent or identical failure 
modes, I think.  Consider two boxes, each containing ten drives.  Creating 
RAID-Z within each box protects against single-drive failures.  Mirroring the 
boxes together protects against single-box failures.

>(Also, expressing "mirror of RAID-Zs" from the CLI would be a bit messy;
>you'd have to introduce parentheses in vdev descriptions or something.)

That doesn't sound so bad, actually.  (An alternative would be to take the SVM 
approach and allow vdevs to be built up in multiple commands.)
 
 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[zfs-discuss] Re: Mirrored Raidz

[Jeb Campbell]

Don't know if you are running current OpenSolaris or can wait for Solaris 10 
11/06 (should be released in November).

Either of those will contain raidz2 (which is like raid6 where you lose 2 
disks).

For max space with some redundancy, I would make one raidz2 vdev of all 8 
disks.  You will get the space of 6 disks, and can you can lose *any* 2 disks.

Don't know if that helps...
 
 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss