Re: [zfs-discuss] Single disk parity

2009-07-10 Thread Haudy Kazemi
Richard Elling wrote: There are many error correcting codes available. RAID2 used Hamming codes, but that's just one of many options out there. Par2 uses configurable strength Reed-Solomon to get multi bit error correction. The par2 source is available, although from a ZFS perspective is

Re: [zfs-discuss] Single disk parity

2009-07-09 Thread Haudy Kazemi
Adding additional data protection options are commendable. On the other hand I feel there are important gaps in the existing feature set that are worthy of a higher priority, not the least of which is the automatic recovery of uberblock / transaction group problems (see Victor Latushkin's

Re: [zfs-discuss] Single disk parity

2009-07-09 Thread Richard Elling
Haudy Kazemi wrote: Adding additional data protection options are commendable. On the other hand I feel there are important gaps in the existing feature set that are worthy of a higher priority, not the least of which is the automatic recovery of uberblock / transaction group problems (see

Re: [zfs-discuss] Single disk parity

2009-07-09 Thread Christian Auby
On Wed, 8 Jul 2009, Moore, Joe wrote: That's true for the worst case, but zfs mitigates that somewhat by batching i/o into a transaction group. This means that i/o is done every 30 seconds (or 5 seconds, depending on the version you're running), allowing multiple writes to be written

Re: [zfs-discuss] Single disk parity

2009-07-08 Thread Mark J Musante
On Wed, 8 Jul 2009, Moore, Joe wrote: The copies code is nice because it tries to put each copy far away from the others. This does have a significant performance impact when on a single spindle, however, because each logical write will be written here and then a disk seek to write it to

Re: [zfs-discuss] Single disk parity

2009-07-08 Thread Richard Elling
Haudy Kazemi wrote: Daniel Carosone wrote: Sorry, don't have a thread reference to hand just now. http://www.opensolaris.org/jive/thread.jspa?threadID=100296 Note that there's little empirical evidence that this is directly applicable to the kinds of errors (single bit, or otherwise)

[zfs-discuss] Single disk parity

2009-07-07 Thread Christian Auby
ZFS is able to detect corruption thanks to checksumming, but for single drives (regular folk-pcs) it doesn't help much unless it can correct them. I've been searching and can't find anything on the topic, so here goes: 1. Can ZFS do parity data on a single drive? e.g. x% parity for all writes,

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Richard Elling
Christian Auby wrote: ZFS is able to detect corruption thanks to checksumming, but for single drives (regular folk-pcs) it doesn't help much unless it can correct them. I've been searching and can't find anything on the topic, so here goes: 1. Can ZFS do parity data on a single drive? e.g. x%

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Louis-Frédéric Feuillette
On Tue, 2009-07-07 at 17:42 -0700, Richard Elling wrote: Christian Auby wrote: ZFS is able to detect corruption thanks to checksumming, but for single drives (regular folk-pcs) it doesn't help much unless it can correct them. I've been searching and can't find anything on the topic, so

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Daniel Carosone
There was a discussion in zfs-code around error-correcting (rather than just -detecting) properties of the checksums currently kept, an of potential additional checksum methods with stronger properties. It came out of another discussion about fletcher2 being both weaker than desired, and

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Christian Auby
You are describing the copies parameter. It really helps to describe it in pictures, rather than words. So I did that. http://blogs.sun.com/relling/entry/zfs_copies_and_data _protection -- richard It's not quite like copies as it's not actually a copy of the data I'm talking about. 10%

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Richard Elling
Christian Auby wrote: You are describing the copies parameter. It really helps to describe it in pictures, rather than words. So I did that. http://blogs.sun.com/relling/entry/zfs_copies_and_data _protection -- richard It's not quite like copies as it's not actually a copy of the data

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Christian Auby
Do you have data to back this up? It's more of a logical observation. The random data corruption I've had up through the years have generally either involved a single sector or two or a full disk failure. 5% parity on a 128KB block size would allow you to lose 6.4KB, or ~10 512 byte sectors.

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Daniel Carosone
Sorry, don't have a thread reference to hand just now. http://www.opensolaris.org/jive/thread.jspa?threadID=100296 Note that there's little empirical evidence that this is directly applicable to the kinds of errors (single bit, or otherwise) that a single failing disk medium would produce.