Re: [zfs-discuss] Scrub and checksum permutations

2012-10-31 Thread Matthew Ahrens
On Thu, Oct 25, 2012 at 2:25 AM, Jim Klimov jimkli...@cos.ru wrote: Hello all, I was describing how raidzN works recently, and got myself wondering: does zpool scrub verify all the parity sectors and the mirror halves? Yes. The ZIO_FLAG_SCRUB instructs the raidz or mirror vdev to read

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-31 Thread Timothy Coalson
On Wed, Oct 31, 2012 at 6:47 PM, Matthew Ahrens mahr...@delphix.com wrote: On Thu, Oct 25, 2012 at 2:25 AM, Jim Klimov jimkli...@cos.ru wrote: Hello all, I was describing how raidzN works recently, and got myself wondering: does zpool scrub verify all the parity sectors and the mirror

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-29 Thread Tomas Forsman
On 28 October, 2012 - Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) sent me these 1,0K bytes: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov I tend to agree that parity calculations likely are faster (even if

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-28 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov I tend to agree that parity calculations likely are faster (even if not all parities are simple XORs - that would be silly for double- or triple-parity sets which may use

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Ray Arachelian
On 10/26/2012 04:29 AM, Karl Wagner wrote: Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. I could understand why it would not store a checksum for a

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Toby Thain
On 27/10/12 11:56 AM, Ray Arachelian wrote: On 10/26/2012 04:29 AM, Karl Wagner wrote: Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. ... Parity is

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Jim Klimov
2012-10-27 20:54, Toby Thain wrote: Parity is very simple to calculate and doesn't use a lot of CPU - just slightly more work than reading all the blocks: read all the stripe blocks on all the drives involved in a stripe, then do a simple XOR operation across all the data. The actual checksums

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Timothy Coalson
On Sat, Oct 27, 2012 at 12:35 PM, Jim Klimov jimkli...@cos.ru wrote: 2012-10-27 20:54, Toby Thain wrote: Parity is very simple to calculate and doesn't use a lot of CPU - just slightly more work than reading all the blocks: read all the stripe blocks on all the drives involved in a stripe,

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-26 Thread Karl Wagner
Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. I could understand why it would not store a checksum for a parity block. It is not really necessary:

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-26 Thread Jim Klimov
2012-10-26 12:29, Karl Wagner wrote: Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. No, for the on-disk sector allocation over M disks, zfs raidzN

[zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Jim Klimov
Hello all, I was describing how raidzN works recently, and got myself wondering: does zpool scrub verify all the parity sectors and the mirror halves? That is, IIRC, the scrub should try to read all allocated blocks and if they are read in OK - fine; if not - fix in-place with redundant data

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Karl Wagner
I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on both disks in a mirrored pair. Logically, it would not make sense not to verify all redundant data. The point of a scrub is to ensure all data is correct. On 2012-10-25 10:25, Jim Klimov

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Karl Wagner I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on both disks in a mirrored pair. Logically, it would not make sense not to

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Jim Klimov
2012-10-25 15:30, Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) пишет: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Karl Wagner I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Logically, yes - I agree this is what we expect to be done. However, at least with the normal ZFS reading pipeline, reads of redundant copies and parities only kick in if the

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Timothy Coalson
On Thu, Oct 25, 2012 at 7:35 AM, Jim Klimov jimkli...@cos.ru wrote: If scrubbing works the way we logically expect it to, it should enforce validation of such combinations for each read of each copy of a block, in order to ensure that parity sectors are intact and can be used for data

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Jim Klimov
2012-10-25 21:17, Timothy Coalson wrote: On Thu, Oct 25, 2012 at 7:35 AM, Jim Klimov jimkli...@cos.ru mailto:jimkli...@cos.ru wrote: If scrubbing works the way we logically expect it to, it should enforce validation of such combinations for each read of each copy of a block, in