For those following along, this is the e-mail I meant to send to the list but instead sent directly to Tonmaus. My mistake, and I apologize for having to
re-send. === Start === My understanding, limited though it may be, is that a scrub touches ALL data that has been written, including the parity data. It confirms the validity of every bit that has been written to the array. Now, there may be an implementation detail that is responsible for the pathology that you observed. More than likely, I'd imagine. Filing a bug may be in order. Since triple parity RAIDZ exists now, you may want to test with that by grabbing a LiveCD or LiveUSB image from genunix.org. Maybe RAIDZ3 has the same (or worse) problems? As for "scrub management", I pointed out the specific responses from Richard where he noted that scrub I/O priority *can* be tuned. How you do that, I'm not sure. Richard, how does one tune scrub I/O priority? Other than that, as I said, I don't think there is a model (publicly available anyway) describing scrub behavior and how it scales with pool size (< 5 TB, 5 TB - 50 TB, > 50 TB, etc.) or data layout (mirror vs. RAIDZ vs. RAIDZ2). ZFS is really that new, that all of this needs to be reconsidered and modeled. Maybe this is something you can contribute to the community? ZFS is a new storage system, not the same old file systems whose behaviors and quirks are well known because of 20+ years of history. We're all writing a new chapter in data storage here, so it is incumbent upon us to share knowledge in order to answer these types of questions. I think the questions I raised in my longer response are also valid and need to be re-considered. There are large pools in production today. So how are people scrubbing these pools? Please post your experiences with scrubbing 100+ TB pools. Tonmaus, maybe you should repost my other questions in a new, separate thread? === End === On Tue, Mar 16, 2010 at 19:41, Tonmaus <sequoiamo...@gmx.net> wrote: > > Are you sure that you didn't also enable > > something which > > does consume lots of CPU such as enabling some sort > > of compression, > > sha256 checksums, or deduplication? > > None of them is active on that pool or in any existing file system. Maybe > the issue is particular to RAIDZ2, which is comparably recent. On that > occasion: does anybody know if ZFS reads all parities during a scrub? > Wouldn't it be sufficient for stale corruption detection to read only one > parity set unless an error occurs there? > > > The main concern that one should have is I/O > > bandwidth rather than CPU > > consumption since "software" based RAID must handle > > the work using the > > system's CPU rather than expecting it to be done by > > some other CPU. > > There are more I/Os and (in the case of mirroring) > > more data > > transferred. > > What I am trying to say is that CPU may become the bottleneck for I/O in > case of parity-secured stripe sets. Mirrors and simple stripe sets have > almost 0 impact on CPU. So far at least my observations. Moreover, x86 > processors not optimized for that kind of work as much as i.e. an Areca > controller with a dedicated XOR chip is, in its targeted field. > > Regards, > > Tonmaus > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > -- "You can choose your friends, you can choose the deals." - Equity Private "If Linux is faster, it's a Solaris bug." - Phil Harman Blog - http://whatderass.blogspot.com/ Twitter - @khyron4eva
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
