On Wed, 2009-09-16 at 14:19 -0700, Richard Elling wrote:
> Actually, I had a ton of data on resilvering which shows mirrors and
> raidz equivalently bottlenecked on the media write bandwidth. However,
> there are other cases which are IOPS bound (or CR bound :-) which
> cover some of the postings here. I think Sommerfeld has some other
> data which could be pertinent.
I'm not sure I have data, but I have anecdotes and observations, and a
few large production pools used for solaris development by me and my
coworkers.
the biggest one (by disk count) takes 80-100 hours to scrub and/or
resilver.
my working hypothesis is that resilver of pools which:
1) have a lot of files, directories, filesystems, and periodic
snapshots
2) have atime updates enabled (default config)
3) have regular (daily) jobs doing large-scale filesystem tree-walks
wind up rewriting most blocks of the dnode files on every tree walk
doing atime updates, and as a result the dnode file (but not most of the
blocks it points to) differs greatly from daily snapshot to daily
snapshot.
as a result, scrub/resilver traversals end up spending most of their
time doing random reads of the dnode files of each snapshot.
here are some bugs that, if fixed, might help:
6678033 resilver code should prefetch
6730737 investigate colocating directory dnodes
- Bill
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
