Stuart Anderson wrote:

On Jun 21, 2009, at 8:57 PM, Richard Elling wrote:

Stuart Anderson wrote:
It is currently taking ~1 week to resilver an x4500 running S10U6,
recently patched with~170M small files on ~170 datasets after a
disk failure/replacement, i.e.,

wow, that is impressive.  There is zero chance of doing that with a
manageable number of UFS file systems.

However, it is a bit disconcerting to have to run with reduced data
protection for an entire week. While I am certainly not going back to
UFS, it seems like it should be at least theoretically possible to do this
several orders of magnitude faster, e.g., what if every block on the
replacement disk had its RAIDZ2 data recomputed from the degraded
array regardless of whether the pool was using it or not. In that case
I would expect it to be able to sequentially reconstruct in the same few
hours it would take a HW RAID controller to do the same RAID6 job.

ZFS reconstruction is done in time order, so the workload is random for
data which has been updated over time.

Nevertheless, in my lab testing, I was not able to create a random-enough
workload to not be write limited on the reconstructing drive.  Anecdotal
evidence shows that some systems are limited by the random reads.

Perhaps there needs to be an option to re-order the loops for
resilvering on pools with lots of small files to resilver in device
order rather than filesystem order?

The information is not there.  Unlike RAID-1/5/6, ZFS does not
require a 1:N mapping of blocks.


scrub: resilver in progress for 53h47m, 30.72% done, 121h19m to go

Is there anything that can be tuned to improve this performance, e.g.,
adding a faster cache device for reading and/or writing?

Resilver tends to be bound by one of two limits:

1. sequential write performance of the resilvering device

2. random I/O performance of the non-resilvering devices


A quick look at iostat leads me to conjecture that the vdev rebuilding is
taking a very low priority compared to ongoing application I/O (NFSD
in this case). Are there any ZFS knobs that control the relative priority of
resilvering to other disk I/O tasks?

Yes, it is low priority.  This is one argument for the competing RFEs:
CR 6592835, resilver needs to go faster
CR 6494473, ZFS needs a way to slow down resilvering

-- richard

_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Reply via email to