On 3/21/2011 3:25 PM, Richard Elling wrote:
On Mar 21, 2011, at 5:09 AM, Edward Ned Harvey wrote:
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
boun...@opensolaris.org] On Behalf Of Richard Elling
How many times do we have to rehash this? The speed of resilver is
dependent on the amount of data, the distribution of data on the
resilvering
device, speed of the resilvering device, and the throttle. It is NOT
dependent
on the number of drives in the vdev.
What the heck? Yes it is. Indirectly. When you say it depends on the
amount of data, speed of resilvering device, etc, what you really mean
(correctly) is that it depends on the total number of used blocks that must
be resilvered on the resilvering device, multiplied by the access time for
the resilvering device. And of course, throttling and usage during resilver
can have a big impact. And various other factors. But the controllable big
factor is the number of blocks used in the degraded vdev.
There is no direct correlation between the number of blocks and resilver time.
Just to be clear here, remember block != slab. Slab is the allocation
unit often seen through the "recordsize" attribute.
The number of data *slabs* directly correlates to resilver time.
So here is how the number of devices in the vdev matter:
If you have your whole pool made of one vdev, then every block in the pool
will be on the resilvering device. You must spend time resilvering every
single block in the whole pool.
If you have the same amount of data, on a pool broken into N smaller vdev's,
then approximately speaking, 1/N of the blocks in the pool must be
resilvered on the resilvering vdev. And therefore the resilver goes
approximately N times faster.
Nope. The resilver time is dependent on the speed of the resilvering disk.
Well, unless my previous posts are completely wrong, I can't see how
resilver time is primarily bounded by speed (i.e bandwidth/throughput)
of the HD for the vast majority of use cases. The IOPS and raw speed
of the underlying backing store help define how fast the workload (i.e.
total used slabs) gets processed. The layout of the vdev, and the
on-disk data distribution, will define the total IOPS required to
resilver the slab workload. Most data distribution/vdev layout
combinations will result in an IOPS-bound resilver disk, not a
bandwidth-saturated resilver disk.
So if you assume the size of the pool or the number of total disks is a
given, determined by outside constraints and design requirements, and then
you faced the decision of how to architect the vdev's in your pool, then
Yes. The number of devices in a vdev do dramatically impact the resilver
time. Only because the number of blocks written in each vdev depend on
these decisions you made earlier.
I do not think it is wise to set the vdev configuration based on a model for
resilver time. Choose the configuration to get the best data protection.
-- richard
Depends on the needs of the end-user. I can certainly see places where
it would be better to build a pool out of RAIDZ2 devices rather than
RAIDZ3 devices. And, of course, the converse. Resilver times should be
a consideration in building your pool, just like performance and disk
costs are. How much you value it, of course, it up to the end-user.
--
Erik Trimble
Java System Support
Mailstop: usca22-123
Phone: x17195
Santa Clara, CA
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
