On Mar 20, 2011, at 14:24, Roy Sigurd Karlsbakk wrote:
>>> It all depends on the number of drives in the VDEV(s), traffic
>>> patterns during resilver, speed VDEV fill, of drives etc. Still,
>>> close to 6 days is a lot. Can you detail your configuration?
>>
>> How many times do we have to rehash this? The speed of resilver is
>> dependent on the amount of data, the distribution of data on the
>> resilvering device, speed of the resilvering device, and the throttle. It is
>> NOT
>> dependent on the number of drives in the vdev.
>
> Thanks for clearing this up - I've been told large VDEVs lead to long
> resilver times, but then, I guess that was wrong.
There was a thread ("Suggested RaidZ configuration...") a little while back
where the topic of IOps and resilver time came up:
http://mail.opensolaris.org/pipermail/zfs-discuss/2010-September/thread.html#44633
I think this message by Erik Trimble is a good summary:
> Scenario 1: I have 5 1TB disks in a raidz1, and I assume I have 128k slab
> sizes. Thus, I have 32k of data for each slab written to each disk. (4x32k
> data + 32k parity for a 128k slab size). So, each IOPS gets to reconstruct
> 32k of data on the failed drive. It thus takes about 1TB/32k = 31e6 IOPS to
> reconstruct the full 1TB drive.
>
> Scenario 2: I have 10 1TB drives in a raidz1, with the same 128k slab
> sizes. In this case, there's only about 14k of data on each drive for a
> slab. This means, each IOPS to the failed drive only write 14k. So, it takes
> 1TB/14k = 71e6 IOPS to complete.
>
> From this, it can be pretty easy to see that the number of required IOPS to
> the resilvered disk goes up linearly with the number of data drives in a
> vdev. Since you're always going to be IOPS bound by the single disk
> resilvering, you have a fixed limit.
http://mail.opensolaris.org/pipermail/zfs-discuss/2010-September/044660.html
Also, a post by Jeff Bonwick on resilvering:
http://blogs.sun.com/bonwick/entry/smokin_mirrors
Between Richard's and Eric's statements, I would say that while resilver time
is not dependent "number of drives in the vdev", the pool configuration can
affect the IOps rate, and /that/ can affect the time it takes to finish a
resilver. Is that a decent summary?
I think maybe the "number of drives in the vdev" perhaps come into play because
that when people have a lot of disks, they often put them into RAIDZ[123]
configurations. So it's just a matter of confusing the (IOps limiting)
configuration with the fact that one may have many disks.
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
