On 28.09.09 22:01, Richard Elling wrote:
On Sep 28, 2009, at 10:31 AM, Victor Latushkin wrote:
Richard Elling wrote:
On Sep 28, 2009, at 3:42 PM, Albert Chin wrote:
On Mon, Sep 28, 2009 at 12:09:03PM -0500, Bob Friesenhahn wrote:
On Mon, 28 Sep 2009, Richard Elling wrote:
Scrub could be faster, but you can try
tar cf - . > /dev/null
If you think about it, validating checksums requires reading the
data.
So you simply need to read the data.
This should work but it does not verify the redundant metadata. For
example, the duplicate metadata copy might be corrupt but the problem
is not detected since it did not happen to be used.
Too bad we cannot scrub a dataset/object.
Can you provide a use case? I don't see why scrub couldn't start and
stop at specific txgs for instance. That won't necessarily get you to a
specific file, though.
With ever increasing disk and pool sizes it takes more and more time
for scrub to complete its job. Let's imagine that you have 100TB pool
with 90TB of data in it, and there's dataset with 10TB that is
critical and another dataset with 80TB that is not that critical and
you can afford loosing some blocks/files there.
Personally, I have three concerns here.
1. Gratuitous complexity, especially inside a pool -- aka creeping
featurism
There's the idea of priority-based resilvering (though not implemented yet, see
http://blogs.sun.com/bonwick/en_US/entry/smokin_mirrors) that can be simply
extended to scrubs as well.
2. Wouldn't a better practice be to use two pools with different
protection
policies? The only protection policy differences inside a pool
are copies.
In other words, I am concerned that people replace good data
protection
practices with scrubs and expecting scrub to deliver better data
protection
(it won't).
It may be better, it may be not... With two pools you split you bandwidth and
IOPS and space and have more entities to care about...
3. Since the pool contains the set of blocks, shared by datasets, it
is not clear
to me that scrubbing a dataset will detect all of the data
corruption failures
which can affect the dataset. I'm thinking along the lines of
phantom writes,
for example.
That is why it may be useful to always scrub pool-wide metadata or have a way to
specifically request it.
4. the time it takes to scrub lots of stuff
...there are four concerns... :-)
For magnetic media, a yearly scrub interval should suffice for most
folks. I know
some folks who scrub monthly. More frequent scrubs won't buy much.
It won't buy you much in term of magnetic media decay discovery. Unfortunately,
there other sources of corruption as well (including phantom writes you are
thinking about), and being able to discover corruption and recover it as quickly
as possible from the backup it a good thing.
Scrubs are also useful for detecting broken hardware. However, normal
activity will also detect broken hardware, so it is better to think of
scrubs as
finding degradation of old data rather than being a hardware checking
service.
So being able to scrub individual dataset would help to run scrubs of
critical data more frequently and faster and schedule scrubs for less
frequently used and/or less important data to happen much less
frequently.
It may be useful to have a way to tell ZFS to scrub pool-wide metadata
only (space maps etc), so that you can build your own schedule of scrubs.
Another interesting idea is to be able to scrub only blocks modified
since last snapshot.
This can be relatively easy to implement. But remember that scrubs are most
useful for finding data which has degraded from the media. In other
words, old
data. New data is not likely to have degraded yet, and since ZFS is COW,
all of
the new data is, well, new.
This is why having the ability to bound the start and end of a scrub by txg
can be easy and perhaps useful.
This requires exporting concept of the transaction group numbers to the user and
i do not see how it is less complex from the user interface perspective than
being able to request scrub of individual dataset, pool-wide metadata or
newly-written data.
regards,
victor
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
