On Jun 11, 2014, at 6:33 AM, Gregory Farnum <g...@inktank.com> wrote:

> On Tue, May 20, 2014 at 6:44 PM, Guang Yang <yguan...@outlook.com> wrote:
>> Hi ceph-devel,
>> Like some users of Ceph, we are using Ceph for a latency sensitive project, 
>> and scrubbing (especially deep-scrubbing) impacts the SLA in a non-trivial 
>> way, as commodity hardware could fail in one way or the other, I think it is 
>> essential to have scrubbing enabled to preserve data durability.
>> 
>> Inspired by how erasure coding backend implement scrubbing[1], I am 
>> wondering if the following changes is valid to somehow reduce the 
>> performance impact from scrubbing:
>> 1. Store the CRC checksum along with each physical copy of the object on 
>> filesystem (via xattr or omap?)
>> 2. For read request, it checks the CRC locally and if it mismatch, redirect 
>> the request to a replica and mark the PG as inconsistent.
> 
> The problem with this is that you need to maintain the CRC across
> partial overwrites of the object. And the real cost of scrubbing isn't
> in the network traffic, it's in the disk reads, which you would have
> to do anyway with this method. :)
Thanks Greg for the response!
Partial update is the right concern if that happens frequently. However, the 
major benefit of this proposal is to postpone the CRC check to READ request 
instead of doing it from within a background job (although we may still need to 
do background check as deep-scrubbing, we can reduce the frequency 
dramatically). By checking the CRC at read time, in-consistent object are 
marked as inconsistent (PG) and further we can trigger a repair for the PG.
> -Greg
> Software Engineer #42 @ http://inktank.com | http://ceph.com
> 
>> 
>> This is just a general idea and details (like append) will need to further 
>> discussed.
>> 
>> By having such, we can schedule the scrubbing less aggresively but still 
>> preserve the durability for read.
>> 
>> Does this make some sense?
>> 
>> [1] http://ceph.com/docs/master/dev/osd_internals/erasure_coding/pgbackend/
>> 
>> Thanks,
>> Guang Yang--
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reply via email to