Thanks Ted and Jeffrey. I noticed HBASE-8763 and I agree and can understand it. Since our internal branch is based on 0.94.11, I wonder if there is according patch for 0.94 of HBASE-8763. ________________________________________ 发件人: Jeffrey Zhong [jzh...@hortonworks.com] 发送时间: 2014年5月28日 2:46 收件人: dev@hbase.apache.org 主题: Re: Two kinds of data inconsistency
Yes, that's one of HBASE-8763 intention. The issues could happen when region moves and we bump up memstore readpoint which would cause the scanner returns inconsistent data as well. One solution is that we could keep mvcc number around but it can't cover all cases such as in replication & data recovery where we apply changes based on the ordering of log change sequence id. Using log sequence id to replace mvcc, we will have a consistent view against snapshot if a client has the read point when opening the scanner. Thanks, -Jeffrey On 5/27/14 9:57 AM, "Ted Yu" <yuzhih...@gmail.com> wrote: >Unification of mvcc and seqId would pave way for solving such >inconsistencies. >See HBASE-8763 > >Cheers > > >On Tue, May 27, 2014 at 3:16 AM, 冯宏华 <fenghong...@xiaomi.com> wrote: > >> Not sure whether there already had similar discussion on it, sorry for >> re-raising if yes. >> >> 1. data inconsistency between master and peer clusters: >> a). write a keyvalue KV1 at a specific coordinate (row, cf, col, ts) >> with value V1 to master cluster A, and since there is active scanner >>while >> flushing, KV1's memstoreTS not set to 0 in the resultant hfile F1 >> b). write KV1 once again with the same coordinate (row, cf, col, ts) >>but >> with different value V2, and no active scanner while flushing this time, >> KV1's memstoreTS is set to 0 in the resultant hfile F2 >> c). two KV1 are replicated to peer cluster serially, no active scanner >> when flushing and they are flushed to two different hfiles both with >> memstoreTS=0 >> >> now, a client reads KV1 from the master cluster will find the value is >> V1 (since its memstoreTS is larger), and when it reads KV1 from peer >> cluster will find the value is V2 (since memstoreTS are equal but the >> latter's seqID is larger) >> >> 2. data inconsistency in different time phases: >> a). write a keyvalue KV1 at a specific coordinate (row, cf, col, ts) >> with value V1 to master cluster A, and since there is active scanner >>while >> flushing, KV1's memstoreTS is not set to 0 in the resultant hfile F1 >> b). write KV1 once again with the same coordinate (row, cf, col, ts) >>but >> with different value V2, and no active scanner while flushing this time, >> KV1's memstoreTS is set to 0 in the resultant hfile F2 >> >> reading KV1 now will find the value is V1 (since its memstoreTS is >> larger) >> >> c). after a while a compact including F1(but not F2) occurs and KV1's >> memstoreTS is set to 0 since no active scanner >> >> reading KV1 now will find the value is V2 (since memstoreTS are equal >> but the latter's seqID is larger) >> >> Keeping mvcc untouched during a keyvalue's whole lifecycle (during >> flush/compact, or failover/HLog-replay) can avoid above two kinds of >>data >> inconsistency, any opinion? >> -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
