Actually, major compaction still doesn't look at memstore so the window for reappearing puts is still there, it's just small. I am +1 on the OP idea :)
On Thu, Mar 14, 2013 at 10:00 AM, lars hofhansl <[email protected]> wrote: > Unless you can guarantee that all KVs affected by a delete marker have > been deleted you cannot remove the delete marker. That's a fact :) > That means in order to remove a delete marker you have to look everywhere > where a potentially affected KV could reside. Currently that is everywhere > and hence only a major compaction can do that. > > Sergey is working on striped compactions where will be able (hopefully) to > limit where we have to look for affected to something smaller than > "everything". > > -- Lars > > > > ________________________________ > From: Ted Yu <[email protected]> > To: [email protected] > Sent: Thursday, March 14, 2013 9:41 AM > Subject: Re: Is it feasible to delete qualified tombstones during minor > compaction? > > That is correct. > > On Thu, Mar 14, 2013 at 9:28 AM, ramkrishna vasudevan < > [email protected]> wrote: > > > So seeing the final patch it means that whatever Liang has asked for is > not > > available right? > > > > Regards > > Ram > > > > On Thu, Mar 14, 2013 at 9:55 PM, Ted Yu <[email protected]> wrote: > > > > > The issue was HBASE-7902 > > > > > > The final patch was just mostly renaming: > > > > > > - Collections.singletonList(memstoreScanner), > > > ScanType.MINOR_COMPACT, > > > + Collections.singletonList(memstoreScanner), > > > ScanType.COMPACT_RETAIN_DELETES, > > > > > > Cheers > > > > > > On Thu, Mar 14, 2013 at 9:22 AM, Anoop John <[email protected]> > > wrote: > > > > > > > Hi > > > > I have seen some issue raised by Sergey regarding removing > > > Delete > > > > markers during minor compaction. Not remembering the issue id and > > details > > > > :( > > > > > > > > -Anoop- > > > > > > > > > > > > On Thu, Mar 14, 2013 at 3:11 PM, 谢良 <[email protected]> wrote: > > > > > > > > > Currently, all tombstones only get cleaned up with major > > compactions. I > > > > > guess the original > > > > > design probably considered that we need to ensure whether there are > > > other > > > > > versions of > > > > > the deleted key in other storefiles which are not included in minor > > > > > compaction? If that's > > > > > true, i have a thought, maybe we could delete a bunch of tombstones > > per > > > > > bloom filter, in > > > > > other words, we can try to query the deleted key in other > > storefiles's > > > > > bloom filter info to determine > > > > > whether we can remove it safely or not. > > > > > I am not familiar with compaction&bloomfilter internal, it's just a > > raw > > > > > thought right now:) > > > > > > > > > > Regards, > > > > > Liang > > > > > > > > > >
