Unless you can guarantee that all KVs affected by a delete marker have been deleted you cannot remove the delete marker. That's a fact :) That means in order to remove a delete marker you have to look everywhere where a potentially affected KV could reside. Currently that is everywhere and hence only a major compaction can do that.
Sergey is working on striped compactions where will be able (hopefully) to limit where we have to look for affected to something smaller than "everything". -- Lars ________________________________ From: Ted Yu <[email protected]> To: [email protected] Sent: Thursday, March 14, 2013 9:41 AM Subject: Re: Is it feasible to delete qualified tombstones during minor compaction? That is correct. On Thu, Mar 14, 2013 at 9:28 AM, ramkrishna vasudevan < [email protected]> wrote: > So seeing the final patch it means that whatever Liang has asked for is not > available right? > > Regards > Ram > > On Thu, Mar 14, 2013 at 9:55 PM, Ted Yu <[email protected]> wrote: > > > The issue was HBASE-7902 > > > > The final patch was just mostly renaming: > > > > - Collections.singletonList(memstoreScanner), > > ScanType.MINOR_COMPACT, > > + Collections.singletonList(memstoreScanner), > > ScanType.COMPACT_RETAIN_DELETES, > > > > Cheers > > > > On Thu, Mar 14, 2013 at 9:22 AM, Anoop John <[email protected]> > wrote: > > > > > Hi > > > I have seen some issue raised by Sergey regarding removing > > Delete > > > markers during minor compaction. Not remembering the issue id and > details > > > :( > > > > > > -Anoop- > > > > > > > > > On Thu, Mar 14, 2013 at 3:11 PM, 谢良 <[email protected]> wrote: > > > > > > > Currently, all tombstones only get cleaned up with major > compactions. I > > > > guess the original > > > > design probably considered that we need to ensure whether there are > > other > > > > versions of > > > > the deleted key in other storefiles which are not included in minor > > > > compaction? If that's > > > > true, i have a thought, maybe we could delete a bunch of tombstones > per > > > > bloom filter, in > > > > other words, we can try to query the deleted key in other > storefiles's > > > > bloom filter info to determine > > > > whether we can remove it safely or not. > > > > I am not familiar with compaction&bloomfilter internal, it's just a > raw > > > > thought right now:) > > > > > > > > Regards, > > > > Liang > > > > > >
