Hi Ted, thanks a lot for the reference. Ted and All, I have additional minor question though. Would a flush in HBase cause cascaded minor compaction? To be more specific, when a minor compaction finishes, the newly generated HFile may make the condition of minor compaction met again, in this case, would HBase trigger another minor compaction?
For example, say we have following files. HFile index 1, 2, 3, 4, 5, 6 HFile size 1, 1, 3, 3, 9, 9 with hbase.hstore.compactionThreshold=3 (i.e., at least 3 files to merge each time) and ratio is 1, then a flush into such file storage initially generates a new HFile of size 1 (the file is indexed by number 0): HFile index 0, 1, 2, 3, 4, 5, 6 HFile size 1, 1, 1, 3, 3, 9, 9 The compaction condition is met for HFile 0, 1, 2 and a minor compaction is trigger to merge file 1 and 2 and 0 to a new file of size 3, as follows, HFile index 2', 3, 4, 5, 6 HFile size 3, 3, 3, 9, 9 Then the condition for minor compaction met again for HFile 2',3,4, would this be triggered and executed immediately or to say would compaction be cascaded in HBase? Regards, Yun On Thu, Apr 11, 2013 at 12:12 PM, Ted Yu <[email protected]> wrote: > For 0.94, take a look at these files: > > 94-hbase tyu$ ls > src/main/java/org/apache/hadoop/hbase/regionserver/compactions > CompactSelection.java CompactionProgress.java CompactionRequest.java > > You can see the revision history for these files. I suggest you start with > recent JIRAs that touched them. > > Cheers > > On Thu, Apr 11, 2013 at 8:04 AM, yun peng <[email protected]> wrote: > > > Sorry to not mention the version. I am looking at version 0.94.2 which > > seems file RatioBasedCompactionPolicy is not yet in. > > regards, > > Yun > > > > > > On Thu, Apr 11, 2013 at 11:00 AM, Ted Yu <[email protected]> wrote: > > > > > Which release are you targeting ? > > > > > > In trunk, take a look at RatioBasedCompactionPolicy which is the > default: > > > > > > public class RatioBasedCompactionPolicy extends CompactionPolicy { > > > > > > Cheers > > > > > > On Thu, Apr 11, 2013 at 7:52 AM, yun peng <[email protected]> > wrote: > > > > > > > Hi, All > > > > I am asking the source code showing the file selection mechanism for > > > minor > > > > compaction in HBase. I am aware that HBase minor compaction selects > > file > > > > based on size and generation time. But I can not find source file > that > > > > realizes this mechanism in HBase. I am interested in the algorithm > > works > > > > behind it. Kindly point to me the source code and I appreciate it. > > > Thanks. > > > > Regards > > > > Yun > > > > > > > > > >
