[ https://issues.apache.org/jira/browse/HBASE-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924147#action_12924147 ]
HBase Review Board commented on HBASE-2462: ------------------------------------------- Message from: "Jonathan Gray" <jg...@apache.org> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1078/ ----------------------------------------------------------- (Updated 2010-10-23 00:48:28.858137) Review request for hbase, stack, Nicolas, Karthik Ranganathan, and Kannan Muthukkaruppan. Changes ------- Working patch. Summary ------- Pulls compaction file selection code into new interface and makes it so it's configurable. Currently is globally configurable but should be easy to make it per-family setting. Also makes the algorithm standalone and testable. Includes a new compaction algorithm based on a new config param 'compactionForce'. See javadoc in compaction classes for explanation. Big test included for new algorithm. Also the TestCompact class includes a neat new way for us to compare compaction algorithms. You specify a bunch of input paramaters and then it runs a simulation and generates statistics. The output looks like: ----- Ran test ----- numPuts=1000000 putSizeRange=1.0KB to 10.0KB numPutsPerGet=10 flushSizeRange=64.0MB to 256.0MB max=10, threshold=3, force=6, factor=0.5 ----- ----- Final Result ----- files=82.2MB, 2.9GB, 898.3MB, 1.3GB memstoreSize=100.8MB totalSize=5.1GB totalThroughput=18.2GB averageFilesPerGet=3.25622 This addresses bug HBASE-2462. http://issues.apache.org/jira/browse/HBASE-2462 Diffs (updated) ----- trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/PriorityCompactionQueue.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1026565 trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactSplitThread.java PRE-CREATION trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionRequestor.java PRE-CREATION trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelector.java PRE-CREATION trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelectorHBase89.java PRE-CREATION trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelectorWithForce.java PRE-CREATION trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java 1026565 trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestPriorityCompactionQueue.java 1026565 trunk/src/test/java/org/apache/hadoop/hbase/regionserver/compact/TestCompact.java PRE-CREATION Diff: http://review.cloudera.org/r/1078/diff Testing ------- TestCompact is passing. Have not run test suite. Thanks, Jonathan > Review compaction heuristic and move compaction code out so standalone and > independently testable > ------------------------------------------------------------------------------------------------- > > Key: HBASE-2462 > URL: https://issues.apache.org/jira/browse/HBASE-2462 > Project: HBase > Issue Type: Improvement > Reporter: stack > Assignee: Jonathan Gray > Priority: Critical > > Anything that improves our i/o profile makes hbase run smoother. Over in > HBASE-2457, good work has been done already describing the tension between > minimizing compactions versus minimizing count of store files. This issue is > about following on from what has been done in 2457 but also, breaking the > hard-to-read compaction code out of Store.java out to a standalone class that > can be the easier tested (and easily analyzed for its performance > characteristics). > If possible, in the refactor, we'd allow specification of alternate merge > sort implementations. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.