[ https://issues.apache.org/jira/browse/HBASE-14468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970117#comment-14970117 ]
Hadoop QA commented on HBASE-14468: ----------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12768121/HBASE-14468-v6.patch against master branch at commit 467bc098a9512afca38356da56d92c351f15b042. ATTACHMENT ID: 12768121 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 1734 checkstyle errors (more than the master's current 1733 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 site{color}. The patch appears to cause mvn post-site goal to fail. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16181//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16181//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16181//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16181//console This message is automatically generated. > Compaction improvements: FIFO compaction policy > ----------------------------------------------- > > Key: HBASE-14468 > URL: https://issues.apache.org/jira/browse/HBASE-14468 > Project: HBase > Issue Type: Improvement > Reporter: Vladimir Rodionov > Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-14468-v1.patch, HBASE-14468-v2.patch, > HBASE-14468-v3.patch, HBASE-14468-v4.patch, HBASE-14468-v5.patch, > HBASE-14468-v6.patch > > > h2. FIFO Compaction > h3. Introduction > FIFO compaction policy selects only files which have all cells expired. The > column family MUST have non-default TTL. > Essentially, FIFO compactor does only one job: collects expired store files. > I see many applications for this policy: > # use it for very high volume raw data which has low TTL and which is the > source of another data (after additional processing). Example: Raw > time-series vs. time-based rollup aggregates and compacted time-series. We > collect raw time-series and store them into CF with FIFO compaction policy, > periodically we run task which creates rollup aggregates and compacts > time-series, the original raw data can be discarded after that. > # use it for data which can be kept entirely in a a block cache (RAM/SSD). > Say we have local SSD (1TB) which we can use as a block cache. No need for > compaction of a raw data at all. > Because we do not do any real compaction, we do not use CPU and IO (disk and > network), we do not evict hot data from a block cache. The result: improved > throughput and latency both write and read. > See: https://github.com/facebook/rocksdb/wiki/FIFO-compaction-style > h3. To enable FIFO compaction policy > For table: > {code} > HTableDescriptor desc = new HTableDescriptor(tableName); > > desc.setConfiguration(DefaultStoreEngine.DEFAULT_COMPACTION_POLICY_CLASS_KEY, > FIFOCompactionPolicy.class.getName()); > {code} > For CF: > {code} > HColumnDescriptor desc = new HColumnDescriptor(family); > > desc.setConfiguration(DefaultStoreEngine.DEFAULT_COMPACTION_POLICY_CLASS_KEY, > FIFOCompactionPolicy.class.getName()); > {code} > Make sure, that table has disabled region splits (either by setting > explicitly DisabledRegionSplitPolicy or by setting > ConstantSizeRegionSplitPolicy and very large max region size). You will have > to increase to a very large number store's blocking file number : > *hbase.hstore.blockingStoreFiles* as well. > > h3. Limitations > Do not use FIFO compaction if : > * Table/CF has MIN_VERSION > 0 > * Table/CF has TTL = FOREVER (HColumnDescriptor.DEFAULT_TTL) -- This message was sent by Atlassian JIRA (v6.3.4#6332)