[ https://issues.apache.org/jira/browse/HBASE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539318#comment-16539318 ]
Hudson commented on HBASE-17215: -------------------------------- Results for branch branch-1.4 [build #381 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/381/]: (x) *{color:red}-1 overall{color}* ---- details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/381//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/381//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/381//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 source release artifact{color} -- See build output for details. > Separate small/large file delete threads in HFileCleaner to accelerate > archived hfile cleanup speed > --------------------------------------------------------------------------------------------------- > > Key: HBASE-17215 > URL: https://issues.apache.org/jira/browse/HBASE-17215 > Project: HBase > Issue Type: Improvement > Reporter: Yu Li > Assignee: Yu Li > Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-17215.patch, HBASE-17215.v2.patch, > HBASE-17215.v3.patch > > > When using PCIe-SSD the flush speed will be really quick, and although we > have per CF flush, we still have the > {{hbase.regionserver.optionalcacheflushinterval}} setting and some other > mechanism to avoid data kept in memory for too long to flush small hfiles. In > our online environment we found the single thread cleaner kept cleaning > earlier flushed small files while large files got no chance, which caused > disk full then many other problems. > Deleting hfiles in parallel with too many threads will also increase the > workload of namenode, so here we propose to separate large/small hfile > cleaner threads just like we do for compaction, and it turned out to work > well in our cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)