[ https://issues.apache.org/jira/browse/HBASE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646037#comment-16646037 ]
Anoop Sam John commented on HBASE-21000: ---------------------------------------- RN for the issue would be better. So 1.5 will be identical to 2.0 wrt the compaction throughput controller right > Default limits for PressureAwareCompactionThroughputController are too low > -------------------------------------------------------------------------- > > Key: HBASE-21000 > URL: https://issues.apache.org/jira/browse/HBASE-21000 > Project: HBase > Issue Type: Improvement > Affects Versions: 1.5.0 > Reporter: Andrew Purtell > Assignee: Andrew Purtell > Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-21000-branch-1.patch, HBASE-21000.patch > > > In PressureAwareCompactionThroughputController: > {code:java} > /** > > > * A throughput controller which uses the follow schema to limit throughput > > > * <ul> > > > * <li>If compaction pressure is greater than 1.0, no limitation.</li> > > > * <li>In off peak hours, use a fixed throughput limitation > > > * {@value #HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_OFFPEAK}</li> > > > * <li>In normal hours, the max throughput is tuned between > > > * {@value #HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_LOWER_BOUND} and > > > * {@value #HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_HIGHER_BOUND}, using the > formula "lower + > > * (higher - lower) * compactionPressure", where compactionPressure is > in range [0.0, 1.0]</li> > > * </ul> > > > */ > {code} > The lower and upper bounds are 10MB/sec and 20MB/sec, respectively: > {code:java} > public static final String > HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_HIGHER_BOUND = > "hbase.hstore.compaction.throughput.higher.bound"; > private static final long > DEFAULT_HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_HIGHER_BOUND = > 20L * 1024 * 1024; > public static final String > HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_LOWER_BOUND = > "hbase.hstore.compaction.throughput.lower.bound"; > private static final long > DEFAULT_HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_LOWER_BOUND = > 10L * 1024 * 1024; > {code} > (In contrast, in PressureAwareFlushThroughputController the lower and upper > bounds are 10x of those limits, at 100MB/sec and 200MB/sec, respectively.) > In fairly light load scenarios we see compaction quickly falls behind and > write clients are backed off or failing due to RegionTooBusy exceptions. > Although compaction throughput becomes unbounded after the store reaches the > blocking file count, in the lead up to this the default settings do not > provide enough bandwidth to stave off blocking. The defaults should be > increased. > I'm not sure what new good defaults make sense. We could start by doubling > them to 20MB/sec and 40MB/sec respectively. Might need to be doubled again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)