Repository: hbase Updated Branches: refs/heads/master 69f5d707b -> 0adcb4096
HBASE-13884 Fix Compactions section in HBase book Project: http://git-wip-us.apache.org/repos/asf/hbase/repo Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/0adcb409 Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/0adcb409 Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/0adcb409 Branch: refs/heads/master Commit: 0adcb40963392de6731311eb2423d2f9b425d52e Parents: 69f5d70 Author: Michael Stack <st...@apache.org> Authored: Tue Mar 27 17:40:31 2018 -0700 Committer: Michael Stack <st...@apache.org> Committed: Tue Mar 27 17:40:35 2018 -0700 ---------------------------------------------------------------------- src/main/asciidoc/_chapters/architecture.adoc | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hbase/blob/0adcb409/src/main/asciidoc/_chapters/architecture.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc index 7be5f48..f35e118 100644 --- a/src/main/asciidoc/_chapters/architecture.adoc +++ b/src/main/asciidoc/_chapters/architecture.adoc @@ -1757,9 +1757,20 @@ These parameters will be explained in context, and then will be given in a table ====== Being Stuck When the MemStore gets too large, it needs to flush its contents to a StoreFile. -However, a Store can only have `hbase.hstore.blockingStoreFiles` files, so the MemStore needs to wait for the number of StoreFiles to be reduced by one or more compactions. -However, if the MemStore grows larger than `hbase.hregion.memstore.flush.size`, it is not able to flush its contents to a StoreFile. -If the MemStore is too large and the number of StoreFiles is also too high, the algorithm is said to be "stuck". The compaction algorithm checks for this "stuck" situation and provides mechanisms to alleviate it. +However, Stores are configured with a bound on the number StoreFiles, +`hbase.hstore.blockingStoreFiles`, and if in excess, the MemStore flush must wait +until the StoreFile count is reduced by one or more compactions. If the MemStore +is too large and the number of StoreFiles is also too high, the algorithm is said +to be "stuck". By default we'll wait on compactions up to +`hbase.hstore.blockingWaitTime` milliseconds. If this period expires, we'll flush +anyways even though we are in excess of the +`hbase.hstore.blockingStoreFiles` count. + +Upping the `hbase.hstore.blockingStoreFiles` count will allow flushes to happen +but a Store with many StoreFiles in will likely have higher read latencies. Try to +figure why Compactions are not keeping up. Is it a write spurt that is bringing +about this situation or is a regular occurance and the cluster is under-provisioned +for the volume of writes? [[exploringcompaction.policy]] ====== The ExploringCompactionPolicy Algorithm
