[ https://issues.apache.org/jira/browse/HBASE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590545#comment-14590545 ]
Hudson commented on HBASE-13876: -------------------------------- SUCCESS: Integrated in HBase-TRUNK #6580 (See [https://builds.apache.org/job/HBase-TRUNK/6580/]) HBASE-13876 Improving performance of HeapMemoryManager (eclark: rev 623fd63827b2953c150597f24c7205737119bebe) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HeapMemoryManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultHeapMemoryTuner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/RollingStatCalculator.java > Improving performance of HeapMemoryManager > ------------------------------------------ > > Key: HBASE-13876 > URL: https://issues.apache.org/jira/browse/HBASE-13876 > Project: HBase > Issue Type: Improvement > Components: hbase, regionserver > Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1 > Reporter: Abhilash > Assignee: Abhilash > Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13876-v2.patch, HBASE-13876-v3.patch, > HBASE-13876-v4.patch, HBASE-13876-v5.patch, HBASE-13876-v6.patch, > HBASE-13876-v7.patch, HBASE-13876.patch > > > I am trying to improve the performance of DefaultHeapMemoryTuner by > introducing some more checks. The current checks under which the > DefaultHeapMemoryTuner works are very rare so I am trying to weaken these > checks to improve its performance. > Check current memstore size and current block cache size. For say if we are > using less than 50% of currently available block cache size we say block > cache is sufficient and same for memstore. This check will be very effective > when server is either load heavy or write heavy. Earlier version just waited > for number of evictions / number of flushes to be zero which are very rare. > Otherwise based on percent change in number of cache misses and number of > flushes we increase / decrease memory provided for caching / memstore. After > doing so, on next call of HeapMemoryTuner we verify that last change has > indeed decreased number of evictions / flush either of which it was expected > to do. We also check that it does not make the other (evictions / flush) > increase much. I am doing this analysis by comparing percent change (which is > basically nothing but normalized derivative) of number of evictions and > number of flushes during last two periods. The main motive for doing this was > that if we have random reads then we will be having a lot of cache misses. > But even after increasing block cache we wont be able to decrease number of > cache misses and we will revert back and eventually we will not waste memory > on block caches. This will also help us ignore random short term spikes in > reads / writes. I have also tried to take care not to tune memory if do do > not have enough hints as unnecessary tuning my slow down the system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)