On Mon, Jul 28, 2014 at 1:50 PM, gabriela montiel < gabriela.mont...@oracle.com> wrote:
> Hi St.Ack, > > We are working with HBase 0.96.1.1-cdh5.0.1 using 10 region servers. We > configured the bucket cache and set the -XX:MaxDirectMemorySize to 32g, > heap size of 12g, and hbase.regionserver.count=40. The bucket cache > configuration is the following (added in hbase-site.xml): > > > <property> > <name>hbase.regionserver.handler.count</name> > <value>40</value> > </property> > <property> > <name>hbase.bucketcache.ioengine</name> > <value>offheap</value> > </property> > <property> > <name>hbase.bucketcache.percentage.in.combinedcache</name> > <value>0.8</value> > </property> > <property> > <name>hbase.bucketcache.size</name> > <value>40000</value> > </property> > > > After restarting HBase, we checked out the HBase process from console we > are able to set the -XX:MaxDirectMemorySize=32, as well as the memory heap > size we configured. However, when we go to the http://<hostname>:60010/conf, > the*hbase.regionserver.count* is still set up to 30 (presumably the default > value) and the bucketcache parameters added (bucketcache.ioengine, > bucketcache.percentage.in.combinedcache, bucketcache.size) are not showed > up, so we are not sure if the parameters are taken into account or not. > > I do not know what hbase.regionserver.count is. Do you mean hbase.regionserver.handler.count? > Is there a way to validate that the bucketcache parameters as well as the > new value for the hbase.regionserver.count are really taken from the > hbase-site.xml we changed? > It is bit awkward in 0.96 but look in your logs. Look for ' LOG.info("Allocating LruBlockCache with maximum size " + StringUtils.humanReadableInt(lruCacheSize));' Should be 0.2 * 12G. Also looking for LOG.info("Started bucket cache"); If you turn on DEBUG even temporarily on the regionserver, you should see stats about your block cache emitted every 5 minutes or so: LOG.debug("BucketCache Stats: " + "failedBlockAdditions=" + this.failedBlockAdditions.get() + ", " + "total=" + StringUtils.byteDesc(totalSize) + ", " + "free=" + StringUtils.byteDesc(freeSize) + ", " + "usedSize=" + StringUtils.byteDesc(usedSize) +", " + "cacheSize=" + StringUtils.byteDesc(cacheSize) +", " + "accesses=" + cacheStats.getRequestCount() + ", " + "hits=" + cacheStats.getHitCount() + ", " + "IOhitsPerSecond=" + cacheStats.getIOHitsPerSecond() + ", " + "IOTimePerHit=" + String.format("%.2f", cacheStats.getIOTimePerHit())+ ", " + "hitRatio=" + (cacheStats.getHitCount() == 0 ? "0," : (StringUtils.formatPercent(cacheStats.getHitRatio(), 2)+ ", ")) + "cachingAccesses=" + cacheStats.getRequestCachingCount() + ", " + "cachingHits=" + cacheStats.getHitCachingCount() + ", " + "cachingHitsRatio=" +(cacheStats.getHitCachingCount() == 0 ? "0," : (StringUtils.formatPercent(cacheStats.getHitCachingRatio(), 2)+ ", ")) + "evictions=" + cacheStats.getEvictionCount() + ", " + "evicted=" + cacheStats.getEvictedCount() + ", " + "evictedPerRun=" + cacheStats.evictedPerEviction()); St.Ack