On Mon, Jul 28, 2014 at 1:50 PM, gabriela montiel <
[email protected]> wrote:
> Hi St.Ack,
>
> We are working with HBase 0.96.1.1-cdh5.0.1 using 10 region servers. We
> configured the bucket cache and set the -XX:MaxDirectMemorySize to 32g,
> heap size of 12g, and hbase.regionserver.count=40. The bucket cache
> configuration is the following (added in hbase-site.xml):
>
>
> <property>
> <name>hbase.regionserver.handler.count</name>
> <value>40</value>
> </property>
> <property>
> <name>hbase.bucketcache.ioengine</name>
> <value>offheap</value>
> </property>
> <property>
> <name>hbase.bucketcache.percentage.in.combinedcache</name>
> <value>0.8</value>
> </property>
> <property>
> <name>hbase.bucketcache.size</name>
> <value>40000</value>
> </property>
>
>
> After restarting HBase, we checked out the HBase process from console we
> are able to set the -XX:MaxDirectMemorySize=32, as well as the memory heap
> size we configured. However, when we go to the http://<hostname>:60010/conf,
> the*hbase.regionserver.count* is still set up to 30 (presumably the default
> value) and the bucketcache parameters added (bucketcache.ioengine,
> bucketcache.percentage.in.combinedcache, bucketcache.size) are not showed
> up, so we are not sure if the parameters are taken into account or not.
>
>
I do not know what hbase.regionserver.count is. Do you mean
hbase.regionserver.handler.count?
> Is there a way to validate that the bucketcache parameters as well as the
> new value for the hbase.regionserver.count are really taken from the
> hbase-site.xml we changed?
>
It is bit awkward in 0.96 but look in your logs.
Look for ' LOG.info("Allocating LruBlockCache with maximum size " +
StringUtils.humanReadableInt(lruCacheSize));' Should be 0.2 * 12G.
Also looking for LOG.info("Started bucket cache");
If you turn on DEBUG even temporarily on the regionserver, you should see
stats about your block cache emitted every 5 minutes or so:
LOG.debug("BucketCache Stats: " +
"failedBlockAdditions=" + this.failedBlockAdditions.get() + ", " +
"total=" + StringUtils.byteDesc(totalSize) + ", " +
"free=" + StringUtils.byteDesc(freeSize) + ", " +
"usedSize=" + StringUtils.byteDesc(usedSize) +", " +
"cacheSize=" + StringUtils.byteDesc(cacheSize) +", " +
"accesses=" + cacheStats.getRequestCount() + ", " +
"hits=" + cacheStats.getHitCount() + ", " +
"IOhitsPerSecond=" + cacheStats.getIOHitsPerSecond() + ", " +
"IOTimePerHit=" + String.format("%.2f",
cacheStats.getIOTimePerHit())+ ", " +
"hitRatio=" + (cacheStats.getHitCount() == 0 ? "0," :
(StringUtils.formatPercent(cacheStats.getHitRatio(), 2)+ ", ")) +
"cachingAccesses=" + cacheStats.getRequestCachingCount() + ", " +
"cachingHits=" + cacheStats.getHitCachingCount() + ", " +
"cachingHitsRatio=" +(cacheStats.getHitCachingCount() == 0 ? "0," :
(StringUtils.formatPercent(cacheStats.getHitCachingRatio(), 2)+
", ")) +
"evictions=" + cacheStats.getEvictionCount() + ", " +
"evicted=" + cacheStats.getEvictedCount() + ", " +
"evictedPerRun=" + cacheStats.evictedPerEviction());
St.Ack
