[
https://issues.apache.org/jira/browse/HBASE-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985218#comment-13985218
]
stack commented on HBASE-10403:
-------------------------------
+1 on notion; especially stuff like a percentage being applied to the
XX:MaxDirectMemorySize for BC use.
We'll need to make sure the old configs continue to work after instituting
these new ones.
On there being less options, I was going to suggest just drop slab cache?
That'd be one less option. i've not looked closely at it but going by Nick
blog post numbers, it does more work to get to same place at least when using
DoubleBlockCache...and for CombinedBlockCache, BucketCache has more options so
lets just concentrate on making it work and easy to use and just let go of
SlabCache? On the other hand, it is looking like there'll be some more BC'
implementations to play with before there will be less (HBASE-8894 and if we
get around to acting on [~vrodionov] suggestions regards cache) so we should
just deal but let us let go of options that have been superseded.
Agree w/ [~zjushch] comments that exposing CBC and DBC will only confuse
(Strike DBC since you can set up BucketCache to do L1/L2 as DBC does?).
Maybe have configs:
hbase.block.cache.l1=LruBlockCache
hbase.block.cache.l2=BucketCache
hbase.block.cache.policy=CBC
... or something like that.
Thinking on it, even simpler is just have hbase.block.cache.impl and pass
LRUBC, or BucketCache, or CBC (they all implement BlockCache interface?) Add a
new class if you want to do a new combination. Keeps it simple.
On the patch:
Looks good. I like the refactorings. We should refactor that setVictim
thing... its awkward. Hack.
+1 moving bucket cache configs into bucket cache class.
Restore the old param below:
- * @param direct true if allocate direct buffer
+ * @param isDirect true if allocate isDirect buffer
(isDirect is a method name, not a param name)
+1 on moving all the sizing into the class the size applies to.
Patch is great. Only thing to decide is what for configs? We have to support
the old.
> Simplify offheap cache configuration
> ------------------------------------
>
> Key: HBASE-10403
> URL: https://issues.apache.org/jira/browse/HBASE-10403
> Project: HBase
> Issue Type: Bug
> Components: io
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Priority: Minor
> Attachments: HBASE-10403.0.patch
>
>
> The BucketCache (HBASE-7404) is a very nice piece of functionality which is
> hidden behind complex configuration. Enabling it currently requires manual
> calculation of L1 cache. It'd be nice to make this easier to use and conform
> better with the existing heap management tools we already have.
> Turning it on currently requires explicitly setting LruBlockCache (L1)
> instance size and IOEngine (L2) size, making sure that L1 size isn't too big
> vs global memstore and total heap. This is further confused by
> hbase.bucketcache.size accepting a percentage of total heap OR explicit size
> in MB. Enabling SlabCache is slightly easier in that it just accepts whatever
> LruBlockCache is provided.
> Turning on BucketCache using off-heap mode could look like:
> hbase-env.sh:
> Set HBASE_REGIONSERVER_OPTS:
> -Xmx5000m
> -XX:MaxDirectMemorySize=15000m
> hbase-site.xml:
> - hbase.regionserver.global.memstore.size = 0.7
> - hbase.regionserver.onheap.blockcache.size = 0.1
> - hbase.regionserver.blockcache.impl = BucketCache
> - hbase.bucketcache.ioengine = offheap
> The result being a CombinedCache instance with 500m LruBlockCache + 15000m
> ByteBufferIOEngine running in direct mode.
> This example does a couple things (mostly for the admin):
> - knows NOT to enable SlabCache
> - s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/
> - maintains the validity of HBaseConfiguration's existing check that global
> MemStore + LruBlockCache == 0.8
> - maps "BucketCache" into meaning "a CombinedCache instance with these
> implementations for L1 and L2."
> - Figures out appropriate values for hbase.bucketcache.size and
> hbase.bucketcache.percentage.in.combinedcache
--
This message was sent by Atlassian JIRA
(v6.2#6252)
