[
https://issues.apache.org/jira/browse/HBASE-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16096195#comment-16096195
]
Anoop Sam John edited comment on HBASE-17819 at 7/21/17 12:45 PM:
------------------------------------------------------------------
BlockCacheKey after align the heap overhead is 32 bytes and even after the
change of ref to byte it will be 32. Still it will be worth doing that as
above 32 GB heap size, compressed ref wont be there and refs might take 8
bytes. Then it will make a diff.
BucketEntry was 64 bytes heap and after the patch it will be 48.
Also we will be removing 40 bytes per entry as we remove blocksByHFile Set.
So the math is
Now - 32 + 64 + 40 + 40 = 176
After patch - 32 + 48 + 40 = 120
Tested with Java Instrumentation
was (Author: anoop.hbase):
BlockCacheKey after align the heap overhead is 32 bytes and even after the
change of ref to byte it will be 32. Still it will be worth doing that as
above 32 GB heap size, compressed ref wont be there and refs might take 8
bytes. Then it will make a diff.
BucketEntry was 64 bytes heap and after the patch it will be 48.
Also we will be removing 40 bytes per entry as we remove blocksByHFile Set.
So the math is
Now - 32 + 64 + 40 + 40 = 176
After patch - 32 + 48 + 40 = 120
> Reduce the heap overhead for BucketCache
> ----------------------------------------
>
> Key: HBASE-17819
> URL: https://issues.apache.org/jira/browse/HBASE-17819
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-17819_V1.patch, HBASE-17819_V2.patch
>
>
> We keep Bucket entry map in BucketCache. Below is the math for heapSize for
> the key , value into this map.
> BlockCacheKey
> ---------------
> String hfileName - Ref - 4
> long offset - 8
> BlockType blockType - Ref - 4
> boolean isPrimaryReplicaBlock - 1
> Total = 12 (Object) + 17 = 29
> BucketEntry
> ------------
> int offsetBase - 4
> int length - 4
> byte offset1 - 1
> byte deserialiserIndex - 1
> long accessCounter - 8
> BlockPriority priority - Ref - 4
> volatile boolean markedForEvict - 1
> AtomicInteger refCount - 16 + 4
> long cachedTime - 8
> Total = 12 (Object) + 51 = 63
> ConcurrentHashMap Map.Entry - 40
> blocksByHFile ConcurrentSkipListSet Entry - 40
> Total = 29 + 63 + 80 = 172
> For 10 million blocks we will end up having 1.6GB of heap size.
> This jira aims to reduce this as much as possible
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
