[
https://issues.apache.org/jira/browse/HBASE-9553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769121#comment-13769121
]
Lars Hofhansl commented on HBASE-9553:
--------------------------------------
The memstore stores small variable sized KVs so slab is essential there.
Not sure a slab is needed or even desired here, as we already have fixed (well
after we do some simple padding) sized chunks for memory. The padding is simple
and low overhead.
Could calculate standard variation of the KV sizes and add that to the HFile's
metadata. Then the padding could be a multiple of the standard deviation,
subject to some maximum (like 2% of the hfile's blocksize or something).
For testing, I would generate data with KVs drawn from a simple size
distribution and then measure the GC as we evict/replace block in the block
cache.
[~vasu.mariy...@gmail.com], this is the idea I was talking about earlier today.
> Pad HFile blocks to a fixed size before placing them into the blockcache
> ------------------------------------------------------------------------
>
> Key: HBASE-9553
> URL: https://issues.apache.org/jira/browse/HBASE-9553
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
>
> In order to make it easy on the garbage collector and to avoid full
> compaction phases we should make sure that all (or at least a large
> percentage) of the HFile blocks as cached in the block cache are exactly the
> same size.
> Currently an HFile block is typically slightly larger than the declared block
> size, as the block will accommodate that last KV on the block. The padding
> would be a ColumnFamily option. In many cases 100 bytes would probably be a
> good value to make all blocks exactly the same size (but of course it depends
> on the max size of the KVs).
> This does not have to be perfect. The more blocks evicted and replaced in the
> block cache are of the exact same size the easier it should be on the GC.
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
