[
https://issues.apache.org/jira/browse/HBASE-20460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479352#comment-16479352
]
Anoop Sam John commented on HBASE-20460:
----------------------------------------
Hi [~stack].. How abt below details?
In hbase-2.0.0, HBASE-15179 made the HBase write path to work off heap. By
default the memstores use MSLAB to avoid memory fragmentation. It creates
bigger fixed sized chunks and memstore cell's data will get copied into these
chunks. These chunks can be pooled also and from 2.0.0 the MSLAB pool is by
default ON. The offheaping make use of the MSLAB pool. It creates the MSLAB
chunks as Direct ByteBuffers and pool them. The max size for such an off heap
chunk pool can be specified using the config
'hbase.regionserver.offheap.global.memstore.size'. Each of the chunk will be of
2MB size by default. When a Cell is added to a memstore, the Cell key, value
bytes get copied into these off heap buffers and a Cell POJO will refer to this
memory area. It can reduce the on heap size occupancy of the memstores to a
great extend and reduce the max heap size requirement for RSs in a write heavy
workload. We track the on heap and off heap size of each of the memstore
separately and same with for Region and Global level. The flush decisions
consider both sizes. At Region level, it consider both sizes sum and compares
against the region flush size (128 MB by default). Globally on heap size
occupancy of all memstores are tracked as well as off heap size. When any of
these sizes breaches the lower mark or the max size, regions are selected for
forced flushes.
We had also changed the RPC server req reading part to make use of the
ByteBufferPool's DBBs. But now , by default Netty RPC server is getting used
which is not having this pool usage. So am not adding that info.
> Doc offheap write-path
> ----------------------
>
> Key: HBASE-20460
> URL: https://issues.apache.org/jira/browse/HBASE-20460
> Project: HBase
> Issue Type: Bug
> Components: documentation, Offheaping
> Reporter: stack
> Priority: Critical
> Fix For: 2.1.0
>
>
> We have an empty section in refguide that needs filling in on how to enable
> offheap write-path, how to know you've set it up right or not, how to tune
> it, and how it relates to direct memory allocation and offheap read-path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
