[
https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15758850#comment-15758850
]
Anastasia Braginsky commented on HBASE-16421:
---------------------------------------------
Hey Again,
Here is the road-map for how we see the CellChunkMap is going to be added to
HBase.
We assume two prerequisites:
1. We need evaluation of how reasonable it would be to use CellChunkMap with
the scans due to the GC. It is going to be accomplished soon by [~ram_krish]
and we will continue by now assuming that CellChunkMaps are valid anyway.
2. The Cell will have the API to get its chunk ID. This will be resolved by
[~anoop.hbase] or [~ram_krish] under HBASE-16437 and HBASE-16438. And you will
just let us know what the API is. What is the status there by the way?
In the following each task is going to be a JIRA, so we try to split the work
and have smaller patches to manage, review, and commit.
*1. MemStoreChunkAllocator Abstraction* Refactoring the separation between
MemStoreChunkPool and (new) MemStoreChunkAllocator. The latter allocates chunks
either from heap or from pool and assigns them the Chunk IDs.
MemStoreChunkAllocator stores the mapping between chunks and their IDs.
*2. CellChunkMap Implementation* Unit-tested only. Not user-observable.
Bringing the code from early versions of HBASE-14921 and adopting to the new
and updated environment.
*3. Flattening segment index (with regular cells) into CellChunkMap upon
in-memory flush* Upon in-memory-flush if the memstore is compacting
(CompactingMemStore) and off-heap MSLAB storage then use CellChunkMap as a
flattened index. Constraints: flattening only for regular cells (no upserted
cells, no big cells). Just assume all cells are regular, the support for
irregular cells will come in following JIRAs.
*4. Copying the upserted cells upon in-memory flush* In time of the in-memory
flush, when a normal-size cell is encountered which is not allocated on MSLAB
(due to upsert) copy it to the MSLAB normal-size chunk.
*5. Support for variable-size chunks in MemStoreChunkAllocator* For future big
cells support. Those chunks are allocated on-heap upon request and released
when unneeded - no pool usage. Assuming there are not much such threads. If the
big cells are a common case for some workload, consider just increasing the
chunk's size to stay off-heap.
*6. Support for flattening with irregular cells*
Any comments?
Thanks!
> Introducing the CellChunkMap as a new additional index variant in the MemStore
> ------------------------------------------------------------------------------
>
> Key: HBASE-16421
> URL: https://issues.apache.org/jira/browse/HBASE-16421
> Project: HBase
> Issue Type: Umbrella
> Reporter: Anastasia Braginsky
> Attachments: CellChunkMapRevived.pdf,
> IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Follow up for HBASE-14921. This is going to be the umbrella JIRA to include
> all the parts of integration of the CellChunkMap to the MemStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
