[
https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593960#comment-15593960
]
ramkrishna.s.vasudevan commented on HBASE-16438:
------------------------------------------------
[~saint....@gmail.com]
The idea was actually to create an index to the cells such that we avoid heap
overhead.
Suppose we have MSLAB and Chunk pool enabled. We do add every cell to a chunk.
Now this is in case of the active segment. Once the segment is either MERGED
(as in HBASE-16608 or we move them to the pipeline) we can actually create an
indexed version of the cell such that they occupy less over head.
If suppose we know to which chunk the cell was copied to, during index creation
we could add the chunkid, length and offset of every cell in to the index array
and we only need to maintain the index array and for any cell retrieval (we do
binary search) on the index. Remember that if we use ChunkPool then we are
going to have a fixed number of chunks and we would be reusing it.
bq.Sequenceid could be used to identify a Cell uniquely. Could we use this to
go indirectly to a chunkid?
A cell can definitely be identified uniquely but how will we know where is the
cell located now? We need a pointer to that to retrive it.
But some points to note before we even do this full fledged is -
-> ChunkPool if not enabled how will we retrieve the chunk?
-> Currently we have a limitation on cell size.
-> Append/Increment does not use MSLAB at all for upsert - so we are at a risk
here?
> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>
> Key: HBASE-16438
> URL: https://issues.apache.org/jira/browse/HBASE-16438
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> For CellChunkMap we may need a cell such that the chunk out of which it was
> created, the id of the chunk be embedded in it so that when doing flattening
> we can use the chunk id as a meta data. More details will follow once the
> initial tasks are completed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
