[ 
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208212#comment-15208212
 ] 

Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------

Hi [~stack],

Thanks for taking a look. It looks like the name “CellBlock" made a confusion.

Please pay attention that we distinguish between the “Cell Data” and the “Cell 
Reference” as it was explained in the attached diagrams.
The “Cell Data” is key, value, tags, sequence ID, etc., it is de facto 
information explaining the given Cell.
However, the “Cell Reference” is any ability to reach this “Cell Data”. The 
“Cell Reference” holds no information explaining the given Cell, but a shortcut 
to where this information can be found. For example, in current MemStore, the 
skip-list holds “Cell References” and MSLAB holds “Cell Data”. 

We didn’t mean to change the way the Cell Data is represented, it is done very 
well via MSLAB. Alternatively, CellData can be allocated directly from Java 
heap or using any Cell Codec/Encoding/Decoding etc.
As it was explained also in the attached documents, all what we meant to do was 
to change the way the shortcut to the Cell in MemStore is working. If it is 
applicable anywhere beside MemStore - great!
The suggested structure uses 4 times less memory than the previous 
ConcurrentSkipListMap, provides no fragmentation and it is ready for 
off-heaping.

Soon, I’ll upload the continuation of this patch, where you’ll see how this 
flat structure is integrated in ImmutableSegment and works well with the 
existing scanners.
Surely, we are also preparing the benchmark to measure the potential.

Thanks,
Anastasia

> Memory optimizations
> --------------------
>
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, 
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, 
> HBASE-14921-V02.patch
>
>
> Memory optimizations including compressed format representation and offheap 
> allocations



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to