[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244228#comment-15244228
]
Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------
[~stack], I assume you are busy with HBaseConf and other stuff, so I appreciate
your effort to look on this patch! Thanks!
\\
bq. Is MemstoreCompactingIterator what used to be a 'snapshot'? You interject
this Iteration step rather than move current MemStore to be a 'snapshot'?
MemstoreCompactingIterator and 'snapshot' are not related. After all, the
diagrams are not good enough and rather confusing, need to improve them (smile)
:)
MemstoreCompactingIterator is an iterator (based on StoreScanner) over the
compaction pipeline segments, only in the context of compaction. It is used to
initiate all the relevant scanners, then to supply the next cell, which
‘‘survives‘‘ the compaction (meaning the cell is not eliminated). Finally, on
iterator.close() all scans are closed so the iteration may be restarted.
With the new code we introduce many variants of compaction, so we may iterate
over the compaction pipeline and compact in three different ways. So it makes
sense to save all this code in different class.
\\
bq. I think Anoop Sam John and ramkrishna.s.vasudevan are doing #2 for you.
Oh! This is great! What is the JIRA number?
\\
bq. I'm interested in how #3 will be done
The plan is to support special very-large variable size Chunks, which are not
going to be reused. In this case we plan to have Chunk per Cell.
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch,
> IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
