[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417138#comment-15417138
]
Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------
The current patch includes fixes to all accepted code review comments. Thank
you [~ram_krish], [~anoop.hbase], and [~stack] for your comments!!
The goal of this patch is finalize the implementation reflecting the design
published on this Jira. Our intent is to isolate non-critical issues to the
following jira's, before the current code becomes unmanageable :) The current
patch will go thru the normal review cycle, of course. In the future, we'll be
shooting for much smaller jira's to converge faster.
Suggested roadmap of taking care about the related issues.
1. Size arithmetic. I'll be publishing a design proposal for this issue here
and on HBASE-16229 soon, let's manage the rest of the discussion there. There
seems to be an elegant solution, which I even have a partial code for. Can post
a suggested patch immediately after HBASE-14921 is committed.
2. In-memory flattening/compaction policy. The current patch circumvents the
"too many open files" with a simple heuristic that prevents too many segments
from piling up. However, we realize that this issue might require more
discussion, so let's open a new jira for it. We can deal with all performance
issues and configuration possibilities there.
3. And finally, the crown jewel - the CellChunkMap project. Its main value is
providing platform for the Off-Heap Memory project. Here too, let us create a
new jira, focusing on the following:
- The CellChunkMap implementation itself (already prototyped but not
integrated yet)
- Related design issues (some refactoring of MemStoreChunkPool, MSLAB and
HeapMSLAB).
- Flattening to CellChunkMap (integrating with new [~anoop.hbase] and
[~ram_krish] code)
- The Big Cells issue.
We might need to manage this project as an umbrella jira, to de-clutter the
space :)
Looking forward to your review comments.
Thank you!
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
> HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch,
> HBASE-14921-V06-CAO.patch, HBASE-14921-V08-CAO.patch,
> HBASE-14921-V09-CAO.patch, InitialCellArrayMapEvaluation.pdf,
> IntroductiontoNewFlatandCompactMemStore.pdf, MemstoreItrCountissue.patch,
> NewCompactingMemStoreFlow.pptx
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
