[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418386#comment-15418386
]
ramkrishna.s.vasudevan commented on HBASE-14921:
------------------------------------------------
bq.1. Size arithmetic. I'll be publishing a design proposal for this issue here
and on HBASE-16229 soon, let's manage the rest of the discussion there. There
seems to be an elegant solution, which I even have a partial code for. Can post
a suggested patch immediately after HBASE-14921 is committed.
Yes. We are infact waiting for gettig in HBASE-16229 after this JIRA is done.
Size arithmetic is important for us for the offheap accounting also.
bq.The current patch circumvents the "too many open files" with a simple
heuristic that prevents too many segments from piling up. However, we realize
that this issue might require more discussion, so let's open a new jira for it.
Makes sense. So were you able to observe the problems that we highlighted in
the previous discussions - We got two problems one is the invalid negative size
for memstore and the 'Too many open files'?
Were you able to get this patch running with the PE tool and the configs that
we had updated in the earlier discussions?
bq.And finally, the crown jewel - the CellChunkMap project. Its main value is
providing platform for the Off-Heap Memory project. Here too, let us create a
new jira, focusing on the following:
Yes. Once you can get the CellChunkMap code into it - we can help you with the
new type of Cell to avoid chunk copying and the MSLAB chunk creation related
code which we have maintained internally.
bq.The Big Cells issue.
+1 to address this. Need to see what best can be done here. May be discuss what
we call as a BigCell.
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
> HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch,
> HBASE-14921-V06-CAO.patch, HBASE-14921-V08-CAO.patch,
> HBASE-14921-V09-CAO.patch, InitialCellArrayMapEvaluation.pdf,
> IntroductiontoNewFlatandCompactMemStore.pdf, MemStoreSizes.pdf,
> MemstoreItrCountissue.patch, NewCompactingMemStoreFlow.pptx
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
