[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387993#comment-15387993
]
Edward Bortnikov commented on HBASE-14921:
------------------------------------------
[~ram_krish], thanks for all the feedback so far. My 2 cents:
1. On-disk fragmentation - agree we might need to merge upon flush (or maybe a
bit before) if the pipeline becomes too big. We don't want to create too many
files, obviously.
2. Flattening without compaction - let's give this feature a second thought,
I'm not all that sure it's worth separating between the two. At least that
would be confusing for the administrator to deal with one more flag. Could you
please bring some hard benchmark numbers that show that the overhead of
compaction doesn't make sense?
Maybe we can solve the fragmentation and compaction problems together? Like, a
new immutable segment is automatically flattened but not merged with other
segments. Once there are enough segments in the pipeline (3?), they are merged,
without any speculative prediction check. Upon flush, the oldest segment goes
to disk - same as now. No flags, no configuration parameters (other than just
using the new Store type).
Opinions? [~stack], please chime in :)
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
> HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch,
> HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf,
> IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
