[
https://issues.apache.org/jira/browse/HBASE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483759#comment-13483759
]
Nicolas Spiegelberg commented on HBASE-6371:
--------------------------------------------
@Lars: you are correct about doing a better job of partitioning newly-written
and stale data. With leveled compaction, the different tiers end up implicitly
becoming different age groups. This was the primary motivation for us.
Also note that we are looking into coprocessor-based compactions, but will
probably utilize that for TSDB-style compactions and other stuff that is more
niche and is questionable if it belongs in the core.
> [89-fb] Tier based compaction
> -----------------------------
>
> Key: HBASE-6371
> URL: https://issues.apache.org/jira/browse/HBASE-6371
> Project: HBase
> Issue Type: Improvement
> Reporter: Akashnil
> Assignee: Liyin Tang
> Labels: noob
> Attachments: HBASE-6371-089fb-commit.patch
>
>
> Currently, the compaction selection is not very flexible and is not sensitive
> to the hotness of the data. Very old data is likely to be accessed less, and
> very recent data is likely to be in the block cache. Both of these
> considerations make it inefficient to compact these files as aggressively as
> other files. In some use-cases, the access-pattern is particularly obvious
> even though there is no way to control the compaction algorithm in those
> cases.
> In the new compaction selection algorithm, we plan to divide the candidate
> files into different levels according to oldness of the data that is present
> in those files. For each level, parameters like compaction ratio, minimum
> number of store-files in each compaction may be different. Number of levels,
> time-ranges, and parameters for each level will be configurable online on a
> per-column family basis.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
