[
https://issues.apache.org/jira/browse/HBASE-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027498#comment-13027498
]
Nicolas Spiegelberg edited comment on HBASE-3842 at 5/1/11 8:57 PM:
--------------------------------------------------------------------
Talked a little with Gary Helmling about this. The main points are:
1. Selection of files for compaction will be decoupled from the actual
compaction/merge itself
2. Splits will be completely decoupled from compactions
3. Since we're making all these changes, it would be nice to add the ability to
short-circuit & instead use custom compaction algorithms.
Here is an initial proposal for the new API.
{code}
/* Called before selecting what files to compact. An sorted candidate set for
compaction is given
and the compaction algorithm will use the remaining contents of the candidate
set (after coprocessor)
as input to the compaction algorithm.
* @param candidates : a mutable copy of StoreFile candidates for this Store.
add/remove within the
coprocessor will affect the candidate set
* @return true to skip default selection algorithm and use all files still in
candidates
* false will use the default selection algorithm, using the current
content of candidates as input
*/
boolean preCompactSelection(byte[] store, List<StoreFile> candidates);
/* Called after selecting what files to compact. Mostly for notification
purposes
* @param candidates : StoreFiles selected for compaction
*/
void postCompactSelection(byte[] store, ImmutableList<StoreFile> selected);
{code}
{code}
/* Called before the compaction merge algorithm should commence. Note that the
coprocessor user can
implement his custom merge algorithm at this point. If you wish to alter the
candidate set, this should
be done in {@link preCompactSelection}.
* @param toCompact : StoreFiles that will be compacted.
* @return null == continue with default compaction algorithm
* @return if not null, assumes a custom compaction algorithm was used and adds
resulting StoreFiles to the Store
*/
List<StoreFile> preCompaction(byte[] store, ImmutableList<StoreFile> toCompact);
/* Called after selecting what files to compact. Mostly for notification
purposes
* @param newFiles : StoreFiles made by this compaction
*/
void postCompaction(byte[] store, ImmutableList<StoreFile> newFiles);
{code}
was (Author: nspiegelberg):
Talked a little with Gary Helmling about this. The main points are:
1. Selection of files for compaction will be decoupled from the actual
compaction/merge itself
2. Splits will be completely decoupled from compactions
3. Since we're making all these changes, it would be nice to add the ability to
short-circuit & instead use custom compaction algorithms.
Here is an initial proposal for the new API.
{code}
/* Called before selecting what files to compact. An sorted candidate set for
compaction is given
and the compaction algorithm will use the remaining contents of the candidate
set (after coprocessor)
as input to the compaction algorithm.
* @param candidates : a mutable copy of StoreFile candidates for this Store.
add/remove within the
coprocessor will affect the candidate set
* @return true to skip default selection algorithm and use all files still in
candidates
* false will use the default selection algorithm, using the current
content of candidates as input
*/
boolean preCompactSelection(byte[] store, List<StoreFile> candidates);
/* Called after selecting what files to compact. Mostly for notification
purposes
* @param candidates : StoreFiles selected for compaction
*/
void postCompactSelection(byte[] store, ImmutableList<StoreFile> selected);
{code}
{code}
/* Called before the compaction merge algorithm should commence. Note that the
coprocessor user can
implement his custom merge algorithm at this point. If you wish to alter the
candidate set, this should
be done in {@link preCompactSelection}.
* @param toCompact : StoreFiles that will be compacted.
* @return null == continue with default compaction algorithm
* @return if not null, assumes a custom compaction algorithm was used and adds
resulting StoreFiles to the Store
*/
List<StoreFile> preCompaction(byte[] store, ImmutableList<StoreFile> toCompact);
/* Called after selecting what files to compact. Mostly for notification
purposes
* @param newFiles : StoreFiles made by this compaction
*/
void postCompaction(byte[] store, List<StoreFile> newFiles);
{code}
> Refactor Coprocessor Compaction API
> -----------------------------------
>
> Key: HBASE-3842
> URL: https://issues.apache.org/jira/browse/HBASE-3842
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors, regionserver
> Affects Versions: 0.92.0
> Reporter: Nicolas Spiegelberg
> Assignee: Nicolas Spiegelberg
> Priority: Minor
> Labels: compaction
> Fix For: 0.92.0
>
>
> After HBASE-3797, the compaction logic flow has been significantly altered.
> Because of this, the current compaction coprocessor API is insufficient for
> gaining full insight into compaction requests/results. Refactor coprocessor
> API after HBASE-3797 is committed to be more extensible and increase
> visibility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
