[
https://issues.apache.org/jira/browse/HBASE-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-7843:
------------------------------------
Attachment: HBASE-7843-v0.patch
I will submit /r/; tests are running (many passed :)).
The idea of the patch is as follows.
Right now, we have CompactionRequest, which combines some logical properties of
compaction, some physical details necessary to execute it (store, region), and
execution details pertaining to CompactSplitThread. We also have
CompactSelection which has a subset of logical properties of CompactionRequest;
until recently it used to manage off-peak compactions, but that was moved, so
it has almost no logic. Also, CompactionRequest can, since recently, be
subclassed by coprocessors.
The main purpose of this patch is to encapsulate the flow of compaction - from
getting files to select from thru selection and compaction.
To that end, and for general cleanup, the following was done.
1) CompactSelection is gone; its few fields are merged into CompactionRequest.
2) HStore, HRegionServer, and HRegion are gone from CompactionRequest via
variety of means, mostly (3).
3) CompactionSplitThread-related logic of CompactionRequest was moved into
CompactionSplitThread's nested class "CompactionRunner".
4) StoreEngine now creates CompactionContext class (could just be called
"Compaction"), which has the request, as well as hidden details of
policy/compactor/etc. that are used to achieve the main purpose of the patch.
This class replaces CompactionRequest in the calls to HRegion and Store
compact(...) methods. I initially wanted to make compaction context the
orchestration class for HRegion/HStore/etc., with protected abstract method for
hidden details, but it seems awkward now, so most of the logic will stay in
HRegion/HStore, who will call CompactionContext::compact instead of
Compactor::compact where appropriate.
4a) For (4) (CompactionContext creation), StoreEngine was made generic on
policy/compactor/sfm types, to allow for protected strongly-typed fields
instead of hidden generic type fields, and to thus avoid casts.
4b) Also due to (4) some methods on CompactionPolicy class are no longer
general and necessary.
5) Fixed issue in HStore::completeCompaction - it works now because it's always
called with one file, but it won't actually work with multiple files. That is
because it's called with single file multiple times by HRegion::compact, and
yet it performs both single-file-replaced logic (move into place), and
multi-file logic (remove files compacting, add new files, etc.).
> enable encapsulating compaction policy/compactor/store file manager
> interaction shennanigans
> --------------------------------------------------------------------------------------------
>
> Key: HBASE-7843
> URL: https://issues.apache.org/jira/browse/HBASE-7843
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Priority: Critical
> Attachments: HBASE-7843-v0.patch
>
>
> To avoid massive casting and/or deciphering of structures traveling between
> SFM, compaction policy, and compactor in non-trivial compaction schemes like
> stripe or level, we need to make interaction between themselves hidden.
> Elsewhere, the changes are being made to coprocessor for compactions that
> will make CompactionRequest a limited-visibility class for users, with
> coprocessors being able to subclass and return it. -This seems like a viable
> solution for the problem at hand too. Policy will (optionally) subclass
> compaction request and return it. Instead of calling something.compact(req),
> req.compact() will be called (with "something" already stored inside req as
> of now), after which, magic will happen.-
> After merging that code I actually see that subclassing compactionrequest in
> both will break the policy, or require bunch of ugly code in coprocessors (a
> subclass for every policy, and telling them apart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
