[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982561#comment-14982561
]
ramkrishna.s.vasudevan commented on HBASE-13082:
------------------------------------------------
After doing the code changes and testing it in a single node cluster found that
having a single List<StoreFile> in the StorefileManager and adding all the
newly compacted files and not removing the older one till the compaction chore
runs - creates some problems like in flush where we try to count the storefiles
count and decide if there are so many files and wait for the compaciton to
compact it. Now if we have one single list we are bound to iterate thro the
list and find if each of the file is already compacted by checking its state
and then return the count based on that. It may be costlier operation and also
considering the fact that all the test cases depend on the count - the patch
becomes bigger. So instead if we maintain two lists one for the actual store
files and other one for the compacted files and the compaction cleaner will
remove the files from the comapcted list - things should work fine. Just
attaching a patch for QA - some more things am just verifying.
> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
> Key: HBASE-13082
> URL: https://issues.apache.org/jira/browse/HBASE-13082
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: ramkrishna.s.vasudevan
> Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt,
> 13082-v4.txt, 13082.txt, 13082.txt, HBASE-13082.pdf, HBASE-13082_1_WIP.patch,
> HBASE-13082_2_WIP.patch, HBASE-13082_3.patch, HBASE-13082_4.patch, gc.png,
> gc.png, gc.png, hits.png, next.png, next.png
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to
> the lock already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make
> the cores wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking
> contract. For example Phoenix does not lock RegionScanner.nextRaw() and
> required in the documentation (not picking on Phoenix, this one is my fault
> as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load.
> RegionScanner operations would keep getting the locks and the
> flushes/compactions would not be able finalize the set of files.
> I'll have a patch soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
