[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15114679#comment-15114679
]
Hudson commented on HBASE-13082:
--------------------------------
SUCCESS: Integrated in HBase-1.3-IT #460 (See
[https://builds.apache.org/job/HBase-1.3-IT/460/])
HBASE-14970 Backport HBASE-13082 and its sub-jira to branch-1 - recommit
(ramkrishna: rev bc0e5fc048de3f18ab07bd5cb06509eb349e951a)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestIOFencing.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/executor/EventType.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileManager.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/example/TestZooKeeperTableArchiveClient.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactedHFilesDischarger.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCompactedHFilesDischarger.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEncryptionKeyRotation.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFileManager.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChangedReadersObserver.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactedHFilesDischargeHandler.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeStoreFileManager.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MockStoreFile.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/executor/ExecutorType.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFileManager.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
> Key: HBASE-13082
> URL: https://issues.apache.org/jira/browse/HBASE-13082
> Project: HBase
> Issue Type: Bug
> Components: Performance, Scanners
> Reporter: Lars Hofhansl
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt,
> 13082-v4.txt, 13082.txt, 13082.txt, HBASE-13082.pdf, HBASE-13082_1.pdf,
> HBASE-13082_12.patch, HBASE-13082_13.patch, HBASE-13082_14.patch,
> HBASE-13082_15.patch, HBASE-13082_16.patch, HBASE-13082_17.patch,
> HBASE-13082_18.patch, HBASE-13082_19.patch, HBASE-13082_1_WIP.patch,
> HBASE-13082_2.pdf, HBASE-13082_2_WIP.patch, HBASE-13082_3.patch,
> HBASE-13082_4.patch, HBASE-13082_9.patch, HBASE-13082_9.patch,
> HBASE-13082_withoutpatch.jpg, HBASE-13082_withpatch.jpg,
> LockVsSynchronized.java, gc.png, gc.png, gc.png, hits.png, next.png, next.png
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to
> the lock already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make
> the cores wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking
> contract. For example Phoenix does not lock RegionScanner.nextRaw() and
> required in the documentation (not picking on Phoenix, this one is my fault
> as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load.
> RegionScanner operations would keep getting the locks and the
> flushes/compactions would not be able finalize the set of files.
> I'll have a patch soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
