[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946193#comment-14946193
]
ramkrishna.s.vasudevan commented on HBASE-13082:
------------------------------------------------
bq.So, bulk load won't show mid-scan... you have to get to the end? That would
be fine.
New files added by bulk load during an on going scan won't be included in the
current scan. The new files will be included only when a new scan is started.
Currently during a course of a scan the bulk loaded files can be included too.
bq.On the patch, can we get more of Lars comments in on what is going on ....
Could we get rid of some of these getReaderLocks too... in hstorefile, in
hstore, etc.... would be good to not let this stuff out if we can.
Yes we can. I have a working version of a patch. Some issues with Merge
regions and splits. Working on them. All the locks will be removed that are
currently in StoreScanner and also the notifyReaders() will also go away
totally. Just because we are updating the readers during a course of a scan we
needed all those locks.
> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
> Key: HBASE-13082
> URL: https://issues.apache.org/jira/browse/HBASE-13082
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: ramkrishna.s.vasudevan
> Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt,
> 13082-v4.txt, 13082.txt, 13082.txt, gc.png, gc.png, gc.png, hits.png,
> next.png, next.png
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to
> the lock already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make
> the cores wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking
> contract. For example Phoenix does not lock RegionScanner.nextRaw() and
> required in the documentation (not picking on Phoenix, this one is my fault
> as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load.
> RegionScanner operations would keep getting the locks and the
> flushes/compactions would not be able finalize the set of files.
> I'll have a patch soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
