[
https://issues.apache.org/jira/browse/HBASE-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174541#comment-13174541
]
Phabricator commented on HBASE-5033:
------------------------------------
Kannan has commented on the revision
"[jira][HBASE-5033][[89-fb]]Opening/Closing store in parallel to reduce region
open/close time".
Liyin-- This looks very good. Some minor comments inlined.
Also: Initially, I was not sure if this parallelization is worth doing for
close. But as you pointed out-- with EvictOnClose turned on, this
parallelization might be useful for close scenario too.
On a separate note, with EvictOnClose, it seems like there are two cases
related to closing a region.
a) cluster shutdown related region closes
b) load balancing related region closes.
For the cluster shutdown case, even if EvictOnClose is true, I think we
should by-pass the evictions to avoid the cost of cleaning up the blocks
individually from the block cache since the server is shutting down anyway. For
the load balancing case or normal compaction related file deletion case, it
makes sense to keep do the evictions on close. I don't think the code currently
distinguishes the two cases... (but I haven't looked carefully to double check).
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:529-541 might
be good to make this block a helper function. This block of code is repeated 4
times with minor differences in params...
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:528 since 30
is repeated a few times, would be better to make a "const" for this too. And I
would set the default much lower... to say 8 or something. Not sure how many
disks is typical.
src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:294 Openner ->
Opener
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:527 plural
form here (maxStoreOpenerThread -> maxStoreOpenerThreads) and similarly in the
other places.
REVISION DETAIL
https://reviews.facebook.net/D933
> Opening/Closing store in parallel to reduce region open/close time
> ------------------------------------------------------------------
>
> Key: HBASE-5033
> URL: https://issues.apache.org/jira/browse/HBASE-5033
> Project: HBase
> Issue Type: Improvement
> Reporter: Liyin Tang
> Assignee: Liyin Tang
> Attachments: D933.1.patch, D933.2.patch, D933.3.patch, D933.4.patch
>
>
> Region servers are opening/closing each store and each store file for every
> store in sequential fashion, which may cause inefficiency to open/close
> regions.
> So this diff is to open/close each store in parallel in order to reduce
> region open/close time. Also it would help to reduce the cluster restart time.
> 1) Opening each store in parallel
> 2) Loading each store file for every store in parallel
> 3) Closing each store in parallel
> 4) Closing each store file for every store in parallel.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
