[ https://issues.apache.org/jira/browse/HBASE-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279118#comment-13279118 ]
stack commented on HBASE-5970: ------------------------------ I like the effect this patch has. You are addressing this comment in updateTimers? {code} // This loop could be expensive. {code} Should this be in the new timer class? {code} + private final ConcurrentSkipListSet<ServerName> serversInUpdatingTimer = + new ConcurrentSkipListSet<ServerName>(); {code} Is this javadoc right? {code} + * Add the server to serversInUpdatingTimer, and wait {@link TimerUpdater} to + * update timers for all regions in transition going against this server. {code} Why is doing this work in the background faster? Is it just that it being inline, it takes a noticeable amount of time? This is a nice improvement. > Improve the AssignmentManager#updateTimer and speed up handling opened event > ---------------------------------------------------------------------------- > > Key: HBASE-5970 > URL: https://issues.apache.org/jira/browse/HBASE-5970 > Project: HBase > Issue Type: Improvement > Components: master > Reporter: chunhui shen > Assignee: chunhui shen > Attachments: 5970v3.patch, HBASE-5970.patch, HBASE-5970v2.patch, > HBASE-5970v3.patch > > > We found handing opened event very slow in the environment with lots of > regions. > The problem is the slow AssignmentManager#updateTimer. > We do the test for bulk assigning 10w (i.e. 100k) regions, the whole process > of bulk assigning took 1 hours. > 2012-05-06 20:31:49,201 INFO > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 100000 > region(s) round-robin across 5 server(s) > 2012-05-06 21:26:32,103 INFO > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done > I think we could do the improvement for the AssignmentManager#updateTimer: > Make a thread do this work. > After the improvement, it took only 4.5mins > 2012-05-07 11:03:36,581 INFO > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 100000 > region(s) across 5 server(s), retainAssignment=true > 2012-05-07 11:07:57,073 INFO > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira