[ https://issues.apache.org/jira/browse/HBASE-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347126#comment-17347126 ]
Bryan Beaudreault commented on HBASE-25894: ------------------------------------------- I'm on a very old EOL'd version of HBase, but recently looked into this there. One thing I noticed is that [CostFromRegionLoadFunction#cost|https://github.com/apache/hbase/blob/741b4b4674be0c500bc7a474739d6481834c5e4a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/CostFromRegionLoadFunction.java#L55-L67] we are calculating the full cost of every region for every server on every iteration of the balancer. On the other hand, our CandidateGenerators typically generate actions which only affect 1 or 2 regions on 1 or 2 servers. As such we should really only need to re-calculate cost for the 2 servers which are affected by an action. This is a lot of wasted work for big clusters with 1000s of regions or 100s of regionservers. I improved this dramatically by caching the costs and invalidating them in tryAction. Here is the patch I wrote for my version of hbase: [https://gist.github.com/bbeaudreault/fca47f8694792be8516049c9d883a581.] It will not apply to HBase 2+, but wanted to contribute it to this Jira incase it's helpful. > Improve the performance for region load and region count related cost > functions > ------------------------------------------------------------------------------- > > Key: HBASE-25894 > URL: https://issues.apache.org/jira/browse/HBASE-25894 > Project: HBase > Issue Type: Sub-task > Components: Balancer, Performance > Reporter: Duo Zhang > Assignee: Duo Zhang > Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > For a large cluster, we have a lot of regions, so computing the whole cost > will be expensive. We should try to remove the unnecessary calculation as > much as possible. -- This message was sent by Atlassian Jira (v8.3.4#803005)