[ https://issues.apache.org/jira/browse/HBASE-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606864#comment-14606864 ]
Lei Chen commented on HBASE-13965: ---------------------------------- Sorry for the delay. I propose the following changes which can be seen illustrated in the attached class diagram. The {{StochasticLoadBalancer}} extends {{BaseLoadBalancer}} which has a class variable of type {{MetricsBalancer}}. The {{MetricsBalancer}} contains a private class variable of type {{MetricsBalancerSource}}. The {{MetricsBalancerSource}} is an interface which defines which metrics can be reported to JMX. This proves to make extension difficult for load balancer implementation specific metrics (e.g. the {{StochasticLoadBalancer}}). Adding metrics to the generic interface is not appropriate being it is used by all load balancers and should not contain any load balancer specific metrics. I propose to create a class extending {{MetricsBalancer}} to provide specific load balancer metrics. To use this class, I propose to add a constructor to {{BaseLoadBalancer}} which allows for the balancer instance metrics class to be passed in. (Thanks [~enis] for code review and giving the constructor suggestion!) In the constructor of {{StochasticLoadBalancer}}, an instance of {{MetricsStochasticBalancer}} is created and passed to a new constructor added to {{BaseLoadBalancer}}, which will use it to replace the default {{MetricsBalancer}}. The function used to add metrics is declared as following: {code} public void updateStochasticCost(String tableName, String costFunctionName, String costFunctionDesc, Double value); {code} In {{MetricsBalancer}}, the {{private final}} class variable {{source}} was previously hardcoded and instantiated in its constructor; I propose a new function {{initSource}} which can be overridden to set this variable. As such, in the subclass {{MetricsStochasticBalancer}}, {{initSource}} will create a {{MetricsStochasticBalancerSource}} instance instead of the default {{MetricsBalancerSource}}. Finally, to give good insight to the internal status of {{StochasticLoadBalancer}}, we are considering adding metrics for each cost function, as well as the overall cost. For example, if the balancing is carried out for table "MyTable1" and the {{StochasticLoadBalancer}} has 3 cost functions "MoveCost", "LocalityCost", and "RegionReplicaHostCost", then 4 metrics will be added to “HBase -> Master -> Balancer” as following: MyTable1_Overall MyTable1_MoveCost MyTable1_LocalityCost MyTable1_RegionReplicaHostCost I'm building the patch, any suggestion is appreciated. > Stochastic Load Balancer JMX Metrics > ------------------------------------ > > Key: HBASE-13965 > URL: https://issues.apache.org/jira/browse/HBASE-13965 > Project: HBase > Issue Type: Improvement > Components: Balancer, metrics > Reporter: Lei Chen > Assignee: Lei Chen > > Today’s default HBase load balancer (the Stochastic load balancer) is cost > function based. The cost function weights are tunable but no visibility into > those cost function results is directly provided. > A driving example is a cluster we have been tuning which has skewed rack size > (one rack has half the nodes of the other few racks). We are tuning the > cluster for uniform response time from all region servers with the ability to > tolerate a rack failure. Balancing LocalityCost, RegionReplicaRack Cost and > RegionCountSkew Cost is difficult without a way to attribute each cost > function’s contribution to overall cost. > What this jira proposes is to provide visibility via JMX into each cost > function of the stochastic load balancer, as well as the overall cost of the > balancing plan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)