Thanks, will take a look. We're using HBase 1.4.0 on AWS EMR. Cheers.
---- Saad On Tue, Mar 20, 2018 at 9:32 PM, Ted Yu <yuzhih...@gmail.com> wrote: > Saad: > You didn't mention the version of hbase you are using. > Please check the version to see if the following were included: > > HBASE-18164 Fast locality computation in balancer > HBASE-16570 Compute region locality in parallel at startup > HBASE-15515 Improve LocalityBasedCandidateGenerator in Balancer > > Cheers > > On Tue, Mar 20, 2018 at 6:11 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > Please consider tuning the following parameters of stochastic load > > balancer : > > > > "hbase.master.balancer.stochastic.maxRunningTime" > > > > default value is 30 seconds. It controls the duration of runtime for > each balanceCluster() > > call. > > > > "hbase.balancer.period" > > > > default is 300 seconds. It controls the maximum time master runs balancer > > for. > > > > You can turn on DEBUG logging and observe the following output in master > > log: > > > > balancer.StochasticLoadBalancer: Finished computing new load balance > plan. Computation took 1200227ms to try 2254 different iterations. Found > a solution that moves 550 regions; Going from a computed cost of > 77.52829271038965 to a new cost of 74.32764924425548 > > > > If you have a dev cluster, you can try different combinations of the > above > > two parameters and get best performance by checking the above log. > > > > Cheers > > > > On Tue, Mar 20, 2018 at 5:18 PM, Saad Mufti <saad.mu...@gmail.com> > wrote: > > > >> Hi, > >> > >> We are using the stochastic load balancer, and have tuned it to do a > >> maximum of 1% of regions in any calculation. But it is way too > >> conservative > >> after that, it moves one region at a time. Is there a way to tell it to > go > >> faster with whatever number of regions it decided to do? I have been > >> looking at the settings in the code but so far a little confused about > >> which exact setting will achieve this. Is it one of the steps settings? > >> > >> Also our cluster was in bad shape for a while as a bunch of region > servers > >> aborted for different reasons. When we stopped everything and brought it > >> back up and enabled the tables, most of the region servers were assigned > >> no > >> regions. Is it for locality reasons that HBase is trying to assign > regions > >> where they were assigned before? Is there a way to tell HBase to ignore > >> that on startup? > >> > >> Thanks. > >> > >> ---- > >> Saad > >> > > > > >
