Hmm...any workaround? I only want to do this: Rebalance the new regions *evenly* to all servers after manually adding splits, so later bulk insertions won't cause contention.
P.S. Looks like two of the region servers which had majority of the regions were down during Major compaction... I guess it had too much data. Jianshi On Tue, Sep 23, 2014 at 12:13 AM, Jianshi Huang <jianshi.hu...@gmail.com> wrote: > Yes, I have access to Master UI, however logs/*.log cannot be opened or > downloaded, must be some security restrictions in the proxy... > > Jianshi > > On Tue, Sep 23, 2014 at 12:06 AM, Ted Yu <yuzhih...@gmail.com> wrote: > >> Do you have access to Master UI ? >> >> <master-address>:60010/logs/ would show you list of log files. >> >> The you can view <master-address>:60010/logs/hbase-<user>-master-XXX.log >> >> Cheers >> >> On Mon, Sep 22, 2014 at 9:00 AM, Jianshi Huang <jianshi.hu...@gmail.com> >> wrote: >> >> > Ah... I don't have access to HMaster logs... I need to ask the admin. >> > >> > Jianshi >> > >> > On Mon, Sep 22, 2014 at 11:49 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> > >> > > bq. assign per-table balancer class >> > > >> > > No that I know of. >> > > Can you pastebin master log involving output from balancer ? >> > > >> > > Cheers >> > > >> > > On Mon, Sep 22, 2014 at 8:29 AM, Jianshi Huang < >> jianshi.hu...@gmail.com> >> > > wrote: >> > > >> > > > Hi Ted, >> > > > >> > > > I moved setBalancerRunning before balancer and run them twice. >> However >> > I >> > > > still got highly skewed region distribution. >> > > > >> > > > I guess it's because of the StochasticLoadBalancer, can I assign >> > > per-table >> > > > balancer class in HBase? >> > > > >> > > > >> > > > Jianshi >> > > > >> > > > On Mon, Sep 22, 2014 at 9:50 PM, Ted Yu <yuzhih...@gmail.com> >> wrote: >> > > > >> > > > > admin.setBalancerRunning() call should precede the call to >> > > > > admin.balancer(). >> > > > > >> > > > > You can inspect master log to see whether regions are being moved >> off >> > > the >> > > > > heavily loaded server. >> > > > > >> > > > > Cheers >> > > > > >> > > > > On Mon, Sep 22, 2014 at 1:42 AM, Jianshi Huang < >> > > jianshi.hu...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Hi Ted and others, >> > > > > > >> > > > > > I did the following after adding splits (without data) to my >> table, >> > > > > however >> > > > > > the region is still very imbalanced (one region server has 221 >> > > regions >> > > > > and >> > > > > > other 50 region servers have about 4~8 regions each). >> > > > > > >> > > > > > admin.balancer() >> > > > > > admin.setBalancerRunning(true, true) >> > > > > > >> > > > > > The balancer class in my HBase cluster is >> > > > > > >> > > > > > org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer >> > > > > > >> > > > > > So, is this behavior expected? Can I assign different balancer >> > class >> > > to >> > > > > my >> > > > > > tables (I don't have HBase admin permission)? Which one should I >> > use? >> > > > > > >> > > > > > I just want HBase to evenly distribute the regions even they >> don't >> > > have >> > > > > > data (that's the purpose of pre-split I think). >> > > > > > >> > > > > > >> > > > > > Jianshi >> > > > > > >> > > > > > >> > > > > > On Sat, Sep 6, 2014 at 12:45 AM, Ted Yu <yuzhih...@gmail.com> >> > wrote: >> > > > > > >> > > > > > > Yes. See the following method in HBaseAdmin: >> > > > > > > >> > > > > > > public boolean balancer() >> > > > > > > >> > > > > > > >> > > > > > > On Fri, Sep 5, 2014 at 9:38 AM, Jianshi Huang < >> > > > jianshi.hu...@gmail.com >> > > > > > >> > > > > > > wrote: >> > > > > > > >> > > > > > > > Thanks Ted! >> > > > > > > > >> > > > > > > > Didn't know I still need to run the 'balancer' command. >> > > > > > > > >> > > > > > > > Is there a way to do it programmatically? >> > > > > > > > >> > > > > > > > Jianshi >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > On Sat, Sep 6, 2014 at 12:29 AM, Ted Yu < >> yuzhih...@gmail.com> >> > > > wrote: >> > > > > > > > >> > > > > > > > > After splitting the region, you may need to run balancer >> to >> > > > spread >> > > > > > the >> > > > > > > > new >> > > > > > > > > regions out. >> > > > > > > > > >> > > > > > > > > Cheers >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Fri, Sep 5, 2014 at 9:25 AM, Jianshi Huang < >> > > > > > jianshi.hu...@gmail.com >> > > > > > > > >> > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > Hi Shahab, >> > > > > > > > > > >> > > > > > > > > > I see, that seems to be the right way... >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > On Sat, Sep 6, 2014 at 12:21 AM, Shahab Yunus < >> > > > > > > shahab.yu...@gmail.com> >> > > > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > Shahab >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > -- >> > > > > > > > > > Jianshi Huang >> > > > > > > > > > >> > > > > > > > > > LinkedIn: jianshi >> > > > > > > > > > Twitter: @jshuang >> > > > > > > > > > Github & Blog: http://huangjs.github.com/ >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > -- >> > > > > > > > Jianshi Huang >> > > > > > > > >> > > > > > > > LinkedIn: jianshi >> > > > > > > > Twitter: @jshuang >> > > > > > > > Github & Blog: http://huangjs.github.com/ >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > -- >> > > > > > Jianshi Huang >> > > > > > >> > > > > > LinkedIn: jianshi >> > > > > > Twitter: @jshuang >> > > > > > Github & Blog: http://huangjs.github.com/ >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Jianshi Huang >> > > > >> > > > LinkedIn: jianshi >> > > > Twitter: @jshuang >> > > > Github & Blog: http://huangjs.github.com/ >> > > > >> > > >> > >> > >> > >> > -- >> > Jianshi Huang >> > >> > LinkedIn: jianshi >> > Twitter: @jshuang >> > Github & Blog: http://huangjs.github.com/ >> > >> > > > > -- > Jianshi Huang > > LinkedIn: jianshi > Twitter: @jshuang > Github & Blog: http://huangjs.github.com/ > -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/
