Re: HBase restart without region reassigning

2016-10-17 Thread Ted Yu
Since you are using 1.1.2, you may want to look at HBASE-14531 which was fixed in 1.1.3 FYI On Mon, Oct 17, 2016 at 9:13 AM, Alexander Ilyin wrote: > I'm restarting it through Ambari. First time I specified a delay between > regionserver restarts, second time I didn't. Not sure whether Ambari u

Re: HBase restart without region reassigning

2016-10-17 Thread Alexander Ilyin
I'm restarting it through Ambari. First time I specified a delay between regionserver restarts, second time I didn't. Not sure whether Ambari uses graceful restart script internally but I can try to use it directly. On Mon, Oct 17, 2016 at 6:00 PM, Jeremy Carroll wrote: > How are you restarting

Re: HBase restart without region reassigning

2016-10-17 Thread Jeremy Carroll
How are you restarting the cluster? From my experience a graceful rolling restart retains locality. For each regionserver (one at a time) run the graceful restart script to retain local blocks. The master configuration option you specified only works on a full cluster reboot (or master reboot) On

Re: HBase restart without region reassigning

2016-10-17 Thread Alexander Ilyin
Hi Dima, These are instances in the cloud and we're using Consul for name resolution. Regarding network settings, your question is a bit broad... Which settings would you recommend to check first? On Mon, Oct 17, 2016 at 5:28 PM, Dima Spivak wrote: > Hey Alexander, > > Could something be amiss

Re: HBase restart without region reassigning

2016-10-17 Thread Dima Spivak
Hey Alexander, Could something be amiss in your network settings? Seeing phantom datanodes could be tripping things up. Are these physical machines or instances in the cloud? On Monday, October 17, 2016, Alexander Ilyin wrote: > Hi, > > We have a 7-node HBase cluster (version 1.1.2) and we chan

HBase restart without region reassigning

2016-10-17 Thread Alexander Ilyin
Hi, We have a 7-node HBase cluster (version 1.1.2) and we change some of its settings from time to time which requires a restart. The problem is that every time after the restart load balancer reassigns the regions making data locality low. To address this issue we tried the settings described he