Re: Unbalanced region server

2012-04-23 Thread Bryan Beaudreault
Unfortunately I don't have any experience with that version. You might try an hdfs rebalance, but I'm not sure of the implications of that for hbase. Hopefully someone else can be of more help. Good luck! Sent from iPhone. On Apr 24, 2012, at 12:07 AM, David Charle wrote: > On Mon, Apr 23, 2

Re: Unbalanced region server

2012-04-23 Thread Stack
On Mon, Apr 23, 2012 at 9:07 PM, David Charle wrote: > Is there any other way to split and reclaim the space ? > One node has more hdfs blocks than others? Is it a problem? Are you coming close to running out of disk space? If you look in namenode log, what happens when it tries to rebalance b

Re: Unbalanced region server

2012-04-23 Thread David Charle
On Mon, Apr 23, 2012 at 8:43 PM, Bryan Beaudreault wrote: > Over time the balancer will move around both existing and new regions. A > few questions: How many tables do you have? Are they all configured > similarly? What version of hbase are you on? > > 70 tables. hdfs://cluster1.com/hbase 24220

Re: Unbalanced region server

2012-04-23 Thread Bryan Beaudreault
Over time the balancer will move around both existing and new regions. A few questions: How many tables do you have? Are they all configured similarly? What version of hbase are you on? Sent from iPhone. On Apr 23, 2012, at 11:39 PM, David Charle wrote: > Does balancer_switch=true; rebalances

Re: Unbalanced region server

2012-04-23 Thread David Charle
Does balancer_switch=true; rebalances any skew in the existing keys or it only does for all new regions ? The requests seems to be oK across all servers. On Mon, Apr 23, 2012 at 8:03 PM, David Charle wrote: > Thanks Bryan. > > Here is the regions count: > > 1: 2616 > 2: 2620 > 3: 2623 > 4: 2617

Re: Unbalanced region server

2012-04-23 Thread David Charle
Thanks Bryan. Here is the regions count: 1: 2616 2: 2620 3: 2623 4: 2617 5: 2617 The skew is in Node 2 where we have the space issue (double the size of other 4 nodes). On Mon, Apr 23, 2012 at 7:50 PM, Bryan Beaudreault wrote: > Do all nodes have the same number of regions? If not you may no

Re: Unbalanced region server

2012-04-23 Thread Bryan Beaudreault
Do all nodes have the same number of regions? If not you may not have the balancer on. You can turn it on using balancer_switch true in the hbase shell. The balancer also doesn't run if there is a region stuck in transition. In this case if your data is growing rapidly I have seen regionservers

Unbalanced region server

2012-04-23 Thread David Charle
Hi What can make a region server unbalanced when it comes to space ? (and possibly requests too). For example; I have 5 node cluster (replication factor of 3); and in which all 4 has same size; where as one node is almost double in size for /hbase. Any help will be appreciated; esp what makes th