Oh, and on top of the above, I just observed that even though bin/hadoop
balancer exits immediately and reports the cluster is fully balanced, I do see
*very* few blocks (1-2 blocks per node) getting moved every time I run
balancer. It feels as if the balancer does actually find some blocks that it
could move around, moves them, but then quickly gets lazy and just exits
claiming the cluster is/was already balanced. I just ran balancer about 10
times and each time it moved a couple of blocks and then exited.
Makes me want to do ugly stuff like:
for ((i=1; i <= 9999; i++)); do echo $i; bin/hadoop balancer; done
...just to get to the point where all 4 nodes have the same number of blocks
and thus the same percentage of disk used...
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
> From: Otis Gospodnetic <[EMAIL PROTECTED]>
> To: core-user@hadoop.apache.org
> Sent: Sunday, May 11, 2008 2:36:24 PM
> Subject: Balancer not balancing 100%?
>
> Hi,
>
> I have 4 identical nodes in a Hadoop cluster (all functioning as DNs). One
> of
> the 4 nodes is a new node that I recently added. I ran the balancer a few
> times
> and it did move some of the blocks from the other 3 nodes to the new node.
> However, the 4 nodes are still not 100% balanced (according to the GUI), even
> though running bin/hadoop balancer says the cluster is balanced:
>
> Time Stamp Iteration# Bytes Already Moved Bytes Left To Move
> Bytes Being Moved
> The cluster is balanced. Exiting...
> Balancing took 666.0 milliseconds
>
>
> The 3 old DNs are about 60% full (around 24K blocks), which the 1 new DN is
> only
> about 50% full (around 21K blocks). I restarted the NN and re-ran the
> balancer,
> bug got the same output: "The cluster is balanced. Exiting..."
>
> Is this a bug or is it somehow possible for a cluster to be balanced, yet
> have
> nodes with different number of blocks?
>
> Thanks,
> Otis
