Load balancing HDFS

[Lior Schachter]

>
> Hi all,
> We currently have a 10 nodes cluster with 6TB per machine.
> We are buying few more nodes and considering to have only 3TB per machine.
>
> By default HDFS assigns blocks according to used capacity, percentage wise.
> This means that old nodes will contain more data.
> We prefer that the nodes (6TB, 3TB) will be balanced by actual used space
> so M/R jobs will work better.
> We don't expect to exceed the 3TB limit (buy more machines).
>
> Thanks,****
>
> Lior
>
>