Hadoop Installation on Multihomed Networks

2014-11-11 Thread Fei Hu
Hi, I am trying to install Hadoop1.0.4 on multihomed networks. I have done it as the link http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html. But it still

[no subject]

2014-11-11 Thread Premal Shah
Hi, We recently upgraded from 1.0.4 to 1.2.1 and now have a ton of DFS balancing to do. On 1.0.4, when I set the bandwidth to 1 or 10 Gbps, the datanodes got working and were using a ton of network to get the nodes balanced. However, looks like 1.2.1 is not honoring the setBandWidth setting, no

Re:

2014-11-11 Thread daemeon reiydelle
I observed what I thought was a similar problem but found that I was actually hitting the physical disks much harder/more efficiently ... I found that I had some spindles with pretty deep queue depths. When I sat back and actually measured, I found that my throughput had substantially increased,

Re:

2014-11-11 Thread Premal Shah
I'm sorry, I send this email without a subject. This is from iostat avg-cpu: %user %nice %system %iowait %steal %idle 1.980.000.340.710.02 96.94 When we run MR jobs, I see bandwidth usage spike to 1Gbps easily. But when there are no jobs running, balancer is