Re: balancer and under replication

2013-01-07 Thread alxsss
org.apache.hadoop.hdfs.server.balancer.Balancer: 0 under utilized nodes: and shuts down with no errors. Thanks. Alex. -Original Message- From: Harsh J To: Sent: Mon, Jan 7, 2013 1:17 pm Subject: Re: balancer and under replication Under normal operation, NN takes care of under-replicated blocks

Re: balancer and under replication

2013-01-07 Thread Patai Sangbutsarakum
Thanks Harsh, you're the first, as usual. Currently, there are 291 active nodes spread in 11 racks. We have rack-awareness enable for a year (work like a champ). I ran fsck throughout HDFS again and noticed that majority of the files that have under repl. blocks are set to have 255 replicas but a

Re: balancer and under replication

2013-01-07 Thread Harsh J
Under normal operation, NN takes care of under-replicated blocks by itself. A file with a replication factor set higher than the cluster's nodes will also register its blocks as under-replicated. A common config mistake here is the mapred.submit.replication, which is a default of 10 (useful for 10