When you brought down the DN, the blocks in it were replicated to the remaining DNs. When the DN was added back, the blocks in it were over replicated, resulting in deletion of the extra replica.
On Mon, Mar 14, 2011 at 7:34 AM, Alex Baranau <alex.barano...@gmail.com>wrote: > Hello, > > As far as I understand, since "hadoop fs -du" command uses Linux' "du" > internally this mean that the number of replicas (at the moment of command > run) affect the result. Is that correct? > > I have the following case. > I have a small (1 master + 5 slaves each with DN, TT & RS) test HBase > cluster with replication set to 2. The tables data size is monitoried with > the help of "hadoop fs -du" command. There's a table which is constantly > written to: data is only added in it. > At some point I decided to reconfigure one of the slaves and shut it down. > After reconfiguration (HBase already marked it as dead one) I brought it up > again. Things went smoothly. However on the table size graph (I drew from > data fetched with "hadoop fs -du" command) I noticed a little spike up on > data size and then it went down to the normal/expected values. Can it be so > that at some point of the taking out/reconfiguring/adding back node > procedure at some point blocks were over-replicated? I'd expect them to be > under-replicated for some time (as DN is down) and I'd expect to see the > inverted spike: small decrease in data amount and then back to "expected" > rate (after all blocks got replicated again). Any ideas? > > Thank you, > > Alex Baranau > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase > -- Regards, Suresh
