Hi guys,
I just faced a weird situation, in which one of my hard disks on DN went
down.
Due to which when I restarted namenode, some of the blocks went missing and
it was saying my namenode is CORRUPT and in safe mode, which doesn't allow
you to add or delete any files on HDFS.
I know , we can cl
You ran into a corrupt files issue, not a namenode corruption (which generally
refers to the fsimage or edits getting corrupted).
Did your files not have adequate replication that they could not withstand the
loss of one DN's disk? What exactly did fsck output? Did all block replicas go
missing
I have a replication factor of 2, because of the reason that I can not
afford 3 replicas on my cluster.
fsck output was saying block replicas missing for some files that was
making Namenode is corrupt
I don't have the output with me. but issue was block replicas were missing.
How can we tackle that
Hi everyone,
Any ideas on how to tackle this kind of situation.
Thanks,
Praveenesh
On Tue, Jan 17, 2012 at 1:02 PM, praveenesh kumar wrote:
> I have a replication factor of 2, because of the reason that I can not
> afford 3 replicas on my cluster.
> fsck output was saying block replicas missing
Hey Praveenesh,
Here's a good article on HDFS by some senior Yahoo!, Facebook, HortonWorks
and eBay engineers that you might find helpful:
http://www.aosabook.org/en/hdfs.html
You may already know that "each block replica on a DataNode is represented
by two files in the DataNode's local, native f
Thanks a lot guys, for such illustrative explanation. I will go through the
links you send and will get back with any doubts I have.
Thanks,
Praveenesh
On Thu, Jan 19, 2012 at 2:17 PM, Sameer Farooqui wrote:
> Hey Praveenesh,
>
> Here's a good article on HDFS by some senior Yahoo!, Facebook, Hor
