I had the same problem a couple of weeks ago with 0.19.1. Had to reformat the cluster too...
On Mon, May 4, 2009 at 3:50 PM, Stas Oskin <stas.os...@gmail.com> wrote: > Hi. > > After rebooting the NameNode server, I found out the NameNode doesn't start > anymore. > > The logs contained this error: > "FSNamesystem initialization failed" > > > I suspected filesystem corruption, so I tried to recover from > SecondaryNameNode. Problem is, it was completely empty! > > I had an issue that might have caused this - the root mount has run out of > space. But, both the NameNode and the SecondaryNameNode directories were on > another mount point with plenty of space there - so it's very strange that > they were impacted in any way. > > Perhaps the logs, which were located on root mount and as a result, could > not be written, have caused this? > > > To get back HDFS running, i had to format the HDFS (including manually > erasing the files from DataNodes). While this reasonable in test > environment > - production-wise it would be very bad. > > Any idea why it happened, and what can be done to prevent it in the future? > I'm using the stable 0.18.3 version of Hadoop. > > Thanks in advance! >
