Re: Why must I wait for NameNode?
There's a bit of an issue if you have no data in your HDFS -- 0 blocks out of 0 is considered 100% reported, so NN leaves safe mode even if there are no DNs talking to it yet. For a fix, please see HDFS-528, included in Cloudera's CDH2. Thanks -Todd On Fri, Mar 19, 2010 at 10:29 AM, Bill Habermaas b...@habermaas.us wrote: At startup, the namenode goes into 'safe' mode to wait for all data nodes to send block reports on data they are holding. This is normal for hadoop and necessary to make sure all replicated data is accounted for across the cluster. It is the nature of the beast to work this way for good reasons. Bill -Original Message- From: Nick Klosterman [mailto:nklos...@ecn.purdue.edu] Sent: Friday, March 19, 2010 1:21 PM To: common-user@hadoop.apache.org Subject: Why must I wait for NameNode? What is the namemode doing upon startup? I have to wait about 1 minute and watch for the namenode dfs usage drop from 100% otherwise the install is unusable. Is this typical? Is something wrong with my install? I've been attempting the Pseudo distributed tutorial example for a while trying to get it to work. I finally discovered that the namenode upon start up is 100% in use and I need to wait about 1 minute before I can use it. Is this typical of hadoop installations? This isn't entirely clear in the tutorial. I believe that a note should be entered if this is typical. This error caused me to get WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: SOMEFILE could only be replicated to 0 nodes, instead of 1 I had written a script to do all of the steps right in a row. Now with a 1 minute wait things work. Is my install atypical or am I doing something wrong that is causing this needed wait time. Thanks, Nick -- Todd Lipcon Software Engineer, Cloudera
Re: Why must I wait for NameNode?
If you don't want to wait then you can do bin/hadoop dfsadmin -safemode leave. And this might be useful for reference. -safemode enter|leave|get|wait: Safe mode maintenance command. Safe mode is a Namenode state in which it 1. does not accept changes to the name space (read-only) 2. does not replicate or delete blocks. Safe mode is entered automatically at Namenode startup, and leaves safe mode automatically when the configured minimum percentage of blocks satisfies the minimum replication condition. Safe mode can also be entered manually, but then it can only be turned off manually as well. Ravi Hadoop @ Yahoo! On 3/19/10 10:29 AM, Bill Habermaas b...@habermaas.us wrote: At startup, the namenode goes into 'safe' mode to wait for all data nodes to send block reports on data they are holding. This is normal for hadoop and necessary to make sure all replicated data is accounted for across the cluster. It is the nature of the beast to work this way for good reasons. Bill -Original Message- From: Nick Klosterman [mailto:nklos...@ecn.purdue.edu] Sent: Friday, March 19, 2010 1:21 PM To: common-user@hadoop.apache.org Subject: Why must I wait for NameNode? What is the namemode doing upon startup? I have to wait about 1 minute and watch for the namenode dfs usage drop from 100% otherwise the install is unusable. Is this typical? Is something wrong with my install? I've been attempting the Pseudo distributed tutorial example for a while trying to get it to work. I finally discovered that the namenode upon start up is 100% in use and I need to wait about 1 minute before I can use it. Is this typical of hadoop installations? This isn't entirely clear in the tutorial. I believe that a note should be entered if this is typical. This error caused me to get WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: SOMEFILE could only be replicated to 0 nodes, instead of 1 I had written a script to do all of the steps right in a row. Now with a 1 minute wait things work. Is my install atypical or am I doing something wrong that is causing this needed wait time. Thanks, Nick Ravi --
