The 3 *-site.xml files are in the /etc/hadoop-0.20/conf directory. I've confirmed that these are the files being used. Thanks again. Cliff
On Mon, Aug 23, 2010 at 10:26 AM, Harsh J <qwertyman...@gmail.com> wrote: > Can you confirm that this is the right configuration your NN is starting > with? > > On Mon, Aug 23, 2010 at 7:19 PM, cliff palmer <palmercl...@gmail.com> > wrote: > > Thanks Harsh, but I am still not sure I understand what is going on. > > The directory specified in the dfs.name.dir property, > > "/var/lib/hadoop-0.20/dfsname", does exist and rights to that directory > have > > been granted to the OS user that is running the Hadoop startup script. > > The directory mentioned in the error message is > > "/var/lib/hadoop-0.20/cache/hadoop/dfs/name". > > I can create this directory and that would (I assume) remove the error, > but > > I want to understand how the name is derived. It's not a child of the > > directory name specified in the dfs.name.dir property. > > Thanks again! > > Cliff > > > > > > On Mon, Aug 23, 2010 at 9:21 AM, Harsh J <qwertyman...@gmail.com> wrote: > > > >> Its checking this directory cause your dfs.name.dir (hdfs-site.xml) > >> has it in its list of dirs to write a copy to: > >> > >> <property> > >> <name>dfs.name.dir</name> > >> <value>/DFS/dfsname,/var/lib/hadoop-0.20/dfsname</value> > >> <property> > >> > >> Remove it from this property if you don't need it. If you need > >> multiple dirs, you should create the path and allocate proper > >> permissions to it so that it may write to it. Its good to use a second > >> path for backup purposes (most have this on the NFS). The path is so > >> because your distribution of hadoop perhaps placed its files around > >> that place and its up in the conf files as some form of default :) > >> > >> On Mon, Aug 23, 2010 at 6:45 PM, cliff palmer <palmercl...@gmail.com> > >> wrote: > >> > The namenode log for a Hadoop-0.20 installation contains this error > >> message: > >> > "/var/lib/hadoop-0.20/cache/hadoop/dfs/name in in an inconsistent > state". > >> > This directory does not exist and I would like to understand why this > >> > particular directory name is required (not what the directory is used > >> for, > >> > but why this particular directory name). The *-site.xml files are > below > >> (IP > >> > addresses have been masked). > >> > Thanks in advance for your help. > >> > Cliff > >> > > >> > core-site.xml: > >> > > >> > <?xml version="1.0"?> > >> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > >> > > >> > <!-- Put site-specific property overrides in this file. --> > >> > > >> > <configuration> > >> > <property> > >> > <name>fs.default.name</name> > >> > <value>hdfs://xxx.xxx.xxx.xxx:8020</value> > >> > <final>true</final> > >> > </property> > >> > <property> > >> > <name>hadoop.tmp.dir</name> > >> > <value>/var/lib/hadoop-0.20/cache/hadoop</value> > >> > </property> > >> > </configuration> > >> > -------- end of core-site.xml ----------- > >> > > >> > hdfs-site.xml: > >> > > >> > <?xml version="1.0"?> > >> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > >> > > >> > <!-- Put site-specific property overrides in this file. --> > >> > > >> > <configuration> > >> > <property> > >> > <name>dfs.replication</name> > >> > <value>3</value> > >> > </property> > >> > <property> > >> > <name>dfs.name.dir</name> > >> > > <value>/DFS/dfsname,/var/lib/hadoop-0.20/dfsname</value> > >> > <property> > >> > </property> > >> > <name>dfs.data.dir</name> > >> > > <value>/DFS1/dfsdata,/DFS2/dfsdata,/DFS3/dfsdata</value> > >> > </property> > >> > <property> > >> > <name>dfs.permissions</name> > >> > <value>false</value> > >> > </property> > >> > <property> > >> > <name>dfs.datanode.du.reserved</name> > >> > <value>20000000000</value> > >> > </property> > >> > <property> > >> > <name>dfs.namenode,plugins</name> > >> > > <value>org.apache.hadoop.thriftfs.NamenodePlugin</value> > >> > </property> > >> > <property> > >> > <name>dfs.datanode.plugins</name> > >> > > <value>org.apache.hadoop.thriftfs.DatanodePlugins</value> > >> > </property> > >> > <property> > >> > <name>dfs.thrift.address</name> > >> > <value>0.0.0.0:9090</value> > >> > </property> > >> > </configuration> > >> > -------- end of core-site.xml ----------- > >> > > >> > mapred-site.xml: > >> > > >> > <?xml version="1.0"?> > >> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > >> > > >> > <!-- Put site-specific property overrides in this file. --> > >> > > >> > <configuration> > >> > <property> > >> > <name>mapred.job.tracker</name> > >> > <value>xxx.xxx.xxx.xxx:8020</value> > >> > </property> > >> > <property> > >> > <name>jobtracker.thrift.address</name> > >> > <value>0.0.0:9290</value> > >> > </property> > >> > <property> > >> > <name>mapred.jobtracker.plugins</name> > >> > > >> > <value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value> > >> > </property> > >> > </configuration> > >> > -------- end of core-site.xml ----------- > >> > > >> > >> > >> > >> -- > >> Harsh J > >> www.harshj.com > >> > > > > > > -- > Harsh J > www.harshj.com >
