Give complete hostname with domain name not just master-node. <property> <name>fs.defaultFS</name> <value>hdfs://master-node.domain.name:9000</value> </property>
Else give IP address also On 12/16/14, Dan Dong <dongda...@gmail.com> wrote: > Hi, Johny, > Yes, they have been turned off from the beginning. Guess the problem is > still in the conf files, it would be helpful if some example *.xml could be > shown. > > Cheers, > Dan > > > 2014-12-15 12:24 GMT-06:00 johny casanova <pcgamer2...@outlook.com>: >> >> do you have selinux and iptables turned off? >> >> ------------------------------ >> Date: Mon, 15 Dec 2014 09:54:41 -0600 >> Subject: Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file >> system" >> From: dongda...@gmail.com >> To: user@hadoop.apache.org >> >> >> Found in the log file: >> 2014-12-12 15:51:10,434 ERROR >> org.apache.hadoop.hdfs.server.namenode.NameNode: >> java.lang.IllegalArgumentException: Does not contain a valid host:port >> authority: file:/// >> at >> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:212) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:244) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:280) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:569) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1479) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488) >> >> But I have set it in core-site.xml already: >> <property> >> <name>fs.defaultFS</name> >> <value>hdfs://master-node:9000</value> >> </property> >> >> Other settings: >> $ cat mapred-site.xml >> <configuration> >> <property> >> <name>mapred.job.tracker</name> >> <value>master-node:9002</value> >> </property> >> <property> >> <name>mapreduce.jobhistory.address</name> >> <value>master-node:10020</value> >> </property> >> <property> >> <name>mapreduce.jobhistory.webapp.address</name> >> <value>master-node:19888</value> >> </property> >> </configuration> >> >> $ cat yarn-site.xml >> <configuration> >> >> <!-- Site specific YARN configuration properties --> >> <property> >> <name>mapreduce.framework.name</name> >> <value>yarn</value> >> </property> >> <property> >> <name>yarn.resourcemanager.address</name> >> <value>master-node:18040</value> >> </property> >> <property> >> <name>yarn.resourcemanager.scheduler.address</name> >> <value>master-node:18030</value> >> </property> >> <property> >> <name>yarn.resourcemanager.webapp.address</name> >> <value>master-node:18088</value> >> </property> >> <property> >> <name>yarn.resourcemanager.resource-tracker.address</name> >> <value>master-node:18025</value> >> </property> >> <property> >> <name>yarn.resourcemanager.admin.address</name> >> <value>master-node:18141</value> >> </property> >> <property> >> <name>yarn.nodemanager.aux-services</name> >> <value>mapreduce_shuffle</value> >> </property> >> <property> >> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> >> <value>org.apache.hadoop.mapred.ShuffleHandler</value> >> </property> >> </configuration> >> >> Cheers, >> Dan >> >> >> 2014-12-15 9:17 GMT-06:00 Dan Dong <dongda...@gmail.com>: >> >> Thank you all, but still the same after change file:/ to file://, and >> HADOOP_CONF_DIR points to the correct position already: >> $ echo $HADOOP_CONF_DIR >> /home/dong/import/hadoop-2.6.0/etc/hadoop >> >> >> 2014-12-15 8:57 GMT-06:00 johny casanova <pcgamer2...@outlook.com>: >> >> Don't you have to use file:// instead of just one /? >> >> ------------------------------ >> From: brahmareddy.batt...@huawei.com >> To: user@hadoop.apache.org >> Subject: RE: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file >> system" >> Date: Sat, 13 Dec 2014 05:48:18 +0000 >> >> >> Hi Dong, >> >> HADOOP_CONF_DIR might be referring to default..you can export >> HADOOP_CONF_DIR where following configuration files are present.. >> >> Thanks & Regards >> Brahma Reddy Battula >> >> >> ------------------------------ >> *From:* Dan Dong [dongda...@gmail.com] >> *Sent:* Saturday, December 13, 2014 3:43 AM >> *To:* user@hadoop.apache.org >> *Subject:* Hadoop 2.6.0: "FileSystem file:/// is not a distributed file >> system" >> >> Hi, >> I installed Hadoop2.6.0 on my cluster with 2 nodes, I got the following >> error when I run: >> $hadoop dfsadmin -report >> FileSystem file:/// is not a distributed file system >> >> What this mean? I have set it in core-site.xml already: >> <property> >> <name>fs.defaultFS</name> >> <value>hdfs://master-node:9000</value> >> </property> >> >> and in hdfs-site.xml: >> <property> >> <name>dfs.namenode.name.dir</name> >> <value>file:/home/dong/hadoop-2.6.0-dist/dfs/name</value> >> <final>true</final> >> </property> >> <property> >> <name>dfs.dataname.data.dir</name> >> <value>file:/home/dong/hadoop-2.6.0-dist/dfs/data</value> >> <final>true</final> >> </property> >> >> The java process are running on master as: >> 10479 SecondaryNameNode >> 10281 NameNode >> 10628 ResourceManager >> >> and on slave: >> 22870 DataNode >> 22991 NodeManager >> >> Any hints? Thanks! >> >> Cheers, >> Dan >> >> >> >
