nevermind i resolved it. the solution was bad instructions on the hadoop site or unclear/misleading instructions.
this is NOT the way to start slave datanode daemons (NOTICE THE SINGULAR DAEMON). $HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script hdfs start datanode this is the correct way to start slave datanode daemons (NOTICE THE PLURAL DAEMONS). $HADOOP_PREFIX/sbin/hadoop-daemons.sh --config $HADOOP_CONF_DIR --script hdfs start datanode On Sun, Jul 27, 2014 at 3:11 AM, Jane Wayne <jane.wayne2...@gmail.com> wrote: > i am following the instructions to setup a multi-node cluster at > http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/ClusterSetup.html > . > > my problem is that when i run the script to start up the slave datanodes, > no slave datanode is started (more on this later). > > i have two nodes so far that i am experimenting with > 1. node1 (this is the namenode) > 2. node2 (this is the datanode) > > on node1 (namenode), i start the namenode daemon as follows. there is no > problem here. > > $HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script > hdfs start namenode > > on node1 (namenode), i start the datanode daemon (on node2) as follows. > > $HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script > hdfs start datanode > > it is at this point that the problems begin: > a) the datanode daemon is started on node1 (instead of node2) > b) the datanode daemon is not started on node2 (the only slave defined) > c) the only job of node1 is to be a namenode, not also a datanode > > my $HADOOP_CONF_DIR/slaves has one single entry (line): > node2 > > my /etc/hosts file looks like the following > 127.0.0.1 localhost 192.168.0.10 node1 > 192.168.0.11 node2 > > why does the script to start the datanode slave daemons not start them and > only start a local slave daemon? > > on node2, when i run the script to start the datanode daemon, it is able > to find node1:8020 and become active as part of the cluster. but this is > strange to me, because this script should be called/executed from the > namenode to start all datanodes on the slaves (not the other way around). > > any idea what i am doing wrong? > >
