Hey Amit, please try HOD or hadoop on demand tool. This will suffice to your
need for creating multiple users on ur cluster.
-Piyush
On Thu, Feb 10, 2011 at 12:42 AM, Kumar, Amit H. <ahku...@odu.edu> wrote:
> Dear All,
>
> I am trying to setup Hadoop for multiple users in a class, on our cluster.
> For some reason I don't seem to get it right. If only one user is running it
> works great.
> I would want to have all of the users submit a Hadoop job to the existing
> DataNode and on the cluster, not sure if this is right.
> Do I need to start a DataNode for every user, if so I was not able to do
> because I ran into issues of port already being used.
> Please advise. Below are few of the config files.
>
> Also I have tired searching for other documents, that tell us to create a
> user "Hadoop" and a group "Hadoop" and then start the daemons as Hadoop
> user. This didn't work for me as well. I am sure I am doing something
> wrong. Could anyone please thrown in some more ideas.
>
> =>List of env changed in Hadoop-env.sh:
> export HADOOP_LOG_DIR=/scratch/$USER/hadoop-logs
> export HADOOP_PID_DIR=/scratch/$USER/.var/hadoop/pids
>
> #cat core-site.xml
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://frontend:9000</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/scratch/${user.name}/hadoop-FS</value>
> <description>A base for other temporary directories.</description>
> </property>
> </configuration>
>
> # cat hdfs-site.xml
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.name.dir</name>
> <value>/scratch/${user.name}/.hadoop/.transaction/.edits</value>
> </property>
> </configuration>
>
> # cat mapred-site.xml
> <configuration>
> <property>
> <name>mapred.job.tracker</name>
> <value>frontend:9001</value>
> </property>
> <property>
> <name>mapreduce.tasktracker.map.tasks.maximum</name>
> <value>2</value>
> </property>
> <property>
> <name>mapreduce.tasktracker.reduce.tasks.maximum</name>
> <value>2</value>
> </property>
> </configuration>
>
>
> Thank you,
> Amit
>
>
>
