Re: question on shuffle and sort

2010-03-30 Thread
yes ,indeed 在 2010-03-31三的 09:56 +0800,Cui tony写道: > Hi, > Did all key-value pairs of the map output, which have the same key, will > be sent to the same reducer tasknode?

a question about automatic restart of the NameNode

2010-03-30 Thread
Hi all, Does automatic restart and failover of the NameNode software to another machine available in hadoop 0.20.2?

JobTracker startup failure when starting hadoop-0.20.0 cluster on Amazon EC2 with contrib/ec2 scripts

2010-03-25 Thread
I downloaded Hadoop 0.20.0 and used the src/contrib/ec2/bin scripts to launch a Hadoop cluster on Amazon EC2, after building a new Hadoop 0.20.0 AMI. I launched an instance with my new Hadoop 0.20.0 AMI, then logged in and ran the following to launch a new cluster: root(/vol/hadoop-0.20.0)> bin/l

a problem when executing wordcount in hadoop cluster environment

2010-03-25 Thread
I have finished configuring the Hadoop in cluster environment as follows: 1. maoh...@maohong-desktop:~/Software/Development/Hadoop/hadoop-0.20.2$ bin/start-all.sh 2. starting namenode, logging to /home/maohong/Software/Development/Hadoop/hadoop-0.20.2/bin/../logs/hadoop-maoho

Re: a problem when starting datanodes

2010-03-23 Thread
10-03-23二的 21:23 +0800,liu chang写道: > On Tue, Mar 23, 2010 at 9:11 PM, 毛宏 wrote: > > I use "file /usr/bin/env" to check if /usr/bin/env is present in my > > system and the answer is yes. > > But why does it still display > > datanode1:/usr/bin/env : bash: No

Re: a problem when starting datanodes

2010-03-23 Thread
yes I can, I am using Ubuntu 9.10 in my namenode, debian 4.0 in my datanode. They all have /usr/bin/env 在 2010-03-23二的 21:23 +0800,liu chang写道: > On Tue, Mar 23, 2010 at 9:11 PM, 毛宏 wrote: > > I use "file /usr/bin/env" to check if /usr/bin/env is present in my

Re: a problem when starting datanodes

2010-03-23 Thread
gt; > > file /bin/env > > > > If you have /bin/env but not /usr/bin/env, you can make a symbolic link for > > it: > > > > ln -s /usr/bin/env /bin/env > > > > You need to execute the command above as root. > > > > Liu Chang > > >

a problem when starting datanodes

2010-03-23 Thread
Hi all, I install Hadoop in three machines, my pc is the namenode, two other pc are the datanodes, but when I execute bin/start-dfs.sh, it displays these two line as follows: datanode1: /usr/bin/env: bash: No such file or directory

a question about tasktracker in hadoop

2010-03-21 Thread
I read from 《Towards Optimizing Hadoop Provisioning in the Cloud 》 saying that "mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum respectively set the maximum number of parallel mappers and reducers that can run on a Hadoop slave". It means that a tasktracker in H