Opps, It seems I added the values to the wrong place first. I added them to the hadoop-0.17.1/conf/hadoop-site.xml
Here they didn't work. I then added the parameters to the nutch job file (nutch-site.xml) which I am running and hadoop cought them up at last. Now things are getting much faster. thanks Alexander 2008/7/31 Gopal Gandhi <[EMAIL PROTECTED]> > I think this is due to the input data of your job exist on one node. > Mappers are launched only on nodes with data (Hadoop calls it "block"). > For reducer, I am not sure why there's only 1 reducer. Anybody can explain > that? > > > > ----- Original Message ---- > From: Alexander Aristov <[EMAIL PROTECTED]> > To: core-user@hadoop.apache.org > Sent: Thursday, July 31, 2008 12:06:59 PM > Subject: how to increase number of reduce tasks > > Hi > > I am running nutch on hadoop 0.17.1. I launch 5 nodes to perform crawling. > > When I look at the job statistics I see that only 1 reduce task is stared > for all steps and hence I do a conclusion that hadoop doesn't consume all > available resources. > > Only one node is extremily busy, other nodes are idle. How can I configure > hadoop to consume all resources? > > I added mapred.map.tasks and mapred.reduce.tasks parameters but they have > no > effect. > I also increased the max number for the mapred tasks, job tracker shows it. > > During all stages map tasks reaches maximum 3, andreduce only 1. > > -- > Best Regards > Alexander Aristov > > > > > -- Best Regards Alexander Aristov
