Hi Di,
The parameters you are defining denote the number of mappers and reducers to
run. In order to increase
the number of tasks that run simultaneously, you need to set the following
parameters:
<name>mapred.tasktracker.map.tasks.maximum</name>
<name>mapred.tasktracker.map.reduce.maximum</name>
Regards,
-Stavros.
________________________________________
From: [email protected] [[email protected]] on behalf of di wang
[[email protected]]
Sent: Thursday, April 12, 2012 4:54 PM
To: [email protected]
Subject: question about changing the number of mappers/reducers of web search
crawling phase
Hi,
I was trying to change the number of mappers and reducers for crawling of web
search benchmark.
But no matter what values I set in the nutch-test/search/conf/mapred-site.xml
(mapred.map.tasks, mapred.reduce.tasks), it always uses 2 mappers and 2
reducers per node.
Is there any other place that I need to change in order to do this?
thanks,
di
