Hi I am running nutch on hadoop 0.17.1. I launch 5 nodes to perform crawling.
When I look at the job statistics I see that only 1 reduce task is stared for all steps and hence I do a conclusion that hadoop doesn't consume all available resources. Only one node is extremily busy, other nodes are idle. How can I configure hadoop to consume all resources? I added mapred.map.tasks and mapred.reduce.tasks parameters but they have no effect. I also increased the max number for the mapred tasks, job tracker shows it. During all stages map tasks reaches maximum 3, andreduce only 1. -- Best Regards Alexander Aristov
