Becides, seems that number of fetcher threads does not affects anything. Same result for 20 and for 1000 threads.
caezar wrote: > > Hi All, > > I have 15 machines in hadoop farm. While fetching, I've got about 10 > pages/s (4000kb/s) per machine. I suppose it is very slow. I've set > mapred.map.tasks and mapred.reduce.tasks to 15. Is this correct? HTTP > timeout is 5 seconds, max reties 2, 0.5 seconds between retries. > fetcher.threads.fetch is 300. How can I tweak the performance? What other > options may affect performance? Should I provide some other information > for you to be able to help me? > > Thanks > -- View this message in context: http://www.nabble.com/Nutch-fetch-performance-tp24203861p24203907.html Sent from the Nutch - User mailing list archive at Nabble.com.
