[ https://issues.apache.org/jira/browse/NUTCH-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Talat UYARER resolved NUTCH-1618. --------------------------------- Resolution: Fixed Thanks [~jnioche] Committed revision 1592218. > Turn speculative execution off for Fetching > ------------------------------------------- > > Key: NUTCH-1618 > URL: https://issues.apache.org/jira/browse/NUTCH-1618 > Project: Nutch > Issue Type: Bug > Components: fetcher > Affects Versions: 2.1, 2.2, 2.3, 2.4 > Reporter: Talat UYARER > Assignee: Talat UYARER > Priority: Minor > Fix For: 2.3 > > Attachments: NUTCH-1618-v2.patch, NUTCH-1618.patch > > > We are using nutch for high volume crawls. We noticed that FetcherJob > ReduceTask fetches some websites multiple times for long lasting queues. I > have discovered the reason of this is > mapred.reduce.tasks.speculative.execution settings in hadoop. 1.x has > speculative execution turned off. I create a patch for 2.x -- This message was sent by Atlassian JIRA (v6.2#6252)