I don't think I configured such things, how can I be sure ? ----- Message d'origine ----- De : Lewis John Mcgibbney Envoyés : 14.02.12 19:18 À : [email protected] Objet : Re: fetcher.max.crawl.delay = -1 doesn't work?
Hi Danicela, Before I try this, have you configured any other overrides for generating or fetching in nutch-site.xml? Thanks On Tue, Feb 14, 2012 at 3:10 PM, Danicela nutch <[email protected]>wrote: > Hi, > > I have in my nutch-site.xml the value fetcher.max.crawl.delay = -1. > > When I try to fetch a site with a robots.txt with a Crawl Delay, it > doesn't work. > > If I put fetcher.max.crawl.delay = 10000, it works. > > I use Nutch 1.2, but according to the changelog, nothing has been changed > about that since then. > > Is this a Nutch bug or I misused something ? > > Another thing, in hadoop.log, the pages which couldn't be fetched are > still marked as "fetching", is this normal ? Shouldn't they be marked as > "dropped" or something ? > > Thanks. > -- *Lewis*
