Redirected URL are fetched wihtout setting any FetchInterval
------------------------------------------------------------
Key: NUTCH-615
URL: https://issues.apache.org/jira/browse/NUTCH-615
Project: Nutch
Issue Type: Bug
Components: fetcher
Affects Versions: 1.0.0
Reporter: Emmanuel Joke
Fix For: 1.0.0
Attachments: NUTCH-615.patch
An url which is redirected result to a new URL. We create a new CrawlDatum for
the new URL within the Fetcher but the FetchInterval was not initialized.
The new url was recorded in the DB with a FetchInterval = 0 and the FetchTime
is never correctly updated to be fetch later in the future. Thus we keep
crawling those URL at each generation.
This patch fix this issue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
