[ http://issues.apache.org/jira/browse/NUTCH-273?page=all ]
Stefan Groschupf updated NUTCH-273:
-----------------------------------
Priority: Blocker (was: Major)
I agree this is a serious problem for any production use of nutch - a blocker
since you end up refetching again and again the same pages.
> When a page is redirected, the original url is NOT updated.
> -----------------------------------------------------------
>
> Key: NUTCH-273
> URL: http://issues.apache.org/jira/browse/NUTCH-273
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.8
> Environment: n/a
> Reporter: Lukas Vlcek
> Priority: Blocker
>
> [Excerpt from maillist, sender: Andrzej Bialecki]
> When a page is redirected, the original url is NOT updated - so, CrawlDB will
> never know that a redirect occured, it won't even know that a fetch
> occured... This looks like a bug.
> In 0.7 this was recorded in the segment, and then it would affect the Page
> status during updatedb. It should do so 0.8, too...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
