[ https://issues.apache.org/jira/browse/NUTCH-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16242295#comment-16242295 ]
Yossi Tamari commented on NUTCH-2456: ------------------------------------- db.update.additions.allowed is set to false, which I guess amounts to the same thing. It's still an issue though, isn't it? > Redirected documents are not indexed > ------------------------------------ > > Key: NUTCH-2456 > URL: https://issues.apache.org/jira/browse/NUTCH-2456 > Project: Nutch > Issue Type: Bug > Components: indexer > Affects Versions: 1.13 > Reporter: Yossi Tamari > Priority: Critical > > If http.redirect.max is set to a positive value, the Fetcher will follow > redirects, creating a new CrawlDatum. > If the redirected URL is fetched and parsed, during indexing for it we have a > special case: dbDatum is null. This means that in > [https://github.com/apache/nutch/blob/6199492f5e1e8811022257c88dbf63f1e1c739d0/src/java/org/apache/nutch/indexer/IndexerMapReduce.java#L259] > the document is not indexed, as it is assumed it only has inlinks (actually > it has everything but dbDatum). > I'm not sure what the correct fix is here. It seems to me the condition > should use AND instead of OR anyway, but I may not understand the original > intent. It is clear that it is too strict as is. > However, the code following that line assumes all 4 objects are not null, so > a patch would need to change more than just the condition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)