[
https://issues.apache.org/jira/browse/NUTCH-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847074#action_12847074
]
Andrzej Bialecki commented on NUTCH-693:
-----------------------------------------
This patch is controversial in the sense that a) Nutch strives to adhere to
Internet standards and netiquette, which says that robots should obey nofollow,
and b) most Nutch users want a well-behaved robot. You are free of course to
modify the source as you did. Therefore I think that this functionality is not
applicable to majority of Nutch users, and I vote -1 on including it in Nutch.
> Add configurable option for treating nofollow behaviour.
> --------------------------------------------------------
>
> Key: NUTCH-693
> URL: https://issues.apache.org/jira/browse/NUTCH-693
> Project: Nutch
> Issue Type: New Feature
> Reporter: Andrew McCall
> Assignee: Otis Gospodnetic
> Priority: Minor
> Attachments: nutch.nofollow.patch
>
>
> For my purposes I'd like to follow links even if they're marked nofollow-
> Ideally I'd like to follow them, but not pass the link juice between them.
> I've attached a patch that adds a configuration element
> parser.html.outlinks.ignore_nofollow which allows the parser to ignore the
> nofollow elements on a page.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
