[
https://issues.apache.org/jira/browse/NUTCH-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548420
]
Matt Kangas commented on NUTCH-585:
-----------------------------------
Simplest path forward... that I can think of:
1) Add a new indexing plugin extension-point for filtering page content.
2) Put your "apriori marked-up content" exclusion logic into a plugin.
3) Someone else figures out a more general-purpose solution later, and swaps
out your plugin at that time.
Ergo, you generalize the interface, and lazy-load the more general
implementation. :-)
> [PARSE-HTML plugin] Block certain parts of HTML code from being indexed
> -----------------------------------------------------------------------
>
> Key: NUTCH-585
> URL: https://issues.apache.org/jira/browse/NUTCH-585
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 0.9.0
> Environment: All operating systems
> Reporter: Andrea Spinelli
> Priority: Minor
>
> We are using nutch to index our own web sites; we would like not to index
> certain parts of our pages, because we know they are not relevant (for
> instance, there are several links to change the background color) and
> generate spurious matches.
> We have modified the plugin so that it ignores HTML code between certain HTML
> comments, like
> <!-- START-IGNORE -->
> ... ignored part ...
> <!-- STOP-IGNORE -->
> We feel this might be useful to someone else, maybe factorizing the comment
> strings as constants in the configuration files (say parser.html.ignore.start
> and parser.html.ignore.stop in nutch-site.xml).
> We are almost ready to contribute our code snippet. Looking forward for any
> expression of interest - or for an explanation why waht we are doing is
> plain wrong!
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
