[ https://issues.apache.org/jira/browse/NUTCH-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199746#comment-13199746 ]
Abhay Dabholkar commented on NUTCH-585: --------------------------------------- I been using this plugin for sometime and recently i wanted to use it to extract text based on url pattern Example: for http://x.y.z/?id=12 white-list will only look for div id=12 for http://x.y.z/?id=13 white-list will only look for div id=13 ref: http://lucene.472066.n3.nabble.com/index-blacklist-whitelist-pluign-for-multiple-set-of-urls-td3711697.html > [PARSE-HTML plugin] Block certain parts of HTML code from being indexed > ----------------------------------------------------------------------- > > Key: NUTCH-585 > URL: https://issues.apache.org/jira/browse/NUTCH-585 > Project: Nutch > Issue Type: Improvement > Affects Versions: 0.9.0 > Environment: All operating systems > Reporter: Andrea Spinelli > Assignee: Markus Jelsma > Priority: Minor > Fix For: 1.5 > > Attachments: blacklist_whitelist_plugin.patch, > nutch-585-excludeNodes.patch, nutch-585-jostens-excludeDIVs.patch > > > We are using nutch to index our own web sites; we would like not to index > certain parts of our pages, because we know they are not relevant (for > instance, there are several links to change the background color) and > generate spurious matches. > We have modified the plugin so that it ignores HTML code between certain HTML > comments, like > <!-- START-IGNORE --> > ... ignored part ... > <!-- STOP-IGNORE --> > We feel this might be useful to someone else, maybe factorizing the comment > strings as constants in the configuration files (say parser.html.ignore.start > and parser.html.ignore.stop in nutch-site.xml). > We are almost ready to contribute our code snippet. Looking forward for any > expression of interest - or for an explanation why waht we are doing is > plain wrong! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira