[jira] Commented: (NUTCH-443) allow parsers to return multiple Parse object, this will speed up the rss parser

[nutch.newbie (JIRA)]

    [ 
https://issues.apache.org/jira/browse/NUTCH-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471998
 ] 

nutch.newbie commented on NUTCH-443:
------------------------------------

Hi.. 

After swaping the parse-plugin.xml i.e. the following way .. (and turning off 
magic detection)

        <mimeType name="text/xml"> 
           <plugin id="parse-rss" /> 
          <plugin id="parse-html" />
        </mimeType> 

Hoping that parse-rss will pick-up the doc firs and not return NPE so out of 25 
RSS URL with 1 round of fetch I managed to escape dedup with only 4 doc being 
indexed all other 21 docs throw NPE .. 

Error parsing: http://rss.cnn.com/rss/cnn_warpcnn.rss: failed(2,200): 
java.lang.NullPointerException
Error parsing: http://rss.cnn.com/rss/cnn_ac360blog.rss: failed(2,200): 
java.lang.NullPointerException
Error parsing: http://rss.cnn.com/rss/cnn_marquee.rss: failed(2,200): 
java.lang.NullPointerException
Error parsing: http://rss.cnn.com/rss/cnn_gupta.rss: failed(2,200): 
java.lang.NullPointerException

I must be doing something sily there must be way to tell nutch to index using 
plugin X.. I thought you do that turning magic off and using plugin-parse.xml 
.. no? am I missing something .. Please let me know..

I am going to try the parse-feed now to see what happens. Issues regarding that 
I will post in Nutch-444

Cheers


> allow parsers to return multiple Parse object, this will speed up the rss 
> parser
> --------------------------------------------------------------------------------
>
>                 Key: NUTCH-443
>                 URL: https://issues.apache.org/jira/browse/NUTCH-443
>             Project: Nutch
>          Issue Type: New Feature
>          Components: fetcher
>    Affects Versions: 0.9.0
>            Reporter: Renaud Richardet
>         Assigned To: Chris A. Mattmann
>            Priority: Minor
>             Fix For: 0.9.0
>
>         Attachments: NUTCH-443-draft-v1.patch, NUTCH-443-draft-v2.patch, 
> NUTCH-443-draft-v3.patch, NUTCH-443-draft-v4.patch, 
> parse-map-core-draft-v1.patch, parse-map-core-untested.patch, parsers.diff
>
>
> allow Parser#parse to return a Map<String,Parse>. This way, the RSS parser 
> can return multiple parse objects, that will all be indexed separately. 
> Advantage: no need to fetch all feed-items separately.
> see the discussion at 
> http://www.nabble.com/RSS-fecter-and-index-individul-how-can-i-realize-this-function-tf3146271.html

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.