[
https://issues.apache.org/jira/browse/NUTCH-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559280#action_12559280
]
Andrzej Bialecki commented on NUTCH-590:
-----------------------------------------
Nutch has a provision to return multiple documents from parsers, however you
are correct that no such provision exists during the indexing step. The
challenge here is that once a ParseResult is stored in a segment the individual
pieces are stored separately, and Nutch doesn't maintain the information about
their relationships. So based on the information managed in Nutch there is no
way to submit a group of documents to Indexer (assuming we rework the API to
accommodate for multiple in/out documents). I'm pretty sure that if you created
a patch that implements this the changes would be substantial across several
Nutch subsystems.
I'm inclined to mark this as "Won't Fix".
> Index multiple docs per call using IndexingFilter extension point
> -----------------------------------------------------------------
>
> Key: NUTCH-590
> URL: https://issues.apache.org/jira/browse/NUTCH-590
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.0.0
> Reporter: Nathaniel Powell
> Fix For: 1.0.0
>
>
> There are many applications where extracting and indexing multiple documents
> from a single HTML web file or other object would be useful. Therefore, it
> would help a lot if the IndexingFilter extension point were modified to pass
> in a list of documents as an argument and return a list (or collection) of
> documents.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
