[ https://issues.apache.org/jira/browse/NUTCH-923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923896#action_12923896 ]
Andrzej Bialecki commented on NUTCH-923: ----------------------------------------- This sounds useful, though the implementation needs to keep the following in mind: * you _assume_ that the lang field will have a nice predictable value, but unless you sanitize the values you can't assume anything... example: one page I saw had a language metadata set to a random string 8kB long with various control chars and '\0'-s. * again, if you don't sanitize and control the total number of unique values in the source field, you could end up with a number of fields approaching infinity, and Solr would melt down... > Multilingual support for Solr-index-mapping > ------------------------------------------- > > Key: NUTCH-923 > URL: https://issues.apache.org/jira/browse/NUTCH-923 > Project: Nutch > Issue Type: Improvement > Components: indexer > Affects Versions: 1.2 > Reporter: Matthias Agethle > Assignee: Markus Jelsma > Priority: Minor > > It would be useful to extend the mapping-possibilites when indexing to solr. > One useful feature would be to use the detected language of the html page > (for example via the language-identifier plugin) and send the content to > corresponding language-aware solr-fields. > The mapping file could be as follows: > <field dest="lang" source="lang"/> > <field dest="title_${lang}" source="title" /> > so that the title-field gets mapped to title_en for English-pages and > tilte_fr for French pages. > What do you think? Could this be useful also to others? > Or are there already other solutions out there? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.