[
https://issues.apache.org/jira/browse/NUTCH-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14011649#comment-14011649
]
Sebastian Nagel commented on NUTCH-1785:
----------------------------------------
+1 (works, looks reasonable). Open points:
* what about truncated content? (not a blocker)
* conversion via {{new String(content.getContent())}} is needless if base64 is
true
* this conversion depends on system's locale, cf. NUTCH-1693
* but which charset shall we use to convert the byte[] into a String if there
exist codepoints >127?
*# the charset used for parsing is not available to indexer (it's in parse
metadata)
*# maybe ASCII is a good choice, cf. comments in sniffCharacterEncoding
(parse-html)
*# in any case (for non-ASCII stuff): the indexing back-ends must consider that
the String in field binaryContent may need recoding
> Ability to index raw content
> ----------------------------
>
> Key: NUTCH-1785
> URL: https://issues.apache.org/jira/browse/NUTCH-1785
> Project: Nutch
> Issue Type: New Feature
> Components: indexer
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.9
>
> Attachments: NUTCH-1785-trunk.patch, NUTCH-1785-trunk.patch,
> NUTCH-1785-trunk.patch, NUTCH-1785-trunk.patch
>
>
> Some use-cases require Nutch to actually write the raw content a configured
> indexing back-end. Since Content is never read, a plugin is out of the
> question and therefore we need to force IndexJob to process Content as well.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
