[ https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Endre Kovacs updated NIFI-6288: ------------------------------- Attachment: message-generator.png simple-flow-overview.png bad-encoded-message.png after-fix-encoded-message.png > character set encoding issue in FetchElasticsearchHttp processor > ---------------------------------------------------------------- > > Key: NIFI-6288 > URL: https://issues.apache.org/jira/browse/NIFI-6288 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Affects Versions: 1.9.2 > Reporter: Endre Kovacs > Assignee: Endre Kovacs > Priority: Major > Labels: easyfix > Attachments: after-fix-encoded-message.png, bad-encoded-message.png, > message-generator.png, simple-flow-overview.png > > > I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch > which have special UTF-8 chars, eg.: characters of foreign languages: > accented chars or Japanese/Chinese chars. > It was working as expected on platforms that have UTF-8 as a default > _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document, > turned to "?" in the fetched, output flow files. > > Taking a look at the source code showed: > - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but > it was not added to > AbstractElasticsearchHttpProcessor in the static initializer block. > - and in the place where the content of the document is written to the > flowfile, : > [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237] > it uses > {code} > out.write(source.toString().getBytes()); > {code} > > which will only work if the JVM's _file.encoding_ is UTF-8. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)