Endre Kovacs created NIFI-6288: ---------------------------------- Summary: character set encoding issue in FetchElasticsearchHttp processor Key: NIFI-6288 URL: https://issues.apache.org/jira/browse/NIFI-6288 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.9.2 Reporter: Endre Kovacs Assignee: Endre Kovacs
I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch which have special UTF-8 chars, eg.: characters of foreign languages: accented chars or Japanese/Chinese chars. It was working as expected on platforms that have UTF-8 as a default _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document, turned to "?" in the fetched, output flow files. Taking a look at the source code showed: - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but it was not added to AbstractElasticsearchHttpProcessor in the static initializer block. - and in the place where the content of the document is written to the flowfile, : [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237] it uses {code} out.write(source.toString().getBytes()); {code} which will only work if the JVM's _file.encoding_ is UTF-8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)