[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: message-generator.png
simple-flow-overview.png
bad-encoded-message.png
after-fix-encoded-message.png
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
> Attachments: after-fix-encoded-message.png, bad-encoded-message.png,
> message-generator.png, simple-flow-overview.png
>
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: image-2019-05-10-13-47-14-725.png
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: image-2019-05-10-13-47-14-789.png
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: (was: image-2019-05-10-13-47-14-789.png)
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: image-2019-05-10-13-47-14-509.png
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: (was: image-2019-05-10-13-47-14-725.png)
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: (was: image-2019-05-10-13-47-14-509.png)
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: (was: image-2019-05-10-13-40-09-396.png)
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-6288) character set encoding issue in FetchElasticsearchHttp processor
[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Kovacs updated NIFI-6288:
---
Attachment: image-2019-05-10-13-40-09-396.png
> character set encoding issue in FetchElasticsearchHttp processor
>
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.9.2
>Reporter: Endre Kovacs
>Assignee: Endre Kovacs
>Priority: Major
> Labels: easyfix
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
