[jira] [Commented] (NIFI-12343) PutElasticsearchJson exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852215#comment-17852215
]
Gregory M. Foreman commented on NIFI-12343:
---
[~Chris S] thank you for doing this!
> PutElasticsearchJson exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12343
> URL: https://issues.apache.org/jira/browse/NIFI-12343
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2, 2.0.0-M2
>Reporter: Gregory M. Foreman
>Assignee: Chris Sampson
>Priority: Major
> Fix For: 2.0.0-M4
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
>
> PutElasticsearchJson throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchJson[id=] No FlowFiles successfully parsed for sending to
> Elasticsearch
> PutElasticsearchJson[id=] Could not read FlowFile content valid JSON.:
> com.fasterxml.jackson.core.exc.StreamConstraintsException: String length
> (20050553) exceeds the maximum length (2000)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12343) PutElasticsearchJson exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850474#comment-17850474
]
ASF subversion and git services commented on NIFI-12343:
Commit cbe5f7db9d678518b9846af1150c9b0e55afae1a in nifi's branch
refs/heads/main from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=cbe5f7db9d ]
NIFI-12343 Added Max JSON Field String Length for Elasticsearch
This closes #8881
Signed-off-by: David Handermann
> PutElasticsearchJson exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12343
> URL: https://issues.apache.org/jira/browse/NIFI-12343
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2, 2.0.0-M2
>Reporter: Gregory M. Foreman
>Assignee: Chris Sampson
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
>
> PutElasticsearchJson throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchJson[id=] No FlowFiles successfully parsed for sending to
> Elasticsearch
> PutElasticsearchJson[id=] Could not read FlowFile content valid JSON.:
> com.fasterxml.jackson.core.exc.StreamConstraintsException: String length
> (20050553) exceeds the maximum length (2000)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12343) PutElasticsearchJson exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849691#comment-17849691
]
Chris Sampson commented on NIFI-12343:
--
This is caused by the fact that the default Jackson {{ObjectMapper}} maximum
string read length is limited to 20MB.
As with the {{JsonTreeReader}} and other components in NiFi, we could allow the
override of this setting when creating the {{ObjectMapper}} in the
{{PutElasticsearchJson}} processor (and other Elasticsearch related processors
where they're using an {{ObjectMapper}} to parse thngs such as the {{query}}
attribute, etc. although they're much less likely to container such large
String values).
A likely workaround for this issue is to use the {{PutElasticsearchRecord}}
processor instead, with a {{JsonTreeReader}} using a larger {{Max String
Length}} setting.
> PutElasticsearchJson exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12343
> URL: https://issues.apache.org/jira/browse/NIFI-12343
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2, 2.0.0-M2
>Reporter: Gregory M. Foreman
>Priority: Major
>
>
> PutElasticsearchJson throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchJson[id=] No FlowFiles successfully parsed for sending to
> Elasticsearch
> PutElasticsearchJson[id=] Could not read FlowFile content valid JSON.:
> com.fasterxml.jackson.core.exc.StreamConstraintsException: String length
> (20050553) exceeds the maximum length (2000)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
