[jira] [Updated] (NIFI-13298) Remove unused instantiated java.util.HashSet from RouteAttribute
[ https://issues.apache.org/jira/browse/NIFI-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-13298: Status: Patch Available (was: In Progress) > Remove unused instantiated java.util.HashSet from RouteAttribute > > > Key: NIFI-13298 > URL: https://issues.apache.org/jira/browse/NIFI-13298 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > In RouteOnAttribute on line 352 a set is instantiated > final Set clones = new HashSet<>(); > and on line 358 its populated > clones.add(cloneFlowFile); > Yet after that nothing is done with it (per Intellij it is updated but > never queried). > This should removed as it is not serving any purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13298 Removed unused instantiated java.util.HashSet from RouteAttribute [nifi]
dan-s1 opened a new pull request, #8883: URL: https://github.com/apache/nifi/pull/8883 # Summary [NIFI-13298](https://issues.apache.org/jira/browse/NIFI-13298) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### UI Contributions - [ ] NiFi is modernizing its UI. Any contributions that update the [current UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui) also need to be implemented in the [new UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi). ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13302) Replace deprecated method authorizeRequests in NiFiRegistrySecurityConfig with API suggestion
[ https://issues.apache.org/jira/browse/NIFI-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-13302: Status: Patch Available (was: In Progress) > Replace deprecated method authorizeRequests in NiFiRegistrySecurityConfig > with API suggestion > - > > Key: NIFI-13302 > URL: https://issues.apache.org/jira/browse/NIFI-13302 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13302 Replaced deprecated method authorizeRequests with method authorizeHttpRequests [nifi]
dan-s1 opened a new pull request, #8882: URL: https://github.com/apache/nifi/pull/8882 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### UI Contributions - [ ] NiFi is modernizing its UI. Any contributions that update the [current UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui) also need to be implemented in the [new UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi). ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13302) Replace deprecated method authorizeRequests in NiFiRegistrySecurityConfig with API suggestion
Daniel Stieglitz created NIFI-13302: --- Summary: Replace deprecated method authorizeRequests in NiFiRegistrySecurityConfig with API suggestion Key: NIFI-13302 URL: https://issues.apache.org/jira/browse/NIFI-13302 Project: Apache NiFi Issue Type: Improvement Reporter: Daniel Stieglitz Assignee: Daniel Stieglitz -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-13265) Remove the instantiation of Object arrays for arguments in ComponentLog log and org.slf4j.Logger statements
[
https://issues.apache.org/jira/browse/NIFI-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849800#comment-17849800
]
Daniel Stieglitz edited comment on NIFI-13265 at 5/27/24 6:25 PM:
--
[~exceptionfactory] The basic fix for this ticket is to remove the
instantiation of an Object[] for a log statement's arguments which by in large
I have taken care of. There are though some log statements which I saw which
seem be superfluous. The prime cases I saw where this a parameter for an
exception object and in some cases there is a parameter for an exception's
message and the last parameter is an exception. Please correct me if I am wrong
but I thought the SimpleProcessLogger can detect when the last parameter is a
Throwable and it emits a message which includes the Throwable's message. Hence
is it necessary to have parameters for the exception or the exception's
message? I didn't think so so in some cases I reworded the log message e.g.
For file
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord.java
line 245 the original code is
{code:java}
getLogger().error("Failed to retrieve content from {} for {} due to {}; routing
to failure", new Object[] {filenameValue, originalFlowFile, e});
{code}
and I changed it to
{code:java}
getLogger().error("Routing to failure since failed to retrieve content from {}
for {} due to ", filenameValue, originalFlowFile, e);
{code}
Is that okay?
was (Author: JIRAUSER294662):
[~exceptionfactory] The basic fix for this ticket is to remove the
instantiation of an Object[] for a log statement's arguments which by in large
I have taken care of. There are though some log statements which I saw which
seem be superfluous. The prime cases I saw where this a parameter for an
exception object and in some cases there is a parameter for an exception's
message and the last parameter is an exception. Please correct me if I am wrong
but I thought the SimpleProcessLogger can detect when the last parameter is a
Throwable and it emits a message which includes the Throwable's message and a
shorter stacktrace. Hence is it necessary to have parameters for the exception
or the exception's message? I didn't think so so in some cases I reworded the
log message e.g.
For file
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord.java
line 245 the original code is
{code:java}
getLogger().error("Failed to retrieve content from {} for {} due to {}; routing
to failure", new Object[] {filenameValue, originalFlowFile, e});
{code}
and I changed it to
{code:java}
getLogger().error("Routing to failure since failed to retrieve content from {}
for {} due to ", filenameValue, originalFlowFile, e);
{code}
Is that okay?
> Remove the instantiation of Object arrays for arguments in ComponentLog log
> and org.slf4j.Logger statements
> ---
>
> Key: NIFI-13265
> URL: https://issues.apache.org/jira/browse/NIFI-13265
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
>
> There are still classes after the changes in NIFI-12075 and NIFI-12076 which
> instantiate an Object array for ComponentLog log statements. This ticket aims
> to remove those instantiations. In addition similar changes should be made to
> org.slf4j.Logger statements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-13265) Remove the instantiation of Object arrays for arguments in ComponentLog log and org.slf4j.Logger statements
[
https://issues.apache.org/jira/browse/NIFI-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849800#comment-17849800
]
Daniel Stieglitz edited comment on NIFI-13265 at 5/27/24 6:18 PM:
--
[~exceptionfactory] The basic fix for this ticket is to remove the
instantiation of an Object[] for a log statement's arguments which by in large
I have taken care of. There are though some log statements which I saw which
seem be superfluous. The prime cases I saw where this a parameter for an
exception object and in some cases there is a parameter for an exception's
message and the last parameter is an exception. Please correct me if I am wrong
but I thought the SimpleProcessLogger can detect when the last parameter is a
Throwable and it emits a message which includes the Throwable's message and a
shorter stacktrace. Hence is it necessary to have parameters for the exception
or the exception's message? I didn't think so so in some cases I reworded the
log message e.g.
For file
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord.java
line 245 the original code is
{code:java}
getLogger().error("Failed to retrieve content from {} for {} due to {}; routing
to failure", new Object[] {filenameValue, originalFlowFile, e});
{code}
and I changed it to
{code:java}
getLogger().error("Routing to failure since failed to retrieve content from {}
for {} due to ", filenameValue, originalFlowFile, e);
{code}
Is that okay?
was (Author: JIRAUSER294662):
[~exceptionfactory] The basic fix for this ticket is to remove the
instantiation of an Object[] for a log statement's arguments which by in large
I have taken care of. There are though some log statements which I saw which
seem be superfluous. The prime cases I saw where this a parameter for an
exception object and in some cases there is a parameter for an exception's
message and the last parameter is an exception. Please correct me if I am wrong
but I thought the SimpleProcessLogger can detect when the last parameter is a
Throwable and it emits a message which includes the Throwable's message and a
shorter stacktrace. Hence is it necessary to have parameters for the exception
or the exception's message? I didn't think so so in some cases I reworded the
log message e.g.
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord.java
line 245
{code:java}
getLogger().error("Failed to retrieve content from {} for {} due to {}; routing
to failure", new Object[] {filenameValue, originalFlowFile, e});
{code}
to
{code:java}
getLogger().error("Routing to failure since failed to retrieve content from {}
for {} due to ", filenameValue, originalFlowFile, e);
{code}
Is that okay?
> Remove the instantiation of Object arrays for arguments in ComponentLog log
> and org.slf4j.Logger statements
> ---
>
> Key: NIFI-13265
> URL: https://issues.apache.org/jira/browse/NIFI-13265
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
>
> There are still classes after the changes in NIFI-12075 and NIFI-12076 which
> instantiate an Object array for ComponentLog log statements. This ticket aims
> to remove those instantiations. In addition similar changes should be made to
> org.slf4j.Logger statements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-13265) Remove the instantiation of Object arrays for arguments in ComponentLog log and org.slf4j.Logger statements
[
https://issues.apache.org/jira/browse/NIFI-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849800#comment-17849800
]
Daniel Stieglitz edited comment on NIFI-13265 at 5/27/24 6:15 PM:
--
[~exceptionfactory] The basic fix for this ticket is to remove the
instantiation of an Object[] for a log statement's arguments which by in large
I have taken care of. There are though some log statements which I saw which
seem be superfluous. The prime cases I saw where this a parameter for an
exception object and in some cases there is a parameter for an exception's
message and the last parameter is an exception. Please correct me if I am wrong
but I thought the SimpleProcessLogger can detect when the last parameter is a
Throwable and it emits a message which includes the Throwable's message and a
shorter stacktrace. Hence is it necessary to have parameters for the exception
or the exception's message? I didn't think so so in some cases I reworded the
log message e.g.
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord.java
line 245
{code:java}
getLogger().error("Failed to retrieve content from {} for {} due to {}; routing
to failure", new Object[] {filenameValue, originalFlowFile, e});
{code}
to
{code:java}
getLogger().error("Routing to failure since failed to retrieve content from {}
for {} due to ", filenameValue, originalFlowFile, e);
{code}
Is that okay?
was (Author: JIRAUSER294662):
[~exceptionfactory] The basic fix for this ticket is to remove the
instantiation of an Object[] for a log statement's arguments which by in large
I have taken care of. There are though some log statements which I saw which
seem be superfluous. The prime cases I saw where this a parameter for an
exception object and in some cases there is a parameter for an exceptions
message and the last parameter is an exception. Please correct me if I am wrong
but I thought the SimpleProcessLogger can detect when the last parameter is a
Throwable and it emits a message which includes the Throwable's message and a
shorter stacktrace. Hence is it necessary to have parameters for the exception
or the exception's message? I didn't think so so in some cases I reworded the
log message e.g.
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord.java
line 245
{code:java}
getLogger().error("Failed to retrieve content from {} for {} due to {}; routing
to failure", new Object[] {filenameValue, originalFlowFile, e});
{code}
to
{code:java}
getLogger().error("Routing to failure since failed to retrieve content from {}
for {} due to ", filenameValue, originalFlowFile, e);
{code}
Is that okay?
> Remove the instantiation of Object arrays for arguments in ComponentLog log
> and org.slf4j.Logger statements
> ---
>
> Key: NIFI-13265
> URL: https://issues.apache.org/jira/browse/NIFI-13265
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
>
> There are still classes after the changes in NIFI-12075 and NIFI-12076 which
> instantiate an Object array for ComponentLog log statements. This ticket aims
> to remove those instantiations. In addition similar changes should be made to
> org.slf4j.Logger statements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-13265) Remove the instantiation of Object arrays for arguments in ComponentLog log and org.slf4j.Logger statements
[
https://issues.apache.org/jira/browse/NIFI-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849800#comment-17849800
]
Daniel Stieglitz commented on NIFI-13265:
-
[~exceptionfactory] The basic fix for this ticket is to remove the
instantiation of an Object[] for a log statement's arguments which by in large
I have taken care of. There are though some log statements which I saw which
seem be superfluous. The prime cases I saw where this a parameter for an
exception object and in some cases there is a parameter for an exceptions
message and the last parameter is an exception. Please correct me if I am wrong
but I thought the SimpleProcessLogger can detect when the last parameter is a
Throwable and it emits a message which includes the Throwable's message and a
shorter stacktrace. Hence is it necessary to have parameters for the exception
or the exception's message? I didn't think so so in some cases I reworded the
log message e.g.
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord.java
line 245
{code:java}
getLogger().error("Failed to retrieve content from {} for {} due to {}; routing
to failure", new Object[] {filenameValue, originalFlowFile, e});
{code}
to
{code:java}
getLogger().error("Routing to failure since failed to retrieve content from {}
for {} due to ", filenameValue, originalFlowFile, e);
{code}
Is that okay?
> Remove the instantiation of Object arrays for arguments in ComponentLog log
> and org.slf4j.Logger statements
> ---
>
> Key: NIFI-13265
> URL: https://issues.apache.org/jira/browse/NIFI-13265
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
>
> There are still classes after the changes in NIFI-12075 and NIFI-12076 which
> instantiate an Object array for ComponentLog log statements. This ticket aims
> to remove those instantiations. In addition similar changes should be made to
> org.slf4j.Logger statements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11766) ElasticSearchClientServiceImpl malformed (nonstandard?) expires attribute when connecting through AWS LoadBalancer
[
https://issues.apache.org/jira/browse/NIFI-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849791#comment-17849791
]
Chris Sampson commented on NIFI-11766:
--
This appears to be a warning logged by the difference between what the
Elasticsearch REST client's use of Apache HttpClient expects with Cookie
Headers vs. what the AWS ALB is sending. See
https://github.com/elastic/support-diagnostics/issues/233, for example.
This appears to be ignorable and could be excluded from the NiFi logs by
changing one's {{logback.xml}} configuration, but that comes at the expense of
also removing logs for these classes/packages that may well be relevant for
other issues.
A suggested approach is to change the unerlying Cookie Handler, e.g.
{code:java}
builder.setRequestConfigCallback(
requestConfigBuilder ->
requestConfigBuilder.setCookieSpec(CookieSpecs.STANDARD));
{code}
which could be done in the {{ElasticSearchClientServiceImpl#setupClient}},
**but** that's likely to cause problems with some of the available
Elasticsearch authentication strategies/functionality, so is probably not the
right answer here.
Instead, this is maybe something that would be handled without warning by the
AWS OpenSearch Client library equivalent, if NiFi implements such an
alternative in the future.
> ElasticSearchClientServiceImpl malformed (nonstandard?) expires attribute
> when connecting through AWS LoadBalancer
> --
>
> Key: NIFI-11766
> URL: https://issues.apache.org/jira/browse/NIFI-11766
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.19.1
> Environment: Ubuntu 22.04, EC2 machine in AWS,
> Java: 11.0.18, AWS
>Reporter: Marek Radwański
>Priority: Minor
>
> Each request to elasticsearch from a processor that utilizes
> ElasticSearchClientServiceImpl to connect to the cluster results in a logged
> Warning entry of:
> 2023-06-30 11:11:42,194 WARN [I/O dispatcher 2]
> o.a.h.c.protocol.ResponseProcessCookies Invalid cookie header: "Set-Cookie:
> AWSALBCORS=m0NDYBGa0v2NsPURJUVvxtso3AQ4Zatbj8EQDO2qLpAYRcHoG4oLoOQkQNhzXAuSxoJhTUD944olr/wNhfszNBS5ENVkivErGMFyys2EfL;
> Expires=Fri, 07 Jul 2023 11:11:42 GMT; Path=/; SameSite=None". Invalid
> 'expires' attribute: Fri, 07 Jul 2023 11:11:42 GMT
> Resulting in millions of those entries accumulating over time on a large
> cluster.
> This seems like a mismatch of cookie standards the underlying HTTP client
> uses and what AWS ALB expects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13265) Remove the instantiation of Object arrays for arguments in ComponentLog log and org.slf4j.Logger statements
[ https://issues.apache.org/jira/browse/NIFI-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-13265: Description: There are still classes after the changes in NIFI-12075 and NIFI-12076 which instantiate an Object array for ComponentLog log statements. This ticket aims to remove those instantiations. In addition similar changes should be made to (was: There are still classes after the changes in NIFI-12075 and NIFI-12076 which instantiate an Object array for ComponentLog log statements. This ticket aims to remove those instantiations.) > Remove the instantiation of Object arrays for arguments in ComponentLog log > and org.slf4j.Logger statements > --- > > Key: NIFI-13265 > URL: https://issues.apache.org/jira/browse/NIFI-13265 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > > There are still classes after the changes in NIFI-12075 and NIFI-12076 which > instantiate an Object array for ComponentLog log statements. This ticket aims > to remove those instantiations. In addition similar changes should be made to -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13265) Remove the instantiation of Object arrays for arguments in ComponentLog log and org.slf4j.Logger statements
[ https://issues.apache.org/jira/browse/NIFI-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-13265: Description: There are still classes after the changes in NIFI-12075 and NIFI-12076 which instantiate an Object array for ComponentLog log statements. This ticket aims to remove those instantiations. In addition similar changes should be made to org.slf4j.Logger statements. (was: There are still classes after the changes in NIFI-12075 and NIFI-12076 which instantiate an Object array for ComponentLog log statements. This ticket aims to remove those instantiations. In addition similar changes should be made to ) > Remove the instantiation of Object arrays for arguments in ComponentLog log > and org.slf4j.Logger statements > --- > > Key: NIFI-13265 > URL: https://issues.apache.org/jira/browse/NIFI-13265 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > > There are still classes after the changes in NIFI-12075 and NIFI-12076 which > instantiate an Object array for ComponentLog log statements. This ticket aims > to remove those instantiations. In addition similar changes should be made to > org.slf4j.Logger statements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12343) PutElasticsearchJson exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson reassigned NIFI-12343:
Assignee: Chris Sampson
> PutElasticsearchJson exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12343
> URL: https://issues.apache.org/jira/browse/NIFI-12343
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2, 2.0.0-M2
>Reporter: Gregory M. Foreman
>Assignee: Chris Sampson
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> PutElasticsearchJson throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchJson[id=] No FlowFiles successfully parsed for sending to
> Elasticsearch
> PutElasticsearchJson[id=] Could not read FlowFile content valid JSON.:
> com.fasterxml.jackson.core.exc.StreamConstraintsException: String length
> (20050553) exceeds the maximum length (2000)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12343) PutElasticsearchJson exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-12343:
-
Status: Patch Available (was: In Progress)
> PutElasticsearchJson exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12343
> URL: https://issues.apache.org/jira/browse/NIFI-12343
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 2.0.0-M2, 1.23.2
>Reporter: Gregory M. Foreman
>Assignee: Chris Sampson
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> PutElasticsearchJson throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchJson[id=] No FlowFiles successfully parsed for sending to
> Elasticsearch
> PutElasticsearchJson[id=] Could not read FlowFile content valid JSON.:
> com.fasterxml.jackson.core.exc.StreamConstraintsException: String length
> (20050553) exceeds the maximum length (2000)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12343 allow override of JSON field max string length for PutElasticsearchJson [nifi]
ChrisSamo632 opened a new pull request, #8881: URL: https://github.com/apache/nifi/pull/8881 # Summary [NIFI-12343](https://issues.apache.org/jira/browse/NIFI-12343) allow override of JSON field max string length for PutElasticsearchJson. Also allow the same override for other Elasticsearch processors that use an `ObjectMapper` to parse input/output. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### UI Contributions - ~[ ] NiFi is modernizing its UI. Any contributions that update the [current UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui) also need to be implemented in the [new UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi).~ ### Licensing - ~[ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)~ - ~[ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files~ ### Documentation - ~[ ] Documentation formatting appears as expected in rendered files~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12343) PutElasticsearchJson exception with JSON strings over 20 MB
[
https://issues.apache.org/jira/browse/NIFI-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849691#comment-17849691
]
Chris Sampson commented on NIFI-12343:
--
This is caused by the fact that the default Jackson {{ObjectMapper}} maximum
string read length is limited to 20MB.
As with the {{JsonTreeReader}} and other components in NiFi, we could allow the
override of this setting when creating the {{ObjectMapper}} in the
{{PutElasticsearchJson}} processor (and other Elasticsearch related processors
where they're using an {{ObjectMapper}} to parse thngs such as the {{query}}
attribute, etc. although they're much less likely to container such large
String values).
A likely workaround for this issue is to use the {{PutElasticsearchRecord}}
processor instead, with a {{JsonTreeReader}} using a larger {{Max String
Length}} setting.
> PutElasticsearchJson exception with JSON strings over 20 MB
> ---
>
> Key: NIFI-12343
> URL: https://issues.apache.org/jira/browse/NIFI-12343
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.23.2, 2.0.0-M2
>Reporter: Gregory M. Foreman
>Priority: Major
>
>
> PutElasticsearchJson throws an exception when reading JSON documents that
> contain string fields over 20 MB:
> {code:java}
> PutElasticsearchJson[id=] No FlowFiles successfully parsed for sending to
> Elasticsearch
> PutElasticsearchJson[id=] Could not read FlowFile content valid JSON.:
> com.fasterxml.jackson.core.exc.StreamConstraintsException: String length
> (20050553) exceeds the maximum length (2000)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-13301) Create NiFi Processor Marketplace App Powered by NiFi Registry
James Guzman (Medel) created NIFI-13301:
---
Summary: Create NiFi Processor Marketplace App Powered by NiFi
Registry
Key: NIFI-13301
URL: https://issues.apache.org/jira/browse/NIFI-13301
Project: Apache NiFi
Issue Type: New Feature
Components: Extensions, NiFi Registry
Environment: Linux: Ubuntu 20.04 or 22.04
Reporter: James Guzman (Medel)
Assignee: James Guzman (Medel)
{*}Description (NiFi){*}: Some people in the community have shown interest in
having a NiFi Processor Marketplace where they can contribute their category of
processors. In particular, I saw some people interested in contributing their
NiFi python processors. This NiFi Processor marketplace could be also applied
toward the community interested in contributing custom NiFi java processors.
* {*}Extra MiNiFi C++{*}: We could even add a section for MiNiFi C++ custom
processors. There is a part of the MiNiFi build process that brings in NiFi
through building the JNI extension. So, that is a way to integrate NiFi
Registry there too.
I have briefly talked with [~bbende] and [~joewitt] asking them questions on
ways to bring custom python processors into production. One of the routes was
through leveraging NiFi Registry in connection with Apache NiFi to streamline
integration. For my use case, I would leverage the NiFi Processor Marketplace
to start contributing my AI/DL "Medical Imaging" Pipeline of custom NiFi Python
Processors where I focused on my master thesis AI/DL for stroke diagnosis.
*High Level Details of NiFi Processor Marketplace App:* For a full stack NiFi
Processor Marketplace App, there are frameworks we can leverage for the UI from
ReactJS (this mainly is used for frontend), H2O.ai Wave (this can be used for
full stack, but probably would mainly be used for the frontend portion) and
other options. in the case that we want to make something more custom to
display each category of processors. This full stack application could interact
wth the NiFi Registry, which is the central location for storage and management
of shared resources across one or more instances of NiFi and/or MiNiFi (C++).
What are some hosting options for Apache projects?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
