[GitHub] [nifi] nathluu commented on pull request #6400: NIFI-10481 Add NiFi configuration to disable zk client ensembleTracker
nathluu commented on PR #6400: URL: https://github.com/apache/nifi/pull/6400#issuecomment-1289937025 Thank @thenatog for helping review and test my PR. To verify the EnsembleTracker option, we need the zookeeper quorum deployed inside the K8S cluster same as with NiFi. Below are the steps for testing: Setup: - Deploy ZK quorum in the AKS - Deploy NiFi to use K8S service FQDN of ZK as the connection string e.g. test-zookeeper.zk-ns.svc.cluster.local:2181 Case 1: EnsembleTracker enabled (default) 1/ Rollout restart all ZK pods (Not NiFi pods) 2/ When all ZK restarted and up again, access the NiFi canvas UI 3/ Verify that the UI will not be load 4/ Rollout restart all of the NiFi pods will resolve the issue Case 2: EnsembleTracker disable (set **nifi.zookeeper.client.ensembleTracker=false** in nifi.properties file) With a working NiFi (canvas is accessible) 1/ Rollout restart all ZK pods (Not NiFi pods) 2/ When all ZK restarted and up again, access the NiFi canvas UI 3/ Verify that the NiFi canvas can be access normally -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] lizhizhou commented on pull request #6416: NIFI-10234 implement PutIoTDB
lizhizhou commented on PR #6416: URL: https://github.com/apache/nifi/pull/6416#issuecomment-1289909895 > Thanks for the recent updates @lizhizhou, I will take a closer look soon. At the moment, there is a merge conflict with `nifi-nar-bundles/pom.xml`, can you take a look and resolve? Resolved. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #6400: NIFI-10481 Add NiFi configuration to disable zk client ensembleTracker
thenatog commented on PR #6400: URL: https://github.com/apache/nifi/pull/6400#issuecomment-1289904465 Thanks @nathluu, this looks good. I tested this out in a 3 node cluster with an external Zookeeper cluster and found things to be working normally. I was not specifically sure how to verify that the ensembleTracker is actually disabled. Is there something which indicates this is now disabled? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on a diff in pull request #6400: NIFI-10481 Add NiFi configuration to disable zk client ensembleTracker
thenatog commented on code in PR #6400:
URL: https://github.com/apache/nifi/pull/6400#discussion_r1003937995
##
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java:
##
@@ -1707,6 +1709,17 @@ public boolean isZooKeeperClientSecure() {
return Boolean.parseBoolean(clientSecure);
}
+public boolean isZookeeperClientWithEnsembleTracker() {
+final String defaultValue =
String.valueOf(DEFAULT_ZOOKEEPER_CLIENT_ENSEMBLE_TRACKER);
+final String withEnsembleTracker =
getProperty(ZOOKEEPER_CLIENT_ENSEMBLE_TRACKER, defaultValue).trim();
+
+if (!"true".equalsIgnoreCase(withEnsembleTracker) &&
!"false".equalsIgnoreCase(withEnsembleTracker)) {
Review Comment:
_Boolean.valueOf()_ can be used instead of comparing against 'true' and
'false' literal string values. I can see this is taken from the
_isZooKeeperClientSecure()_ method, but this may not be the best approach to
validating the property input.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] thenatog commented on a diff in pull request #6400: NIFI-10481 Add NiFi configuration to disable zk client ensembleTracker
thenatog commented on code in PR #6400:
URL: https://github.com/apache/nifi/pull/6400#discussion_r1003937995
##
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java:
##
@@ -1707,6 +1709,17 @@ public boolean isZooKeeperClientSecure() {
return Boolean.parseBoolean(clientSecure);
}
+public boolean isZookeeperClientWithEnsembleTracker() {
+final String defaultValue =
String.valueOf(DEFAULT_ZOOKEEPER_CLIENT_ENSEMBLE_TRACKER);
+final String withEnsembleTracker =
getProperty(ZOOKEEPER_CLIENT_ENSEMBLE_TRACKER, defaultValue).trim();
+
+if (!"true".equalsIgnoreCase(withEnsembleTracker) &&
!"false".equalsIgnoreCase(withEnsembleTracker)) {
Review Comment:
_Boolean.valueOf()_ can be used instead of comparing against 'true' and
'false' literal string values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] BoyuLi4 commented on pull request #6569: NIFI-10692 Fixed Possible Array Ordering Permutations Problem in Tests
BoyuLi4 commented on PR #6569: URL: https://github.com/apache/nifi/pull/6569#issuecomment-1289883630 HI, could you please re-run the checks? I have seen that all small parts of the failed checks were passed actually, but in the end it ran out of time... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10554) NiFi Registry SSL Protocol Error
[ https://issues.apache.org/jira/browse/NIFI-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623477#comment-17623477 ] Irudya Raj commented on NIFI-10554: --- [~exceptionfactory] Thanks for your time. I resolved the issue by concatenating root and server certificates then it works. > NiFi Registry SSL Protocol Error > > > Key: NIFI-10554 > URL: https://issues.apache.org/jira/browse/NIFI-10554 > Project: Apache NiFi > Issue Type: Bug > Components: NiFi Registry >Affects Versions: 1.17.0 > Environment: Ubuntu 20.04 > Java 11 > nifi-registry-1.17.0 >Reporter: Irudya Raj >Priority: Major > > I have setup nifi registry 1.17.0 with truststore and keystore and can see > the log file that it has started on SSL using port 18433 but when I accessed > the application from browser, it reported SSL protocol error. > I looked at the log file and found SSL factory was created with http.1 > protocol. Is this causing issue? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10680) Dependency enforcer fails after banned bouncycastle version change
[
https://issues.apache.org/jira/browse/NIFI-10680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-10680.
-
Resolution: Fixed
> Dependency enforcer fails after banned bouncycastle version change
> --
>
> Key: NIFI-10680
> URL: https://issues.apache.org/jira/browse/NIFI-10680
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.19.0
>Reporter: Kalmár Róbert
>Assignee: Joe Witt
>Priority: Major
> Fix For: 1.19.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The change in banned dependencies introduced with NIFI-10614 causes
> Nifi-Registry build to fail with some profiles.
> {code:java}
> [INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven-version) @
> nifi-registry-ranger-plugin ---
> [ERROR] Rule 3: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> Found Banned Dependency: org.bouncycastle:bcprov-jdk15on:jar:1.67
> Found Banned Dependency: org.bouncycastle:bcpkix-jdk15on:jar:1.67 {code}
> ozone-client seems to include them transitively:
> {code:java}
> [INFO] +- org.apache.ozone:ozone-client:jar:1.2.1:compile
> [INFO] | \- org.apache.ozone:ozone-common:jar:1.2.1:compile
> [INFO] | +- org.apache.ozone:hdds-common:jar:1.2.1:compile
> [INFO] | | +- org.bouncycastle:bcpkix-jdk15on:jar:1.67:compile {code}
> Profiles used in build:
> {code:java}
> -Pinclude-ranger,include-atlas,include-hive3,include-rules,include-sql-reporting,include-hadoop-aws,include-hadoop-azure,include-hadoop-cloud-storage,include-hadoop-gcp,include-graph,include-grpc,include-accumulo,include-hadoop-ozone,include-asn1,include-aws,include-snowflake,include-media
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-10691) Duplicated dependencies in nifi-standard-processors pom.xml
[ https://issues.apache.org/jira/browse/NIFI-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10691. - Fix Version/s: 1.19.0 Resolution: Fixed > Duplicated dependencies in nifi-standard-processors pom.xml > --- > > Key: NIFI-10691 > URL: https://issues.apache.org/jira/browse/NIFI-10691 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 1.19.0 > > > There are two dependencies duplicated in > nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml. > # > org.bouncycastle > bcprov-jdk18on > (defined on both lines 148-151 and lines 193-196) > # > org.bouncycastle > bcpkix-jdk18on > (defined on both lines 156-159 and lines 197-200) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10691) Duplicated dependencies in nifi-standard-processors pom.xml
[ https://issues.apache.org/jira/browse/NIFI-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623454#comment-17623454 ] ASF subversion and git services commented on NIFI-10691: Commit c92a30734fd1f5edab19d84db1c832be5fb4efa5 in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c92a30734f ] NIFI-10680 Corrected Bouncy Castle references in optional profiles NIFI-10691 Removed duplicate Bouncy Castle dependencies from nifi-standard-processors This closes #6571 Signed-off-by: David Handermann > Duplicated dependencies in nifi-standard-processors pom.xml > --- > > Key: NIFI-10691 > URL: https://issues.apache.org/jira/browse/NIFI-10691 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > > There are two dependencies duplicated in > nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml. > # > org.bouncycastle > bcprov-jdk18on > (defined on both lines 148-151 and lines 193-196) > # > org.bouncycastle > bcpkix-jdk18on > (defined on both lines 156-159 and lines 197-200) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10680) Dependency enforcer fails after banned bouncycastle version change
[
https://issues.apache.org/jira/browse/NIFI-10680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623453#comment-17623453
]
ASF subversion and git services commented on NIFI-10680:
Commit c92a30734fd1f5edab19d84db1c832be5fb4efa5 in nifi's branch
refs/heads/main from Joe Witt
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c92a30734f ]
NIFI-10680 Corrected Bouncy Castle references in optional profiles
NIFI-10691 Removed duplicate Bouncy Castle dependencies from
nifi-standard-processors
This closes #6571
Signed-off-by: David Handermann
> Dependency enforcer fails after banned bouncycastle version change
> --
>
> Key: NIFI-10680
> URL: https://issues.apache.org/jira/browse/NIFI-10680
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.19.0
>Reporter: Kalmár Róbert
>Assignee: Joe Witt
>Priority: Major
> Fix For: 1.19.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The change in banned dependencies introduced with NIFI-10614 causes
> Nifi-Registry build to fail with some profiles.
> {code:java}
> [INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven-version) @
> nifi-registry-ranger-plugin ---
> [ERROR] Rule 3: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> Found Banned Dependency: org.bouncycastle:bcprov-jdk15on:jar:1.67
> Found Banned Dependency: org.bouncycastle:bcpkix-jdk15on:jar:1.67 {code}
> ozone-client seems to include them transitively:
> {code:java}
> [INFO] +- org.apache.ozone:ozone-client:jar:1.2.1:compile
> [INFO] | \- org.apache.ozone:ozone-common:jar:1.2.1:compile
> [INFO] | +- org.apache.ozone:hdds-common:jar:1.2.1:compile
> [INFO] | | +- org.bouncycastle:bcpkix-jdk15on:jar:1.67:compile {code}
> Profiles used in build:
> {code:java}
> -Pinclude-ranger,include-atlas,include-hive3,include-rules,include-sql-reporting,include-hadoop-aws,include-hadoop-azure,include-hadoop-cloud-storage,include-hadoop-gcp,include-graph,include-grpc,include-accumulo,include-hadoop-ozone,include-asn1,include-aws,include-snowflake,include-media
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6571: NIFI-10680 cleanup ozone-client bouncycastle references now that jdk1…
exceptionfactory closed pull request #6571: NIFI-10680 cleanup ozone-client bouncycastle references now that jdk1… URL: https://github.com/apache/nifi/pull/6571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10691) Duplicated dependencies in nifi-standard-processors pom.xml
[ https://issues.apache.org/jira/browse/NIFI-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623450#comment-17623450 ] David Handermann commented on NIFI-10691: - Thanks for noting this issue [~dstiegli1], I am going to resolve it as part of merging in other Bouncy Castle changes for NIFI-10680. > Duplicated dependencies in nifi-standard-processors pom.xml > --- > > Key: NIFI-10691 > URL: https://issues.apache.org/jira/browse/NIFI-10691 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > > There are two dependencies duplicated in > nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml. > # > org.bouncycastle > bcprov-jdk18on > (defined on both lines 148-151 and lines 193-196) > # > org.bouncycastle > bcpkix-jdk18on > (defined on both lines 156-159 and lines 197-200) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] BoyuLi4 commented on pull request #6569: NIFI-10692 Fixed Possible Array Ordering Permutations Problem in Tests
BoyuLi4 commented on PR #6569: URL: https://github.com/apache/nifi/pull/6569#issuecomment-1289721507 Sorry, this is just a draft without the ISSUE part. . I opened it by mistake yesterday. But, I have updated the ISSUE section with the tool I use and the steps just now. Hope this will helps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10692) Possible Array Ordering Permutations Problem in Tests
[
https://issues.apache.org/jira/browse/NIFI-10692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Boyu Li updated NIFI-10692:
---
Description:
Following the problem of flakiness that occurred in the project
{code:java}
nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services{code}
test file
{code:java}
org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes{code}
with below three tests
{code:java}
testBasicTypes
testRecursive
testInheritance{code}
All three above tests use the same file, `JASN1ReadRecordTester` as the tester.
The test flakiness is due to comparisons between two ArrayList outputs from
Class
`SimpleRecordSchema` which use ArrayList to store one of the parameter
`List`
However, ArrayList does not guarantee entry orders, its object is an unordered
set of name/value pairs, and internal permutations may occur in the output as a
String.
*Steps to reproduce the failure:*
The flakiness was found with
[NonDex.|https://github.com/TestingResearchIllinois/NonDex)]
After installing NonDex and run the following code.
{code:java}
cd nifi
mvn install -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services -am
-DskipTests
mvn -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services
edu.illinois:nondex-
maven-plugin:1.1.3-SNAPSHOT:nondex -
Dtest=org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes#testBasicTypes
{code}
was:
Following the problem of flakiness that occurred in the project
`nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services`
test file
`org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes`
with below three tests
```
testBasicTypes
testRecursive
testInheritance
```
All three above tests use the same file, `JASN1ReadRecordTester` as the tester.
The test flakiness is due to comparisons between two ArrayList outputs from
Class
`SimpleRecordSchema` which use ArrayList to store one of the parameter
`List`
However, ArrayList does not guarantee entry orders, its object is an unordered
set of name/value pairs, and internal permutations may occur in the output as a
String.
*Steps to reproduce the failure:*
The flakiness was found with
[NonDex.|https://github.com/TestingResearchIllinois/NonDex)]
After installing NonDex and run the following code.
{code:java}
cd nifi
mvn install -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services -am
-DskipTests
mvn -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services
edu.illinois:nondex-
maven-plugin:1.1.3-SNAPSHOT:nondex -
Dtest=org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes#testBasicTypes
{code}
> Possible Array Ordering Permutations Problem in Tests
> -
>
> Key: NIFI-10692
> URL: https://issues.apache.org/jira/browse/NIFI-10692
> Project: Apache NiFi
> Issue Type: Test
> Components: NiFi Registry
>Affects Versions: 1.18.0
> Environment: Apache Maven 3.6.3
> Java version: 11.0.16, vendor: Ubuntu, runtime:
> /usr/lib/jvm/java-11-openjdk-amd64
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "5.4.0-124-generic", arch: "amd64", family: "unix"
>Reporter: Boyu Li
>Priority: Minor
>
> Following the problem of flakiness that occurred in the project
> {code:java}
> nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services{code}
> test file
> {code:java}
> org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes{code}
> with below three tests
> {code:java}
> testBasicTypes
> testRecursive
> testInheritance{code}
> All three above tests use the same file, `JASN1ReadRecordTester` as the
> tester.
> The test flakiness is due to comparisons between two ArrayList outputs from
> Class
> `SimpleRecordSchema` which use ArrayList to store one of the parameter
> `List`
> However, ArrayList does not guarantee entry orders, its object is an
> unordered set of name/value pairs, and internal permutations may occur in the
> output as a String.
>
> *Steps to reproduce the failure:*
> The flakiness was found with
> [NonDex.|https://github.com/TestingResearchIllinois/NonDex)]
> After installing NonDex and run the following code.
> {code:java}
> cd nifi
> mvn install -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services -am
> -DskipTests
> mvn -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services
> edu.illinois:nondex-
> maven-plugin:1.1.3-SNAPSHOT:nondex -
>
> Dtest=org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes#testBasicTypes
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-10692) Possible Array Ordering Permutations Problem in Tests
Boyu Li created NIFI-10692:
--
Summary: Possible Array Ordering Permutations Problem in Tests
Key: NIFI-10692
URL: https://issues.apache.org/jira/browse/NIFI-10692
Project: Apache NiFi
Issue Type: Test
Components: NiFi Registry
Affects Versions: 1.18.0
Environment: Apache Maven 3.6.3
Java version: 11.0.16, vendor: Ubuntu, runtime:
/usr/lib/jvm/java-11-openjdk-amd64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.4.0-124-generic", arch: "amd64", family: "unix"
Reporter: Boyu Li
Following the problem of flakiness that occurred in the project
`nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services`
test file
`org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes`
with below three tests
```
testBasicTypes
testRecursive
testInheritance
```
All three above tests use the same file, `JASN1ReadRecordTester` as the tester.
The test flakiness is due to comparisons between two ArrayList outputs from
Class
`SimpleRecordSchema` which use ArrayList to store one of the parameter
`List`
However, ArrayList does not guarantee entry orders, its object is an unordered
set of name/value pairs, and internal permutations may occur in the output as a
String.
*Steps to reproduce the failure:*
The flakiness was found with
[NonDex.|https://github.com/TestingResearchIllinois/NonDex)]
After installing NonDex and run the following code.
{code:java}
cd nifi
mvn install -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services -am
-DskipTests
mvn -pl nifi-nar-bundles/nifi-asn1-bundle/nifi-asn1-services
edu.illinois:nondex-
maven-plugin:1.1.3-SNAPSHOT:nondex -
Dtest=org.apache.nifi.jasn1.TestJASN1RecordReaderWithComplexTypes#testBasicTypes
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-10690) Update readme of nifi-minifi.git and disable travis CI for old minifi java codebase
[ https://issues.apache.org/jira/browse/NIFI-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-10690. - Resolution: Won't Fix repository is already listed as 'archived'/read-only. Users that end up there will be confused but our website should clear it up. > Update readme of nifi-minifi.git and disable travis CI for old minifi java > codebase > --- > > Key: NIFI-10690 > URL: https://issues.apache.org/jira/browse/NIFI-10690 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10689) UI - fix encoding for provider name
[ https://issues.apache.org/jira/browse/NIFI-10689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Margot Tien updated NIFI-10689: --- Status: Patch Available (was: Open) > UI - fix encoding for provider name > --- > > Key: NIFI-10689 > URL: https://issues.apache.org/jira/browse/NIFI-10689 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.18.0 >Reporter: Margot Tien >Assignee: Margot Tien >Priority: Minor > Attachments: image (7).png > > Time Spent: 10m > Remaining Estimate: 0h > > Fix encoding for the Provider name in the Update Parameter Context dialog. > !image (7).png|width=549,height=313! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10691) Duplicated dependencies in nifi-standard-processors pom.xml
Daniel Stieglitz created NIFI-10691: --- Summary: Duplicated dependencies in nifi-standard-processors pom.xml Key: NIFI-10691 URL: https://issues.apache.org/jira/browse/NIFI-10691 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Daniel Stieglitz Assignee: Daniel Stieglitz There are two dependencies duplicated in nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml. # org.bouncycastle bcprov-jdk18on (defined on both lines 148-151 and lines 193-196) # org.bouncycastle bcpkix-jdk18on (defined on both lines 156-159 and lines 197-200) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10690) Update readme of nifi-minifi.git and disable travis CI for old minifi java codebase
Joe Witt created NIFI-10690: --- Summary: Update readme of nifi-minifi.git and disable travis CI for old minifi java codebase Key: NIFI-10690 URL: https://issues.apache.org/jira/browse/NIFI-10690 Project: Apache NiFi Issue Type: Task Reporter: Joe Witt Assignee: Joe Witt -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-3262) PutElasticsearch support for _routing/_parent
[
https://issues.apache.org/jira/browse/NIFI-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623427#comment-17623427
]
Chris Sampson commented on NIFI-3262:
-
FYI, {{routing}} can be set as a [request
parameter|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk-api-query-params]
using a dynamic property on the {{PutElasticsearchRecord}} processor, although
that sets the value for the entire {{_bulk}} operation, not each individual
document contained within (where one might want to set different routing for
each document).
{{parent}} hasn't been supported in elasticsearch since [version
5.x|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-bulk.html#bulk-parent]
- this is simply a field added to the bulk header, so no need for nifi to
validate/prevent its use, just that users will need to expect and deal with
errors returned by elasticsearch if they use the field (or any other that's
unsupported) with newer versions.
> PutElasticsearch support for _routing/_parent
> -
>
> Key: NIFI-3262
> URL: https://issues.apache.org/jira/browse/NIFI-3262
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Josh Harrison
>Priority: Major
>
> Due to the structure of the current nifi Elasticsearch indexing processors,
> it doesn't appear possible to specify explicit _routing or _parent fields
> when using any of the PutElasticsearch processors (PutElasticsearch,
> PutElasticsearchHTTP, PutElasticsearch5).
> The ability to optionally specify the _parent and/or _routing parameters that
> can be present in a normal bulk index request would be really valuable.
> An alternative to adding a handled attribute would be adding support for the
> bulk indexing syntax as defined on
> https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html
> - a two line structure where line one contains the indexing instructions in
> a JSON object, and line two contains the content of the object. Each line
> ends with an explicit newline character (\n). This would allow bulk indexing,
> update, delete, etc operations to be carried out through nifi.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10067) Support script in Elasticsearch update operation
[
https://issues.apache.org/jira/browse/NIFI-10067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623425#comment-17623425
]
Chris Sampson commented on NIFI-10067:
--
The {{script}} field needs to be [used in
place|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-update]
of the {{doc}} source for an {{update}} operation, i.e. ignore any other
content for the Record (should this result in a failure with appropriate log?)
The {{dynamic_templates}} field needs to be [used as a
map|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-dynamic-templates]
(json object) within the bulk header, not a string (parse attribute string as
json then output a field value)
Other specified {{BULK:}} fields should be used as fields with values taken
directly from the attributes.
> Support script in Elasticsearch update operation
>
>
> Key: NIFI-10067
> URL: https://issues.apache.org/jira/browse/NIFI-10067
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Stijn Caerts
>Priority: Major
> Labels: elasticsearch
>
> Right now, all update operations on Elasticsearch with the
> PutElasticsearchRecord/Json processors use [the partial document update
> approach|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#_update_part_of_a_document].
> The content of the flow file is added as the {{doc}} in the update request
> by the
> [ElasticSearchClientServiceImpl|https://github.com/apache/nifi/blob/d79b3908360265eb0d50a5ce298a00c3054cdf82/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service/src/main/java/org/apache/nifi/elasticsearch/ElasticSearchClientServiceImpl.java#L281].
> Can there also be provided an option to [add a script to the update
> request|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#update-api-desc]?
> Using script updates [is supported in the Elasticsearch {{_bulk}}
> API|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-update],
> so it should be possible to add this feature.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] ottobackwards commented on pull request #6569: NIFI-XX Fixed Possible Array Ordering Permutations Problem in Tests
ottobackwards commented on PR #6569: URL: https://github.com/apache/nifi/pull/6569#issuecomment-1289611606 They are using an analysis tool that flags these kinds of things even if you don't see issues I think -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #6569: NIFI-XX Fixed Possible Array Ordering Permutations Problem in Tests
joewitt commented on PR #6569: URL: https://github.com/apache/nifi/pull/6569#issuecomment-1289607580 I've approved this to run but looking at the code it strikes me that we should be ensuring the equals check for RecordSchema objects works correctly rather than having to tweak it in this test. What is missing/incorrect in the current equals check of RecordSchema? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mtien-apache opened a new pull request, #6576: NIFI-10689: fix encoding for the provider name
mtien-apache opened a new pull request, #6576: URL: https://github.com/apache/nifi/pull/6576 # Summary [NIFI-10689](https://issues.apache.org/jira/browse/NIFI-10689) Fix encoding for the provider name in the Update Parameter Context dialog (UI). # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-10047) Refactor Groovy Tests to Java
[ https://issues.apache.org/jira/browse/NIFI-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Sampson reassigned NIFI-10047: Assignee: David Handermann (was: Chris Sampson) > Refactor Groovy Tests to Java > - > > Key: NIFI-10047 > URL: https://issues.apache.org/jira/browse/NIFI-10047 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Extensions, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > > A number of component modules include unit tests written in Groovy, while the > majority of tests are written in Java. Although Groovy has some advantages > for testing in particular, the lack of consistency across the framework > presents several maintenance challenges. > Groovy is similar enough to Java that it is possible to read with minimal > effort, but writing idiomatic Groovy requires a greater understanding of the > language. Some unit tests have leveraged Groovy to bypass method visibility > constraints, which violates standard class and method contracts. Compiling > and running tests in Groovy requires additional Maven configuration and > plugin execution, which contributes to the overall runtime of continuous > integration workflows. Using Java as the standard language for both > implementation and tests also makes it easier for contributors to maintain > and review changes. > For these reasons, existing Groovy test classes should be rewritten in Java, > with the exception of scripting components. > Refactoring and rewriting tests should be done in logical groups of work to > avoid missing important test functions in the conversion process. Some of the > modules with larger numbers of Groovy tests include the following: > * nifi-toolkit-encrypt-config > * nifi-toolkit-admin > * nifi-registry-core > * nifi-security-utils > * nifi-elasticsearch-restapi-processors > * nifi-elasticsearch-client-service > * nifi-framework-cluster > * nifi-framework-core > * nifi-web-api > * nifi-lookup-services > * nifi-standard-processors -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10687) PutElasticsearchRecord/Json incorrectly try to use empty String as document _id
[
https://issues.apache.org/jira/browse/NIFI-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson reassigned NIFI-10687:
Assignee: Chris Sampson
> PutElasticsearchRecord/Json incorrectly try to use empty String as document
> _id
> ---
>
> Key: NIFI-10687
> URL: https://issues.apache.org/jira/browse/NIFI-10687
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{PutElasticsearchRecord}} and {{PutElasticsearchJson}} will try to pass an
> empty/blank String as the {{_id}} of the document to be created in
> Elasticsearch if the processor is, for example, configured to use an
> attribute from the FlowFile or a field from the Record that doesn't exist.
> To re-create:
> * set the ID field to point at a non-existent Field using Expression
> Language, e.g. {{${notExist}}}
> * input a FlowFile where the attribute/Record Path equates to a non-existent
> Field (i.e. returns an empty String, but not {{null}})
> * observe that the attempt to send a document to Elasticsearch uses the empty
> String rather than {{null}}
> Setting the {{_id}} in NiFi to {{null}} in such circumstances would be
> preferable/expected as this would cause Elasticsearch to auto-generate an
> {{_id}} when indexing.
> Other NiFi processor fields may also be affected, e.g. for {{@timestamp}}
> (others should be considered, but some fields should legitimately equate to
> an empty/blank String and cause an error - {{index}}, for example)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10687) PutElasticsearchRecord/Json incorrectly try to use empty String as document _id
[
https://issues.apache.org/jira/browse/NIFI-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-10687:
-
Status: Patch Available (was: Open)
> PutElasticsearchRecord/Json incorrectly try to use empty String as document
> _id
> ---
>
> Key: NIFI-10687
> URL: https://issues.apache.org/jira/browse/NIFI-10687
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{PutElasticsearchRecord}} and {{PutElasticsearchJson}} will try to pass an
> empty/blank String as the {{_id}} of the document to be created in
> Elasticsearch if the processor is, for example, configured to use an
> attribute from the FlowFile or a field from the Record that doesn't exist.
> To re-create:
> * set the ID field to point at a non-existent Field using Expression
> Language, e.g. {{${notExist}}}
> * input a FlowFile where the attribute/Record Path equates to a non-existent
> Field (i.e. returns an empty String, but not {{null}})
> * observe that the attempt to send a document to Elasticsearch uses the empty
> String rather than {{null}}
> Setting the {{_id}} in NiFi to {{null}} in such circumstances would be
> preferable/expected as this would cause Elasticsearch to auto-generate an
> {{_id}} when indexing.
> Other NiFi processor fields may also be affected, e.g. for {{@timestamp}}
> (others should be considered, but some fields should legitimately equate to
> an empty/blank String and cause an error - {{index}}, for example)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 opened a new pull request, #6575: NIFI-10687 improve blank Elasticsearch document _id handling for PutElasticsearch processors
ChrisSamo632 opened a new pull request, #6575: URL: https://github.com/apache/nifi/pull/6575 # Summary [NIFI-10687](https://issues.apache.org/jira/browse/NIFI-NIFI-10687) set Elasticsearch document _id to null if ID attribute evaluated to blank String for PutElasticsearchRecord or PutElasticsearchJson NIFI-10687 use @timestamp default value if @timestamp record path evaluates to blank String in PutElasticsearchRecord # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [x] JDK 11 - [ ] JDK 17 ### Licensing - ~[ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)~ - ~[ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files~ ### Documentation - ~[ ] Documentation formatting appears as expected in rendered files~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10689) UI - fix encoding for provider name
Margot Tien created NIFI-10689: -- Summary: UI - fix encoding for provider name Key: NIFI-10689 URL: https://issues.apache.org/jira/browse/NIFI-10689 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.18.0 Reporter: Margot Tien Assignee: Margot Tien Attachments: image (7).png Fix encoding for the Provider name in the Update Parameter Context dialog. !image (7).png|width=549,height=313! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10612) Create new Expression Language method isJson to determine whether an attribute is JSON
[ https://issues.apache.org/jira/browse/NIFI-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-10612: Status: Patch Available (was: Open) > Create new Expression Language method isJson to determine whether an > attribute is JSON > -- > > Key: NIFI-10612 > URL: https://issues.apache.org/jira/browse/NIFI-10612 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Create a new Expression Language method isJson to determine whether > attributes are JSON or not. The addition of this method provides a short > circuit of failure before attempting a JsonPath which otherwise would result > in flow files yielding if the attribute is not JSON as detailed in NIFI-10396 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10612) Create new Expression Language method isJson to determine whether an attribute is JSON
[ https://issues.apache.org/jira/browse/NIFI-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-10612: Status: In Progress (was: Patch Available) > Create new Expression Language method isJson to determine whether an > attribute is JSON > -- > > Key: NIFI-10612 > URL: https://issues.apache.org/jira/browse/NIFI-10612 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Create a new Expression Language method isJson to determine whether > attributes are JSON or not. The addition of this method provides a short > circuit of failure before attempting a JsonPath which otherwise would result > in flow files yielding if the attribute is not JSON as detailed in NIFI-10396 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10047) Refactor Groovy Tests to Java
[ https://issues.apache.org/jira/browse/NIFI-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Sampson reassigned NIFI-10047: Assignee: Chris Sampson (was: David Handermann) > Refactor Groovy Tests to Java > - > > Key: NIFI-10047 > URL: https://issues.apache.org/jira/browse/NIFI-10047 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Extensions, NiFi Registry >Reporter: David Handermann >Assignee: Chris Sampson >Priority: Minor > > A number of component modules include unit tests written in Groovy, while the > majority of tests are written in Java. Although Groovy has some advantages > for testing in particular, the lack of consistency across the framework > presents several maintenance challenges. > Groovy is similar enough to Java that it is possible to read with minimal > effort, but writing idiomatic Groovy requires a greater understanding of the > language. Some unit tests have leveraged Groovy to bypass method visibility > constraints, which violates standard class and method contracts. Compiling > and running tests in Groovy requires additional Maven configuration and > plugin execution, which contributes to the overall runtime of continuous > integration workflows. Using Java as the standard language for both > implementation and tests also makes it easier for contributors to maintain > and review changes. > For these reasons, existing Groovy test classes should be rewritten in Java, > with the exception of scripting components. > Refactoring and rewriting tests should be done in logical groups of work to > avoid missing important test functions in the conversion process. Some of the > modules with larger numbers of Groovy tests include the following: > * nifi-toolkit-encrypt-config > * nifi-toolkit-admin > * nifi-registry-core > * nifi-security-utils > * nifi-elasticsearch-restapi-processors > * nifi-elasticsearch-client-service > * nifi-framework-cluster > * nifi-framework-core > * nifi-web-api > * nifi-lookup-services > * nifi-standard-processors -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10612) Create new Expression Language method isJson to determine whether an attribute is JSON
[ https://issues.apache.org/jira/browse/NIFI-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-10612: Status: Patch Available (was: Open) > Create new Expression Language method isJson to determine whether an > attribute is JSON > -- > > Key: NIFI-10612 > URL: https://issues.apache.org/jira/browse/NIFI-10612 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Create a new Expression Language method isJson to determine whether > attributes are JSON or not. The addition of this method provides a short > circuit of failure before attempting a JsonPath which otherwise would result > in flow files yielding if the attribute is not JSON as detailed in NIFI-10396 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10303) GetElasticsearch processor invokes elasticsearch GET api without an identifier when flow file is null
[
https://issues.apache.org/jira/browse/NIFI-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson reassigned NIFI-10303:
Assignee: Chris Sampson
> GetElasticsearch processor invokes elasticsearch GET api without an
> identifier when flow file is null
> -
>
> Key: NIFI-10303
> URL: https://issues.apache.org/jira/browse/NIFI-10303
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.16.3
>Reporter: Vijaya Gorla
>Assignee: Chris Sampson
>Priority: Major
> Labels: elasticsearch
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When GetElasticsearch processor has an incoming connection, and the document
> id is an expression, this results in expression evaluating to null when the
> processor runs without a flow file. This in turn results in elasticsearch GET
> api being invoked without an ID and elastic throwing an error.
> Also noticed that other processors handle this scenario by short-circuiting
> at the beginning of onTrigger method
> {code:java}
> FlowFile input = null;
> if (context.hasIncomingConnection()) {
> input = session.get();
> // If we have no FlowFile, and all incoming connections are
> self-loops then we can continue on.
> // However, if we have no FlowFile and we have connections coming
> from other Processors, then
> // we know that we should run only if we have a FlowFile.
> if (input == null && context.hasNonLoopConnection()) {
> return;
> }
> }
> {code}
> However GetElasticsearch does not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10303) GetElasticsearch processor invokes elasticsearch GET api without an identifier when flow file is null
[
https://issues.apache.org/jira/browse/NIFI-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-10303:
-
Status: Patch Available (was: Open)
> GetElasticsearch processor invokes elasticsearch GET api without an
> identifier when flow file is null
> -
>
> Key: NIFI-10303
> URL: https://issues.apache.org/jira/browse/NIFI-10303
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.16.3
>Reporter: Vijaya Gorla
>Assignee: Chris Sampson
>Priority: Major
> Labels: elasticsearch
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When GetElasticsearch processor has an incoming connection, and the document
> id is an expression, this results in expression evaluating to null when the
> processor runs without a flow file. This in turn results in elasticsearch GET
> api being invoked without an ID and elastic throwing an error.
> Also noticed that other processors handle this scenario by short-circuiting
> at the beginning of onTrigger method
> {code:java}
> FlowFile input = null;
> if (context.hasIncomingConnection()) {
> input = session.get();
> // If we have no FlowFile, and all incoming connections are
> self-loops then we can continue on.
> // However, if we have no FlowFile and we have connections coming
> from other Processors, then
> // we know that we should run only if we have a FlowFile.
> if (input == null && context.hasNonLoopConnection()) {
> return;
> }
> }
> {code}
> However GetElasticsearch does not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] dan-s1 opened a new pull request, #6574: [NIFI-10612] Initial check in of isJson code.
dan-s1 opened a new pull request, #6574: URL: https://github.com/apache/nifi/pull/6574 # Summary [NIFI-10612](https://issues.apache.org/jira/browse/NIFI-10612) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] ChrisSamo632 opened a new pull request, #6573: NIFI-10303 route GetElasticsearch to failure if _id is blank after attribute evaluation
ChrisSamo632 opened a new pull request, #6573: URL: https://github.com/apache/nifi/pull/6573 # Summary [NIFI-10303](https://issues.apache.org/jira/browse/NIFI-10303) route GetElasticsearch to failure if _id is blank after attribute evaluation # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [x] JDK 11 - [ ] JDK 17 ### Licensing - ~[ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)~ - ~[ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files~ ### Documentation - ~[ ] Documentation formatting appears as expected in rendered files~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] UcanInfosec commented on pull request #6570: NIFI-10686 Update questdb-core to 6.5.3-jdk8
UcanInfosec commented on PR #6570: URL: https://github.com/apache/nifi/pull/6570#issuecomment-1289514709 When I ran this on the branch, the checks ran successfully on the 2nd try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-10688) Update javax.json-api and javax.json to 1.1.4
[ https://issues.apache.org/jira/browse/NIFI-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-10688: - Assignee: Mike R > Update javax.json-api and javax.json to 1.1.4 > - > > Key: NIFI-10688 > URL: https://issues.apache.org/jira/browse/NIFI-10688 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update javax.json-api and javax.json to 1.1.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 opened a new pull request, #6572: NIFI-10688 Update javax.json-api & javax.json
mr1716 opened a new pull request, #6572: URL: https://github.com/apache/nifi/pull/6572 # Summary [NIFI-10688](https://issues.apache.org/jira/browse/NIFI-10688) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-10688) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10688) Update javax.json-api and javax.json to 1.1.4
Mike R created NIFI-10688: - Summary: Update javax.json-api and javax.json to 1.1.4 Key: NIFI-10688 URL: https://issues.apache.org/jira/browse/NIFI-10688 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.18.0 Reporter: Mike R Update javax.json-api and javax.json to 1.1.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-10303) GetElasticsearch processor invokes elasticsearch GET api without an identifier when flow file is null
[
https://issues.apache.org/jira/browse/NIFI-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17618800#comment-17618800
]
Chris Sampson edited comment on NIFI-10303 at 10/24/22 7:31 PM:
I don't think the {{input == null}} shotcut is appropriate here as it's
legitimate that no input FlowFile may be preset for {{GetElasticsearch}}.
However, you're right that it shouldn't try to execute a {{GET}} to
Elasticsearch if no {{_id}} (and/or {{index}} or {{type}} where used) is
present once any Expression Language has been evaluated for the processor's run
{{onTrigger}}. Maybe the most appropriate thing to do would be to:
* route the input FlowFile to {{failure}} (if there was a non-null {{input}}),
so that it can be handled appropriately by the Flow and the Flow maintainers
can identify why no {{_id}} was present for the call
* log an error and {{yield}} the processor if there was no {{input}} FlowFile
at all, e.g. the processor is scheduled to run automatically on a Cron Schedule
(or whatever) with an {{_id}} that might be a Parameter or Variable Registry
entry (or just a blank string) - this should generate Bulletins for the Flow
maintainers to spot and fix as it's a Flow configuration issue at that point
was (Author: chris s):
I don't think the {{input == null}} shotcut is appropriate here as it's
legitimate that no input FlowFile may be preset for {{GetElasticsearch}}.
However, you're right that it shouldn't try to execute a {{GET}} to
Elasticsearch if no {{_id}} (and/or {{index}} or {{type}} where used) is
present once any Expression Language has been evaluated for the processor's run
{{onTrigger}}. Maybe the most appropriate thing to do would be to:
* route the input FlowFile to {{failure}} (if there was a non-null {{input}}),
so that it can be handled appropriately by the Flow and the Flow maintainers
can identify why no {{_id}} was present for the call
* log and error and {{yield}} the processor if there was no {{input}} FlowFile
at all, e.g. the processor is scheduled to run automatically on a Cron Schedule
(or whatever) with an {{_id}} that might be a Parameter or Variable Registry
entry (or just a blank string) - this should generate Bulletins for the Flow
maintainers to spot and fix as it's a Flow configuration issue at that point
> GetElasticsearch processor invokes elasticsearch GET api without an
> identifier when flow file is null
> -
>
> Key: NIFI-10303
> URL: https://issues.apache.org/jira/browse/NIFI-10303
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.16.3
>Reporter: Vijaya Gorla
>Priority: Major
> Labels: elasticsearch
>
> When GetElasticsearch processor has an incoming connection, and the document
> id is an expression, this results in expression evaluating to null when the
> processor runs without a flow file. This in turn results in elasticsearch GET
> api being invoked without an ID and elastic throwing an error.
> Also noticed that other processors handle this scenario by short-circuiting
> at the beginning of onTrigger method
> {code:java}
> FlowFile input = null;
> if (context.hasIncomingConnection()) {
> input = session.get();
> // If we have no FlowFile, and all incoming connections are
> self-loops then we can continue on.
> // However, if we have no FlowFile and we have connections coming
> from other Processors, then
> // we know that we should run only if we have a FlowFile.
> if (input == null && context.hasNonLoopConnection()) {
> return;
> }
> }
> {code}
> However GetElasticsearch does not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10687) PutElasticsearchRecord/Json incorrectly try to use empty String as document _id
[
https://issues.apache.org/jira/browse/NIFI-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623360#comment-17623360
]
Chris Sampson commented on NIFI-10687:
--
Similar kind of problem to NIFI-10303 (affecting {{GetElasticsearch}})
> PutElasticsearchRecord/Json incorrectly try to use empty String as document
> _id
> ---
>
> Key: NIFI-10687
> URL: https://issues.apache.org/jira/browse/NIFI-10687
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Chris Sampson
>Priority: Major
>
> {{PutElasticsearchRecord}} and {{PutElasticsearchJson}} will try to pass an
> empty/blank String as the {{_id}} of the document to be created in
> Elasticsearch if the processor is, for example, configured to use an
> attribute from the FlowFile or a field from the Record that doesn't exist.
> To re-create:
> * set the ID field to point at a non-existent Field using Expression
> Language, e.g. {{${notExist}}}
> * input a FlowFile where the attribute/Record Path equates to a non-existent
> Field (i.e. returns an empty String, but not {{null}})
> * observe that the attempt to send a document to Elasticsearch uses the empty
> String rather than {{null}}
> Setting the {{_id}} in NiFi to {{null}} in such circumstances would be
> preferable/expected as this would cause Elasticsearch to auto-generate an
> {{_id}} when indexing.
> Other NiFi processor fields may also be affected, e.g. for {{@timestamp}}
> (others should be considered, but some fields should legitimately equate to
> an empty/blank String and cause an error - {{index}}, for example)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-10687) PutElasticsearchRecord/Json incorrectly try to use empty String as document _id
Chris Sampson created NIFI-10687:
Summary: PutElasticsearchRecord/Json incorrectly try to use empty
String as document _id
Key: NIFI-10687
URL: https://issues.apache.org/jira/browse/NIFI-10687
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.18.0
Reporter: Chris Sampson
{{PutElasticsearchRecord}} and {{PutElasticsearchJson}} will try to pass an
empty/blank String as the {{_id}} of the document to be created in
Elasticsearch if the processor is, for example, configured to use an attribute
from the FlowFile or a field from the Record that doesn't exist.
To re-create:
* set the ID field to point at a non-existent Field using Expression Language,
e.g. {{${notExist}}}
* input a FlowFile where the attribute/Record Path equates to a non-existent
Field (i.e. returns an empty String, but not {{null}})
* observe that the attempt to send a document to Elasticsearch uses the empty
String rather than {{null}}
Setting the {{_id}} in NiFi to {{null}} in such circumstances would be
preferable/expected as this would cause Elasticsearch to auto-generate an
{{_id}} when indexing.
Other NiFi processor fields may also be affected, e.g. for {{@timestamp}}
(others should be considered, but some fields should legitimately equate to an
empty/blank String and cause an error - {{index}}, for example)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-8214) Extend PutElasticsearchHttp / Elastic URL to take ff-Attributes
[ https://issues.apache.org/jira/browse/NIFI-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623354#comment-17623354 ] Chris Sampson commented on NIFI-8214: - Discussion started in [dev mailing list|https://lists.apache.org/thread/4pmrk6zp3pxyvz60p1j191fp2lwh2sz7] > Extend PutElasticsearchHttp / Elastic URL to take ff-Attributes > --- > > Key: NIFI-8214 > URL: https://issues.apache.org/jira/browse/NIFI-8214 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.12.1 >Reporter: ROGIER TIMMERMANS >Priority: Minor > > The PutElasticsearchHttp processor's URL field only allows Variable Registry > scope; this makes it less flexible than ea. InvokeHttp as this allows ff > attributes as well. > Suggesting a minor improvement/enhancement here. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 commented on pull request #6544: NIFI-9398 add verification to ElasticSearchClientService (with integration tests) and Elasticsearch REST API processors
ChrisSamo632 commented on PR #6544: URL: https://github.com/apache/nifi/pull/6544#issuecomment-1289476430 > Marked as `Draft` until I've completed more manual verifications via the NiFi UI. Added integration-tests for `nifi-elasticsearch-restapi-processors` to cover some Processor Verification steps (shares Testcontainer setup with `nifi-elasticsearch-client-service` using a new common `nifi-elasticsearch-integration-test-utils` module) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9437) Flowfile Expiration does not exceed 24 days
[ https://issues.apache.org/jira/browse/NIFI-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Gough updated NIFI-9437: --- Fix Version/s: 1.19.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Flowfile Expiration does not exceed 24 days > --- > > Key: NIFI-9437 > URL: https://issues.apache.org/jira/browse/NIFI-9437 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.14.0, 1.15.0 >Reporter: Tim Chermak >Assignee: Timea Barna >Priority: Minor > Fix For: 1.19.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > We discovered setting a FlowFile Expiration on a queue for anything over 24 > days is ignored and not 'aged out of the flow". So, if FlowFile Expiration is > set to anything beyond 24 days, the file does not automatically expire and > remains in the queue. > > Can this be fixed so that there is no limit? Also, perhaps the Expiration > setting can have other criteria/strategy for expiring, as indicated in an old > ticket NiFi-372 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-9437) Flowfile Expiration does not exceed 24 days
[ https://issues.apache.org/jira/browse/NIFI-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623346#comment-17623346 ] ASF subversion and git services commented on NIFI-9437: --- Commit 7c33516aeebf138dc2e2a82880f9bd7f49c7e61f in nifi's branch refs/heads/main from Timea Barna [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7c33516aee ] NIFI-9437 Flowfile Expiration cannot exceed 24 days NIFI-9437 Addressing review comments, correcting setter in StatelessFlowQueue class Signed-off-by: Nathan Gough This closes #6558. > Flowfile Expiration does not exceed 24 days > --- > > Key: NIFI-9437 > URL: https://issues.apache.org/jira/browse/NIFI-9437 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.14.0, 1.15.0 >Reporter: Tim Chermak >Assignee: Timea Barna >Priority: Minor > Time Spent: 1h 10m > Remaining Estimate: 0h > > We discovered setting a FlowFile Expiration on a queue for anything over 24 > days is ignored and not 'aged out of the flow". So, if FlowFile Expiration is > set to anything beyond 24 days, the file does not automatically expire and > remains in the queue. > > Can this be fixed so that there is no limit? Also, perhaps the Expiration > setting can have other criteria/strategy for expiring, as indicated in an old > ticket NiFi-372 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog closed pull request #6558: NIFI-9437 Flowfile Expiration cannot exceed 24 days
thenatog closed pull request #6558: NIFI-9437 Flowfile Expiration cannot exceed 24 days URL: https://github.com/apache/nifi/pull/6558 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #6558: NIFI-9437 Flowfile Expiration cannot exceed 24 days
thenatog commented on PR #6558: URL: https://github.com/apache/nifi/pull/6558#issuecomment-1289447736 +1 looks good, will merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10680) Dependency enforcer fails after banned bouncycastle version change
[
https://issues.apache.org/jira/browse/NIFI-10680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10680:
Fix Version/s: 1.19.0
> Dependency enforcer fails after banned bouncycastle version change
> --
>
> Key: NIFI-10680
> URL: https://issues.apache.org/jira/browse/NIFI-10680
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.19.0
>Reporter: Kalmár Róbert
>Assignee: Joe Witt
>Priority: Major
> Fix For: 1.19.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The change in banned dependencies introduced with NIFI-10614 causes
> Nifi-Registry build to fail with some profiles.
> {code:java}
> [INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven-version) @
> nifi-registry-ranger-plugin ---
> [ERROR] Rule 3: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> Found Banned Dependency: org.bouncycastle:bcprov-jdk15on:jar:1.67
> Found Banned Dependency: org.bouncycastle:bcpkix-jdk15on:jar:1.67 {code}
> ozone-client seems to include them transitively:
> {code:java}
> [INFO] +- org.apache.ozone:ozone-client:jar:1.2.1:compile
> [INFO] | \- org.apache.ozone:ozone-common:jar:1.2.1:compile
> [INFO] | +- org.apache.ozone:hdds-common:jar:1.2.1:compile
> [INFO] | | +- org.bouncycastle:bcpkix-jdk15on:jar:1.67:compile {code}
> Profiles used in build:
> {code:java}
> -Pinclude-ranger,include-atlas,include-hive3,include-rules,include-sql-reporting,include-hadoop-aws,include-hadoop-azure,include-hadoop-cloud-storage,include-hadoop-gcp,include-graph,include-grpc,include-accumulo,include-hadoop-ozone,include-asn1,include-aws,include-snowflake,include-media
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] joewitt commented on pull request #6571: NIFI-10680 cleanup ozone-client bouncycastle references now that jdk1…
joewitt commented on PR #6571: URL: https://github.com/apache/nifi/pull/6571#issuecomment-1289426451 Built with mvn clean install -T 1C -Pinclude-grpc,contrib-check,include-ranger,include-atlas,include-hive3,include-rules,include-sql-reporting,include-hadoop-aws,include-hadoop-azure,include-hadoop-cloud-storage,include-hadoop-gcp,include-graph,include-accumulo,include-hadoop-ozone,include-asn1,include-aws,include-snowflake,include-media -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt opened a new pull request, #6571: NIFI-10680 cleanup ozone-client bouncycastle references now that jdk1…
joewitt opened a new pull request, #6571: URL: https://github.com/apache/nifi/pull/6571 …5on artifacts are blocked # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] chihhanyu opened a new pull request, #5135: NIFI-8611: GCP BigQuery processors support using designate project resource for ingestion
chihhanyu opened a new pull request, #5135:
URL: https://github.com/apache/nifi/pull/5135
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Please refer to JIRA ticket for details:
https://issues.apache.org/jira/browse/NIFI-8611
This change is already used in our production environment which can properly
support using different source project and ingestion project in the property of
GCP processors.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [x] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #5135: NIFI-8611: GCP BigQuery processors support using designate project resource for ingestion
joewitt commented on PR #5135: URL: https://github.com/apache/nifi/pull/5135#issuecomment-1289396457 @chihhanyu I apologize. I completely dropped this ball and didn't see your notes. You did the right things. If you are still interested in this please re-submit the PR. Actually I'll re-open this one but i'm guessing rebasing is needed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-8859) Improve Github Actions Integration and remove Travis
[ https://issues.apache.org/jira/browse/NIFI-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-8859. Resolution: Fixed > Improve Github Actions Integration and remove Travis > > > Key: NIFI-8859 > URL: https://issues.apache.org/jira/browse/NIFI-8859 > Project: Apache NiFi > Issue Type: Task > Components: NiFi Registry >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-8338) conduct 1.13.2 release
[ https://issues.apache.org/jira/browse/NIFI-8338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-8338. Resolution: Fixed > conduct 1.13.2 release > -- > > Key: NIFI-8338 > URL: https://issues.apache.org/jira/browse/NIFI-8338 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.13.2 >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-7787) Release Apache NiFi 1.12.1
[ https://issues.apache.org/jira/browse/NIFI-7787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-7787. Resolution: Fixed > Release Apache NiFi 1.12.1 > -- > > Key: NIFI-7787 > URL: https://issues.apache.org/jira/browse/NIFI-7787 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Trivial > Fix For: 1.12.1 > > > Items in release: > NIFI-7167/mimetype: git cherry-pick 9975df8962b1f5eac7f3e1df1b69d2c64b06a40b > NIFI-7730: git cherry-pick c3cab48325a8c013f267c9ccc3a375ceb0e3c5e9 > NIFI-7767: git cherry-pick 1e6619b91f2d9a7af2d7195464d4a0f5b595b93d > NIFI-7778: git cherry-pick e884b3cdb2bf2df7c00c914edf520394081f3628 > NIFI-7740: git cherry-pick 45470b0984ab83750155e9c7a540c79bfe862817 > NIFI-7779: git cherry-pick 943904c12c56a1f658ae7144e7c2bbb29d746659 > NIFI-7762: git cherry-pick bdead4d040dab7c7b23328dcd3208627183b3506 > NIFI-7661: git cherry-pick 87685dd4e16f0ab6edda571a7bde819b14117dc7 > NIFI-6767: git cherry-pick 483af3a46023a8c7d44af634b70372c8bb75d0e3 > NIFI-7309: git cherry-pick 861b83f31429645abf84ba25ebe3f4af1908ed7d > NIFI-7714: git cherry-pick ca43615702eabddc6f7bbf280f0d181d7aa0b0d4 > NIFI-7766: git cherry-pick bdfe1f23705a3df86eafed12ac8839ee76662730 > NIFI-7760: git cherry-pick aaab9ee0cfbd52cba2fd7ecdcada39c96b1169cc > NIFI-7751: git cherry-pick d0c0f9704c7b9261b84f662373dc289f81d55bf1 > NIFI-7758: git cherry-pick 11a4127a9f89474a42f182c0b83e365a6b468e71 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (NIFI-9622) There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinat
[ https://issues.apache.org/jira/browse/NIFI-9622 ] Yuriy Malygin deleted comment on NIFI-9622: - was (Author: _ph): Same problem when trying to start cluster (three nodes, version 1.18.0) with SASL/SSL security features in external (not embedded) Zookeeper cluster and NiFi. > There is currently no Cluster Coordinator. This often happens upon restart of > NiFi when running an embedded ZooKeeper. Will register this node to become > the active Cluster Coordinator and will attempt to connect to cluster again. > -- > > Key: NIFI-9622 > URL: https://issues.apache.org/jira/browse/NIFI-9622 > Project: Apache NiFi > Issue Type: Bug > Components: Docker >Reporter: Tarun Bali >Priority: Blocker > > Hi Community, > I am trying to run nifi through docker compose but getting below WARN: > There is currently no Cluster Coordinator. This often happens upon restart of > NiFi when running an embedded ZooKeeper. Will register this node to become > the active Cluster Coordinator and will attempt to connect to cluster again > Attempted to register Leader Election for role 'Cluster Coordinator' but this > role is already registered. > > The NiFI UI is in invalid state and saying that The Flow Controller is > initializing the Data Flow. > Below are my docker-compose config: > > zookeeper: > hostname: zookeeper > image: zookeeper:latest > container_name: zookeeper > ports: > - "2181:2181" > networks: > net_pet: > ipv4_address: 172.27.1.15 > environment: > - ZOOKEEPER_HOST_NAME=zookeeper > - ZOOKEEPER_CONNECTION_TIMEOUT_MS=6000 > - STREAMS_ZK_SESSION_TIMEOUT_SEC=60 > - ALLOW_ANONYMOUS_LOGIN=yes > nifi: > image: apache/nifi:latest > ports: > - 8080 > environment: > - NIFI_WEB_HTTP_PORT=8080 > - NIFI_CLUSTER_IS_NODE=true > - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082 > - NIFI_ZK_CONNECT_STRING=172.27.1.15:2181 > - NIFI_ELECTION_MAX_WAIT=1 min > - NIFI_SENSITIVE_PROPS_KEY=tarunbali > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-8497) Add a SlackRecordSink controller service
[ https://issues.apache.org/jira/browse/NIFI-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emilio Setiadarma reassigned NIFI-8497: --- Assignee: Emilio Setiadarma > Add a SlackRecordSink controller service > > > Key: NIFI-8497 > URL: https://issues.apache.org/jira/browse/NIFI-8497 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Assignee: Emilio Setiadarma >Priority: Major > > For components that use a RecordSinkService controller service > (QueryNiFiReportingTask, PutRecord, e.g.), it would be nice to have a > RecordSinkService that sends to Slack like the PutSlack processor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10686) Update questdb-core to version 6.5.3-jdk8
[ https://issues.apache.org/jira/browse/NIFI-10686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-10686: - Assignee: Mike R > Update questdb-core to version 6.5.3-jdk8 > - > > Key: NIFI-10686 > URL: https://issues.apache.org/jira/browse/NIFI-10686 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update questdb-core to version 6.5.3-jdk8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10680) Dependency enforcer fails after banned bouncycastle version change
[
https://issues.apache.org/jira/browse/NIFI-10680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623289#comment-17623289
]
Joe Witt commented on NIFI-10680:
-
Working on this now
> Dependency enforcer fails after banned bouncycastle version change
> --
>
> Key: NIFI-10680
> URL: https://issues.apache.org/jira/browse/NIFI-10680
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.19.0
>Reporter: Kalmár Róbert
>Assignee: Joe Witt
>Priority: Major
>
> The change in banned dependencies introduced with NIFI-10614 causes
> Nifi-Registry build to fail with some profiles.
> {code:java}
> [INFO] --- maven-enforcer-plugin:3.1.0:enforce (enforce-maven-version) @
> nifi-registry-ranger-plugin ---
> [ERROR] Rule 3: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> Found Banned Dependency: org.bouncycastle:bcprov-jdk15on:jar:1.67
> Found Banned Dependency: org.bouncycastle:bcpkix-jdk15on:jar:1.67 {code}
> ozone-client seems to include them transitively:
> {code:java}
> [INFO] +- org.apache.ozone:ozone-client:jar:1.2.1:compile
> [INFO] | \- org.apache.ozone:ozone-common:jar:1.2.1:compile
> [INFO] | +- org.apache.ozone:hdds-common:jar:1.2.1:compile
> [INFO] | | +- org.bouncycastle:bcpkix-jdk15on:jar:1.67:compile {code}
> Profiles used in build:
> {code:java}
> -Pinclude-ranger,include-atlas,include-hive3,include-rules,include-sql-reporting,include-hadoop-aws,include-hadoop-azure,include-hadoop-cloud-storage,include-hadoop-gcp,include-graph,include-grpc,include-accumulo,include-hadoop-ozone,include-asn1,include-aws,include-snowflake,include-media
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-9622) There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Clust
[ https://issues.apache.org/jira/browse/NIFI-9622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623287#comment-17623287 ] Yuriy Malygin commented on NIFI-9622: - Same problem when trying to start cluster (three nodes, version 1.18.0) with SASL/SSL security features in external (not embedded) Zookeeper cluster and NiFi. > There is currently no Cluster Coordinator. This often happens upon restart of > NiFi when running an embedded ZooKeeper. Will register this node to become > the active Cluster Coordinator and will attempt to connect to cluster again. > -- > > Key: NIFI-9622 > URL: https://issues.apache.org/jira/browse/NIFI-9622 > Project: Apache NiFi > Issue Type: Bug > Components: Docker >Reporter: Tarun Bali >Priority: Blocker > > Hi Community, > I am trying to run nifi through docker compose but getting below WARN: > There is currently no Cluster Coordinator. This often happens upon restart of > NiFi when running an embedded ZooKeeper. Will register this node to become > the active Cluster Coordinator and will attempt to connect to cluster again > Attempted to register Leader Election for role 'Cluster Coordinator' but this > role is already registered. > > The NiFI UI is in invalid state and saying that The Flow Controller is > initializing the Data Flow. > Below are my docker-compose config: > > zookeeper: > hostname: zookeeper > image: zookeeper:latest > container_name: zookeeper > ports: > - "2181:2181" > networks: > net_pet: > ipv4_address: 172.27.1.15 > environment: > - ZOOKEEPER_HOST_NAME=zookeeper > - ZOOKEEPER_CONNECTION_TIMEOUT_MS=6000 > - STREAMS_ZK_SESSION_TIMEOUT_SEC=60 > - ALLOW_ANONYMOUS_LOGIN=yes > nifi: > image: apache/nifi:latest > ports: > - 8080 > environment: > - NIFI_WEB_HTTP_PORT=8080 > - NIFI_CLUSTER_IS_NODE=true > - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082 > - NIFI_ZK_CONNECT_STRING=172.27.1.15:2181 > - NIFI_ELECTION_MAX_WAIT=1 min > - NIFI_SENSITIVE_PROPS_KEY=tarunbali > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] UcanInfosec opened a new pull request, #6570: NIFI-10686 Update questdb-core to 6.5.3-jdk8
UcanInfosec opened a new pull request, #6570: URL: https://github.com/apache/nifi/pull/6570 # Summary [NIFI-10686](https://issues.apache.org/jira/browse/NIFI-10686) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-10686) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X]Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ]JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1414: MINIFICPP-1927 Fix ExecuteProcess command argument issue and refactor
fgerlits commented on code in PR #1414:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1414#discussion_r1003070951
##
extensions/standard-processors/processors/ExecuteProcess.cpp:
##
@@ -30,42 +30,36 @@
#include "core/TypedValues.h"
#include "utils/gsl.h"
-#if defined(__clang__)
-#pragma clang diagnostic push
-#pragma clang diagnostic ignored "-Wsign-compare"
-#pragma clang diagnostic ignored "-Wunused-result"
-#elif defined(__GNUC__) || defined(__GNUG__)
-#pragma GCC diagnostic push
-#pragma GCC diagnostic ignored "-Wsign-compare"
-#pragma GCC diagnostic ignored "-Wunused-result"
-#endif
-
using namespace std::literals::chrono_literals;
-namespace org {
-namespace apache {
-namespace nifi {
-namespace minifi {
-namespace processors {
+namespace org::apache::nifi::minifi::processors {
#ifndef WIN32
Review Comment:
this `ifndef` could be moved to the top, to avoid compiling the included
headers
##
extensions/standard-processors/processors/ExecuteProcess.h:
##
@@ -30,6 +29,7 @@
#include
#include
#include
+#include
#ifndef WIN32
Review Comment:
since the whole thing is not compiled for Windows, this `#ifndef` is
unnecessary
##
extensions/standard-processors/processors/ExecuteProcess.cpp:
##
@@ -74,172 +68,226 @@ void ExecuteProcess::initialize() {
setSupportedRelationships(relationships());
}
-void ExecuteProcess::onTrigger(core::ProcessContext *context,
core::ProcessSession *session) {
+void ExecuteProcess::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory* /*session_factory*/) {
+ gsl_Expects(context);
std::string value;
- std::shared_ptr flow_file;
- if (context->getProperty(Command, value, flow_file)) {
-this->_command = value;
+ if (context->getProperty(Command.getName(), value)) {
+command_ = value;
}
- if (context->getProperty(CommandArguments, value, flow_file)) {
-this->_commandArgument = value;
+ if (context->getProperty(CommandArguments.getName(), value)) {
+command_argument_ = value;
}
- if (context->getProperty(WorkingDir, value, flow_file)) {
-this->_workingDir = value;
+ if (context->getProperty(WorkingDir.getName(), value)) {
+working_dir_ = value;
}
if (auto batch_duration =
context->getProperty(BatchDuration)) {
-_batchDuration = batch_duration->getMilliseconds();
-logger_->log_debug("Setting _batchDuration");
+batch_duration_ = batch_duration->getMilliseconds();
+logger_->log_debug("Setting batch duration to %d milliseconds",
batch_duration_.count());
}
if (context->getProperty(RedirectErrorStream.getName(), value)) {
-_redirectErrorStream =
org::apache::nifi::minifi::utils::StringUtils::toBool(value).value_or(false);
+redirect_error_stream_ =
org::apache::nifi::minifi::utils::StringUtils::toBool(value).value_or(false);
}
- this->_fullCommand = _command + " " + _commandArgument;
- if (_fullCommand.length() == 0) {
-yield();
+ full_command_ = command_ + " " + command_argument_;
+}
+
+bool ExecuteProcess::changeWorkdir() const {
+ if (working_dir_.length() > 0 && working_dir_ != ".") {
+if (chdir(working_dir_.c_str()) != 0) {
+ logger_->log_error("Execute Command can not chdir %s", working_dir_);
+ return false;
+}
+ }
+ return true;
+}
+
+std::vector ExecuteProcess::readArgs() const {
+ std::vector args;
+ std::string current_param;
+ bool in_escaped = false;
+ auto currentParamShouldBeAppended = [&](std::size_t i) {
+bool current_char_is_escaped_apostrophe = full_command_[i] == '\"' &&
in_escaped && i > 0 && full_command_[i - 1] == '\\';
+bool whitespace_in_escaped_block = full_command_[i] == ' ' && in_escaped;
+bool non_special_character = full_command_[i] != '\\' && full_command_[i]
!= '\"' && full_command_[i] != ' ';
+return current_char_is_escaped_apostrophe || whitespace_in_escaped_block
|| non_special_character;
+ };
+
+ for (std::size_t i = 0; i < full_command_.size(); ++i) {
+if (currentParamShouldBeAppended(i)) {
+ current_param += full_command_[i];
+} else if (full_command_[i] == '\"' && (!in_escaped || i == 0 ||
full_command_[i - 1] != '\\')) {
+ in_escaped = !in_escaped;
+} else if (full_command_[i] == ' ' && !in_escaped) {
+ if (!current_param.empty()) {
+args.push_back(current_param);
+ }
+ current_param.clear();
+}
+ }
Review Comment:
This logic is very complex, and it does not treat backslashes as I would
expect. For example, on an input of `a \" b \" c \\" d \\\" e " f \ \\ \g`,
the output is 4 arguments, `a`, ` b " c " d " e `, `f` and `g`.
Why don't we use `std::quoted` instead? That seems to do what we need:
https://godbolt.org/z/PzYPE71Gx
##
extensions/standard-processors/processors/ExecuteProcess.cpp:
##
@@ -74,172 +68,226 @@ void ExecuteProcess::initialize() {
setSupportedRelationships(relationships());
}
-void ExecuteProcess
[GitHub] [nifi] joewitt commented on pull request #6531: NIFI-10648 Upgrade Apache Commons Text to 1.10.0
joewitt commented on PR #6531: URL: https://github.com/apache/nifi/pull/6531#issuecomment-1289185595 No. It will be in a 1.18.1 or 1.19.0 release. No firm plan for either at this moment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6460: NIFI-10562 Added MongoDB testcontainers to support integration testing.
exceptionfactory commented on code in PR #6460:
URL: https://github.com/apache/nifi/pull/6460#discussion_r1003429425
##
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/gridfs/PutGridFS.java:
##
@@ -212,9 +267,10 @@ private boolean canUploadFile(ProcessContext context,
FlowFile input, String buc
Document query;
if (uniqueness.equals(UNIQUE_BOTH.getValue())) {
-query = new Document().append("filename",
fileName).append("md5", hash);
+query = new Document().append("filename", fileName)
+.append("metadata", new Document(hashMetadataName,
hash));
} else if (uniqueness.equals(UNIQUE_HASH.getValue())) {
-query = new Document().append("md5", hash);
+query = new Document().append("metadata", new
Document(hashMetadataName, hash));
Review Comment:
Given the potential complication, it seems better to focus this PR on the
integration testing against current MongoDB versions, and then address the
hashing change separately, which could include adjusting the integration tests
to cover both scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6460: NIFI-10562 Added MongoDB testcontainers to support integration testing.
exceptionfactory commented on code in PR #6460:
URL: https://github.com/apache/nifi/pull/6460#discussion_r1003428262
##
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/gridfs/PutGridFS.java:
##
@@ -212,9 +267,10 @@ private boolean canUploadFile(ProcessContext context,
FlowFile input, String buc
Document query;
if (uniqueness.equals(UNIQUE_BOTH.getValue())) {
-query = new Document().append("filename",
fileName).append("md5", hash);
+query = new Document().append("filename", fileName)
+.append("metadata", new Document(hashMetadataName,
hash));
} else if (uniqueness.equals(UNIQUE_HASH.getValue())) {
-query = new Document().append("md5", hash);
+query = new Document().append("metadata", new
Document(hashMetadataName, hash));
Review Comment:
Thanks for the reply @MikeThomsen. In order to avoid breaking existing users
of PutGridFS, it seems like we either need to add server version checking, or
perhaps a new configurable strategy property along the lines that you
mentioned. Either way, we should have a path forward that maintains the current
functionality, but also provides a new option that moves away from the
deprecated `md5` approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #6400: NIFI-10481 Add NiFi configuration to disable zk client ensembleTracker
thenatog commented on PR #6400: URL: https://github.com/apache/nifi/pull/6400#issuecomment-1289174562 Will review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10683) Update JNA to 5.12.1
[ https://issues.apache.org/jira/browse/NIFI-10683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10683. - Fix Version/s: 1.19.0 Resolution: Fixed > Update JNA to 5.12.1 > > > Key: NIFI-10683 > URL: https://issues.apache.org/jira/browse/NIFI-10683 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 1.19.0 > > Time Spent: 20m > Remaining Estimate: 0h > > h3. Update JNA to 5.12.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10683) Update JNA to 5.12.1
[ https://issues.apache.org/jira/browse/NIFI-10683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623230#comment-17623230 ] ASF subversion and git services commented on NIFI-10683: Commit 753cb1b9df781232d953c6602200aca332f4d019 in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=753cb1b9df ] NIFI-10683 Upgraded JNA to 5.12.1 for Windows Event Log This closes #6568 Signed-off-by: David Handermann > Update JNA to 5.12.1 > > > Key: NIFI-10683 > URL: https://issues.apache.org/jira/browse/NIFI-10683 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > h3. Update JNA to 5.12.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10683) Update JNA to 5.12.1
[ https://issues.apache.org/jira/browse/NIFI-10683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10683: Priority: Minor (was: Major) > Update JNA to 5.12.1 > > > Key: NIFI-10683 > URL: https://issues.apache.org/jira/browse/NIFI-10683 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > h3. Update JNA to 5.12.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6568: NIFI-10683 Update JNA to 5.12.1
exceptionfactory closed pull request #6568: NIFI-10683 Update JNA to 5.12.1 URL: https://github.com/apache/nifi/pull/6568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10686) Update questdb-core to version 6.5.3-jdk8
Mike R created NIFI-10686: - Summary: Update questdb-core to version 6.5.3-jdk8 Key: NIFI-10686 URL: https://issues.apache.org/jira/browse/NIFI-10686 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.18.0 Reporter: Mike R Update questdb-core to version 6.5.3-jdk8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6293: [NIFI-10349] Add maximum object age property to list s3
turcsanyip commented on code in PR #6293:
URL: https://github.com/apache/nifi/pull/6293#discussion_r1003404736
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/ListS3.java:
##
@@ -217,6 +218,15 @@ public class ListS3 extends AbstractS3Processor implements
VerifiableProcessor {
.defaultValue("0 sec")
.build();
+public static final PropertyDescriptor MAX_AGE = new Builder()
+.name("max-age")
+.displayName("Maximum Object Age")
+.description("The maximum age that an S3 object must be in order
to be considered; any object older than this amount of time (according to last
modification date) will be ignored")
Review Comment:
@marcocharlie I had a look at the code and I think there is a typo here:
```suggestion
.description("The maximum age that an S3 object can be in order
to be considered; any object older than this amount of time (according to last
modification date) will be ignored")
```
or `may be in order to be considered`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10235:
Status: In Progress (was: Patch Available)
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Assignee: David Handermann
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10685) PutDatabaseRecord Upsert does not quote columns in conflict clause (postgres)
[
https://issues.apache.org/jira/browse/NIFI-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mgerbig updated NIFI-10685:
---
Description:
Nifi's PutDatabaseRecord does not quote columns in the conflict clause when
using PostgreSQLDatabaseAdapter. This prevents the usage of upsert (and
possibly insert ignore) statements as soon as at least one column of the
primary key contains at least one uppercase character.
h2. Root Cause
Postgres lowercases all unquoted keys, as described in the manual, which causes
this bug:
{quote}Quoting an identifier also makes it case-sensitive, whereas unquoted
names are always folded to lower case. For example, the identifiers
{{{}FOO{}}}, {{{}foo{}}}, and {{"foo"}} are considered the same by PostgreSQL,
but {{"Foo"}} and {{"FOO"}} are different from these three and each other. (The
folding of unquoted names to lower case in PostgreSQL is incompatible with the
SQL standard, which says that unquoted names should be folded to upper case.
Thus, {{foo}} should be equivalent to {{"FOO"}} not {{"foo"}} according to the
standard. If you want to write portable applications you are advised to always
quote a particular name or never quote it.)
{quote}
see
[https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS]
h2. Reproduce behaviour
Create a table in your *PostgreSQL* database
{code:java}
CREATE TABLE test_table
(
"TIME_PERIOD" date NOT NULL,
"R0102" numeric,
"R0203" numeric,
"R0304" numeric,
"R0405" numeric,
"R0506" numeric,
"R0607" numeric,
"R0708" numeric,
"R0809" numeric,
"R0910" numeric,
modified_at_utc date DEFAULT timezone('UTC'::text, CURRENT_TIMESTAMP(0)),
CONSTRAINT test_table PRIMARY KEY ("TIME_PERIOD")
) {code}
Use attached avro to write to the table using a PutDatabaseRecord Processor
configured with UPSERT.
Nifi will throw following Exception in the Bulletin Board
{code:java}
PutDatabaseRecord[id=...] Failed to put Records to database for
FlowFile[filename=redacted.avro]. Routing to failure.:
java.sql.BatchUpdateException: Batch entry 0 INSERT INTO
"public"."test_table"("TIME_PERIOD", "R0102", "R0203", "R0304", "R0405",
"R0506", "R0607", "R0708", "R0809", "R0910") VALUES ('2022-10-21 +00',
'2.97'::numeric, '3.2'::numeric, '3.19'::numeric, '3.19'::numeric,
'3.22'::numeric, '3.33'::numeric, '3.33'::numeric, '3.4'::numeric,
'3.38'::numeric) ON CONFLICT (TIME_PERIOD) DO UPDATE SET ("TIME_PERIOD",
"R0102", "R0203", "R0304", "R0405", "R0506", "R0607", "R0708", "R0809",
"R0910") = (EXCLUDED."TIME_PERIOD", EXCLUDED."R0102", EXCLUDED."R0203",
EXCLUDED."R0304", EXCLUDED."R0405", EXCLUDED."R0506", EXCLUDED."R0607",
EXCLUDED."R0708", EXCLUDED."R0809", EXCLUDED."R0910") was aborted: ERROR:
column "time_period" does not exist
Position: 347 Call getNextException to see other errors in the batch.
- Caused by: org.postgresql.util.PSQLException: ERROR: column "time_period"
does not exist
Position: 347 {code}
h3. Generated SQL Statement
Nifi generates and issues following SQL Statement to the database:
{code:java}
INSERT INTO "public"."test_table"("TIME_PERIOD", "R0102", "R0203", "R0304",
"R0405", "R0506", "R0607", "R0708", "R0809", "R0910") VALUES (?, ?, ?, ?, ?, ?,
?, ?, ?, ?) ON CONFLICT (TIME_PERIOD) DO UPDATE SET ("TIME_PERIOD", "R0102",
"R0203", "R0304", "R0405", "R0506", "R0607", "R0708", "R0809", "R0910") =
(EXCLUDED."TIME_PERIOD", EXCLUDED."R0102", EXCLUDED."R0203", EXCLUDED."R0304",
EXCLUDED."R0405", EXCLUDED."R0506", EXCLUDED."R0607", EXCLUDED."R0708",
EXCLUDED."R0809", EXCLUDED."R0910") {code}
This statement only works with quoted _TIME_PERIOD_ in the _ON CONFLICT_ clause
due to the behaviour of postgres described in Section _Root Cause_
was:
Nifi's PutDatabaseRecord does not quote columns in the conflict clause when
using PostgreSQLDatabaseAdapter. This prevents the usage of upsert (and
possibly insert ignore) statements as soon as at least one column of the
primary key contains at least one uppercase character.{*}{*}
h2. Root Cause
Postgres lowercases all unquoted keys, as described in the manual, which causes
this bug:
{quote}Quoting an identifier also makes it case-sensitive, whereas unquoted
names are always folded to lower case. For example, the identifiers
{{{}FOO{}}}, {{{}foo{}}}, and {{"foo"}} are considered the same by PostgreSQL,
but {{"Foo"}} and {{"FOO"}} are different from these three and each other. (The
folding of unquoted names to lower case in PostgreSQL is incompatible with the
SQL standard, which says that unquoted names should be folded to upper case.
Thus, {{foo}} should be equivalent to {{"FOO"}} not {{"foo"}} according to the
standard. If you want to write portable applications you are advised to always
quote a particular name or never quote it.)
{quote}
see
[https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS]
h2. R
[jira] [Resolved] (NIFI-10554) NiFi Registry SSL Protocol Error
[ https://issues.apache.org/jira/browse/NIFI-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10554. - Resolution: Cannot Reproduce Hi [~irudayambics] closing this for now, please feel free to reopen this issue if you are able to provide additional error and environment details. > NiFi Registry SSL Protocol Error > > > Key: NIFI-10554 > URL: https://issues.apache.org/jira/browse/NIFI-10554 > Project: Apache NiFi > Issue Type: Bug > Components: NiFi Registry >Affects Versions: 1.17.0 > Environment: Ubuntu 20.04 > Java 11 > nifi-registry-1.17.0 >Reporter: Irudya Raj >Priority: Major > > I have setup nifi registry 1.17.0 with truststore and keystore and can see > the log file that it has started on SSL using port 18433 but when I accessed > the application from browser, it reported SSL protocol error. > I looked at the log file and found SSL factory was created with http.1 > protocol. Is this causing issue? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10332) Add ID_TOKEN_LOGOUT support for general OpenID connect server, e.g. Keycloak
[
https://issues.apache.org/jira/browse/NIFI-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough reassigned NIFI-10332:
---
Assignee: Nathan Gough
> Add ID_TOKEN_LOGOUT support for general OpenID connect server, e.g. Keycloak
>
>
> Key: NIFI-10332
> URL: https://issues.apache.org/jira/browse/NIFI-10332
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core UI
>Affects Versions: 1.17.0
> Environment: NiFi 1.17.0, Keycloak 18.0.1
>Reporter: macdoor615
>Assignee: Nathan Gough
>Priority: Major
> Attachments: image-2022-08-09-16-56-25-791.png
>
>
> I deploy a NiFi 1.170 and authenticate with OpenID connect. Authentication
> server is Keycloak 18.0.1.
> I can log in and I can use UI properly.
> But when I logout. I get an error, can not redirect to NiFi UI or keycloak
> login UI
> !image-2022-08-09-16-56-25-791.png|width=782,height=347!
> [https://36.133.55.100:8943/realms/zznode/protocol/openid-connect/logout?post_logout_redirect_uri=https%3A%2F%2F36.138.166.203%3A18089%2Fhb3-dmz-repos-000-nifi%2Fnifi-api%2F..%2Fnifi%2Flogout-complete]
> I made some investigation into source code. I found NiFi only support
> ID_TOKEN_LOGOUT for okta service. Keycloak and other Authentication server
> can not be supported.
> Keycloak say it is compliance OpenID connect spec.
> I modified a few lines of source code. Let it support ID_TOKEN_LOGOUT for
> keycloak. Now I can log out NiFi and redirect to keycloak login UI, and than
> login NiFi again.
> I suggest making nifi to support ID_TOKEN_LOGOUT in later version for general
> OpenID connect server.
> I modified the file,
> [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/OIDCAccessResource.java]
> start from line 403
> {code:java}
> private String determineLogoutMethod(String oidcDiscoveryUrl) {
> Matcher accessTokenMatcher =
> REVOKE_ACCESS_TOKEN_LOGOUT_FORMAT.matcher(oidcDiscoveryUrl);
> Matcher idTokenMatcher =
> ID_TOKEN_LOGOUT_FORMAT.matcher(oidcDiscoveryUrl);
>
> if (accessTokenMatcher.find()) {
> return REVOKE_ACCESS_TOKEN_LOGOUT;
> } else {
> return ID_TOKEN_LOGOUT;
> }
> }
>
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-10606) Connection to Nifi API only succeeds sometimes(invalid token)
[
https://issues.apache.org/jira/browse/NIFI-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-10606.
-
Resolution: Information Provided
Closing based on current answer provided, feel free to reopen if the answer
does not resolve the problem.
> Connection to Nifi API only succeeds sometimes(invalid token)
> -
>
> Key: NIFI-10606
> URL: https://issues.apache.org/jira/browse/NIFI-10606
> Project: Apache NiFi
> Issue Type: Bug
> Components: Security
>Affects Versions: 1.16.3
>Reporter: Andreas Adamides
>Assignee: David Handermann
>Priority: Major
>
> Sometimes the connection to Nifi API succeeds, sometimes it doesn't.
>
> When it doesn't, I get this:
>
> {code:java}
> nipyapi.nifi.rest.ApiException: (401)61Reason: Unauthorized62HTTP response
> headers: HTTPHeaderDict({'Date': 'Thu, 06 Oct 2022 13:06:14 GMT',
> 'Content-Length': '0', 'Connection': 'keep-alive', 'Set-Cookie':
> 'AWSALB=jc7aAv/roBICNC7Bpwr3Ks/bZBi2sCnRsIxUxipL1ytibmVcosZZYqP0bw5226jWB9apcQ5iEpGLbwMc0JjKKqykbR69hmT9OJA8EwtvVIoLFBRABlYwzvq7yXFK;
> Expires=Thu, 13 Oct 2022 13:06:14 GMT; Path=/,
> AWSALBCORS=jc7aAv/roBICNC7Bpwr3Ks/bZBi2sCnRsIxUxipL1ytibmVcosZZYqP0bw5226jWB9apcQ5iEpGLbwMc0JjKKqykbR69hmT9OJA8EwtvVIoLFBRABlYwzvq7yXFK;
> Expires=Thu, 13 Oct 2022 13:06:14 GMT; Path=/; SameSite=None; Secure,
> __Secure-Request-Token=0d8151fd-782a-480d-9b51-506bf8dedde4; Path=/; Secure',
> 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors
> 'self'", 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options':
> 'nosniff', 'Strict-Transport-Security': 'max-age=3154', 'Expires': 'Thu,
> 01 Jan 1970 00:00:00 GMT', 'WWW-Authenticate': 'Bearer error="invalid_token",
> error_description="An error occurred while attempting to decode the Jwt:
> Signed JWT rejected: Another algorithm expected, or no matching key(s)
> found", error_uri="https://tools.ietf.org/html/rfc6750#section-3.1";',
> 'Server': 'Jetty(9.4.46.v20220331)'}) {code}
> And the Nifi CLuster node user log entry:
>
> {code:java}
> 2022-10-05 15:38:48,529 ERROR [NiFi Web Server-22]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> org.springframework.security.oauth2.server.resource.InvalidBearerTokenException:
> An error occurred while attempting to decode the Jwt: Signed JWT rejected:
> Another algorithm expected, or no matching key(s) found. Returning Internal
> Server Error response.
> org.springframework.security.oauth2.server.resource.InvalidBearerTokenException:
> An error occurred while attempting to decode the Jwt: Signed JWT rejected:
> Another algorithm expected, or no matching key(s) found
> at
> org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.getJwt(JwtAuthenticationProvider.java:101)
> at
> org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.authenticate(JwtAuthenticationProvider.java:88)
> at
> org.apache.nifi.web.api.AccessResource.getAccessStatus(AccessResource.java:252)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
> at
> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
> at
> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
>
[GitHub] [nifi] abarrak commented on pull request #6531: NIFI-10648 Upgrade Apache Commons Text to 1.10.0
abarrak commented on PR #6531: URL: https://github.com/apache/nifi/pull/6531#issuecomment-1289127366 Is this released yet @thenatog ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10681) Update Mockito to 3.12.4
[ https://issues.apache.org/jira/browse/NIFI-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10681: Fix Version/s: 1.19.0 Affects Version/s: (was: 1.18.0) Labels: dependency-upgrade (was: ) Status: Patch Available (was: Open) > Update Mockito to 3.12.4 > > > Key: NIFI-10681 > URL: https://issues.apache.org/jira/browse/NIFI-10681 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.19.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > h3. Update Mockito to 3.12.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10681) Update Mockito to 3.12.4
[ https://issues.apache.org/jira/browse/NIFI-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10681: Priority: Minor (was: Major) > Update Mockito to 3.12.4 > > > Key: NIFI-10681 > URL: https://issues.apache.org/jira/browse/NIFI-10681 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > h3. Update Mockito to 3.12.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10681) Update Mockito to 3.12.4
[ https://issues.apache.org/jira/browse/NIFI-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623210#comment-17623210 ] ASF subversion and git services commented on NIFI-10681: Commit b88b1d0d7ee6bcaf90469716ebd5ff83a3090070 in nifi's branch refs/heads/main from UcanInfosec [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b88b1d0d7e ] NIFI-10681 Upgraded Mockito from 3.11.2 to 3.12.4 This closes #6566 Signed-off-by: David Handermann > Update Mockito to 3.12.4 > > > Key: NIFI-10681 > URL: https://issues.apache.org/jira/browse/NIFI-10681 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > h3. Update Mockito to 3.12.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6566: NIFI-10681 Update Mockito to 3.12.4
exceptionfactory closed pull request #6566: NIFI-10681 Update Mockito to 3.12.4 URL: https://github.com/apache/nifi/pull/6566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7823) Retrieve groups from OIDC token using KeyClock
[ https://issues.apache.org/jira/browse/NIFI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-7823: --- Fix Version/s: 1.19.0 Status: Patch Available (was: Open) > Retrieve groups from OIDC token using KeyClock > -- > > Key: NIFI-7823 > URL: https://issues.apache.org/jira/browse/NIFI-7823 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Reporter: Raz Dobkies >Assignee: Malthe Borch >Priority: Major > Fix For: 1.19.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Nifi will can retrueve groups from a token if we will develop > KeyCloakUserGroupProvider. > Keycloak does implement a lot of related functionality, including groups, > users, and such. It would be great to have a first-class integration > available. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-7823) Retrieve groups from OIDC token claim
[ https://issues.apache.org/jira/browse/NIFI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-7823: --- Summary: Retrieve groups from OIDC token claim (was: Retrieve groups from OIDC token using KeyClock) > Retrieve groups from OIDC token claim > - > > Key: NIFI-7823 > URL: https://issues.apache.org/jira/browse/NIFI-7823 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Reporter: Raz Dobkies >Assignee: Malthe Borch >Priority: Major > Fix For: 1.19.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Nifi will can retrueve groups from a token if we will develop > KeyCloakUserGroupProvider. > Keycloak does implement a lot of related functionality, including groups, > users, and such. It would be great to have a first-class integration > available. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-7823) Retrieve groups from OIDC token using KeyClock
[ https://issues.apache.org/jira/browse/NIFI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-7823: --- Issue Type: Improvement (was: Wish) > Retrieve groups from OIDC token using KeyClock > -- > > Key: NIFI-7823 > URL: https://issues.apache.org/jira/browse/NIFI-7823 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Reporter: Raz Dobkies >Assignee: Malthe Borch >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Nifi will can retrueve groups from a token if we will develop > KeyCloakUserGroupProvider. > Keycloak does implement a lot of related functionality, including groups, > users, and such. It would be great to have a first-class integration > available. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-7823) Retrieve groups from OIDC token using KeyClock
[ https://issues.apache.org/jira/browse/NIFI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623207#comment-17623207 ] ASF subversion and git services commented on NIFI-7823: --- Commit 831a11d0b5b118002e9969778d3b164401761a7d in nifi's branch refs/heads/main from Malthe Borch [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=831a11d0b5 ] NIFI-7823 Added groups mapping from OIDC token claim This closes #6454 Signed-off-by: David Handermann > Retrieve groups from OIDC token using KeyClock > -- > > Key: NIFI-7823 > URL: https://issues.apache.org/jira/browse/NIFI-7823 > Project: Apache NiFi > Issue Type: Wish > Components: Security >Reporter: Raz Dobkies >Assignee: Malthe Borch >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Nifi will can retrueve groups from a token if we will develop > KeyCloakUserGroupProvider. > Keycloak does implement a lot of related functionality, including groups, > users, and such. It would be great to have a first-class integration > available. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6454: NIFI-7823 Add groups available via OIDC token to IDP user group service
exceptionfactory closed pull request #6454: NIFI-7823 Add groups available via OIDC token to IDP user group service URL: https://github.com/apache/nifi/pull/6454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10684) Update unboundid-ldapsdk to at least 4.0.5
[ https://issues.apache.org/jira/browse/NIFI-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R updated NIFI-10684: -- Fix Version/s: (was: 1.18.0) Affects Version/s: 1.18.0 > Update unboundid-ldapsdk to at least 4.0.5 > -- > > Key: NIFI-10684 > URL: https://issues.apache.org/jira/browse/NIFI-10684 > Project: Apache NiFi > Issue Type: Wish >Affects Versions: 1.18.0 >Reporter: Mike R >Priority: Major > > Update unboundid-ldapsdk to at least 4.0.5, since the version used currently > is 3.2.1. This component has a CVE with a score of 9.8 open against it. > Starting in 4.0.5, the CVE is remediated. > https://nvd.nist.gov/vuln/detail/CVE-2018-1000134 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10682) When Ghosted Component reloaded, it may not evaluate parameters
[ https://issues.apache.org/jira/browse/NIFI-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-10682: --- Resolution: Fixed Status: Resolved (was: Patch Available) > When Ghosted Component reloaded, it may not evaluate parameters > --- > > Key: NIFI-10682 > URL: https://issues.apache.org/jira/browse/NIFI-10682 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.19.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > On startup, if the flow indicates that NiFi should load version X of a > component, and Version X is not available, it will load the version that is > available - if exactly 1 version does exist. But if 0 versions or multiple > versions exist, it will not load the component at all. Instead, it will make > the component "Ghosted" and unusable. > When this happens, the component's properties are all marked sensitive, > because we are unable to instantiate the component in order to check the > sensitivity of components. This is very much intentional. > We also offer the ability to Change Version, if multiple versions do exist, > or if a new nar is added to the auto-loaded extensions directory. However, > once this happens, the component is not evaluating Parameters properly. > To replicate: > # Create an SSLContext Service. Configure using parameters for keystore and > truststore. > 2. Shutdown > 3. Copy in lib from other version of nifi for ssl-context-service-nar > 4. Update flow.json.gz to point to wrong version of sslcontextservice (for > example change version 1.18.0 to 1.5.0, which does not exist in the > distribution) > 5. Restart > 6. Service is now invalid. Change Version to valid version. > 7. The controller service will be invalid. Configure it and click Apply. This > will re-run validation. > 8. Enable the service. > 9. The Controller Service will now throw Exceptions and remain in an Enabling > state. > Even though the Validation is performed properly, the parameters are not > properly evaluated when attempting to use the component. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10682) When Ghosted Component reloaded, it may not evaluate parameters
[ https://issues.apache.org/jira/browse/NIFI-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623200#comment-17623200 ] ASF subversion and git services commented on NIFI-10682: Commit d64574b56556b91905b91c781237343d50f083c9 in nifi's branch refs/heads/main from markap14 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d64574b565 ] NIFI-10682: When determining effective property values, ensure that we always use the up-to-date version of Property Descriptors (#6567) This closes #6567 > When Ghosted Component reloaded, it may not evaluate parameters > --- > > Key: NIFI-10682 > URL: https://issues.apache.org/jira/browse/NIFI-10682 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.19.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > On startup, if the flow indicates that NiFi should load version X of a > component, and Version X is not available, it will load the version that is > available - if exactly 1 version does exist. But if 0 versions or multiple > versions exist, it will not load the component at all. Instead, it will make > the component "Ghosted" and unusable. > When this happens, the component's properties are all marked sensitive, > because we are unable to instantiate the component in order to check the > sensitivity of components. This is very much intentional. > We also offer the ability to Change Version, if multiple versions do exist, > or if a new nar is added to the auto-loaded extensions directory. However, > once this happens, the component is not evaluating Parameters properly. > To replicate: > # Create an SSLContext Service. Configure using parameters for keystore and > truststore. > 2. Shutdown > 3. Copy in lib from other version of nifi for ssl-context-service-nar > 4. Update flow.json.gz to point to wrong version of sslcontextservice (for > example change version 1.18.0 to 1.5.0, which does not exist in the > distribution) > 5. Restart > 6. Service is now invalid. Change Version to valid version. > 7. The controller service will be invalid. Configure it and click Apply. This > will re-run validation. > 8. Enable the service. > 9. The Controller Service will now throw Exceptions and remain in an Enabling > state. > Even though the Validation is performed properly, the parameters are not > properly evaluated when attempting to use the component. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mcgilman merged pull request #6567: NIFI-10682: When determining effective property values, ensure that w…
mcgilman merged PR #6567: URL: https://github.com/apache/nifi/pull/6567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a diff in pull request #1436: MINIFICPP-1963 improve exception logging
adam-markovics commented on code in PR #1436:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1436#discussion_r1003327948
##
libminifi/src/Exception.cpp:
##
@@ -0,0 +1,41 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "Exception.h"
+
+#ifndef WIN32
+#include
+#endif // !WIN32
+
+#include
+
+namespace org::apache::nifi::minifi {
+std::string getCurrentExceptionTypeName() {
+#ifndef WIN32
+ const std::type_info* exception_type = abi::__cxa_current_exception_type();
+ if (exception_type) {
+return exception_type->name();
+ }
+#endif // !WIN32
+ try {
+std::rethrow_exception(std::current_exception());
+ } catch (const std::exception& ex) {
+return typeid(ex).name();
+ } catch (...) { }
+
+ return {};
Review Comment:
So, this function returns empty string on Windows for exceptions that are
not subclasses of `std::exception`? Is there a way to improve this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] marcocharlie commented on pull request #6293: [NIFI-10349] Add maximum object age property to list s3
marcocharlie commented on PR #6293: URL: https://github.com/apache/nifi/pull/6293#issuecomment-1289025283 Hi @MikeThomsen, how can we merge this PR to main branch? Have we missed something? cc @mronci -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6565: NIFI-10516: PutORC with Java 17 fails when using SNAPPY Compression
exceptionfactory commented on PR #6565: URL: https://github.com/apache/nifi/pull/6565#issuecomment-1289025385 Thanks for the reply @mark-bathori. Although it will take more effort for a user to adjust a bootstrap.conf setting, it seems like the best solution in this case due to the nature of the workaround. There is an open issue for the Snappy Java project to change the behavior, so I recommend referencing this issue URL in the comment, and note that it applies to any component that makes use of snappy-java. https://github.com/xerial/snappy-java/issues/348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10516) PutORC with Java 17 fails when using SNAPPY Compression
[
https://issues.apache.org/jira/browse/NIFI-10516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623167#comment-17623167
]
David Handermann commented on NIFI-10516:
-
There is an open Snappy Issue 348 describing this problem on Java 17.
> PutORC with Java 17 fails when using SNAPPY Compression
> ---
>
> Key: NIFI-10516
> URL: https://issues.apache.org/jira/browse/NIFI-10516
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Adam Turley
>Assignee: Mark Bathori
>Priority: Major
> Attachments: snappy-error.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When using SNAPPY compression with PutORC the task fails with the following
> error:
> "Failed to write due to java.lang.NoClassDefFoundError: Could not initialize
> class io.airlift.compress.snappy.UnsafeUtil:{}
> Task completes successfully when using "NONE" as compression.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on pull request #6416: NIFI-10234 implement PutIoTDB
exceptionfactory commented on PR #6416: URL: https://github.com/apache/nifi/pull/6416#issuecomment-1289013596 Thanks for the recent updates @lizhizhou, I will take a closer look soon. At the moment, there is a merge conflict with `nifi-nar-bundles/pom.xml`, can you take a look and resolve? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
