[jira] [Created] (MINIFICPP-1990) Transferring FlowFiles in Python Processors
Kondakov Artem created MINIFICPP-1990: - Summary: Transferring FlowFiles in Python Processors Key: MINIFICPP-1990 URL: https://issues.apache.org/jira/browse/MINIFICPP-1990 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Affects Versions: 0.12.0 Reporter: Kondakov Artem Fix For: 0.13.0 Hello. There is a problem with the processing of flow-files in python processors, which are connected using the setting nifi.python.processor.dir=./minifi-python/ If you need to split one file into several, then you have to create new files in the Python processor and be sure to transfer the original one to REL_FAILURE. Otherwise, if I do not pass the original file to REL_FAILURE, then the processing result is rolled back and nothing happens It seems to me that it is necessary to make it possible for Python processors to add various relations, and not just REL_FAILURE REL_SUCCESS. Or add an additional possible relationship REL_ORIGIN to which you can pass the original file That. if you look at the metrics for these processors, you see TransferredToFaillure > 0, although the processors processed without errors -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-9656) Publish a Java 11 docker image
[ https://issues.apache.org/jira/browse/NIFI-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-9656: -- Summary: Publish a Java 11 docker image (was: Publish an alternative Java 11 docker image) > Publish a Java 11 docker image > -- > > Key: NIFI-9656 > URL: https://issues.apache.org/jira/browse/NIFI-9656 > Project: Apache NiFi > Issue Type: Wish > Components: Docker >Affects Versions: 1.15.3 > Environment: Docker >Reporter: Christoph Nölle >Assignee: Kevin Doran >Priority: Minor > > There have been requests before to update the official Docker image to Java > 11 (such as [NIFI-9188|https://issues.apache.org/jira/browse/NIFI-9188]). As > I understand, this would be a breaking change for many users, though, and > therefore has been postponed for Nifi 2.0. On the other hand, it would be > quite desirable if one could use Java 11 features in custom processors > (without maintaining a custom Docker image). > A solution could be to keep Java 8 for the default image (e.g. tagged > "1.15.3" = latest) and provide a second image, e.g. "1.15.3-openjdk-11". The > Dockerfile has already been adapted to support building with a Java 11 base > image in [https://github.com/apache/nifi/pull/4460]. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-9656) Publish a Java 11 docker image
[ https://issues.apache.org/jira/browse/NIFI-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-9656: -- Fix Version/s: 1.19.0 Status: Patch Available (was: In Progress) > Publish a Java 11 docker image > -- > > Key: NIFI-9656 > URL: https://issues.apache.org/jira/browse/NIFI-9656 > Project: Apache NiFi > Issue Type: Wish > Components: Docker >Affects Versions: 1.15.3 > Environment: Docker >Reporter: Christoph Nölle >Assignee: Kevin Doran >Priority: Minor > Fix For: 1.19.0 > > > There have been requests before to update the official Docker image to Java > 11 (such as [NIFI-9188|https://issues.apache.org/jira/browse/NIFI-9188]). As > I understand, this would be a breaking change for many users, though, and > therefore has been postponed for Nifi 2.0. On the other hand, it would be > quite desirable if one could use Java 11 features in custom processors > (without maintaining a custom Docker image). > A solution could be to keep Java 8 for the default image (e.g. tagged > "1.15.3" = latest) and provide a second image, e.g. "1.15.3-openjdk-11". The > Dockerfile has already been adapted to support building with a Java 11 base > image in [https://github.com/apache/nifi/pull/4460]. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10820) Change base Docker images for NiFi
[ https://issues.apache.org/jira/browse/NIFI-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-10820: --- Status: Patch Available (was: Open) > Change base Docker images for NiFi > -- > > Key: NIFI-10820 > URL: https://issues.apache.org/jira/browse/NIFI-10820 > Project: Apache NiFi > Issue Type: Sub-task > Components: Docker >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Major > Fix For: 1.19.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Pretty much the same work that was done for minifi under NIFI-10701, but for > nifi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] kevdoran opened a new pull request, #6673: [NIFI-10820] [NIFI-9656] Update Docker base image to eclipse-temurin:11
kevdoran opened a new pull request, #6673: URL: https://github.com/apache/nifi/pull/6673 # Summary [NIFI-10820](https://issues.apache.org/jira/browse/NIFI-10820) [NIFI-9656](https://issues.apache.org/jira/browse/NIFI-9656) This work is based on changes @briansolo1985 made for minifi in #6587, to change all Docker images from openjdk (no longer maintained) to eclipse-temurin (based on OpenJDK). In addition, this changes the base image to Java 11. Lastly, this opens the door for multi-arch Docker images, i.e., amd64 and arm64. ## Testing The following command can be used to build all dockermaven images: ``` mvn -T2C clean install -DskipTests \ && mvn install -Pdocker -pl nifi-toolkit/nifi-toolkit-assembly,nifi-docker/dockermaven,nifi-registry/nifi-registry-docker-maven/dockermaven \ && mvn install -Pdocker,targz -pl minifi/minifi-assembly,minifi/minifi-docker,minifi/minifi-c2/minifi-c2-assembly,minifi/minifi-c2/minifi-c2-docker ``` For convenience in testing, the following images have been published to Docker Hub using my normal build process and 1.18.0 artifacts: - [kevdoran/nifi:1.18.0](https://hub.docker.com/r/kevdoran/nifi/tags) - [kevdoran/nifi-toolkit:1.18.0](https://hub.docker.com/r/kevdoran/nifi-toolkit/tags) - [kevdoran/nifi-registry:1.18.0](https://hub.docker.com/r/kevdoran/nifi-registry/tags) - [kevdoran/nifi-minifi:1.18.0](https://hub.docker.com/r/kevdoran/nifi-minifi/tags) - [kevdoran/nifi-minifi-c2:1.18.0](https://hub.docker.com/r/kevdoran/nifi-minifi-c2/tags) Note that all of these now support arm64 platforms in addition to amd64! Please verify these docker hub as you normally might use them. --- # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10818) MergeContent 1.17.0 Bin no longer "make room for the new one" as documented
[ https://issues.apache.org/jira/browse/NIFI-10818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635090#comment-17635090 ] Noe commented on NIFI-10818: Will this fix be available in 1.17.x? > MergeContent 1.17.0 Bin no longer "make room for the new one" as documented > --- > > Key: NIFI-10818 > URL: https://issues.apache.org/jira/browse/NIFI-10818 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Noe >Assignee: Mark Payne >Priority: Critical > Fix For: 1.19.0 > > Attachments: MergeContent_Bug.xml > > Time Spent: 0.5h > Remaining Estimate: 0h > > MergeContent bin no longer "make room for the new one" This feature works in > 1.15.2, but does not work in 1.17.0 > The bug is demostrated in the attached template. The queue before > MergeContent will back up do to new bins not being created for correlation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10833) Fix grammar error in ListenHTTP log msg
[
https://issues.apache.org/jira/browse/NIFI-10833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough resolved NIFI-10833.
-
Fix Version/s: 1.19.0
Resolution: Fixed
> Fix grammar error in ListenHTTP log msg
> ---
>
> Key: NIFI-10833
> URL: https://issues.apache.org/jira/browse/NIFI-10833
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
> Fix For: 1.19.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> _getLogger().warn("failed to receive{+}*d*{+} acknowledgment for HOLD_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10833) Fix grammar error in ListenHTTP log msg
[
https://issues.apache.org/jira/browse/NIFI-10833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635051#comment-17635051
]
ASF subversion and git services commented on NIFI-10833:
Commit 7a449dbcee668b39d85f850e549ae0a4d88be01b in nifi's branch
refs/heads/main from Arpad Boda
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7a449dbcee ]
NIFI-10833 - Fix grammar error in ListenHTTP log msg
> Fix grammar error in ListenHTTP log msg
> ---
>
> Key: NIFI-10833
> URL: https://issues.apache.org/jira/browse/NIFI-10833
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> _getLogger().warn("failed to receive{+}*d*{+} acknowledgment for HOLD_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] thenatog merged pull request #6669: NIFI-10833 - Fix grammar error in ListenHTTP log msg
thenatog merged PR #6669: URL: https://github.com/apache/nifi/pull/6669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10787) Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
[
https://issues.apache.org/jira/browse/NIFI-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough resolved NIFI-10787.
-
Fix Version/s: 1.19.0
Resolution: Fixed
> Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
> --
>
> Key: NIFI-10787
> URL: https://issues.apache.org/jira/browse/NIFI-10787
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.18.0
>Reporter: Ahsan
>Assignee: Seda Dogan
>Priority: Blocker
> Fix For: 1.19.0
>
> Attachments: index.png, stacktrace_nifi.txt,
> stacktrace_nifi_registry.txt
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Hi,
>
> So we recently updated to Nifi 1.18.0 and registry to 1.18.0.
> Some portions of our flows were for no reason not "Commitable" any more.
> Attached are the stacktraces from nifi and nifi-registry, when we click the
> commit local changes button in nifi.
>
> Thinking this is a problem on our end, we debugged the issue and found out
> the following:
> The method in
> "src/main/java/org/apache/nifi/registry/flow/mapping/NiFiRegistryFlowMapper.java"
> below is where things trip and we get a NPE.
> {code:java}
> private String getRegistryUrl(final FlowRegistryClientNode registry) {
> return
> registry.getComponentType().equals("org.apache.nifi.registry.flow.NifiRegistryFlowRegistryClient")
> ? registry.getRawPropertyValue(registry.getPropertyDescriptor("URL")) : "";
> } {code}
> If you note the call "registry.getPropertyDescriptor("URL")" with the
> hard-coded string "URL", this is failing although the property is there BUT
> with the name in small case "url".
> I say this is because if we look at the class
> {color:#6a8759}"NifiRegistryFlowRegistryClient", {color}the url property is
> described as following:
> {code:java}
> public final static PropertyDescriptor PROPERTY_URL = new
> PropertyDescriptor.Builder()
> .name("url")
> .displayName("URL")
> .description("URL of the NiFi Registry")
> .addValidator(StandardValidators.URL_VALIDATOR)
> .required(true)
> .build();{code}
> And if you note the property name is described with small case "url". Hence
> PropertyDescriptor which bases its hash on the "name" property fails when we
> search with uppercase "URL".
> {code:java}
> // hash def of
> nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java
> @Override
> public int hashCode() {
> return 287 + this.name.hashCode() * 47;
> } {code}
> Hope I have helped here. Can someone fix this issue. We cannot commit in our
> registry currently because of the NPE.
>
> Just in case the debug stacktrace is important showing the src
> PropertyDescription being used to search for in the map, I attach it here:
>
> !index.png!
>
> Regards
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10787) Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
[
https://issues.apache.org/jira/browse/NIFI-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635043#comment-17635043
]
ASF subversion and git services commented on NIFI-10787:
Commit 009d641576499c5cdfb940a422996bbdce7dfda3 in nifi's branch
refs/heads/main from sedadgn
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=009d641576 ]
NIFI-10787 - Cannot commit flows to nifi registry after updating our nifi
release to 1.18.0
NifiRegistryFlowRegistryClient defines the PropertyDescriptor PROPERTY_URL
name as small case "url". The map bases on the name property of the
PropertyDescriptor object. Here searching with uppercase value of "URL" causes
the map lookup to fail and cause a NPE later on. Therefore, it is changed as
"url"
NIFI-10787 - Added constant for property descriptor "url" in
NiFiRegistryFlowMapper to make it more clear.
NIFI-10787 - Added change to unit test.
NIFI-10787 - Updated unit test to validate that NiFi registry url is being set
and retrieved.
Signed-off-by: Nathan Gough
This closes #6655.
> Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
> --
>
> Key: NIFI-10787
> URL: https://issues.apache.org/jira/browse/NIFI-10787
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.18.0
>Reporter: Ahsan
>Assignee: Seda Dogan
>Priority: Blocker
> Attachments: index.png, stacktrace_nifi.txt,
> stacktrace_nifi_registry.txt
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Hi,
>
> So we recently updated to Nifi 1.18.0 and registry to 1.18.0.
> Some portions of our flows were for no reason not "Commitable" any more.
> Attached are the stacktraces from nifi and nifi-registry, when we click the
> commit local changes button in nifi.
>
> Thinking this is a problem on our end, we debugged the issue and found out
> the following:
> The method in
> "src/main/java/org/apache/nifi/registry/flow/mapping/NiFiRegistryFlowMapper.java"
> below is where things trip and we get a NPE.
> {code:java}
> private String getRegistryUrl(final FlowRegistryClientNode registry) {
> return
> registry.getComponentType().equals("org.apache.nifi.registry.flow.NifiRegistryFlowRegistryClient")
> ? registry.getRawPropertyValue(registry.getPropertyDescriptor("URL")) : "";
> } {code}
> If you note the call "registry.getPropertyDescriptor("URL")" with the
> hard-coded string "URL", this is failing although the property is there BUT
> with the name in small case "url".
> I say this is because if we look at the class
> {color:#6a8759}"NifiRegistryFlowRegistryClient", {color}the url property is
> described as following:
> {code:java}
> public final static PropertyDescriptor PROPERTY_URL = new
> PropertyDescriptor.Builder()
> .name("url")
> .displayName("URL")
> .description("URL of the NiFi Registry")
> .addValidator(StandardValidators.URL_VALIDATOR)
> .required(true)
> .build();{code}
> And if you note the property name is described with small case "url". Hence
> PropertyDescriptor which bases its hash on the "name" property fails when we
> search with uppercase "URL".
> {code:java}
> // hash def of
> nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java
> @Override
> public int hashCode() {
> return 287 + this.name.hashCode() * 47;
> } {code}
> Hope I have helped here. Can someone fix this issue. We cannot commit in our
> registry currently because of the NPE.
>
> Just in case the debug stacktrace is important showing the src
> PropertyDescription being used to search for in the map, I attach it here:
>
> !index.png!
>
> Regards
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] thenatog closed pull request #6655: NIFI-10787 Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
thenatog closed pull request #6655: NIFI-10787 Cannot commit flows to nifi registry after updating our nifi release to 1.18.0 URL: https://github.com/apache/nifi/pull/6655 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #6655: NIFI-10787 Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
thenatog commented on PR #6655: URL: https://github.com/apache/nifi/pull/6655#issuecomment-1317800133 +1 will merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] priyanka-28 opened a new pull request, #6672: NIFI-10834 Fixed Flaky Test failing with NonDex
priyanka-28 opened a new pull request, #6672: URL: https://github.com/apache/nifi/pull/6672 # Summary [NIFI-10834](https://issues.apache.org/jira/browse/NIFI-10834) Tests under TestPutSplunkHTTP express non-deterministic behavior as the order of the query parameters in the URL changes. The fix is generating all possible permutations of the query parameter and then checking the generated permutations with the value passed in the 'Assert' function to ensure deterministic behavior. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6416: NIFI-10234 implement PutIoTDB
exceptionfactory commented on PR #6416: URL: https://github.com/apache/nifi/pull/6416#issuecomment-1317747117 Thanks for the updates @lizhizhou, the record handling approach appears to be improved, I will take a closer look at this soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10835) Improve Performance of TestControlRate
[
https://issues.apache.org/jira/browse/NIFI-10835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10835:
Status: Patch Available (was: Open)
> Improve Performance of TestControlRate
> --
>
> Key: NIFI-10835
> URL: https://issues.apache.org/jira/browse/NIFI-10835
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions, Tools and Build
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{TestControlRate}} class takes over 12 seconds to run due to the use of
> Thread.sleep() across multiple test methods.
> The test class can be improved by implementing a method to override the
> current time. The default implementation should maintain the current behavior
> of returning {{System.currentTimeMillis()}} but the extended test class can
> override the implementation to increment current time, allowing the processor
> logic to function as designed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6671: NIFI-10835 Improve performance of TestControlRate
exceptionfactory opened a new pull request, #6671: URL: https://github.com/apache/nifi/pull/6671 # Summary [NIFI-10835](https://issues.apache.org/jira/browse/NIFI-10835) Improves the performance of `TestControlRate`, reducing the overall runtime from over 12 seconds to less than one second. The implementation updates `ControlRate` and `TimedBuffer` classes to use a `LongSupplier` that can be overridden to increment the current time in test methods. This approaches preserves current behavior of `ControlRate` and supports a `ConfigurableControlRate` extension in the test class, avoiding the need for `Thread.sleep()`. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10835) Improve Performance of TestControlRate
David Handermann created NIFI-10835:
---
Summary: Improve Performance of TestControlRate
Key: NIFI-10835
URL: https://issues.apache.org/jira/browse/NIFI-10835
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions, Tools and Build
Reporter: David Handermann
Assignee: David Handermann
The {{TestControlRate}} class takes over 12 seconds to run due to the use of
Thread.sleep() across multiple test methods.
The test class can be improved by implementing a method to override the current
time. The default implementation should maintain the current behavior of
returning {{System.currentTimeMillis()}} but the extended test class can
override the implementation to increment current time, allowing the processor
logic to function as designed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10818) MergeContent 1.17.0 Bin no longer "make room for the new one" as documented
[ https://issues.apache.org/jira/browse/NIFI-10818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635019#comment-17635019 ] ASF subversion and git services commented on NIFI-10818: Commit d7f2eb7c263aa9511dba5a2f5760f779ff1a12f9 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d7f2eb7c26 ] NIFI-10818: Fixed bug in which oldest bin in MergeContent was not being evicted in order to make room for a new bin when using a correlation attribute. Added additional tests for JoinEnrichment to ensure that the changes had no adverse effects on that processor. Found a bug when running that test in MockProcessSession so also addressed the bug (ConcurrentModificationException). Signed-off-by: Matthew Burgess This closes #6668 > MergeContent 1.17.0 Bin no longer "make room for the new one" as documented > --- > > Key: NIFI-10818 > URL: https://issues.apache.org/jira/browse/NIFI-10818 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Noe >Assignee: Mark Payne >Priority: Critical > Fix For: 1.19.0 > > Attachments: MergeContent_Bug.xml > > Time Spent: 0.5h > Remaining Estimate: 0h > > MergeContent bin no longer "make room for the new one" This feature works in > 1.15.2, but does not work in 1.17.0 > The bug is demostrated in the attached template. The queue before > MergeContent will back up do to new bins not being created for correlation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10818) MergeContent 1.17.0 Bin no longer "make room for the new one" as documented
[ https://issues.apache.org/jira/browse/NIFI-10818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-10818: Resolution: Fixed Status: Resolved (was: Patch Available) > MergeContent 1.17.0 Bin no longer "make room for the new one" as documented > --- > > Key: NIFI-10818 > URL: https://issues.apache.org/jira/browse/NIFI-10818 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Noe >Assignee: Mark Payne >Priority: Critical > Fix For: 1.19.0 > > Attachments: MergeContent_Bug.xml > > Time Spent: 0.5h > Remaining Estimate: 0h > > MergeContent bin no longer "make room for the new one" This feature works in > 1.15.2, but does not work in 1.17.0 > The bug is demostrated in the attached template. The queue before > MergeContent will back up do to new bins not being created for correlation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 closed pull request #6668: NIFI-10818: Fixed bug in which oldest bin in MergeContent was not bei…
mattyb149 closed pull request #6668: NIFI-10818: Fixed bug in which oldest bin in MergeContent was not bei… URL: https://github.com/apache/nifi/pull/6668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #6668: NIFI-10818: Fixed bug in which oldest bin in MergeContent was not bei…
mattyb149 commented on PR #6668: URL: https://github.com/apache/nifi/pull/6668#issuecomment-1317715562 +1 LGTM, thanks for the fix! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10834) TestPutSplunkHTTP Test Failures on NonDex
[
https://issues.apache.org/jira/browse/NIFI-10834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanka Awatramani updated NIFI-10834:
---
Summary: TestPutSplunkHTTP Test Failures on NonDex (was: Fixed Flaky Test
failing with NonDex)
> TestPutSplunkHTTP Test Failures on NonDex
> --
>
> Key: NIFI-10834
> URL: https://issues.apache.org/jira/browse/NIFI-10834
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Environment: openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
> Apache Maven 3.6.3;
>Reporter: Priyanka Awatramani
>Priority: Trivial
> Attachments: image-2022-11-16-15-34-06-074.png,
> image-2022-11-16-15-36-02-621.png
>
>
> The test
> *org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParameters*
> and
> *org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParametersFromFlowFile*
> passed using normal maven-test, but showed Non-deterministic behavior under
> NonDex([https://github.com/TestingResearchIllinois/NonDex)] and thus failed.
> Some of the error messages are:
>
> !image-2022-11-16-15-34-06-074.png!
>
> !image-2022-11-16-15-36-02-621.png!
>
> and after some probing, I found the output from the test is not deterministic
> as the order of query parameters in the URL changes.
>
> Steps to reproduce the failure:
> Install Nondex([https://github.com/TestingResearchIllinois/NonDex)] in the
> environment.
> Then cd to nifi repository, and run the following:
> {code:java}
> mvn install -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> -am -DskipTests
> mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParameters
>
> mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParametersFromFlowFile{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-10834) Fixed Flaky Test failing with NonDex
Priyanka Awatramani created NIFI-10834:
--
Summary: Fixed Flaky Test failing with NonDex
Key: NIFI-10834
URL: https://issues.apache.org/jira/browse/NIFI-10834
Project: Apache NiFi
Issue Type: Bug
Components: NiFi Registry
Environment: openjdk version "1.8.0_342";
OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
Apache Maven 3.6.3;
Reporter: Priyanka Awatramani
Attachments: image-2022-11-16-15-34-06-074.png,
image-2022-11-16-15-36-02-621.png
The test
*org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParameters*
and
*org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParametersFromFlowFile*
passed using normal maven-test, but showed Non-deterministic behavior under
NonDex([https://github.com/TestingResearchIllinois/NonDex)] and thus failed.
Some of the error messages are:
!image-2022-11-16-15-34-06-074.png!
!image-2022-11-16-15-36-02-621.png!
and after some probing, I found the output from the test is not deterministic
as the order of query parameters in the URL changes.
Steps to reproduce the failure:
Install Nondex([https://github.com/TestingResearchIllinois/NonDex)] in the
environment.
Then cd to nifi repository, and run the following:
{code:java}
mvn install -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors -am
-DskipTests
mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
edu.illinois:nondex-maven-plugin:1.1.2:nondex
-Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParameters
mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
edu.illinois:nondex-maven-plugin:1.1.2:nondex
-Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParametersFromFlowFile{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[ https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz reassigned NIFI-6498: -- Assignee: Daniel Stieglitz > XMLTransform processor: XSLT doesn't give detail error message with line > number > --- > > Key: NIFI-6498 > URL: https://issues.apache.org/jira/browse/NIFI-6498 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.10.0, 1.9.2 >Reporter: Raymond >Assignee: Daniel Stieglitz >Priority: Major > Attachments: > 0001-Add-support-for-Saxon-error-messages-and-functions.patch, > 0001-Add-support-for-Saxon-error-messages-and-functions.patch > > > When something is wrong with the XSLT in the XMLTransform processor you get > an error, but this is only a general error without any details or the line > number where the parsing error occurs. For example when using Saxon from > example I get the following error: > Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element > type "xsl:tomplate" must be terminated by the matching end-tag > "". > When using NiFi the following error is printed: > 11:07:43 CEST > ERROR > 2b4c1dab-3a87-175f-f817-3f3c72af4102 > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform > StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default, > section=1], offset=0, > length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380] > due to IOException thrown from > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException: > java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported: org.apache.nifi.processor.exception.ProcessException: IOException > thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: > java.io.IOException: java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported > The error in NiFi gives almost no information for debugging. Would be a big > improvement if the Saxon error message is catched and add to the NiFi error > message (bulletin board). > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[ https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635014#comment-17635014 ] David Handermann commented on NIFI-6498: [~dstiegli1] The changes in the patch are now stale because the Saxon version has been upgraded. If you are interested in taking a look at the changes that remain necessary, that would be helpful. > XMLTransform processor: XSLT doesn't give detail error message with line > number > --- > > Key: NIFI-6498 > URL: https://issues.apache.org/jira/browse/NIFI-6498 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.10.0, 1.9.2 >Reporter: Raymond >Priority: Major > Attachments: > 0001-Add-support-for-Saxon-error-messages-and-functions.patch, > 0001-Add-support-for-Saxon-error-messages-and-functions.patch > > > When something is wrong with the XSLT in the XMLTransform processor you get > an error, but this is only a general error without any details or the line > number where the parsing error occurs. For example when using Saxon from > example I get the following error: > Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element > type "xsl:tomplate" must be terminated by the matching end-tag > "". > When using NiFi the following error is printed: > 11:07:43 CEST > ERROR > 2b4c1dab-3a87-175f-f817-3f3c72af4102 > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform > StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default, > section=1], offset=0, > length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380] > due to IOException thrown from > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException: > java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported: org.apache.nifi.processor.exception.ProcessException: IOException > thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: > java.io.IOException: java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported > The error in NiFi gives almost no information for debugging. Would be a big > improvement if the Saxon error message is catched and add to the NiFi error > message (bulletin board). > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[ https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635005#comment-17635005 ] Daniel Stieglitz edited comment on NIFI-6498 at 11/16/22 9:17 PM: -- [~pvillard] [~joewitt] It looks like a Github PR was never submitted. Can I submit one using the attached changes? was (Author: JIRAUSER294662): [~pvillard] It looks like a Github PR was never submitted. Can I submit one using the attached changes? > XMLTransform processor: XSLT doesn't give detail error message with line > number > --- > > Key: NIFI-6498 > URL: https://issues.apache.org/jira/browse/NIFI-6498 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.10.0, 1.9.2 >Reporter: Raymond >Priority: Major > Attachments: > 0001-Add-support-for-Saxon-error-messages-and-functions.patch, > 0001-Add-support-for-Saxon-error-messages-and-functions.patch > > > When something is wrong with the XSLT in the XMLTransform processor you get > an error, but this is only a general error without any details or the line > number where the parsing error occurs. For example when using Saxon from > example I get the following error: > Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element > type "xsl:tomplate" must be terminated by the matching end-tag > "". > When using NiFi the following error is printed: > 11:07:43 CEST > ERROR > 2b4c1dab-3a87-175f-f817-3f3c72af4102 > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform > StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default, > section=1], offset=0, > length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380] > due to IOException thrown from > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException: > java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported: org.apache.nifi.processor.exception.ProcessException: IOException > thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: > java.io.IOException: java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported > The error in NiFi gives almost no information for debugging. Would be a big > improvement if the Saxon error message is catched and add to the NiFi error > message (bulletin board). > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ZhewenFu commented on pull request #6525: NIFI-10639 Fix testAddedRowsTwoTables
ZhewenFu commented on PR #6525: URL: https://github.com/apache/nifi/pull/6525#issuecomment-1317673439 > I'm not sure how this addresses the flakiness of the test. Is this test failing frequently in Github Actions? I don't quite understand how running the runner a second time and verifying no FlowFiles were transferred fixes the test. * Sorry for the late response. I detected that the test results are flaky using the NonDex tool for catching non-deterministic/flaky tests (https://github.com/TestingResearchIllinois/NonDex). You may reproduce the nondeterministic behavior using the following steps: Clone the repo and cd into it. `mvn install -pl modules/configuration-governance -am -DskipTests` `mvn -pl modules/configuration-governance test -Dtest=com.ibm.cloud.platform_services.configuration_governance.v1.model.RuleRequestTest#testRuleRequest` `mvn -pl modules/configuration-governance edu.illinois:nondex-maven-plugin:1.1.2:nondex -Dtest=com.ibm.cloud.platform_services.configuration_governance.v1.model.RuleRequestTest#testRuleRequest` * I think I made a mistake in my previous descriptions. I think the real solve of flaky tests lies on the line `runner.setProperty(QueryDatabaseTableRecord.MAX_ROWS_PER_FLOW_FILE, "0");` due to some inconsistent or asychronized reading/writing. But I am not sure about this for now and still diving into the code to see what's the exact cause of this problem. I will let you know when I find more about it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9822) Update ConsumeKafkaRecord to allow writing out of the Kafka record key
[ https://issues.apache.org/jira/browse/NIFI-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9822: - Fix Version/s: 1.19.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Update ConsumeKafkaRecord to allow writing out of the Kafka record key > -- > > Key: NIFI-9822 > URL: https://issues.apache.org/jira/browse/NIFI-9822 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Paul Grey >Priority: Major > Fix For: 1.19.0 > > Time Spent: 5.5h > Remaining Estimate: 0h > > The ConsumeKafkaRecord processors are among the most commonly used in NiFi, > as they provide a very efficient mechanism for consuming structured data from > Kafka. The down side to these processors that they do not support writing out > the Kafka record's key. This was done because we wanted to bundle the > records' values together into a single FlowFile, and we didn't have a good > way to include the key. > For users who don't care about the key, this works great. For users that do > need the key, they often are forced to use the non-record-oriented > ConsumeKafka, which adds the key as an attribute. But this means that the key > may need to be hex-encoded, which makes it less usable and it means that we > are creating a FlowFile per kafka record. > We should improve this by introducing some new properties to the > ConsumeKafkaRecord processors: > - Output Strategy. This property should have the following values: > - Write Value Only - This should be the default value in order to maintain > backward compatibility and should behave the same as it does now. > - Use Wrapper - If selected, records that are provided to the Record Writer > should be wrapped in a wrapper element that contains 4 keys: "key" (the kafka > record key), "value" (the kafka record value), "headers" (a Map type of field > with Strings as both the keys and values), and "metadata" (should include > topic, partition, offset, timestamp, checksum). > If the Output Strategy selected is "Use Wrapper", we should provide the > following properties: > - Key Format - Allowable Values of (String, Byte Array, Record) > - Key Record Reader - if Key Format = "Record" then should allow specifying > a Record Reader for the key. Should be dependent on Key Format = Record. > Additionally, if the headers and the Kafka record key should only be added as > attributes if using an Output Strategy of "Write Value Only." As a result, > the following existing properties should be made dependent on using an Output > Strategy of "Write Value Only": > - Headers to Add as Attributes (Regex) > - Key Attribute Encoding > It will also be important to update the additionalDetails.html to explain the > differences between the two output modes, and provide examples, including > when one strategy should be preferred over the other. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-9822) Update ConsumeKafkaRecord to allow writing out of the Kafka record key
[ https://issues.apache.org/jira/browse/NIFI-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635012#comment-17635012 ] ASF subversion and git services commented on NIFI-9822: --- Commit 30facedc43c5e859f8a82318bb4340d99654eab2 in nifi's branch refs/heads/main from greyp9 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=30facedc43 ] NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key (#6131) * NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key * publisher wrapper record; property naming and display order; doc updates > Update ConsumeKafkaRecord to allow writing out of the Kafka record key > -- > > Key: NIFI-9822 > URL: https://issues.apache.org/jira/browse/NIFI-9822 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Paul Grey >Priority: Major > Time Spent: 5.5h > Remaining Estimate: 0h > > The ConsumeKafkaRecord processors are among the most commonly used in NiFi, > as they provide a very efficient mechanism for consuming structured data from > Kafka. The down side to these processors that they do not support writing out > the Kafka record's key. This was done because we wanted to bundle the > records' values together into a single FlowFile, and we didn't have a good > way to include the key. > For users who don't care about the key, this works great. For users that do > need the key, they often are forced to use the non-record-oriented > ConsumeKafka, which adds the key as an attribute. But this means that the key > may need to be hex-encoded, which makes it less usable and it means that we > are creating a FlowFile per kafka record. > We should improve this by introducing some new properties to the > ConsumeKafkaRecord processors: > - Output Strategy. This property should have the following values: > - Write Value Only - This should be the default value in order to maintain > backward compatibility and should behave the same as it does now. > - Use Wrapper - If selected, records that are provided to the Record Writer > should be wrapped in a wrapper element that contains 4 keys: "key" (the kafka > record key), "value" (the kafka record value), "headers" (a Map type of field > with Strings as both the keys and values), and "metadata" (should include > topic, partition, offset, timestamp, checksum). > If the Output Strategy selected is "Use Wrapper", we should provide the > following properties: > - Key Format - Allowable Values of (String, Byte Array, Record) > - Key Record Reader - if Key Format = "Record" then should allow specifying > a Record Reader for the key. Should be dependent on Key Format = Record. > Additionally, if the headers and the Kafka record key should only be added as > attributes if using an Output Strategy of "Write Value Only." As a result, > the following existing properties should be made dependent on using an Output > Strategy of "Write Value Only": > - Headers to Add as Attributes (Regex) > - Key Attribute Encoding > It will also be important to update the additionalDetails.html to explain the > differences between the two output modes, and provide examples, including > when one strategy should be preferred over the other. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 merged pull request #6131: NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key
markap14 merged PR #6131: URL: https://github.com/apache/nifi/pull/6131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9656) Publish an alternative Java 11 docker image
[ https://issues.apache.org/jira/browse/NIFI-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635011#comment-17635011 ] Kevin Doran commented on NIFI-9656: --- A number of factors have tipped the scale in favor of changing the NiFi base image over maintaining backwards compatibility: * Many users requesting images for arm64 platform * The current base image, openjdk:8-jre is no longer being maintained Given this, there is a lot of support for moving eclipse-temurin as a new base image, which is one of the popular successors to the openjdk images and it already support multiarch builds. As this will disrupt some users, it seems like a good time to also move to Java 11. Lastly, I think I've found a way to restructure our Docker source / build process to make it much easier to build and publish image variants without a lot of duplication. I'll be looking into this so we can offer Java 17 variants soon. > Publish an alternative Java 11 docker image > --- > > Key: NIFI-9656 > URL: https://issues.apache.org/jira/browse/NIFI-9656 > Project: Apache NiFi > Issue Type: Wish > Components: Docker >Affects Versions: 1.15.3 > Environment: Docker >Reporter: Christoph Nölle >Assignee: Kevin Doran >Priority: Minor > > There have been requests before to update the official Docker image to Java > 11 (such as [NIFI-9188|https://issues.apache.org/jira/browse/NIFI-9188]). As > I understand, this would be a breaking change for many users, though, and > therefore has been postponed for Nifi 2.0. On the other hand, it would be > quite desirable if one could use Java 11 features in custom processors > (without maintaining a custom Docker image). > A solution could be to keep Java 8 for the default image (e.g. tagged > "1.15.3" = latest) and provide a second image, e.g. "1.15.3-openjdk-11". The > Dockerfile has already been adapted to support building with a Java 11 base > image in [https://github.com/apache/nifi/pull/4460]. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog commented on a diff in pull request #6649: NIFI-10785 Allow publishing AMQP message with null header value
thenatog commented on code in PR #6649:
URL: https://github.com/apache/nifi/pull/6649#discussion_r1024500229
##
nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/PublishAMQP.java:
##
@@ -108,10 +106,22 @@ public class PublishAMQP extends
AbstractAMQPProcessor {
.addValidator(StandardValidators.SINGLE_CHAR_VALIDATOR)
.required(false)
.build();
+
+public static final PropertyDescriptor IGNORE_HEADER_WITH_NULL_VALUE = new
PropertyDescriptor.Builder()
Review Comment:
The default value might also be 'true' to allow null headers by default? I'm
actually not sure if there's a great reason not to allow null headers in the
first place, so potentially this property isn't required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-9656) Publish an alternative Java 11 docker image
[ https://issues.apache.org/jira/browse/NIFI-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran reassigned NIFI-9656: - Assignee: Kevin Doran > Publish an alternative Java 11 docker image > --- > > Key: NIFI-9656 > URL: https://issues.apache.org/jira/browse/NIFI-9656 > Project: Apache NiFi > Issue Type: Wish > Components: Docker >Affects Versions: 1.15.3 > Environment: Docker >Reporter: Christoph Nölle >Assignee: Kevin Doran >Priority: Minor > > There have been requests before to update the official Docker image to Java > 11 (such as [NIFI-9188|https://issues.apache.org/jira/browse/NIFI-9188]). As > I understand, this would be a breaking change for many users, though, and > therefore has been postponed for Nifi 2.0. On the other hand, it would be > quite desirable if one could use Java 11 features in custom processors > (without maintaining a custom Docker image). > A solution could be to keep Java 8 for the default image (e.g. tagged > "1.15.3" = latest) and provide a second image, e.g. "1.15.3-openjdk-11". The > Dockerfile has already been adapted to support building with a Java 11 base > image in [https://github.com/apache/nifi/pull/4460]. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog commented on pull request #6649: NIFI-10785 Allow publishing AMQP message with null header value
thenatog commented on PR #6649: URL: https://github.com/apache/nifi/pull/6649#issuecomment-1317657443 Thanks Nandor, I verified PublishAMQP can publish null header values and then ConsumeAMQP will retrieve them an interpret them correctly. I also verified that a warning is thrown when 'Ignore Null Header = true' but suggested a change to the naming and usage. Otherwise, looks good to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on a diff in pull request #6649: NIFI-10785 Allow publishing AMQP message with null header value
thenatog commented on code in PR #6649:
URL: https://github.com/apache/nifi/pull/6649#discussion_r1024497984
##
nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/PublishAMQP.java:
##
@@ -108,10 +106,22 @@ public class PublishAMQP extends
AbstractAMQPProcessor {
.addValidator(StandardValidators.SINGLE_CHAR_VALIDATOR)
.required(false)
.build();
+
+public static final PropertyDescriptor IGNORE_HEADER_WITH_NULL_VALUE = new
PropertyDescriptor.Builder()
Review Comment:
Currently with 'Ignore Null Header = true', the processor throws an error,
which is a little counter-intuitive because we told the processor to ignore the
null header but it throws a warning?
Perhaps this would be better as 'Allow Null Header', and flipping the
true/false condition, such that the warning message is thrown when 'Allow Null
Header = false' and a null header is provided.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10832) Create PutSalesforceRecord processor
[ https://issues.apache.org/jira/browse/NIFI-10832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér updated NIFI-10832: -- Status: Patch Available (was: Open) > Create PutSalesforceRecord processor > > > Key: NIFI-10832 > URL: https://issues.apache.org/jira/browse/NIFI-10832 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Processor able to create new entries in Salesforce -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10832) Create PutSalesforceRecord processor
[ https://issues.apache.org/jira/browse/NIFI-10832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér reassigned NIFI-10832: - Assignee: Lehel Boér > Create PutSalesforceRecord processor > > > Key: NIFI-10832 > URL: https://issues.apache.org/jira/browse/NIFI-10832 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Processor able to create new entries in Salesforce -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] Lehel44 opened a new pull request, #6670: NIFI-10832: Create PutSalesforceRecord processor
Lehel44 opened a new pull request, #6670: URL: https://github.com/apache/nifi/pull/6670 # Summary [NIFI-10832](https://issues.apache.org/jira/browse/NIFI-10832) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-10832) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10833) Fix grammar error in ListenHTTP log msg
[
https://issues.apache.org/jira/browse/NIFI-10833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635008#comment-17635008
]
ASF subversion and git services commented on NIFI-10833:
Commit ffcfbc8dc73840c1e501cad94bbcf21e41a17008 in nifi's branch
refs/heads/NIFI-10833 from Arpad Boda
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ffcfbc8dc7 ]
NIFI-10833 - Fix grammar error in ListenHTTP log msg
> Fix grammar error in ListenHTTP log msg
> ---
>
> Key: NIFI-10833
> URL: https://issues.apache.org/jira/browse/NIFI-10833
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
>
> _getLogger().warn("failed to receive{+}*d*{+} acknowledgment for HOLD_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] arpadboda opened a new pull request, #6669: NIFI-10833 - Fix grammar error in ListenHTTP log msg
arpadboda opened a new pull request, #6669: URL: https://github.com/apache/nifi/pull/6669 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10833) Fix grammar error in ListenHTTP log msg
Arpad Boda created NIFI-10833:
-
Summary: Fix grammar error in ListenHTTP log msg
Key: NIFI-10833
URL: https://issues.apache.org/jira/browse/NIFI-10833
Project: Apache NiFi
Issue Type: Improvement
Reporter: Arpad Boda
Assignee: Arpad Boda
_getLogger().warn("failed to receive{+}*d*{+} acknowledgment for HOLD_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] adamdebreceni opened a new pull request, #1453: MINIFICPP-1989 - Discard old controller service provider from FlowController
adamdebreceni opened a new pull request, #1453: URL: https://github.com/apache/nifi-minifi-cpp/pull/1453 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-1989) Circular reference in controllerservice
Adam Debreceni created MINIFICPP-1989: - Summary: Circular reference in controllerservice Key: MINIFICPP-1989 URL: https://issues.apache.org/jira/browse/MINIFICPP-1989 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Adam Debreceni Assignee: Adam Debreceni FlowController tried to clear its own controller_map_ but it is a ForwardingControllerServiceProvider. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10832) Create PutSalesforceRecord processor
Lehel Boér created NIFI-10832: - Summary: Create PutSalesforceRecord processor Key: NIFI-10832 URL: https://issues.apache.org/jira/browse/NIFI-10832 Project: Apache NiFi Issue Type: New Feature Reporter: Lehel Boér Processor able to create new entries in Salesforce -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10831) Enable JWT auth with Elasticsearch
Chris Sampson created NIFI-10831: Summary: Enable JWT auth with Elasticsearch Key: NIFI-10831 URL: https://issues.apache.org/jira/browse/NIFI-10831 Project: Apache NiFi Issue Type: Improvement Reporter: Chris Sampson Elasticsearch allows for [JWT|https://www.elastic.co/guide/en/elasticsearch/reference/current/jwt-realm.html] auth, NiFi should be updated to allow for such connectivity. The [StandardOauth2AccessTokenProvider|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-oauth2-provider-nar/1.18.0/org.apache.nifi.oauth2.StandardOauth2AccessTokenProvider/index.html] could be used to obtain a JWT from an OAuth2/OIDC provider to be passed to Elasticsearch along with a Service/API Key. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 commented on pull request #6662: NIFI-10776 add NONE and PKI AuthorizationSchemes for ElasticSearchClientService
ChrisSamo632 commented on PR #6662: URL: https://github.com/apache/nifi/pull/6662#issuecomment-1317638709 > I agree it would be helpful to move this pull request forward to get the integration test improvements. I would also note that we should avoid unit testing exact error message strings, instead it is better to assert that a message contains a particular keyword. > > The potential for additional schemes is a good point in favor of these additions. Can you take a look at simplifying the custom validation method? I could see the `PKI` scheme checking for the SSL Context Service being set, but giving it another look, I don't think the other checks are necessary because of the way custom validation hides the other properties. @exceptionfactory I've simplified the `customValidate` logic (largely returning things back to how they are currently on `main` for `BASIC` and `API_KEY` schemes), also addressed your other comments For the additional `AuthorizationSchemes`: - [NIFI-10830 - Kerberos](https://issues.apache.org/jira/browse/NIFI-10830) - [NIFI-10831 - JWT](https://issues.apache.org/jira/browse/NIFI-10831) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10830) Enable Kerberos auth with Elasticsearch
Chris Sampson created NIFI-10830: Summary: Enable Kerberos auth with Elasticsearch Key: NIFI-10830 URL: https://issues.apache.org/jira/browse/NIFI-10830 Project: Apache NiFi Issue Type: Improvement Reporter: Chris Sampson Elasticsearch allows for [Kerberos|https://www.elastic.co/guide/en/elasticsearch/reference/current/kerberos-realm.html] auth to be configured. The NiFi Kerberos components could be used to interface with such an Elasticsearch setup (e.g. [KerberosKeytabUserService|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kerberos-user-service-nar/1.18.0/org.apache.nifi.kerberos.KerberosKeytabUserService/index.html], [KerberosPasswordUserService|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kerberos-user-service-nar/1.18.0/org.apache.nifi.kerberos.KerberosPasswordUserService/index.html], [KerberosTicketCacheUserService|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kerberos-user-service-nar/1.18.0/org.apache.nifi.kerberos.KerberosTicketCacheUserService/index.html]) as appropriate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[ https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635005#comment-17635005 ] Daniel Stieglitz commented on NIFI-6498: [~pvillard] It looks like a Github PR was never submitted. Can I submit one using the attached changes? > XMLTransform processor: XSLT doesn't give detail error message with line > number > --- > > Key: NIFI-6498 > URL: https://issues.apache.org/jira/browse/NIFI-6498 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.10.0, 1.9.2 >Reporter: Raymond >Priority: Major > Attachments: > 0001-Add-support-for-Saxon-error-messages-and-functions.patch, > 0001-Add-support-for-Saxon-error-messages-and-functions.patch > > > When something is wrong with the XSLT in the XMLTransform processor you get > an error, but this is only a general error without any details or the line > number where the parsing error occurs. For example when using Saxon from > example I get the following error: > Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element > type "xsl:tomplate" must be terminated by the matching end-tag > "". > When using NiFi the following error is printed: > 11:07:43 CEST > ERROR > 2b4c1dab-3a87-175f-f817-3f3c72af4102 > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform > StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default, > section=1], offset=0, > length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380] > due to IOException thrown from > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException: > java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported: org.apache.nifi.processor.exception.ProcessException: IOException > thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: > java.io.IOException: java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported > The error in NiFi gives almost no information for debugging. Would be a big > improvement if the Saxon error message is catched and add to the NiFi error > message (bulletin board). > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10818) MergeContent 1.17.0 Bin no longer "make room for the new one" as documented
[ https://issues.apache.org/jira/browse/NIFI-10818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-10818: -- Fix Version/s: 1.19.0 Assignee: Mark Payne Status: Patch Available (was: Open) > MergeContent 1.17.0 Bin no longer "make room for the new one" as documented > --- > > Key: NIFI-10818 > URL: https://issues.apache.org/jira/browse/NIFI-10818 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Noe >Assignee: Mark Payne >Priority: Critical > Fix For: 1.19.0 > > Attachments: MergeContent_Bug.xml > > Time Spent: 10m > Remaining Estimate: 0h > > MergeContent bin no longer "make room for the new one" This feature works in > 1.15.2, but does not work in 1.17.0 > The bug is demostrated in the attached template. The queue before > MergeContent will back up do to new bins not being created for correlation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 opened a new pull request, #6668: NIFI-10818: Fixed bug in which oldest bin in MergeContent was not bei…
markap14 opened a new pull request, #6668: URL: https://github.com/apache/nifi/pull/6668 …ng evicted in order to make room for a new bin when using a correlation attribute. Added additional tests for JoinEnrichment to ensure that the changes had no adverse effects on that processor. Found a bug when running that test in MockProcessSession so also addressed the bug (ConcurrentModificationException). # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10829) Adding the parent folder name as an argument to the ConsumeEWS processor
[
https://issues.apache.org/jira/browse/NIFI-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634985#comment-17634985
]
David Handermann commented on NIFI-10829:
-
Thanks for raising this issue [~dataminer.accolade]. The {{ConsumeEWS}}
Processor uses a Microsoft library that is no longer maintained, so making
changes to this processor is probably not the best approach going forward. It
would be better to consider implementing this feature in a new Processor that
makes use of a supported approach to interacting with Exchange.
> Adding the parent folder name as an argument to the ConsumeEWS processor
>
>
> Key: NIFI-10829
> URL: https://issues.apache.org/jira/browse/NIFI-10829
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
>Reporter: dataminer.accolade
>Priority: Major
> Attachments: image-2022-11-16-17-32-21-819.png
>
>
> I want to be able to pass the root folder as an argument to the ConsumeEWS
> processor.
> [Source|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ConsumeEWS.java#L393]
> !image-2022-11-16-17-32-21-819.png!
> Primarily I am interested in
> [PublicFoldersRoot|https://learn.microsoft.com/en-us/dotnet/api/microsoft.exchange.webservices.data.wellknownfoldername?view=exchange-ews-api]
> making PublicFoldersRoot selectable will require the implementation of the
> deep search because, [deep traversal queries cannot be performed on public
> folders.|https://learn.microsoft.com/en-us/exchange/client-developer/web-service-reference/findfolder-operation?redirectedfrom=MSDN]
> Alternatively it would be nice to pass the path of the folder instead of a
> folder name with a specific root to avoid the usage of deep search.
> I would implement the new feature if it is desired and if it makes sense for
> the project.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] thenatog commented on pull request #6655: NIFI-10787 Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
thenatog commented on PR #6655: URL: https://github.com/apache/nifi/pull/6655#issuecomment-1317461633 After speaking with exceptionfactory, I have updated this PR with a different way of doing the unit test which does not require pom changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6649: NIFI-10785 Allow publishing AMQP message with null header value
nandorsoma commented on code in PR #6649:
URL: https://github.com/apache/nifi/pull/6649#discussion_r1024349399
##
nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/test/java/org/apache/nifi/amqp/processors/PublishAMQPTest.java:
##
@@ -134,14 +124,10 @@ public void
validateSuccessWithHeaderWithCommaPublishToSuccess() throws Exceptio
final Map headerMap = msg1.getProps().getHeaders();
-final Object foo = headerMap.get("foo");
-final Object foo2 = headerMap.get("foo2");
-final Object foo3 = headerMap.get("foo3");
-
-assertEquals("(bar,bar)", foo.toString());
-assertEquals("bar2", foo2.toString());
-assertNull(foo3);
-
+assertEquals(2, headerMap.size());
+assertEquals("(bar,bar)", headerMap.get("foo").toString());
Review Comment:
Good idea, thanks @thenatog! I added a minor modification, though. I'm
creating the maps locally because you cannot use the same map for both tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10826) Separate Compile and Verify Steps for GitHub Workflow
[
https://issues.apache.org/jira/browse/NIFI-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-10826:
Fix Version/s: 1.19.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Separate Compile and Verify Steps for GitHub Workflow
> -
>
> Key: NIFI-10826
> URL: https://issues.apache.org/jira/browse/NIFI-10826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.19.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The current GitHub ci-workflow configuration runs a single Maven {{verify}}
> command to run through the standard lifecycle phases for building the
> project. Separating initial code compilation from unit testing will provide a
> clearer indication of the time for code compilation and the time for unit
> testing. Unit tests can be more prone to errors when running in parallel, but
> running the {{compile}} command with multiple threads should be more reliable
> and provide some performance optimization for runners with multiple cores.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10826) Separate Compile and Verify Steps for GitHub Workflow
[
https://issues.apache.org/jira/browse/NIFI-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634958#comment-17634958
]
ASF subversion and git services commented on NIFI-10826:
Commit 9e3987d04e78148a36c46f42a451290fe5813061 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9e3987d04e ]
NIFI-10826 Separated Maven Compile and Verify in ci-workflow
Signed-off-by: Matthew Burgess
This closes #
> Separate Compile and Verify Steps for GitHub Workflow
> -
>
> Key: NIFI-10826
> URL: https://issues.apache.org/jira/browse/NIFI-10826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The current GitHub ci-workflow configuration runs a single Maven {{verify}}
> command to run through the standard lifecycle phases for building the
> project. Separating initial code compilation from unit testing will provide a
> clearer indication of the time for code compilation and the time for unit
> testing. Unit tests can be more prone to errors when running in parallel, but
> running the {{compile}} command with multiple threads should be more reliable
> and provide some performance optimization for runners with multiple cores.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] mattyb149 closed pull request #6666: NIFI-10826 Separate Maven Compile and Verify in ci-workflow
mattyb149 closed pull request #: NIFI-10826 Separate Maven Compile and Verify in ci-workflow URL: https://github.com/apache/nifi/pull/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #6666: NIFI-10826 Separate Maven Compile and Verify in ci-workflow
mattyb149 commented on PR #: URL: https://github.com/apache/nifi/pull/#issuecomment-1317428637 +1 LGTM and all checks pass, thanks for the improvement! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7392) Create a ValidateJSON Processor
[ https://issues.apache.org/jira/browse/NIFI-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634945#comment-17634945 ] ASF subversion and git services commented on NIFI-7392: --- Commit bd89d66539f5ea83e8c8d02449f9da3ec4c156be in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bd89d66539 ] NIFI-7392 Corrected TestValidationJson for Windows build > Create a ValidateJSON Processor > --- > > Key: NIFI-7392 > URL: https://issues.apache.org/jira/browse/NIFI-7392 > Project: Apache NiFi > Issue Type: New Feature >Reporter: r65535 >Assignee: r65535 >Priority: Minor > Fix For: 1.19.0 > > Time Spent: 8h 40m > Remaining Estimate: 0h > > I've written a ValidateJson processor which uses JSON schemas to validate > FlowFiles (spec here: [http://json-schema.org/]) > This processor ensures the content isn't modified - and only routed. The > ValidateRecord processor rewrites the content. > The JSON schema spec provide several nice to have features that avro schemas > don't support - like pattern matching values. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10188) ConsumeEWS java.lang.ClassNotFoundException: javax.xml.ws.http.HTTPException
[ https://issues.apache.org/jira/browse/NIFI-10188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634928#comment-17634928 ] dataminer.accolade commented on NIFI-10188: --- I had the same problem with version 1.18.0 > ConsumeEWS java.lang.ClassNotFoundException: javax.xml.ws.http.HTTPException > - > > Key: NIFI-10188 > URL: https://issues.apache.org/jira/browse/NIFI-10188 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.2 > Environment: Java 11 > Cent OS 7 >Reporter: RAVINARAYAN SINGH >Priority: Major > > I was trying to use ConsumeEWS processor with self hosted exchange server > and i got following error > ConsumeEWS[id=327442d0-569a-1856--cabbf76f] Processing halted: > yielding [1 sec]: java.lang.NoClassDefFoundError: > javax/xml/ws/http/HTTPException > - Caused by: java.lang.ClassNotFoundException: javax.xml.ws.http.HTTPException -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10829) Adding the parent folder name as an argument to the ConsumeEWS processor
dataminer.accolade created NIFI-10829: - Summary: Adding the parent folder name as an argument to the ConsumeEWS processor Key: NIFI-10829 URL: https://issues.apache.org/jira/browse/NIFI-10829 Project: Apache NiFi Issue Type: New Feature Components: Core Framework Reporter: dataminer.accolade Attachments: image-2022-11-16-17-32-21-819.png I want to be able to pass the root folder as an argument to the ConsumeEWS processor. [Source|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ConsumeEWS.java#L393] !image-2022-11-16-17-32-21-819.png! Primarily I am interested in [PublicFoldersRoot|https://learn.microsoft.com/en-us/dotnet/api/microsoft.exchange.webservices.data.wellknownfoldername?view=exchange-ews-api] making PublicFoldersRoot selectable will require the implementation of the deep search because, [deep traversal queries cannot be performed on public folders.|https://learn.microsoft.com/en-us/exchange/client-developer/web-service-reference/findfolder-operation?redirectedfrom=MSDN] Alternatively it would be nice to pass the path of the folder instead of a folder name with a specific root to avoid the usage of deep search. I would implement the new feature if it is desired and if it makes sense for the project. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1447: MINIFICPP-1976 - Do not store a whole Relationship object for each transfer
adamdebreceni commented on code in PR #1447:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1447#discussion_r1024230985
##
libminifi/include/core/ProcessSession.h:
##
@@ -155,16 +156,25 @@ class ProcessSession : public ReferenceContainer {
std::shared_ptr snapshot;
};
+ using Relationships = utils::ValueIdProvider;
+
+ Relationships relationships_;
Review Comment:
removed `ValueCompressor`, using a pointer into a cache set
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (MINIFICPP-1988) Reduce the usage of shared_ptr and shared ownership in FlowController, Repositories, and global services
Marton Szasz created MINIFICPP-1988: --- Summary: Reduce the usage of shared_ptr and shared ownership in FlowController, Repositories, and global services Key: MINIFICPP-1988 URL: https://issues.apache.org/jira/browse/MINIFICPP-1988 Project: Apache NiFi MiNiFi C++ Issue Type: Epic Reporter: Marton Szasz - FlowController - C2 client - repositories - thread pool and scheduling? - flow files - processors and connections - other components -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10763) Update mysql-binlog-connector-java to 0.27.5
[ https://issues.apache.org/jira/browse/NIFI-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10763. - Fix Version/s: 1.19.0 Resolution: Fixed > Update mysql-binlog-connector-java to 0.27.5 > > > Key: NIFI-10763 > URL: https://issues.apache.org/jira/browse/NIFI-10763 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 1.19.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Update mysql-binlog-connector-java to 0.27.5 from 0.26.1 to ensure > dependencies are updated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10763) Update mysql-binlog-connector-java to 0.27.5
[ https://issues.apache.org/jira/browse/NIFI-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10763: Labels: dependency-upgrade (was: ) > Update mysql-binlog-connector-java to 0.27.5 > > > Key: NIFI-10763 > URL: https://issues.apache.org/jira/browse/NIFI-10763 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.19.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Update mysql-binlog-connector-java to 0.27.5 from 0.26.1 to ensure > dependencies are updated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6667: NIFI-10763 Update mysql-binlog-connector-java To 0.27.5
exceptionfactory closed pull request #6667: NIFI-10763 Update mysql-binlog-connector-java To 0.27.5 URL: https://github.com/apache/nifi/pull/6667 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10763) Update mysql-binlog-connector-java to 0.27.5
[ https://issues.apache.org/jira/browse/NIFI-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634918#comment-17634918 ] ASF subversion and git services commented on NIFI-10763: Commit 83ff168ac151cfe73039ea073c738bccce1f1642 in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=83ff168ac1 ] NIFI-10763 Upgraded mysql-binlog-connector-java from 0.26.1 to 0.27.5 This closes #6667 Signed-off-by: David Handermann > Update mysql-binlog-connector-java to 0.27.5 > > > Key: NIFI-10763 > URL: https://issues.apache.org/jira/browse/NIFI-10763 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Update mysql-binlog-connector-java to 0.27.5 from 0.26.1 to ensure > dependencies are updated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 commented on pull request #6337: NIFI-7392: Add ValidateJson processor to standard bundle
dan-s1 commented on PR #6337: URL: https://github.com/apache/nifi/pull/6337#issuecomment-1317276694 @exceptionfactory Thanks again for all the suggestions you made for reviewing and approving! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-1893) Add processor for validating JSON
[
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-1893:
---
Priority: Minor (was: Major)
> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Minor
> Attachments: image-2018-12-07-18-02-52-813.png
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
> name: "Test",
> timestamp: 1463499695,
> tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
> },
> fields: {
> "cpu": 10.2,
> "load": 15.6
> }
> }
> Could be validated successfully against the following "schema":
> {
> "type": "object",
> "required": ["name", "tags", "timestamp", "fields"],
> "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
> }
> }
> There is at least one ASF-friendly library that could be used for
> implementation: https://github.com/everit-org/json-schema
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-1893) Add processor for validating JSON
[
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-1893.
Fix Version/s: 1.19.0
Resolution: Duplicate
> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Minor
> Fix For: 1.19.0
>
> Attachments: image-2018-12-07-18-02-52-813.png
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
> name: "Test",
> timestamp: 1463499695,
> tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
> },
> fields: {
> "cpu": 10.2,
> "load": 15.6
> }
> }
> Could be validated successfully against the following "schema":
> {
> "type": "object",
> "required": ["name", "tags", "timestamp", "fields"],
> "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
> }
> }
> There is at least one ASF-friendly library that could be used for
> implementation: https://github.com/everit-org/json-schema
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-8179) Standalone JSON validator processor
[ https://issues.apache.org/jira/browse/NIFI-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8179: --- Priority: Minor (was: Major) > Standalone JSON validator processor > --- > > Key: NIFI-8179 > URL: https://issues.apache.org/jira/browse/NIFI-8179 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: John Carole >Priority: Minor > Fix For: 1.19.0 > > > A standalone processor that reads a json, makes sure its a valid json and > then reroute the flow file to success or failure based on the validation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-8179) Standalone JSON validator processor
[ https://issues.apache.org/jira/browse/NIFI-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8179: --- Summary: Standalone JSON validator processor (was: NiFi standalon json validator processor) > Standalone JSON validator processor > --- > > Key: NIFI-8179 > URL: https://issues.apache.org/jira/browse/NIFI-8179 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: John Carole >Priority: Major > Fix For: 1.19.0 > > > A standalone processor that reads a json, makes sure its a valid json and > then reroute the flow file to success or failure based on the validation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-8179) NiFi standalon json validator processor
[ https://issues.apache.org/jira/browse/NIFI-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-8179. Fix Version/s: 1.19.0 Resolution: Fixed > NiFi standalon json validator processor > --- > > Key: NIFI-8179 > URL: https://issues.apache.org/jira/browse/NIFI-8179 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: John Carole >Priority: Major > Fix For: 1.19.0 > > > A standalone processor that reads a json, makes sure its a valid json and > then reroute the flow file to success or failure based on the validation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-7392) Create a ValidateJSON Processor
[ https://issues.apache.org/jira/browse/NIFI-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-7392. Fix Version/s: 1.19.0 Resolution: Fixed > Create a ValidateJSON Processor > --- > > Key: NIFI-7392 > URL: https://issues.apache.org/jira/browse/NIFI-7392 > Project: Apache NiFi > Issue Type: New Feature >Reporter: r65535 >Assignee: r65535 >Priority: Minor > Fix For: 1.19.0 > > Time Spent: 8.5h > Remaining Estimate: 0h > > I've written a ValidateJson processor which uses JSON schemas to validate > FlowFiles (spec here: [http://json-schema.org/]) > This processor ensures the content isn't modified - and only routed. The > ValidateRecord processor rewrites the content. > The JSON schema spec provide several nice to have features that avro schemas > don't support - like pattern matching values. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-7392) Create a ValidateJSON Processor
[ https://issues.apache.org/jira/browse/NIFI-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-7392: --- Issue Type: New Feature (was: Improvement) > Create a ValidateJSON Processor > --- > > Key: NIFI-7392 > URL: https://issues.apache.org/jira/browse/NIFI-7392 > Project: Apache NiFi > Issue Type: New Feature >Reporter: r65535 >Assignee: r65535 >Priority: Minor > Time Spent: 8.5h > Remaining Estimate: 0h > > I've written a ValidateJson processor which uses JSON schemas to validate > FlowFiles (spec here: [http://json-schema.org/]) > This processor ensures the content isn't modified - and only routed. The > ValidateRecord processor rewrites the content. > The JSON schema spec provide several nice to have features that avro schemas > don't support - like pattern matching values. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-7392) Create a ValidateJSON Processor
[ https://issues.apache.org/jira/browse/NIFI-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634906#comment-17634906 ] ASF subversion and git services commented on NIFI-7392: --- Commit a702a8c2154e76054002643939f40f5db6112eff in nifi's branch refs/heads/main from r65535 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a702a8c215 ] NIFI-7392 Added ValidateJson to Standard Processors This closes #6337 Co-authored-by: dan-s1 Co-authored-by: r65535 Signed-off-by: David Handermann > Create a ValidateJSON Processor > --- > > Key: NIFI-7392 > URL: https://issues.apache.org/jira/browse/NIFI-7392 > Project: Apache NiFi > Issue Type: Improvement >Reporter: r65535 >Assignee: r65535 >Priority: Minor > Time Spent: 8.5h > Remaining Estimate: 0h > > I've written a ValidateJson processor which uses JSON schemas to validate > FlowFiles (spec here: [http://json-schema.org/]) > This processor ensures the content isn't modified - and only routed. The > ValidateRecord processor rewrites the content. > The JSON schema spec provide several nice to have features that avro schemas > don't support - like pattern matching values. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6337: NIFI-7392: Add ValidateJson processor to standard bundle
exceptionfactory closed pull request #6337: NIFI-7392: Add ValidateJson processor to standard bundle URL: https://github.com/apache/nifi/pull/6337 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] r65535 commented on pull request #6337: NIFI-7392: Add ValidateJson processor to standard bundle
r65535 commented on PR #6337: URL: https://github.com/apache/nifi/pull/6337#issuecomment-1317257062 Thanks for helping us get this over the line @exceptionfactory! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1981) Decrease C2 heartbeat frequency
[ https://issues.apache.org/jira/browse/MINIFICPP-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1981. - Resolution: Fixed > Decrease C2 heartbeat frequency > --- > > Key: MINIFICPP-1981 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1981 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.13.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The default 250 milliseconds heartbeat interval is unnecessarily frequent, > and places a large burden on the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1978) MergeContent should flush bins even when they don't exactly reach the max size.
[ https://issues.apache.org/jira/browse/MINIFICPP-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1978: Resolution: Fixed Status: Resolved (was: Patch Available) > MergeContent should flush bins even when they don't exactly reach the max > size. > --- > > Key: MINIFICPP-1978 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1978 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Adam Debreceni >Priority: Major > Fix For: 0.13.0 > > Time Spent: 1h > Remaining Estimate: 0h > > It looks like MergeContent will fill up Bins until they reach the max group > size, and reject further flow files when that would push the bin over the max > size. If the last flow file doesn't exactly reach the max group size, the bin > is not flushed, but it also cannot accept new flow files. We should come up > with a way to flush "almost full" bins in a timely manner. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] sedadgn commented on a diff in pull request #6655: NIFI-10787 Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
sedadgn commented on code in PR #6655:
URL: https://github.com/apache/nifi/pull/6655#discussion_r1024192634
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/registry/flow/mapping/NiFiRegistryFlowMapper.java:
##
@@ -193,7 +193,7 @@ public InstantiatedVersionedProcessGroup
mapProcessGroup(final ProcessGroup grou
// This is specific for the {@code NifiRegistryFlowRegistryClient}, purely
for backward compatibility
private String getRegistryUrl(final FlowRegistryClientNode registry) {
-return
registry.getComponentType().equals("org.apache.nifi.registry.flow.NifiRegistryFlowRegistryClient")
? registry.getRawPropertyValue(registry.getPropertyDescriptor("URL")) : "";
+return
registry.getComponentType().equals("org.apache.nifi.registry.flow.NifiRegistryFlowRegistryClient")
? registry.getRawPropertyValue(registry.getPropertyDescriptor("url")) : "";
Review Comment:
Hello, I added new constant in this class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-10828) Adding REST endpoint for NiFi for providing more information on versioned flows
Simon Bence created NIFI-10828: -- Summary: Adding REST endpoint for NiFi for providing more information on versioned flows Key: NIFI-10828 URL: https://issues.apache.org/jira/browse/NIFI-10828 Project: Apache NiFi Issue Type: Improvement Reporter: Simon Bence Assignee: Simon Bence The FlowRegistryClient is capable to serve more information about a flow than the listing of the flows in the bucket providing. This capability is not exposed however, not part of the NiFi REST API. I would think, this additional information would open a range of possibilities for the UI to provide a more satisfying user experience. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 commented on pull request #6667: NiFi-10763 Update mysql-binlog-connector-java To 0.27.5
mr1716 commented on PR #6667: URL: https://github.com/apache/nifi/pull/6667#issuecomment-1317164008 All of the checks including JDK 11 were successful in my local branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6337: NIFI-7392: Add ValidateJson processor to standard bundle
exceptionfactory commented on PR #6337: URL: https://github.com/apache/nifi/pull/6337#issuecomment-1317125619 Thanks for the work on this @r65535, and thanks for the updates @dan-s1. I will take a look at the updates today, it appears close to completion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10827) OIDC Logout Doesn't Work
[ https://issues.apache.org/jira/browse/NIFI-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10827. - Fix Version/s: 1.19.0 Assignee: Nathan Gough Resolution: Fixed Thanks for reporting this issue [~ruckc]. NIFI-10332 describes the same issue, and was just recently fixed, so the resolution will be included in upcoming releases. > OIDC Logout Doesn't Work > > > Key: NIFI-10827 > URL: https://issues.apache.org/jira/browse/NIFI-10827 > Project: Apache NiFi > Issue Type: Bug > Components: Security >Affects Versions: 1.18.0 >Reporter: Curtis W Ruck >Assignee: Nathan Gough >Priority: Major > Fix For: 1.19.0 > > Attachments: openid-configuration.json > > > With keycloak 20, nifi 1.18.0, logouts don't succeed. This results in a > failure to logout. > The keycloak error is `Missing parameters: id_token_hint` and it appears to > not be getting the EndSessionEndpoint in > [OIDCAccessResource:360-370|https://github.com/apache/nifi/blob/rel/nifi-1.18.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/OIDCAccessResource.java#L360-L370]. > The attached configuration is what is provided to nifi. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a diff in pull request #1447: MINIFICPP-1976 - Do not store a whole Relationship object for each transfer
adam-markovics commented on code in PR #1447:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1447#discussion_r1024083843
##
libminifi/include/core/ProcessSession.h:
##
@@ -155,16 +156,25 @@ class ProcessSession : public ReferenceContainer {
std::shared_ptr snapshot;
};
+ using Relationships = utils::ValueIdProvider;
+
+ Relationships relationships_;
Review Comment:
We only instantiate `ValueCompressor` with id being 8 bytes, so this
doesn't apply now. Also this member should be private as no subclasses use it,
so invalid pointer dereferencing is not possible (by someone else) if usage is
consistent within this class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] KalmanJantner commented on a diff in pull request #6589: NIFI-10710 implement processor for AWS Polly, Textract, Translate, Tr…
KalmanJantner commented on code in PR #6589:
URL: https://github.com/apache/nifi/pull/6589#discussion_r1023592279
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/pom.xml:
##
@@ -117,6 +117,26 @@
1.19.0-SNAPSHOT
provided
+
+com.amazonaws
+aws-java-sdk-translate
+1.12.328
Review Comment:
Fixed thanks.
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/ml/AwsMLJobStatusGetter.java:
##
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.aws.ml;
+
+import com.amazonaws.AmazonWebServiceClient;
+import com.amazonaws.ClientConfiguration;
+import com.amazonaws.ResponseMetadata;
+import com.amazonaws.auth.AWSCredentials;
+import com.amazonaws.http.SdkHttpMetadata;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.module.SimpleModule;
+import java.io.BufferedWriter;
+import java.io.OutputStreamWriter;
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processors.aws.AbstractAWSCredentialsProviderProcessor;
+
+abstract public class AwsMLJobStatusGetter
+extends AbstractAWSCredentialsProviderProcessor {
Review Comment:
Updated, thanks.
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/ml/AwsMLJobStatusGetter.java:
##
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.aws.ml;
+
+import com.amazonaws.AmazonWebServiceClient;
+import com.amazonaws.ClientConfiguration;
+import com.amazonaws.ResponseMetadata;
+import com.amazonaws.auth.AWSCredentials;
+import com.amazonaws.http.SdkHttpMetadata;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.module.SimpleModule;
+import java.io.BufferedWriter;
+import java.io.OutputStreamWriter;
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processors.aws.AbstractAWSCredentialsProviderProcessor;
+
+abstract public class AwsMLJobStatusGetter
Review Comment:
Updated thanks.
##
[jira] [Created] (NIFI-10827) OIDC Logout Doesn't Work
Curtis W Ruck created NIFI-10827: Summary: OIDC Logout Doesn't Work Key: NIFI-10827 URL: https://issues.apache.org/jira/browse/NIFI-10827 Project: Apache NiFi Issue Type: Bug Components: Security Affects Versions: 1.18.0 Reporter: Curtis W Ruck Attachments: openid-configuration.json With keycloak 20, nifi 1.18.0, logouts don't succeed. This results in a failure to logout. The keycloak error is `Missing parameters: id_token_hint` and it appears to not be getting the EndSessionEndpoint in [OIDCAccessResource:360-370|https://github.com/apache/nifi/blob/rel/nifi-1.18.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/OIDCAccessResource.java#L360-L370]. The attached configuration is what is provided to nifi. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-1987) Configuring a metrics for Processors for C2 Protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634848#comment-17634848 ] Gábor Gyimesi commented on MINIFICPP-1987: -- We could implement a solution to use a regex filter in the C2 properties for defining which processor metrics we want to publish. The filter would something look like: processors/.* just to specify that this filter refers only to the processor metrics. > Configuring a metrics for Processors for C2 Protocol > > > Key: MINIFICPP-1987 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1987 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Priority: Major > Fix For: 0.13.0 > > > Hello. > I tried to setup receiving metrics using the C2 protocol for my processors. > The documentation describes that you must explicitly set the metric classes, > for example like this > nifi.c2.root.class.definitions.metrics.metrics.processorMetrics.classes=ListenHTTPMetrics,PostElasticsearchMetrics,InvokeHTTPMetrics,BinFilesMetrics,MergeContentMetrics... > It turns out a complex configuration, if I want to receive data on all my > processors, I need to update the list of these classes every time if I add a > new or remove type of processor. > If I try to configure classes for all possible processors, then I get 50+ > errors in the logs, , because not all processor types are involved, for > example > "No metric defined for PublishKafkaMetrics" > "No metric defined for ConsumeKafkaMetrics" > etc... > It seems to me that it is worth making one universal metric class for all > processes, or lower the error logging level when searching for a metric class > to warning -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10763) Update mysql-binlog-connector-java to 0.27.5
[ https://issues.apache.org/jira/browse/NIFI-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R updated NIFI-10763: -- Priority: Minor (was: Major) > Update mysql-binlog-connector-java to 0.27.5 > > > Key: NIFI-10763 > URL: https://issues.apache.org/jira/browse/NIFI-10763 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > > Update mysql-binlog-connector-java to 0.27.5 from 0.26.1 to ensure > dependencies are updated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 opened a new pull request, #6667: NiFi-10763 Update mysql-binlog-connector-java To 0.27.5
mr1716 opened a new pull request, #6667: URL: https://github.com/apache/nifi/pull/6667 # Summary [NIFI-10763](https://issues.apache.org/jira/browse/NIFI-10763) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-10763) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-10763) Update mysql-binlog-connector-java to 0.27.5
[ https://issues.apache.org/jira/browse/NIFI-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-10763: - Assignee: Mike R > Update mysql-binlog-connector-java to 0.27.5 > > > Key: NIFI-10763 > URL: https://issues.apache.org/jira/browse/NIFI-10763 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update mysql-binlog-connector-java to 0.27.5 from 0.26.1 to ensure > dependencies are updated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1987) Configuring a metrics for Processors for C2 Protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kondakov Artem updated MINIFICPP-1987: -- Summary: Configuring a metrics for Processors for C2 Protocol (was: Metrics for Processors via C2 Protocol) > Configuring a metrics for Processors for C2 Protocol > > > Key: MINIFICPP-1987 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1987 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.12.0 >Reporter: Kondakov Artem >Priority: Major > Fix For: 0.13.0 > > > Hello. > I tried to setup receiving metrics using the C2 protocol for my processors. > The documentation describes that you must explicitly set the metric classes, > for example like this > nifi.c2.root.class.definitions.metrics.metrics.processorMetrics.classes=ListenHTTPMetrics,PostElasticsearchMetrics,InvokeHTTPMetrics,BinFilesMetrics,MergeContentMetrics... > It turns out a complex configuration, if I want to receive data on all my > processors, I need to update the list of these classes every time if I add a > new or remove type of processor. > If I try to configure classes for all possible processors, then I get 50+ > errors in the logs, , because not all processor types are involved, for > example > "No metric defined for PublishKafkaMetrics" > "No metric defined for ConsumeKafkaMetrics" > etc... > It seems to me that it is worth making one universal metric class for all > processes, or lower the error logging level when searching for a metric class > to warning -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-1987) Metrics for Processors via C2 Protocol
Kondakov Artem created MINIFICPP-1987: - Summary: Metrics for Processors via C2 Protocol Key: MINIFICPP-1987 URL: https://issues.apache.org/jira/browse/MINIFICPP-1987 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.12.0 Reporter: Kondakov Artem Fix For: 0.13.0 Hello. I tried to setup receiving metrics using the C2 protocol for my processors. The documentation describes that you must explicitly set the metric classes, for example like this nifi.c2.root.class.definitions.metrics.metrics.processorMetrics.classes=ListenHTTPMetrics,PostElasticsearchMetrics,InvokeHTTPMetrics,BinFilesMetrics,MergeContentMetrics... It turns out a complex configuration, if I want to receive data on all my processors, I need to update the list of these classes every time if I add a new or remove type of processor. If I try to configure classes for all possible processors, then I get 50+ errors in the logs, , because not all processor types are involved, for example "No metric defined for PublishKafkaMetrics" "No metric defined for ConsumeKafkaMetrics" etc... It seems to me that it is worth making one universal metric class for all processes, or lower the error logging level when searching for a metric class to warning -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] r65535 commented on pull request #6337: NIFI-7392: Add ValidateJson processor to standard bundle
r65535 commented on PR #6337: URL: https://github.com/apache/nifi/pull/6337#issuecomment-1316944568 @dan-s1 - thanks for making the changes to this PR! 拾 @exceptionfactory - do you think we could get this merged in for the 1.19.0 release? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1451: MINIFICPP-1962 Implement communication between process groups through ports
adamdebreceni commented on code in PR #1451:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1451#discussion_r1023930246
##
extensions/standard-processors/tests/unit/YamlProcessGroupParserTests.cpp:
##
@@ -102,30 +102,190 @@ TEST_CASE("Cannot connect processors from different
groups", "[YamlProcessGroupP
}
SECTION("Connecting processors in their child/parent group") {
-Conn1.source = UnresolvedProc{Child1_Proc1.id};
-Conn1.destination = UnresolvedProc{Child1_Port1.id};
+Conn1.source = Proc{Child1_Proc1.id, Child1_Proc1.name,
ConnectionFailure::UNRESOLVED_SOURCE};
+Conn1.destination = Proc{Child1_Port1.id, Child1_Port1.name,
ConnectionFailure::UNRESOLVED_DESTINATION};
-Child1_Conn1.source = UnresolvedProc{Proc1.id};
-Child1_Conn1.destination = UnresolvedProc{Port1.id};
+Child1_Conn1.source = Proc{Proc1.id, Proc1.name,
ConnectionFailure::UNRESOLVED_SOURCE};
+Child1_Conn1.destination = Proc{Port1.id, Proc1.name,
ConnectionFailure::UNRESOLVED_DESTINATION};
}
SECTION("Connecting processors between their own and their child/parent
group") {
Conn1.source = Proc1;
-Conn1.destination = UnresolvedProc{Child1_Port1.id};
+Conn1.destination = Proc{Child1_Port1.id, Child1_Port1.name,
ConnectionFailure::UNRESOLVED_DESTINATION};
-Child1_Conn1.source = UnresolvedProc{Port1.id};
+Child1_Conn1.source = Proc{Port1.id, Port1.name,
ConnectionFailure::UNRESOLVED_SOURCE};
Child1_Conn1.destination = Child1_Proc1;
}
SECTION("Connecting processors in a sibling group") {
Conn1.source = Proc1;
Conn1.destination = Port1;
-Child1_Conn1.source = UnresolvedProc{Child2_Proc1.id};
-Child1_Conn1.destination = UnresolvedProc{Child2_Port1.id};
+Child1_Conn1.source = Proc{Child2_Proc1.id, Child2_Proc1.name,
ConnectionFailure::UNRESOLVED_SOURCE};
+Child1_Conn1.destination = Proc{Child2_Port1.id, Child2_Port1.name,
ConnectionFailure::UNRESOLVED_DESTINATION};
}
auto root = config.getRootFromPayload(pattern.serialize().join("\n"));
verifyProcessGroup(*root, pattern);
}
+
+TEST_CASE("Processor can communicate with root process group's input port",
"[YamlProcessGroupParser4]") {
+ auto pattern = Group("root")
+.With({Conn{"Conn1",
+Proc{"----0001", "Proc1"},
+InputPort{"----0002", "Port1"}}})
+.With({Proc{"----0001", "Proc1"}})
+.With({
+ Group("Child1")
+ .With({InputPort{"----0002", "Port1"}})
Review Comment:
isn't this a child group's port, not the root process group's port?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] michael81877 commented on pull request #6560: NIFI-10676 USE_SPECIFIED_OR_COMPATIBLE_OR_GHOST on flow load from bytes
michael81877 commented on PR #6560: URL: https://github.com/apache/nifi/pull/6560#issuecomment-1316776212 @markap14 : Do you see any other changes needed for this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10821) Upgrade Netty to 4.1.85
[ https://issues.apache.org/jira/browse/NIFI-10821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-10821: -- Fix Version/s: 1.19.0 > Upgrade Netty to 4.1.85 > --- > > Key: NIFI-10821 > URL: https://issues.apache.org/jira/browse/NIFI-10821 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.19.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty [4.1.85|https://netty.io/news/2022/11/09/4-1-85-Final.html] includes > several minor bug fixes and improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10821) Upgrade Netty to 4.1.85
[ https://issues.apache.org/jira/browse/NIFI-10821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-10821: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Netty to 4.1.85 > --- > > Key: NIFI-10821 > URL: https://issues.apache.org/jira/browse/NIFI-10821 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 20m > Remaining Estimate: 0h > > Netty [4.1.85|https://netty.io/news/2022/11/09/4-1-85-Final.html] includes > several minor bug fixes and improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10821) Upgrade Netty to 4.1.85
[ https://issues.apache.org/jira/browse/NIFI-10821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634756#comment-17634756 ] ASF subversion and git services commented on NIFI-10821: Commit 01351911067c12b550776dc0c4552b51db7a5637 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0135191106 ] NIFI-10821 Upgraded Netty from 4.1.84 to 4.1.85 Signed-off-by: Pierre Villard This closes #6665. > Upgrade Netty to 4.1.85 > --- > > Key: NIFI-10821 > URL: https://issues.apache.org/jira/browse/NIFI-10821 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 10m > Remaining Estimate: 0h > > Netty [4.1.85|https://netty.io/news/2022/11/09/4-1-85-Final.html] includes > several minor bug fixes and improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
