[jira] [Updated] (NIFI-10891) TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sopan Phaltankar updated NIFI-10891:
Description:
{code:java}
org.apache.nifi.processors.standard.TestMergeContent.testDefragmentMultipleMingledSegments{code}
This is a flaky test, it can pass mvn test while but when run using the tool
[NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex is
a tool that will introduce non-determinism in certain java collections.
The test shows below:
{code:java}
[ERROR] Failures:
[ERROR] TestMergeContent.testDefragmentMultipleMingledSegments:1028 FlowFile
content differs from input at byte 0 with input having value 65 and FlowFile
having value 78
{code}
*Steps to reproduce the failure:*
# Run the following command in nifi:
# First, build the module:
{noformat}
mvn clean install -pl
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -DskipTests
-Drat.skip -am {noformat}
# Then run the test using
[NonDex|https://github.com/TestingResearchIllinois/NonDex]
{noformat}
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
-Dtest=org.apache.nifi.processors.standard.TestMergeContent#testDefragmentMultipleMingledSegments{noformat}
The result will be saved under the module folder in .nondex
was:
{code:java}
org.apache.nifi.processors.standard.TestMergeContent{code}
This is a flaky test, it can pass mvn test while but when run using the tool
[NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex is
a tool that will introduce non-determinism in certain java collections.
The test shows below:
{code:java}
[ERROR] Failures:
[ERROR] TestMergeContent.testDefragmentMultipleMingledSegments:1028 FlowFile
content differs from input at byte 0 with input having value 65 and FlowFile
having value 78
{code}
*Steps to reproduce the failure:*
# Run the following command in nifi:
# First, build the module:
{noformat}
mvn clean install -pl
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -DskipTests
-Drat.skip -am {noformat}
# Then run the test using
[NonDex|https://github.com/TestingResearchIllinois/NonDex]
{noformat}
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
-Dtest=org.apache.nifi.processors.standard.TestMergeContent#testDefragmentMultipleMingledSegments{noformat}
The result will be saved under the module folder in .nondex
> TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic
> HashMap
> --
>
> Key: NIFI-10891
> URL: https://issues.apache.org/jira/browse/NIFI-10891
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Attachments: org.apache.nifi.processors.standard.TestMergeContent.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.standard.TestMergeContent.testDefragmentMultipleMingledSegments{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR] Failures:
> [ERROR] TestMergeContent.testDefragmentMultipleMingledSegments:1028
> FlowFile content differs from input at byte 0 with input having value 65 and
> FlowFile having value 78
> {code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn clean install -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -DskipTests
> -Drat.skip -am {noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> -Dtest=org.apache.nifi.processors.standard.TestMergeContent#testDefragmentMultipleMingledSegments{noformat}
> The result will be saved under the module folder in .nondex
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] greyp9 opened a new pull request, #6739: NIFI-10916 - Controller Service allowable values dropdown list should…
greyp9 opened a new pull request, #6739: URL: https://github.com/apache/nifi/pull/6739 … be sorted ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 8 - [x] JDK 11 - [x] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10909) Upgrade Jackson-databind to 2.14.1
[ https://issues.apache.org/jira/browse/NIFI-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10909: Priority: Minor (was: Major) > Upgrade Jackson-databind to 2.14.1 > -- > > Key: NIFI-10909 > URL: https://issues.apache.org/jira/browse/NIFI-10909 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > A new version of Jackson-databind has been released. [Jackson Release 2.14.1 > · FasterXML/jackson Wiki > (github.com)|https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1] > This is an update -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10909) Upgrade Jackson-databind to 2.14.1
[ https://issues.apache.org/jira/browse/NIFI-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10909: Labels: dependency-upgrade (was: ) Status: Patch Available (was: Open) > Upgrade Jackson-databind to 2.14.1 > -- > > Key: NIFI-10909 > URL: https://issues.apache.org/jira/browse/NIFI-10909 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 10m > Remaining Estimate: 0h > > A new version of Jackson-databind has been released. [Jackson Release 2.14.1 > · FasterXML/jackson Wiki > (github.com)|https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1] > This is an update -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10909) Upgrade Jackson-databind to 2.14.1
[ https://issues.apache.org/jira/browse/NIFI-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-10909: --- Assignee: David Handermann > Upgrade Jackson-databind to 2.14.1 > -- > > Key: NIFI-10909 > URL: https://issues.apache.org/jira/browse/NIFI-10909 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: David Handermann >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > A new version of Jackson-databind has been released. [Jackson Release 2.14.1 > · FasterXML/jackson Wiki > (github.com)|https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1] > This is an update -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6738: NIFI-10909 Upgrade Jackson from 2.14.0 to 2.14.1
exceptionfactory opened a new pull request, #6738: URL: https://github.com/apache/nifi/pull/6738 # Summary [NIFI-10909](https://issues.apache.org/jira/browse/NIFI-10909) Upgrades Jackson JSON dependencies from 2.14.0 to [2.14.1](https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1). Version 2.14.1 includes several minor bug fixes, including addressing a heap space usage issue with instances of ObjectMapper. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10917) Upgrade Spring Framework to 5.3.24 and Spring Security to 5.8.0
[ https://issues.apache.org/jira/browse/NIFI-10917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10917: Status: Patch Available (was: In Progress) > Upgrade Spring Framework to 5.3.24 and Spring Security to 5.8.0 > --- > > Key: NIFI-10917 > URL: https://issues.apache.org/jira/browse/NIFI-10917 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Spring Framework > [5.3.24|https://github.com/spring-projects/spring-framework/releases/tag/v5.3.24] > contains a small number of features and bug fixes over version 5.3.23. > Spring Security > [5.8.0|https://github.com/spring-projects/spring-security/releases/tag/5.8.0] > includes a variety of incremental feature improvements and transitive > dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6737: NIFI-10917 Upgrade Spring Framework from 5.3.23 to 5.3.24
exceptionfactory opened a new pull request, #6737: URL: https://github.com/apache/nifi/pull/6737 # Summary [NIFI-10917](https://issues.apache.org/jira/browse/NIFI-10917) Upgrades Spring Framework dependencies from 5.3.23 to [5.3.24](https://github.com/spring-projects/spring-framework/releases/tag/v5.3.24) and also upgrades the following: - Spring Security from 5.7.5 to [5.8.0](https://github.com/spring-projects/spring-security/releases/tag/5.8.0) - Spring Boot from 2.7.4 to [2.7.6](https://github.com/spring-projects/spring-boot/releases/tag/v2.7.6) for Registry # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10918) If NiFi Registry URL changes, flows containing inner versioned flows cannot be fetched
[
https://issues.apache.org/jira/browse/NIFI-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641582#comment-17641582
]
Mark Payne commented on NIFI-10918:
---
Addressed this issue by changing the logic over how to resolve a Registry
Client for a given internal versioned flow.
Previously, we asked the client if it was applicable for the Remote Flow's URL.
If not, we would not use the client.
We will now create a List of all Registry Clients, and sort them based on
whether or not the client reports being able to handle the Remote Flow's URL.
This will cause us to use the most appropriate client first, when applicable.
However, we will now attempt to fetch the flow from all clients, until one of
them succeeds.
This is acceptable because:
* We will only interact with Registry instances that are have been configured
for use
* Authorization will be performed against all requests
* We will only fetch the flow if it has the same Bucket ID and Flow ID. So
even if it is stored in a different registry, we should have no problem feeling
confident that it's the correct flow.
This also provides a few additional benefits, in that we no longer have to
rewrite the URLs of flows when any of the following occurs:
* The hostname of the registry is changes
* The port that the registry is listening on changes
* If replicating flows from one registry to another
> If NiFi Registry URL changes, flows containing inner versioned flows cannot
> be fetched
> --
>
> Key: NIFI-10918
> URL: https://issues.apache.org/jira/browse/NIFI-10918
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I was performing some testing of integration with NiFi Registry. I started
> with an insecure registry running on
> [http://localhost:18080|http://localhost:18080/]
> I created a simple dataflow, called 'Inner Flow'. I then versioned it.
> Afterwards I put its parent Process Group ('Outer Flow') under Version
> Control.
> Everything worked as expected. I then wanted to verify that things still
> worked when secured. I secured my registry, changing the URL to
> [https://localhost:18443|https://localhost:18443/]
> Unfortunately, this resulted in no longer being able to check out the flow
> 'Outer Flow'. It failed because when the flow was created, it stored the
> coordinates of 'Inner Flow' as [http://localhost:18080/] but now it can
> no longer find any Registry Client that can handle
> [http://localhost:18080/...] since the client has now been reconfigured to
> use the HTTPS based URL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] markap14 opened a new pull request, #6736: NIFI-10918: When fetching a flow from a Flow Registry, if it referenc…
markap14 opened a new pull request, #6736: URL: https://github.com/apache/nifi/pull/6736 …es any 'internal versioned flows' instead of requiring that we have a client configured for the appropriate URL, attempt to fetch the flow from each client. We will start with the clients that do report that they can handle the URL but will try others as well. As soon as we successfully fetch the flow, we stop. # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10918) If NiFi Registry URL changes, flows containing inner versioned flows cannot be fetched
Mark Payne created NIFI-10918:
-
Summary: If NiFi Registry URL changes, flows containing inner
versioned flows cannot be fetched
Key: NIFI-10918
URL: https://issues.apache.org/jira/browse/NIFI-10918
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 1.20.0, 1.19.1
I was performing some testing of integration with NiFi Registry. I started with
an insecure registry running on [http://localhost:18080|http://localhost:18080/]
I created a simple dataflow, called 'Inner Flow'. I then versioned it.
Afterwards I put its parent Process Group ('Outer Flow') under Version Control.
Everything worked as expected. I then wanted to verify that things still worked
when secured. I secured my registry, changing the URL to
[https://localhost:18443|https://localhost:18443/]
Unfortunately, this resulted in no longer being able to check out the flow
'Outer Flow'. It failed because when the flow was created, it stored the
coordinates of 'Inner Flow' as [http://localhost:18080/] but now it can no
longer find any Registry Client that can handle [http://localhost:18080/...]
since the client has now been reconfigured to use the HTTPS based URL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-10917) Upgrade Spring Framework to 5.3.24 and Spring Security to 5.8.0
David Handermann created NIFI-10917: --- Summary: Upgrade Spring Framework to 5.3.24 and Spring Security to 5.8.0 Key: NIFI-10917 URL: https://issues.apache.org/jira/browse/NIFI-10917 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, NiFi Registry Reporter: David Handermann Assignee: David Handermann Spring Framework [5.3.24|https://github.com/spring-projects/spring-framework/releases/tag/v5.3.24] contains a small number of features and bug fixes over version 5.3.23. Spring Security [5.8.0|https://github.com/spring-projects/spring-security/releases/tag/5.8.0] includes a variety of incremental feature improvements and transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10916) Controller Service allowable values dropdown list should be sorted
[ https://issues.apache.org/jira/browse/NIFI-10916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey updated NIFI-10916: - Description: When a processor (or controller service) configuration property allows for the selection of a controller service (via combobox), the available values should be sorted, to facilitate selection of the intended controller service. See attachment. > Controller Service allowable values dropdown list should be sorted > -- > > Key: NIFI-10916 > URL: https://issues.apache.org/jira/browse/NIFI-10916 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Attachments: NIFI-10916.png > > > When a processor (or controller service) configuration property allows for > the selection of a controller service (via combobox), the available values > should be sorted, to facilitate selection of the intended controller service. > See attachment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10916) Controller Service allowable values dropdown list should be sorted
[ https://issues.apache.org/jira/browse/NIFI-10916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey updated NIFI-10916: - Attachment: NIFI-10916.png > Controller Service allowable values dropdown list should be sorted > -- > > Key: NIFI-10916 > URL: https://issues.apache.org/jira/browse/NIFI-10916 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Attachments: NIFI-10916.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10916) Controller Service allowable values dropdown list should be sorted
Paul Grey created NIFI-10916: Summary: Controller Service allowable values dropdown list should be sorted Key: NIFI-10916 URL: https://issues.apache.org/jira/browse/NIFI-10916 Project: Apache NiFi Issue Type: Improvement Reporter: Paul Grey Assignee: Paul Grey -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10241) Add comments tooltip for Controller Services and Reporting Tasks
[ https://issues.apache.org/jira/browse/NIFI-10241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nissim Shiman updated NIFI-10241: - Description: Add tooltip enabled Comments icon for Controller Services and Reporting Tasks. This will alert nifi administrators that the controller service/reporting task has a special comment relating to it. This icon should be near the Usage icon that is visible when the list of available controller services/reporting tasks are displayed (i.e. Global Menu -> Controller Settings -> Management Controller Services and Global Menu -> Controller Settings -> Reporting Tasks) was: Currently, both Processors and Processor Groups have a tooltip enabled icon that alerts nifi graph users that a comment exists for the given Processor/Processor Group. These are seen at their respective bottom right corners Add similar functionality for Controller Services and Reporting Tasks as well. This icon should be next to the Usage (and Alert) icons that are visible when the list of available controller services/reporting tasks are displayed (i.e. Global Menu -> Controller Settings -> Management Controller Services and Global Menu -> Controller Settings -> Reporting Tasks) > Add comments tooltip for Controller Services and Reporting Tasks > > > Key: NIFI-10241 > URL: https://issues.apache.org/jira/browse/NIFI-10241 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > > Add tooltip enabled Comments icon for Controller Services and Reporting Tasks. > This will alert nifi administrators that the controller service/reporting > task has a special comment relating to it. > This icon should be near the Usage icon that is visible when the list of > available controller services/reporting tasks are displayed > (i.e. Global Menu -> Controller Settings -> Management Controller Services > and > Global Menu -> Controller Settings -> Reporting Tasks) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10908) Weird JDBC Connection Closed Error
[ https://issues.apache.org/jira/browse/NIFI-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-10908: --- Assignee: David Handermann > Weird JDBC Connection Closed Error > -- > > Key: NIFI-10908 > URL: https://issues.apache.org/jira/browse/NIFI-10908 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.3 >Reporter: Cemre Mengu >Assignee: David Handermann >Priority: Trivial > Attachments: image-2022-11-30-16-38-53-210.png, > image-2022-11-30-20-43-26-441.png > > > Using hikari connection pool, I am getting weird `connection is closed` error > for a flow during sql updates. There is nothing wrong with the data or query > as far as I can see so I am confused why I am getting this error. I have many > flows but the only one with the error is this one. > Any ideas what can be causing this error? > !image-2022-11-30-16-38-53-210.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10908) Weird JDBC Connection Closed Error
[ https://issues.apache.org/jira/browse/NIFI-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641557#comment-17641557 ] David Handermann commented on NIFI-10908: - Thanks for the positive feedback and description of the resolution [~cemremengu], glad to help! > Weird JDBC Connection Closed Error > -- > > Key: NIFI-10908 > URL: https://issues.apache.org/jira/browse/NIFI-10908 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.3 >Reporter: Cemre Mengu >Priority: Trivial > Attachments: image-2022-11-30-16-38-53-210.png, > image-2022-11-30-20-43-26-441.png > > > Using hikari connection pool, I am getting weird `connection is closed` error > for a flow during sql updates. There is nothing wrong with the data or query > as far as I can see so I am confused why I am getting this error. I have many > flows but the only one with the error is this one. > Any ideas what can be causing this error? > !image-2022-11-30-16-38-53-210.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10899) Apply SameSite Attribute to Cookies
[
https://issues.apache.org/jira/browse/NIFI-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10899:
Status: Patch Available (was: In Progress)
> Apply SameSite Attribute to Cookies
> ---
>
> Key: NIFI-10899
> URL: https://issues.apache.org/jira/browse/NIFI-10899
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Security
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The standard {{Authorization-Bearer}} cookie includes the
> [SameSite|https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite]
> attribute in the {{Set-Cookie}} response header, but other cookies for CSRF
> mitigation, logout processing, and external authentication service
> integration do not apply the attribute when setting cookies.
> The Java Servlet
> [Cookie|https://docs.oracle.com/javaee/7/api/javax/servlet/http/Cookie.html]
> does not support the {{SameSite}} attribute, but the NiFi
> {{StandardApplicationCookieService}} uses the Spring Response Cookie Builder,
> which supports the attribute and is capable of applying it to {{Set-Cookie}}
> headers. Direct use of the Java Servlet {{Cookie}} should be replaced with
> the implementation approach that supports setting the {{SameSite}} attribute
> to avoid warnings in modern browsers. In absence of the {{SameSite}}
> attribute, browsers default to {{{}Lax{}}}, but this can be changed to
> {{Strict}} in most cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6735: NIFI-10899 Add SameSite Policy to Application Cookies
exceptionfactory opened a new pull request, #6735: URL: https://github.com/apache/nifi/pull/6735 - Added __Secure prefix to Application Cookie Names # Summary [NIFI-10899](https://issues.apache.org/jira/browse/NIFI-10899) Adds the `SameSite` attribute to all application cookies. The current security configuration applies `SameSite=Strict` to the `__Secure-Authorization-Bearer` Cookie, but no other cookies have the `SameSite` policy applied. The changes implemented add a `SameSitePolicy` enumeration and update the `ApplicationCookieName` enumeration to indicate the policy that will be applied. The implementation applies a `SameSite` attribute to all application cookies to avoid browser warnings and fallback to implicit behavior. The `__Secure-Authorization-Bearer` and `__Secure-Request-Token` cookies are limited to first-party access, so both of those cookies have the `Strict` policy configured. The Logout, SAML, and OIDC cookies require a value of `None` for the `SameSite` attribute because authentication and logout involves redirects between NiFi and external identity providers. Additional changes include adding the `__Secure` [prefix](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#cookie_prefixes) to the the Logout, SAML, and OIDC cookie names . # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10908) Weird JDBC Connection Closed Error
[ https://issues.apache.org/jira/browse/NIFI-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cemre Mengu updated NIFI-10908: --- Priority: Trivial (was: Critical) > Weird JDBC Connection Closed Error > -- > > Key: NIFI-10908 > URL: https://issues.apache.org/jira/browse/NIFI-10908 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.3 >Reporter: Cemre Mengu >Priority: Trivial > Attachments: image-2022-11-30-16-38-53-210.png, > image-2022-11-30-20-43-26-441.png > > > Using hikari connection pool, I am getting weird `connection is closed` error > for a flow during sql updates. There is nothing wrong with the data or query > as far as I can see so I am confused why I am getting this error. I have many > flows but the only one with the error is this one. > Any ideas what can be causing this error? > !image-2022-11-30-16-38-53-210.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10908) Weird JDBC Connection Closed Error
[ https://issues.apache.org/jira/browse/NIFI-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cemre Mengu resolved NIFI-10908. Resolution: Invalid > Weird JDBC Connection Closed Error > -- > > Key: NIFI-10908 > URL: https://issues.apache.org/jira/browse/NIFI-10908 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.3 >Reporter: Cemre Mengu >Priority: Trivial > Attachments: image-2022-11-30-16-38-53-210.png, > image-2022-11-30-20-43-26-441.png > > > Using hikari connection pool, I am getting weird `connection is closed` error > for a flow during sql updates. There is nothing wrong with the data or query > as far as I can see so I am confused why I am getting this error. I have many > flows but the only one with the error is this one. > Any ideas what can be causing this error? > !image-2022-11-30-16-38-53-210.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10908) Weird JDBC Connection Closed Error
[ https://issues.apache.org/jira/browse/NIFI-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641545#comment-17641545 ] Cemre Mengu commented on NIFI-10908: Hi [~exceptionfactory] thanks for your quick response! After more research, I discovered that some of the data is coming in garbled and those characters can be a \0 sometimes which postgres does not like. Filtering those seems to solve the problem. Apologies for the noise. I would also like to take this opportunity to thank for your continuous hard work on NiFi! > Weird JDBC Connection Closed Error > -- > > Key: NIFI-10908 > URL: https://issues.apache.org/jira/browse/NIFI-10908 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.3 >Reporter: Cemre Mengu >Priority: Critical > Attachments: image-2022-11-30-16-38-53-210.png, > image-2022-11-30-20-43-26-441.png > > > Using hikari connection pool, I am getting weird `connection is closed` error > for a flow during sql updates. There is nothing wrong with the data or query > as far as I can see so I am confused why I am getting this error. I have many > flows but the only one with the error is this one. > Any ideas what can be causing this error? > !image-2022-11-30-16-38-53-210.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10891) TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641537#comment-17641537
]
ASF subversion and git services commented on NIFI-10891:
Commit 8c74f54b9d4eb6a41b47094b4eec3effbd62e452 in nifi's branch
refs/heads/support/nifi-1.19 from sopan98
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8c74f54b9d ]
NIFI-10891 Replaced HashMap with LinkedHashMap in BinFiles
- Resolved non-deterministic behavior in
TestMergeContent.testDefragmentMultipleMingledSegments
This closes #6727
Signed-off-by: David Handermann
> TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic
> HashMap
> --
>
> Key: NIFI-10891
> URL: https://issues.apache.org/jira/browse/NIFI-10891
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Attachments: org.apache.nifi.processors.standard.TestMergeContent.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.standard.TestMergeContent{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR] Failures:
> [ERROR] TestMergeContent.testDefragmentMultipleMingledSegments:1028
> FlowFile content differs from input at byte 0 with input having value 65 and
> FlowFile having value 78
> {code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn clean install -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -DskipTests
> -Drat.skip -am {noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> -Dtest=org.apache.nifi.processors.standard.TestMergeContent#testDefragmentMultipleMingledSegments{noformat}
> The result will be saved under the module folder in .nondex
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10910) Update amqp-client to 5.16.0
[ https://issues.apache.org/jira/browse/NIFI-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641539#comment-17641539 ] ASF subversion and git services commented on NIFI-10910: Commit 8e0eeded009e9899941de740f36a74f71709161e in nifi's branch refs/heads/support/nifi-1.19 from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8e0eeded00 ] NIFI-10910 Upgraded amqp-client from 5.15.0 to 5.16.0 This closes #6734 Signed-off-by: David Handermann > Update amqp-client to 5.16.0 > > > Key: NIFI-10910 > URL: https://issues.apache.org/jira/browse/NIFI-10910 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > Update ampq-client from 5.15.0 to 5.16.0. [Releases · > rabbitmq/rabbitmq-java-client > (github.com)|https://github.com/rabbitmq/rabbitmq-java-client/releases] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10859) Update metrics-jvm to 4.1.33
[ https://issues.apache.org/jira/browse/NIFI-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641533#comment-17641533 ] ASF subversion and git services commented on NIFI-10859: Commit ec813fc445ffc2d621287f54c99654c59b1f5dd5 in nifi's branch refs/heads/support/nifi-1.19 from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ec813fc445 ] NIFI-10859 Upgraded metrics-jvm from 4.1.2 to 4.1.33 This closes #6721 Signed-off-by: David Handermann > Update metrics-jvm to 4.1.33 > > > Key: NIFI-10859 > URL: https://issues.apache.org/jira/browse/NIFI-10859 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 1.20.0, 1.19.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Update metrics-jvm to 4.2.13 from 4.1.2. It can be found at > /nifi-commons/nifi-metrics/pom.xml > release notes https://github.com/dropwizard/metrics/releases -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10901) PublishKafka headers not sent in ProducerRecord
[ https://issues.apache.org/jira/browse/NIFI-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641540#comment-17641540 ] ASF subversion and git services commented on NIFI-10901: Commit 6808a5944d328305ba67babd3e86cfd5e783bcf5 in nifi's branch refs/heads/support/nifi-1.19 from greyp9 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6808a5944d ] NIFI-10901 - PublishKafka headers not sent in ProducerRecord (#6731) > PublishKafka headers not sent in ProducerRecord > --- > > Key: NIFI-10901 > URL: https://issues.apache.org/jira/browse/NIFI-10901 > Project: Apache NiFi > Issue Type: Bug >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10765) Add better error reporting for JASN1Reader
[ https://issues.apache.org/jira/browse/NIFI-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641538#comment-17641538 ] ASF subversion and git services commented on NIFI-10765: Commit c36c4c0bbce4d3de690b58611a979373ce3d6ff0 in nifi's branch refs/heads/support/nifi-1.19 from Tamas Palfy [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c36c4c0bbc ] NIFI-10765 Added better error reporting in JASN1Reader This closes #6703 Signed-off-by: David Handermann > Add better error reporting for JASN1Reader > -- > > Key: NIFI-10765 > URL: https://issues.apache.org/jira/browse/NIFI-10765 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The JASN1Reader relies on https://github.com/beanit/asn1bean to compile ASN > schema files and to deserialize ASN data files. > The schema compilation step can fail for many different reasons and the error > is not reported to the user properly (or in any way really) so understanding > and fixing errors in the schema file is very hard currently. > We should report meaningful error messages to the bulletin and/or the > controller service validation error indicator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-6428) CaptureChangeMySQL throws IOException with BIGIN event due to lingering 'inTransaction' instance variable
[ https://issues.apache.org/jira/browse/NIFI-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641522#comment-17641522 ] ASF subversion and git services commented on NIFI-6428: --- Commit 0a09aa57fd9d69626cc6b2a1ae24fb515f1d4944 in nifi's branch refs/heads/support/nifi-1.19 from sssqhai [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0a09aa57fd ] NIFI-6428 Add 'inTransaction' value to state,fix bug Signed-off-by: Matthew Burgess This closes #6036 > CaptureChangeMySQL throws IOException with BIGIN event due to lingering > 'inTransaction' instance variable > - > > Key: NIFI-6428 > URL: https://issues.apache.org/jira/browse/NIFI-6428 > Project: Apache NiFi > Issue Type: Bug >Reporter: Koji Kawamura >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Because of not clearing 'inTransaction' instance variable when the processor > stops, when a user clears processor state and restart it with a specific > initial binlog position, then if BEGIN event is received, the processor > throws an IOException. > The processor should reset 'inTransaction' to false, and also other instance > variables when it start with an empty processor state. > The issue was reported at NiFi user ML. > [https://mail-archives.apache.org/mod_mbox/nifi-users/201907.mbox/%3C2019070919385393266224%40geekplus.com.cn%3E] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10834) TestPutSplunkHTTP Test Failures on NonDex
[
https://issues.apache.org/jira/browse/NIFI-10834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641532#comment-17641532
]
ASF subversion and git services commented on NIFI-10834:
Commit 03933c64253536e9422444f6640233a67147f634 in nifi's branch
refs/heads/support/nifi-1.19 from priyanka-28
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=03933c6425 ]
NIFI-10834 Updated TestPutSplunkHTTP to avoid non-deterministic tests
This closes #6672
Signed-off-by: David Handermann
> TestPutSplunkHTTP Test Failures on NonDex
> --
>
> Key: NIFI-10834
> URL: https://issues.apache.org/jira/browse/NIFI-10834
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Environment: openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
> Apache Maven 3.6.3;
>Reporter: Priyanka Awatramani
>Assignee: Priyanka Awatramani
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Attachments: image-2022-11-16-15-34-06-074.png,
> image-2022-11-16-15-36-02-621.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The test
> *org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParameters*
> and
> *org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParametersFromFlowFile*
> passed using normal maven-test, but showed Non-deterministic behavior under
> NonDex([https://github.com/TestingResearchIllinois/NonDex)] and thus failed.
> Some of the error messages are:
>
> !image-2022-11-16-15-34-06-074.png!
>
> !image-2022-11-16-15-36-02-621.png!
>
> and after some probing, I found the output from the test is not deterministic
> as the order of query parameters in the URL changes.
>
> Steps to reproduce the failure:
> Install Nondex([https://github.com/TestingResearchIllinois/NonDex)] in the
> environment.
> Then cd to nifi repository, and run the following:
> {code:java}
> mvn install -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> -am -DskipTests
> mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParameters
>
> mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParametersFromFlowFile{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10871) Intermittent CSRF HTTP 403 in Clustered Deployments
[
https://issues.apache.org/jira/browse/NIFI-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641534#comment-17641534
]
ASF subversion and git services commented on NIFI-10871:
Commit 4aa86b850db6ed605f1d987127eb08cf15915534 in nifi's branch
refs/heads/support/nifi-1.19 from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4aa86b850d ]
NIFI-10871 Skipped CSRF processing for replicated HTTP requests
- Updated Security Filter Configuration to avoid unnecessary CSRF Request Token
generation for requests replicated between cluster nodes
This closes #6715.
Signed-off-by: Tamas Palfy
> Intermittent CSRF HTTP 403 in Clustered Deployments
>
>
> Key: NIFI-10871
> URL: https://issues.apache.org/jira/browse/NIFI-10871
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI, Security
>Affects Versions: 1.14.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> NiFi 1.14.0 introduced Cross-Site Request Forgery mitigation as part of
> updates to support JSON Web Token resolution using HttpOnly Session cookies.
> The standard Spring Security
> [CsrfFilter|https://github.com/spring-projects/spring-security/blob/5.7.5/web/src/main/java/org/springframework/security/web/csrf/CsrfFilter.java]
> includes a Request Matcher property to control whether filtering operations
> should be applied, but the CsrfFilter checks the Request Matcher after
> generating and saving a new token.
> Standalone deployments of NiFi can reuse the CSRF Request Token when the HTTP
> request includes the value in a {{Cookie}} header, but the NiFi HTTP Request
> Replicator removes the CSRF Request Token cookie before sending the request
> to other cluster nodes.
> As a result of these implementation details, NiFi cluster nodes receiving
> replicated HTTP requests generate and return a new CSRF Request Token. The
> NiFi user interface receives the new CSRF Request Token and uses it to set
> the custom {{Request-Token}} HTTP Header on subsequent requests. This is not
> an issue for HTTP GET requests, but requests using methods such as POST, PUT,
> or DELETE can return an HTTP 403 Forbidden response from the Spring Security
> CsrfFilter due to receiving mismatched {{__Secure-Request-Token}} Cookie and
> {{Request-Token}} Header values.
> This issue is intermittent because it depends on the web browser
> simultaneously receiving an HTTP response with a new Secure-Request-Token
> Cookie while preparing to send a new HTTP request with a {{Request-Token}}
> Header that contains the value from the previously received cookie.
> Resolving the problem should include adjusting the behavior of the CsrfFilter
> to avoid setting a new cookie on requests that do not require filtering.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10890) TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641536#comment-17641536
]
ASF subversion and git services commented on NIFI-10890:
Commit e6dfea6352f5ffabbbcafde10745c4fc59a64fc1 in nifi's branch
refs/heads/support/nifi-1.19 from sopan98
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e6dfea6352 ]
NIFI-10890 Replaced HashMap with LinkedHashMap in JsonRowRecordReader
- Updated unit test based on deterministic behavior of LinkedHashMap
This closes #6726
Signed-off-by: David Handermann
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses
> non-deterministic HashMap
>
>
> Key: NIFI-10890
> URL: https://issues.apache.org/jira/browse/NIFI-10890
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Attachments: org.apache.nifi.json.TestJsonTreeRowRecordReader.txt
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.json.TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR]
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved:433
> expected: City, state=MS, zipCode=1, country=USA,
> account=MapRecord[{balance=4750.89, id=42}]}]> but was: name=John Doe, address=123 My Street, city=My City, state=MS, zipCode=1,
> country=USA, account=MapRecord[{id=42, balance=4750.89}]}]>{code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn install -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
> -DskipTests -Drat.skip -am{noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
>
> -Dtest=org.apache.nifi.json.TestJsonTreeRowRecordReader#testReadRawRecordFieldOrderPreserved
> {noformat}
> The result will be saved under the module folder in .nondex
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-7190) CaptureChangeMySQL processor doesn't emit DDL events if SQL statement begins with a comment
[
https://issues.apache.org/jira/browse/NIFI-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641529#comment-17641529
]
ASF subversion and git services commented on NIFI-7190:
---
Commit 0ed8114dcb05eefecb8f6c1234449f6f1a362d47 in nifi's branch
refs/heads/support/nifi-1.19 from Vasily Makarov
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0ed8114dcb ]
NIFI-7190 CaptureChangeMySQL processor remove comments from normalized query
This closes #6711
Co-authored-by: Vasily Makarov
Co-authored-by: Matt Burgess
Signed-off-by: David Handermann
> CaptureChangeMySQL processor doesn't emit DDL events if SQL statement begins
> with a comment
> ---
>
> Key: NIFI-7190
> URL: https://issues.apache.org/jira/browse/NIFI-7190
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.2
>Reporter: Przemyslaw Dubaniewicz
>Assignee: Makarov Vasiliy Nicolaevich
>Priority: Minor
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The existing check for DDL Events doesn't take into account SQL statements
> that begin with a comment, in file:
> _nifi/nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/src/main/java/org/apache/nifi/cdc/mysql/processors/CaptureChangeMySQL.java:821_
> {code:java}
> // Check for DDL events (alter table, e.g.). Normalize the query to do string
> matching on the type of change
> String normalizedQuery = sql.toLowerCase().trim().replaceAll(" {2,}", " ");
> if (normalizedQuery.startsWith("alter table")
> || normalizedQuery.startsWith("alter ignore table")
> || normalizedQuery.startsWith("create table")
> || normalizedQuery.startsWith("truncate table")
> || normalizedQuery.startsWith("rename table")
> || normalizedQuery.startsWith("drop table")
> || normalizedQuery.startsWith("drop database")) {code}
> SQL commands such as:
> {code:sql}
> /* ApplicationName=DataGrip 2019.2.4 */ alter table test_table
> drop column test_column
> {code}
> won't evaluate to true.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10873) GenerateFlowFile: flowfiles in a batch are not unique
[ https://issues.apache.org/jira/browse/NIFI-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641525#comment-17641525 ] ASF subversion and git services commented on NIFI-10873: Commit e0b4feac575a9c241b19bacdfd3d166015d43eda in nifi's branch refs/heads/support/nifi-1.19 from Arpad Boda [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e0b4feac57 ] NIFI-10873 - GenerateFlowFile: flowfiles in a batch are not unique This closes #6717. Signed-off-by: Peter Turcsanyi > GenerateFlowFile: flowfiles in a batch are not unique > - > > Key: NIFI-10873 > URL: https://issues.apache.org/jira/browse/NIFI-10873 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.19.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 1.20.0, 1.19.1 > > Time Spent: 40m > Remaining Estimate: 0h > > When GenerateFlowFile is configured to generate unique flowfiles, the > flowfiles aren't unique in case the batch size is greater than one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10886) Upgrade PostgreSQL JDBC Driver to 42.4.3
[ https://issues.apache.org/jira/browse/NIFI-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641528#comment-17641528 ] ASF subversion and git services commented on NIFI-10886: Commit 83d9f9916a687e143d9157a10fd8a4b749ed71d2 in nifi's branch refs/heads/support/nifi-1.19 from dependabot[bot] [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=83d9f9916a ] NIFI-10886 Upgraded PostgreSQL JDBC Driver from 42.4.1 to 42.4.3 This closes #6713 Signed-off-by: David Handermann > Upgrade PostgreSQL JDBC Driver to 42.4.3 > > > Key: NIFI-10886 > URL: https://issues.apache.org/jira/browse/NIFI-10886 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry, Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > The PostgreSQL JDBC Driver used for NiFi Registry integration tests should be > upgraded to 42.4.3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10875) Test org.apache.nifi.processors.solr.TestQuerySolr.testAttributes Fails on NonDex
[
https://issues.apache.org/jira/browse/NIFI-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641531#comment-17641531
]
ASF subversion and git services commented on NIFI-10875:
Commit 2e6f8ecbe00561b3a348666d82146004f7624b56 in nifi's branch
refs/heads/support/nifi-1.19 from Yihen Liu
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2e6f8ecbe0 ]
NIFI-10875 Changed TestQuerySolr to check for path elements
- Changes avoid non-deterministic expectations for query parameters
This closes #6718
Signed-off-by: David Handermann
> Test org.apache.nifi.processors.solr.TestQuerySolr.testAttributes Fails on
> NonDex
> -
>
> Key: NIFI-10875
> URL: https://issues.apache.org/jira/browse/NIFI-10875
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Yichen
>Assignee: Yichen
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.solr.TestQuerySolr.testAttributes {code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.The
> test shows below:
>
> {code:java}
> ---
> Test set: org.apache.nifi.processors.solr.TestQuerySolr
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.426 s <<<
> FAILURE! - in org.apache.nifi.processors.solr.TestQuerySolr
> org.apache.nifi.processors.solr.TestQuerySolr.testAttributes Time elapsed:
> 0.205 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: expected:
> but was:
>
> at
> org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
> at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
> at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1142)
> at
> org.apache.nifi.processors.solr.TestQuerySolr.testAttributes(TestQuerySolr.java:606)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
> at
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
> at
> org.junit.platform.engine.support.hier
[jira] [Commented] (NIFI-10866) Refactor Kafka 1.0 and 2.0 Components using nifi-kafka-shared
[
https://issues.apache.org/jira/browse/NIFI-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641519#comment-17641519
]
ASF subversion and git services commented on NIFI-10866:
Commit 3caad988be9d2e70ae5c1ee94d5c4a6bbe3c6b96 in nifi's branch
refs/heads/support/nifi-1.19 from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3caad988be ]
NIFI-10866 Refactored Kafka 1.0 and 2.0 using nifi-kafka-shared
Signed-off-by: Joe Gresock
This closes #6710.
> Refactor Kafka 1.0 and 2.0 Components using nifi-kafka-shared
> -
>
> Key: NIFI-10866
> URL: https://issues.apache.org/jira/browse/NIFI-10866
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The Kafka 1.0 and 2.0 components should be refactored to use property and
> validation components in {{nifi-kafka-shared}}.
> This refactoring will reduce duplicate code and also bring additional
> security measures to the validation of custom sasl.jaas.config properties
> across all Kafka components
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10850) Query Parameters property in InvokeAWSGatewayApi should support FlowFile attributes in EL
[ https://issues.apache.org/jira/browse/NIFI-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641521#comment-17641521 ] ASF subversion and git services commented on NIFI-10850: Commit 2834aab432ed1c43627a54e8445fba3ab00de366 in nifi's branch refs/heads/support/nifi-1.19 from Peter Turcsanyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2834aab432 ] NIFI-10850: Fixed Query Parameters property in InvokeAWSGatewayApi should support FlowFile attributes in EL Signed-off-by: Pierre Villard This closes #6692. > Query Parameters property in InvokeAWSGatewayApi should support FlowFile > attributes in EL > - > > Key: NIFI-10850 > URL: https://issues.apache.org/jira/browse/NIFI-10850 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Labels: aws > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > Query Parameters property declares FlowFile Attributes support but it is not > handled in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10872) Fix broken Java Mail links in ConsumeIMAP and ConsumePOP3 documentation
[ https://issues.apache.org/jira/browse/NIFI-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641527#comment-17641527 ] ASF subversion and git services commented on NIFI-10872: Commit bfa59b204115196ce563ada99634a29f32a142ed in nifi's branch refs/heads/support/nifi-1.19 from Nandor Soma Abonyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bfa59b2041 ] NIFI-10872 Fix broken Java Mail links in ConsumeIMAP and ConsumePOP3 documentation This closes #6723 Signed-off-by: David Handermann > Fix broken Java Mail links in ConsumeIMAP and ConsumePOP3 documentation > --- > > Key: NIFI-10872 > URL: https://issues.apache.org/jira/browse/NIFI-10872 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Labels: docuentation > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > Java mail project moved from [https://javaee.github.io/javamail/] to > [https://jakartaee.github.io/mail-api/] because it was moved to the Eclipse > Foundation as part of the [Eclipse Enterprise for > Java|https://projects.eclipse.org/projects/ee4j] project. > Links on the new project pages are broken, so we need to use the old site. > Unfortunately, the documentation is not versioned there, so we need to use > the "latest", which might list properties that are not supported. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10882) ElasticSearchClientService should not allow both BASIC and API_KEY properties to be set simultaneously
[
https://issues.apache.org/jira/browse/NIFI-10882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641530#comment-17641530
]
ASF subversion and git services commented on NIFI-10882:
Commit d5028315592028b91f793ab5d1d48fee2754075b in nifi's branch
refs/heads/support/nifi-1.19 from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d502831559 ]
NIFI-10882 Set credentials for ElasticSearchClientService based on
AuthorizationScheme
This closes #6722
Signed-off-by: David Handermann
> ElasticSearchClientService should not allow both BASIC and API_KEY properties
> to be set simultaneously
> --
>
> Key: NIFI-10882
> URL: https://issues.apache.org/jira/browse/NIFI-10882
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Major
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> While the {{ElasticSearchClientService}} will show/hide Controller Service
> properties based on the currently selected {{AuthorizationScheme}}, it is
> possible for a user to set {{BASIC}} credentials (e.g.
> {{Username}}/{{Password}}) then later switch to {{API_KEY}} auth without
> removing the {{BASIC}} properties - this leads to multiple {{Authorization}}
> headers being configured in requests to Elasticsearch, which is unexpected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10785) Allow publishing AMQP message with null header value
[ https://issues.apache.org/jira/browse/NIFI-10785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641535#comment-17641535 ] ASF subversion and git services commented on NIFI-10785: Commit 71f41b96baf602859c2d1ed9383ff1fd13268719 in nifi's branch refs/heads/support/nifi-1.19 from Nandor Soma Abonyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=71f41b96ba ] NIFI-10785 Allow publishing AMQP message with null header value NIFI-10785 addressing review comment NIFI-10785 addressing review comments (remove unnecessary property to ignore null headers) Signed-off-by: Nathan Gough This closes #6649. > Allow publishing AMQP message with null header value > > > Key: NIFI-10785 > URL: https://issues.apache.org/jira/browse/NIFI-10785 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Labels: amqp > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Since after NIFI-10317 ConsumeAMQP is able to handle null header value it > makes sense to support it in PublishAMQP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10913) Release NiFi 1.19.1
[ https://issues.apache.org/jira/browse/NIFI-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641541#comment-17641541 ] ASF subversion and git services commented on NIFI-10913: Commit eb0d07fda3f41512a79339d3fa53f3eb4e98d0b1 in nifi's branch refs/heads/support/nifi-1.19 from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=eb0d07fda3 ] NIFI-10913 updating versions post commit landing > Release NiFi 1.19.1 > --- > > Key: NIFI-10913 > URL: https://issues.apache.org/jira/browse/NIFI-10913 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.19.1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10874) Multiple levels of versioned flows might not able to load from repository
[ https://issues.apache.org/jira/browse/NIFI-10874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641526#comment-17641526 ] ASF subversion and git services commented on NIFI-10874: Commit 69e4788ffdf455f585c5f7b987353b87cbda in nifi's branch refs/heads/support/nifi-1.19 from Simon Bence [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=69e478 ] NIFI-10874 Fixed issue with multiple levels of versioned flows This closes #6719 Signed-off-by: David Handermann > Multiple levels of versioned flows might not able to load from repository > - > > Key: NIFI-10874 > URL: https://issues.apache.org/jira/browse/NIFI-10874 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 50m > Remaining Estimate: 0h > > With the refactoring of the registry (changing it to a component), several > necessary change was done, including the "generification" of the way included > versioned flows are handled (recognised). > Previous implementation was depending on an URI, which was not descriptive > enough for the new concept. Storage location has been introduced and now > registry client implementations are capable for determine if they are capable > to load a given flow based on this. > As a fallback mechanism, the URI was kept for the NiFiRegistry based > implementation. A bug has been introduced with this mechanism, namely: the > ProcessGroupResouce#createProcessGroup method works with version information > from the UI, which has no knowledge of this. In order to guarantee that the > given information is stored during flow creation (here: loading flow from > external source) we need to guarantee the transition of this information. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10845) JsonQueryElasticsearch processors are not outputting an empty flow file for a combined response with output_no_hits set to true
[ https://issues.apache.org/jira/browse/NIFI-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641523#comment-17641523 ] ASF subversion and git services commented on NIFI-10845: Commit 51395bddd18877b2324a7d9858a8694ea8aa83b0 in nifi's branch refs/heads/support/nifi-1.19 from Ryan Van Den Bos [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=51395bddd1 ] NIFI-10845 - JsonQueryElasticsearch processors are not outputting an empty flow file for a combined response with output_no_hits set to true Signed-off-by: Chris Sampson This closes #6701 > JsonQueryElasticsearch processors are not outputting an empty flow file for a > combined response with output_no_hits set to true > > > Key: NIFI-10845 > URL: https://issues.apache.org/jira/browse/NIFI-10845 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Ryan >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > If you set OUTPUT_NO_HITS to 'true' and SEARCH_RESULTS_SPLIT to 'Per Query' > it does not output an empty flow file if no hits are found. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-5819) Support SQLServer sql_variant type
[
https://issues.apache.org/jira/browse/NIFI-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641524#comment-17641524
]
ASF subversion and git services commented on NIFI-5819:
---
Commit db2bf5f64530e66763894efcfa702d2b4bae1e39 in nifi's branch
refs/heads/support/nifi-1.19 from Matt Burgess
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=db2bf5f645 ]
NIFI-5819: Support SQLServer sql_variant type
This closes #6699.
Signed-off-by: Peter Turcsanyi
> Support SQLServer sql_variant type
> --
>
> Key: NIFI-5819
> URL: https://issues.apache.org/jira/browse/NIFI-5819
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
>Reporter: Charlie Meyer
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If i run a query using executesqlrecord against a mssql database such as
> {{SELECT SERVERPROPERTY ('ProductVersion') AS MajorVersion}} nifi logs the
> following stack trace:
> {code:java}
> { "cause": { "cause": null, "stackTrace": [ { "methodName": "createSchema",
> "fileName": "JdbcCommon.java", "lineNumber": 677, "className":
> "org.apache.nifi.processors.standard.util.JdbcCommon", "nativeMethod": false
> }, { "methodName": "writeResultSet", "fileName": "RecordSqlWriter.java",
> "lineNumber": 68, "className":
> "org.apache.nifi.processors.standard.sql.RecordSqlWriter", "nativeMethod":
> false }, { "methodName": "lambda$onTrigger$1", "fileName":
> "ExecuteSqlRecordWithoutSwallowingErrors.java", "lineNumber": 362,
> "className":
> "com.civitaslearning.collect.nifi.processor.ExecuteSqlRecordWithoutSwallowingErrors",
> "nativeMethod": false }, { "methodName": "write", "fileName":
> "StandardProcessSession.java", "lineNumber": 2648, "className":
> "org.apache.nifi.controller.repository.StandardProcessSession",
> "nativeMethod": false }, { "methodName": "onTrigger", "fileName":
> "ExecuteSqlRecordWithoutSwallowingErrors.java", "lineNumber": 360,
> "className":
> "com.civitaslearning.collect.nifi.processor.ExecuteSqlRecordWithoutSwallowingErrors",
> "nativeMethod": false }, { "methodName": "onTrigger", "fileName":
> "AbstractProcessor.java", "lineNumber": 27, "className":
> "org.apache.nifi.processor.AbstractProcessor", "nativeMethod": false }, {
> "methodName": "onTrigger", "fileName": "StandardProcessorNode.java",
> "lineNumber": 1165, "className":
> "org.apache.nifi.controller.StandardProcessorNode", "nativeMethod": false },
> { "methodName": "invoke", "fileName": "ConnectableTask.java", "lineNumber":
> 203, "className": "org.apache.nifi.controller.tasks.ConnectableTask",
> "nativeMethod": false }, { "methodName": "run", "fileName":
> "TimerDrivenSchedulingAgent.java", "lineNumber": 117, "className":
> "org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1",
> "nativeMethod": false }, { "methodName": "call", "fileName":
> "Executors.java", "lineNumber": 511, "className":
> "java.util.concurrent.Executors$RunnableAdapter", "nativeMethod": false }, {
> "methodName": "runAndReset", "fileName": "FutureTask.java", "lineNumber":
> 308, "className": "java.util.concurrent.FutureTask", "nativeMethod": false },
> { "methodName": "access$301", "fileName": "ScheduledThreadPoolExecutor.java",
> "lineNumber": 180, "className":
> "java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask",
> "nativeMethod": false }, { "methodName": "run", "fileName":
> "ScheduledThreadPoolExecutor.java", "lineNumber": 294, "className":
> "java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask",
> "nativeMethod": false }, { "methodName": "runWorker", "fileName":
> "ThreadPoolExecutor.java", "lineNumber": 1149, "className":
> "java.util.concurrent.ThreadPoolExecutor", "nativeMethod": false }, {
> "methodName": "run", "fileName": "ThreadPoolExecutor.java", "lineNumber":
> 624, "className": "java.util.concurrent.ThreadPoolExecutor$Worker",
> "nativeMethod": false }, { "methodName": "run", "fileName": "Thread.java",
> "lineNumber": 748, "className": "java.lang.Thread", "nativeMethod": false }
> ], "message": "createSchema: Unknown SQL type -156 / sql_variant (table:
> NiFi_ExecuteSQL_Record, column: MajorVersion) cannot be converted to Avro
> type", "localizedMessage": "createSchema: Unknown SQL type -156 / sql_variant
> (table: NiFi_ExecuteSQL_Record, column: MajorVersion) cannot be converted to
> Avro type", "suppressed": [] }, "stackTrace": [ { "methodName":
> "lambda$onTrigger$1", "fileName":
> "ExecuteSqlRecordWithoutSwallowingErrors.java", "lineNumber": 364,
> "className":
> "com.civitaslearning.collect.nifi.processor.ExecuteSqlRecordWithoutSwallowingErrors",
> "nativeMethod": false }, { "
[jira] [Updated] (NIFI-10845) JsonQueryElasticsearch processors are not outputting an empty flow file for a combined response with output_no_hits set to true
[ https://issues.apache.org/jira/browse/NIFI-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10845: Fix Version/s: 1.19.1 > JsonQueryElasticsearch processors are not outputting an empty flow file for a > combined response with output_no_hits set to true > > > Key: NIFI-10845 > URL: https://issues.apache.org/jira/browse/NIFI-10845 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Ryan >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > If you set OUTPUT_NO_HITS to 'true' and SEARCH_RESULTS_SPLIT to 'Per Query' > it does not output an empty flow file if no hits are found. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10882) ElasticSearchClientService should not allow both BASIC and API_KEY properties to be set simultaneously
[
https://issues.apache.org/jira/browse/NIFI-10882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10882:
Fix Version/s: 1.19.1
> ElasticSearchClientService should not allow both BASIC and API_KEY properties
> to be set simultaneously
> --
>
> Key: NIFI-10882
> URL: https://issues.apache.org/jira/browse/NIFI-10882
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Major
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> While the {{ElasticSearchClientService}} will show/hide Controller Service
> properties based on the currently selected {{AuthorizationScheme}}, it is
> possible for a user to set {{BASIC}} credentials (e.g.
> {{Username}}/{{Password}}) then later switch to {{API_KEY}} auth without
> removing the {{BASIC}} properties - this leads to multiple {{Authorization}}
> headers being configured in requests to Elasticsearch, which is unexpected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10874) Multiple levels of versioned flows might not able to load from repository
[ https://issues.apache.org/jira/browse/NIFI-10874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10874: Fix Version/s: 1.19.1 > Multiple levels of versioned flows might not able to load from repository > - > > Key: NIFI-10874 > URL: https://issues.apache.org/jira/browse/NIFI-10874 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 50m > Remaining Estimate: 0h > > With the refactoring of the registry (changing it to a component), several > necessary change was done, including the "generification" of the way included > versioned flows are handled (recognised). > Previous implementation was depending on an URI, which was not descriptive > enough for the new concept. Storage location has been introduced and now > registry client implementations are capable for determine if they are capable > to load a given flow based on this. > As a fallback mechanism, the URI was kept for the NiFiRegistry based > implementation. A bug has been introduced with this mechanism, namely: the > ProcessGroupResouce#createProcessGroup method works with version information > from the UI, which has no knowledge of this. In order to guarantee that the > given information is stored during flow creation (here: loading flow from > external source) we need to guarantee the transition of this information. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-7190) CaptureChangeMySQL processor doesn't emit DDL events if SQL statement begins with a comment
[
https://issues.apache.org/jira/browse/NIFI-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-7190:
---
Fix Version/s: 1.19.1
> CaptureChangeMySQL processor doesn't emit DDL events if SQL statement begins
> with a comment
> ---
>
> Key: NIFI-7190
> URL: https://issues.apache.org/jira/browse/NIFI-7190
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.2
>Reporter: Przemyslaw Dubaniewicz
>Assignee: Makarov Vasiliy Nicolaevich
>Priority: Minor
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The existing check for DDL Events doesn't take into account SQL statements
> that begin with a comment, in file:
> _nifi/nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/src/main/java/org/apache/nifi/cdc/mysql/processors/CaptureChangeMySQL.java:821_
> {code:java}
> // Check for DDL events (alter table, e.g.). Normalize the query to do string
> matching on the type of change
> String normalizedQuery = sql.toLowerCase().trim().replaceAll(" {2,}", " ");
> if (normalizedQuery.startsWith("alter table")
> || normalizedQuery.startsWith("alter ignore table")
> || normalizedQuery.startsWith("create table")
> || normalizedQuery.startsWith("truncate table")
> || normalizedQuery.startsWith("rename table")
> || normalizedQuery.startsWith("drop table")
> || normalizedQuery.startsWith("drop database")) {code}
> SQL commands such as:
> {code:sql}
> /* ApplicationName=DataGrip 2019.2.4 */ alter table test_table
> drop column test_column
> {code}
> won't evaluate to true.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-6428) CaptureChangeMySQL throws IOException with BIGIN event due to lingering 'inTransaction' instance variable
[ https://issues.apache.org/jira/browse/NIFI-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-6428: --- Fix Version/s: 1.19.1 > CaptureChangeMySQL throws IOException with BIGIN event due to lingering > 'inTransaction' instance variable > - > > Key: NIFI-6428 > URL: https://issues.apache.org/jira/browse/NIFI-6428 > Project: Apache NiFi > Issue Type: Bug >Reporter: Koji Kawamura >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Because of not clearing 'inTransaction' instance variable when the processor > stops, when a user clears processor state and restart it with a specific > initial binlog position, then if BEGIN event is received, the processor > throws an IOException. > The processor should reset 'inTransaction' to false, and also other instance > variables when it start with an empty processor state. > The issue was reported at NiFi user ML. > [https://mail-archives.apache.org/mod_mbox/nifi-users/201907.mbox/%3C2019070919385393266224%40geekplus.com.cn%3E] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10871) Intermittent CSRF HTTP 403 in Clustered Deployments
[
https://issues.apache.org/jira/browse/NIFI-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10871:
Fix Version/s: 1.19.1
> Intermittent CSRF HTTP 403 in Clustered Deployments
>
>
> Key: NIFI-10871
> URL: https://issues.apache.org/jira/browse/NIFI-10871
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI, Security
>Affects Versions: 1.14.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> NiFi 1.14.0 introduced Cross-Site Request Forgery mitigation as part of
> updates to support JSON Web Token resolution using HttpOnly Session cookies.
> The standard Spring Security
> [CsrfFilter|https://github.com/spring-projects/spring-security/blob/5.7.5/web/src/main/java/org/springframework/security/web/csrf/CsrfFilter.java]
> includes a Request Matcher property to control whether filtering operations
> should be applied, but the CsrfFilter checks the Request Matcher after
> generating and saving a new token.
> Standalone deployments of NiFi can reuse the CSRF Request Token when the HTTP
> request includes the value in a {{Cookie}} header, but the NiFi HTTP Request
> Replicator removes the CSRF Request Token cookie before sending the request
> to other cluster nodes.
> As a result of these implementation details, NiFi cluster nodes receiving
> replicated HTTP requests generate and return a new CSRF Request Token. The
> NiFi user interface receives the new CSRF Request Token and uses it to set
> the custom {{Request-Token}} HTTP Header on subsequent requests. This is not
> an issue for HTTP GET requests, but requests using methods such as POST, PUT,
> or DELETE can return an HTTP 403 Forbidden response from the Spring Security
> CsrfFilter due to receiving mismatched {{__Secure-Request-Token}} Cookie and
> {{Request-Token}} Header values.
> This issue is intermittent because it depends on the web browser
> simultaneously receiving an HTTP response with a new Secure-Request-Token
> Cookie while preparing to send a new HTTP request with a {{Request-Token}}
> Header that contains the value from the previously received cookie.
> Resolving the problem should include adjusting the behavior of the CsrfFilter
> to avoid setting a new cookie on requests that do not require filtering.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10834) TestPutSplunkHTTP Test Failures on NonDex
[
https://issues.apache.org/jira/browse/NIFI-10834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10834:
Fix Version/s: 1.19.1
> TestPutSplunkHTTP Test Failures on NonDex
> --
>
> Key: NIFI-10834
> URL: https://issues.apache.org/jira/browse/NIFI-10834
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Environment: openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
> Apache Maven 3.6.3;
>Reporter: Priyanka Awatramani
>Assignee: Priyanka Awatramani
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Attachments: image-2022-11-16-15-34-06-074.png,
> image-2022-11-16-15-36-02-621.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The test
> *org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParameters*
> and
> *org.apache.nifi.processors.splunk.TestPutSplunkHTTP.testHappyPathWithCustomQueryParametersFromFlowFile*
> passed using normal maven-test, but showed Non-deterministic behavior under
> NonDex([https://github.com/TestingResearchIllinois/NonDex)] and thus failed.
> Some of the error messages are:
>
> !image-2022-11-16-15-34-06-074.png!
>
> !image-2022-11-16-15-36-02-621.png!
>
> and after some probing, I found the output from the test is not deterministic
> as the order of query parameters in the URL changes.
>
> Steps to reproduce the failure:
> Install Nondex([https://github.com/TestingResearchIllinois/NonDex)] in the
> environment.
> Then cd to nifi repository, and run the following:
> {code:java}
> mvn install -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> -am -DskipTests
> mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParameters
>
> mvn -pl nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.splunk.TestPutSplunkHTTP#testHappyPathWithCustomQueryParametersFromFlowFile{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-5819) Support SQLServer sql_variant type
[
https://issues.apache.org/jira/browse/NIFI-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-5819:
---
Fix Version/s: 1.19.1
> Support SQLServer sql_variant type
> --
>
> Key: NIFI-5819
> URL: https://issues.apache.org/jira/browse/NIFI-5819
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
>Reporter: Charlie Meyer
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If i run a query using executesqlrecord against a mssql database such as
> {{SELECT SERVERPROPERTY ('ProductVersion') AS MajorVersion}} nifi logs the
> following stack trace:
> {code:java}
> { "cause": { "cause": null, "stackTrace": [ { "methodName": "createSchema",
> "fileName": "JdbcCommon.java", "lineNumber": 677, "className":
> "org.apache.nifi.processors.standard.util.JdbcCommon", "nativeMethod": false
> }, { "methodName": "writeResultSet", "fileName": "RecordSqlWriter.java",
> "lineNumber": 68, "className":
> "org.apache.nifi.processors.standard.sql.RecordSqlWriter", "nativeMethod":
> false }, { "methodName": "lambda$onTrigger$1", "fileName":
> "ExecuteSqlRecordWithoutSwallowingErrors.java", "lineNumber": 362,
> "className":
> "com.civitaslearning.collect.nifi.processor.ExecuteSqlRecordWithoutSwallowingErrors",
> "nativeMethod": false }, { "methodName": "write", "fileName":
> "StandardProcessSession.java", "lineNumber": 2648, "className":
> "org.apache.nifi.controller.repository.StandardProcessSession",
> "nativeMethod": false }, { "methodName": "onTrigger", "fileName":
> "ExecuteSqlRecordWithoutSwallowingErrors.java", "lineNumber": 360,
> "className":
> "com.civitaslearning.collect.nifi.processor.ExecuteSqlRecordWithoutSwallowingErrors",
> "nativeMethod": false }, { "methodName": "onTrigger", "fileName":
> "AbstractProcessor.java", "lineNumber": 27, "className":
> "org.apache.nifi.processor.AbstractProcessor", "nativeMethod": false }, {
> "methodName": "onTrigger", "fileName": "StandardProcessorNode.java",
> "lineNumber": 1165, "className":
> "org.apache.nifi.controller.StandardProcessorNode", "nativeMethod": false },
> { "methodName": "invoke", "fileName": "ConnectableTask.java", "lineNumber":
> 203, "className": "org.apache.nifi.controller.tasks.ConnectableTask",
> "nativeMethod": false }, { "methodName": "run", "fileName":
> "TimerDrivenSchedulingAgent.java", "lineNumber": 117, "className":
> "org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1",
> "nativeMethod": false }, { "methodName": "call", "fileName":
> "Executors.java", "lineNumber": 511, "className":
> "java.util.concurrent.Executors$RunnableAdapter", "nativeMethod": false }, {
> "methodName": "runAndReset", "fileName": "FutureTask.java", "lineNumber":
> 308, "className": "java.util.concurrent.FutureTask", "nativeMethod": false },
> { "methodName": "access$301", "fileName": "ScheduledThreadPoolExecutor.java",
> "lineNumber": 180, "className":
> "java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask",
> "nativeMethod": false }, { "methodName": "run", "fileName":
> "ScheduledThreadPoolExecutor.java", "lineNumber": 294, "className":
> "java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask",
> "nativeMethod": false }, { "methodName": "runWorker", "fileName":
> "ThreadPoolExecutor.java", "lineNumber": 1149, "className":
> "java.util.concurrent.ThreadPoolExecutor", "nativeMethod": false }, {
> "methodName": "run", "fileName": "ThreadPoolExecutor.java", "lineNumber":
> 624, "className": "java.util.concurrent.ThreadPoolExecutor$Worker",
> "nativeMethod": false }, { "methodName": "run", "fileName": "Thread.java",
> "lineNumber": 748, "className": "java.lang.Thread", "nativeMethod": false }
> ], "message": "createSchema: Unknown SQL type -156 / sql_variant (table:
> NiFi_ExecuteSQL_Record, column: MajorVersion) cannot be converted to Avro
> type", "localizedMessage": "createSchema: Unknown SQL type -156 / sql_variant
> (table: NiFi_ExecuteSQL_Record, column: MajorVersion) cannot be converted to
> Avro type", "suppressed": [] }, "stackTrace": [ { "methodName":
> "lambda$onTrigger$1", "fileName":
> "ExecuteSqlRecordWithoutSwallowingErrors.java", "lineNumber": 364,
> "className":
> "com.civitaslearning.collect.nifi.processor.ExecuteSqlRecordWithoutSwallowingErrors",
> "nativeMethod": false }, { "methodName": "write", "fileName":
> "StandardProcessSession.java", "lineNumber": 2648, "className":
> "org.apache.nifi.controller.repository.StandardProcessSession",
> "nativeMethod": false }, { "methodName": "onTrigger", "fileName":
> "ExecuteSqlRecordWithoutSwallowingErrors.java", "lineNumber": 360,
> "className":
> "com.civitaslearning.collect.n
[jira] [Updated] (NIFI-10873) GenerateFlowFile: flowfiles in a batch are not unique
[ https://issues.apache.org/jira/browse/NIFI-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10873: Fix Version/s: 1.19.1 > GenerateFlowFile: flowfiles in a batch are not unique > - > > Key: NIFI-10873 > URL: https://issues.apache.org/jira/browse/NIFI-10873 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.19.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 1.20.0, 1.19.1 > > Time Spent: 40m > Remaining Estimate: 0h > > When GenerateFlowFile is configured to generate unique flowfiles, the > flowfiles aren't unique in case the batch size is greater than one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10910) Update amqp-client to 5.16.0
[ https://issues.apache.org/jira/browse/NIFI-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10910: Fix Version/s: 1.19.1 > Update amqp-client to 5.16.0 > > > Key: NIFI-10910 > URL: https://issues.apache.org/jira/browse/NIFI-10910 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > Update ampq-client from 5.15.0 to 5.16.0. [Releases · > rabbitmq/rabbitmq-java-client > (github.com)|https://github.com/rabbitmq/rabbitmq-java-client/releases] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10886) Upgrade PostgreSQL JDBC Driver to 42.4.3
[ https://issues.apache.org/jira/browse/NIFI-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10886: Fix Version/s: 1.19.1 > Upgrade PostgreSQL JDBC Driver to 42.4.3 > > > Key: NIFI-10886 > URL: https://issues.apache.org/jira/browse/NIFI-10886 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry, Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > The PostgreSQL JDBC Driver used for NiFi Registry integration tests should be > upgraded to 42.4.3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10872) Fix broken Java Mail links in ConsumeIMAP and ConsumePOP3 documentation
[ https://issues.apache.org/jira/browse/NIFI-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10872: Fix Version/s: 1.19.1 > Fix broken Java Mail links in ConsumeIMAP and ConsumePOP3 documentation > --- > > Key: NIFI-10872 > URL: https://issues.apache.org/jira/browse/NIFI-10872 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Labels: docuentation > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > Java mail project moved from [https://javaee.github.io/javamail/] to > [https://jakartaee.github.io/mail-api/] because it was moved to the Eclipse > Foundation as part of the [Eclipse Enterprise for > Java|https://projects.eclipse.org/projects/ee4j] project. > Links on the new project pages are broken, so we need to use the old site. > Unfortunately, the documentation is not versioned there, so we need to use > the "latest", which might list properties that are not supported. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10891) TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10891:
Fix Version/s: 1.19.1
> TestMergeContent.testDefragmentMultipleMingledSegments uses non-deterministic
> HashMap
> --
>
> Key: NIFI-10891
> URL: https://issues.apache.org/jira/browse/NIFI-10891
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Attachments: org.apache.nifi.processors.standard.TestMergeContent.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.standard.TestMergeContent{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR] Failures:
> [ERROR] TestMergeContent.testDefragmentMultipleMingledSegments:1028
> FlowFile content differs from input at byte 0 with input having value 65 and
> FlowFile having value 78
> {code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn clean install -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -DskipTests
> -Drat.skip -am {noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> -Dtest=org.apache.nifi.processors.standard.TestMergeContent#testDefragmentMultipleMingledSegments{noformat}
> The result will be saved under the module folder in .nondex
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10765) Add better error reporting for JASN1Reader
[ https://issues.apache.org/jira/browse/NIFI-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10765: Fix Version/s: 1.19.1 > Add better error reporting for JASN1Reader > -- > > Key: NIFI-10765 > URL: https://issues.apache.org/jira/browse/NIFI-10765 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The JASN1Reader relies on https://github.com/beanit/asn1bean to compile ASN > schema files and to deserialize ASN data files. > The schema compilation step can fail for many different reasons and the error > is not reported to the user properly (or in any way really) so understanding > and fixing errors in the schema file is very hard currently. > We should report meaningful error messages to the bulletin and/or the > controller service validation error indicator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10890) TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses non-deterministic HashMap
[
https://issues.apache.org/jira/browse/NIFI-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10890:
Fix Version/s: 1.19.1
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved uses
> non-deterministic HashMap
>
>
> Key: NIFI-10890
> URL: https://issues.apache.org/jira/browse/NIFI-10890
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Sopan Phaltankar
>Assignee: Sopan Phaltankar
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Attachments: org.apache.nifi.json.TestJsonTreeRowRecordReader.txt
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.json.TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved{code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.
> The test shows below:
> {code:java}
> [ERROR]
> TestJsonTreeRowRecordReader.testReadRawRecordFieldOrderPreserved:433
> expected: City, state=MS, zipCode=1, country=USA,
> account=MapRecord[{balance=4750.89, id=42}]}]> but was: name=John Doe, address=123 My Street, city=My City, state=MS, zipCode=1,
> country=USA, account=MapRecord[{id=42, balance=4750.89}]}]>{code}
> *Steps to reproduce the failure:*
> # Run the following command in nifi:
> # First, build the module:
> {noformat}
> mvn install -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
> -DskipTests -Drat.skip -am{noformat}
> # Then run the test using
> [NonDex|https://github.com/TestingResearchIllinois/NonDex]
> {noformat}
> mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -pl
> nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services
>
> -Dtest=org.apache.nifi.json.TestJsonTreeRowRecordReader#testReadRawRecordFieldOrderPreserved
> {noformat}
> The result will be saved under the module folder in .nondex
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10875) Test org.apache.nifi.processors.solr.TestQuerySolr.testAttributes Fails on NonDex
[
https://issues.apache.org/jira/browse/NIFI-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10875:
Fix Version/s: 1.19.1
> Test org.apache.nifi.processors.solr.TestQuerySolr.testAttributes Fails on
> NonDex
> -
>
> Key: NIFI-10875
> URL: https://issues.apache.org/jira/browse/NIFI-10875
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Yichen
>Assignee: Yichen
>Priority: Trivial
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.solr.TestQuerySolr.testAttributes {code}
> This is a flaky test, it can pass mvn test while but when run using the tool
> [NonDex|https://github.com/TestingResearchIllinois/NonDex], it fails. NonDex
> is a tool that will introduce non-determinism in certain java collections.The
> test shows below:
>
> {code:java}
> ---
> Test set: org.apache.nifi.processors.solr.TestQuerySolr
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.426 s <<<
> FAILURE! - in org.apache.nifi.processors.solr.TestQuerySolr
> org.apache.nifi.processors.solr.TestQuerySolr.testAttributes Time elapsed:
> 0.205 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: expected:
> but was:
>
> at
> org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
> at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
> at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1142)
> at
> org.apache.nifi.processors.solr.TestQuerySolr.testAttributes(TestQuerySolr.java:606)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
> at
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
> at
> org.
[jira] [Updated] (NIFI-10859) Update metrics-jvm to 4.1.33
[ https://issues.apache.org/jira/browse/NIFI-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10859: Fix Version/s: 1.19.1 > Update metrics-jvm to 4.1.33 > > > Key: NIFI-10859 > URL: https://issues.apache.org/jira/browse/NIFI-10859 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.18.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 1.20.0, 1.19.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Update metrics-jvm to 4.2.13 from 4.1.2. It can be found at > /nifi-commons/nifi-metrics/pom.xml > release notes https://github.com/dropwizard/metrics/releases -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10895) MiNiFi C2 - Implement UPDATE/PROPERTIES command
[ https://issues.apache.org/jira/browse/NIFI-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10895: Fix Version/s: (was: 1.20.0) > MiNiFi C2 - Implement UPDATE/PROPERTIES command > --- > > Key: NIFI-10895 > URL: https://issues.apache.org/jira/browse/NIFI-10895 > Project: Apache NiFi > Issue Type: New Feature > Components: C2, MiNiFi >Reporter: Ferenc Erdei >Assignee: Ferenc Erdei >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Implement a UPDATE/PROPERTIES command which will allow the agent to update > bootstrap properties using the C2 protocol. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10850) Query Parameters property in InvokeAWSGatewayApi should support FlowFile attributes in EL
[ https://issues.apache.org/jira/browse/NIFI-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10850: Fix Version/s: 1.19.1 > Query Parameters property in InvokeAWSGatewayApi should support FlowFile > attributes in EL > - > > Key: NIFI-10850 > URL: https://issues.apache.org/jira/browse/NIFI-10850 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Minor > Labels: aws > Fix For: 1.20.0, 1.19.1 > > Time Spent: 20m > Remaining Estimate: 0h > > Query Parameters property declares FlowFile Attributes support but it is not > handled in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10866) Refactor Kafka 1.0 and 2.0 Components using nifi-kafka-shared
[
https://issues.apache.org/jira/browse/NIFI-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10866:
Fix Version/s: 1.19.1
> Refactor Kafka 1.0 and 2.0 Components using nifi-kafka-shared
> -
>
> Key: NIFI-10866
> URL: https://issues.apache.org/jira/browse/NIFI-10866
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 1.20.0, 1.19.1
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The Kafka 1.0 and 2.0 components should be refactored to use property and
> validation components in {{nifi-kafka-shared}}.
> This refactoring will reduce duplicate code and also bring additional
> security measures to the validation of custom sasl.jaas.config properties
> across all Kafka components
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10785) Allow publishing AMQP message with null header value
[ https://issues.apache.org/jira/browse/NIFI-10785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10785: Fix Version/s: 1.19.1 > Allow publishing AMQP message with null header value > > > Key: NIFI-10785 > URL: https://issues.apache.org/jira/browse/NIFI-10785 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Labels: amqp > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Since after NIFI-10317 ConsumeAMQP is able to handle null header value it > makes sense to support it in PublishAMQP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10887) Improve Performance of ReplaceText processor
[
https://issues.apache.org/jira/browse/NIFI-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10887:
Fix Version/s: (was: 1.20.0)
> Improve Performance of ReplaceText processor
>
>
> Key: NIFI-10887
> URL: https://issues.apache.org/jira/browse/NIFI-10887
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Labels: performance
> Attachments: ReplaceText-LiteralReplace-0msRunDuration.png,
> ReplaceText-LiteralReplace-25msRunDuration.png,
> ReplaceText-LiteralReplace-AfterChanges.png,
> ReplaceText-LiteralReplace-BeforeChanges.png,
> ReplaceText-RegexReplace-AfterChanges.png,
> ReplaceText-RegexReplace-BeforeChanges.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When performing some tests with the ReplaceText processor, I found that it
> seemed to be quite a bit slower than I expected, especially when using a
> Replacement Strategy of "Literal Replace" and when using a lot of small
> FlowFiles.
> As a result, I performed some profiling and identified a few areas that could
> use some improvement:
> * When using the Literal Replace strategy, we find matches using
> {{Pattern.compile(Pattern.quote(...));}} and then using
> {{{}Pattern.matcher(...).find(){}}}. This is quite inefficient compared to
> just using {{String.indexOf(...)}} and accounted for approximately 30% of the
> time spent in the processor.
> * A significant amount of time was spent flushing the write buffer, as it
> flushes to disk when finished writing to each individual FlowFile. Even when
> we set a Run Duration > 0 ms, we flush for each FlowFile. This flush() gets
> delegated all the way down to the FileOutputStream. However, when using
> ProcessSession.append(), we intercept this with a NonFlushableOutputStream.
> We should do this when calling ProcessSession.write() as well. While it makes
> sense to flush data from the Processor layer's buffer, there's no need to
> flush past the session layer until the session is committed.
> * A decent bit of time was spent in the session's get() method calling
> {{{}final Set set =
> unacknowledgedFlowFiles.computeIfAbsent(connection.getFlowFileQueue(), k ->
> new HashSet<>());{}}}. The time here was spent in StandardFlowFileQueue's
> hashCode() method, which is the JVM default. We can easily implement
> hashCode() to just return the hashCode of the identifier, which is a String.
> This is a pre-computed hashcode so provides constant time of 0 ms (with the
> exception of the method call itself) so eliminates the expense here.
> * When using a Run Duration > 0 ms, we can hold InputStreams open by
> processing multiple FlowFiles in a given Session. This can also significantly
> improve performance. As such, we should make the default run duration 25 ms
> instead of 0 ms.
> * A common pattern with ReplaceText is to prepend text to the beginning of a
> FlowFile, or line. And then use another ReplaceText to append text to the end
> of a FlowFile, or line. We should have a strategy for "Surround" that allow
> us to both Prepend text and Append text. This will result in double the
> performance for this use case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10888) Improve performance of Record Readers when inferring schema of small FlowFiles
[
https://issues.apache.org/jira/browse/NIFI-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10888:
Fix Version/s: (was: 1.20.0)
> Improve performance of Record Readers when inferring schema of small FlowFiles
> --
>
> Key: NIFI-10888
> URL: https://issues.apache.org/jira/browse/NIFI-10888
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Labels: performance
> Attachments: InferSchema-AfterChanges.png,
> InferSchema-BeforeChanges.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When we infer the schema of a FlowFile, the Record Reader has to read all of
> the data in the FlowFile in order to infer the schema accurately. As a
> result, when we use a Record Reader, by default, we must parse the entire
> FlowFile, then seek back to the beginning of it, and parse the entire
> FlowFile again in order to return the records.
> It turns out that for smaller FlowFiles, the most expensive part of this
> cycle is actually seeking back to the beginning of the FlowFile (via
> {{{}InputStream.reset(){}}}). When {{InputStream.reset()}} is called, it
> closes the current InputStream and opens a new one, reading from the Content
> Repository again, causing a disk seek.
> Instead, if {{InputStream.mark()}} is called, we should use a
> BufferedInputStream under the hood, and if {{reset()}} is then called, we
> should call {{BufferedInputStream.reset()}} if the number of bytes consumed
> since mark is less than or equal to the read limit. We should then use
> {{{}InputStream.mark(1024 * 1024){}}}.
> Effectively, we should buffer up to 1 MB worth of content when inferring a
> schema. As a result, we can avoid that extra disk seek. For FlowFiles larger
> than 1 MB, this will not make a difference in performance. However, for
> larger FlowFiles it is less of a concern, simply because we are performing
> the seek less frequently (i.e., if we have 10 FlowFiles, each 50 MB vs. 1000
> FlowFiles each 5 KB, we end up seeking 100x less frequently in the case of
> larger FlowFiles).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10901) PublishKafka headers not sent in ProducerRecord
[ https://issues.apache.org/jira/browse/NIFI-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10901: Fix Version/s: 1.20.0 (was: 2.0.0) > PublishKafka headers not sent in ProducerRecord > --- > > Key: NIFI-10901 > URL: https://issues.apache.org/jira/browse/NIFI-10901 > Project: Apache NiFi > Issue Type: Bug >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Major > Fix For: 1.20.0, 1.19.1 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10915) Add additionaDetails information to extension manifest
Bryan Bende created NIFI-10915: -- Summary: Add additionaDetails information to extension manifest Key: NIFI-10915 URL: https://issues.apache.org/jira/browse/NIFI-10915 Project: Apache NiFi Issue Type: Improvement Reporter: Bryan Bende Assignee: Bryan Bende Fix For: nifi-nar-maven-plugin-1.4.0 The extension-manifest.xml contains all of the information used to generate the documentation of a component, but one missing part is the additionalDetails.html that can optionally be provided. This ticket is to figure out how to convey the additionaDetails information through the extension-manifest. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10177) Nifi Registry logout via OIDC
[
https://issues.apache.org/jira/browse/NIFI-10177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641510#comment-17641510
]
ASF subversion and git services commented on NIFI-10177:
Commit 844751cec0cbef47f60f861df38819522ccf9286 in nifi's branch
refs/heads/main from Emilio Setiadarma
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=844751cec0 ]
NIFI-10177: Implemented ID token logout and revoke access token logout for NiFi
Registry when using OIDC/OAuth 2.0 providers
NIFI-10177: Addressed latest PR reviews. Reworded comments in the logout
endpoint, use nifi registry properties to configure HTTP client timeouts for
OIDC logout request, used NiFiUserUtils.getNiFiUserIdentity to retrieve
identity used to delete the key
Signed-off-by: Nathan Gough
This closes #6637.
> Nifi Registry logout via OIDC
> -
>
> Key: NIFI-10177
> URL: https://issues.apache.org/jira/browse/NIFI-10177
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
>Affects Versions: 1.16.3
>Reporter: kim myungwon
>Assignee: Emilio Setiadarma
>Priority: Major
> Attachments: image-2022-06-29-12-41-52-164.png,
> image-2022-06-29-12-42-48-430.png, image-2022-06-29-12-43-25-441.png,
> image-2022-06-29-12-43-48-726.png
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> I am trying to login and logout via {*}OIDC{*}.
> Login via OIDC is well. but *{color:#de350b}logout via OIDC is not
> working.{color}*
>
> {color:#172b4d}When I logout, NiFi Registry shows "Please contact your System
> Administrator." error message.{color}
> !image-2022-06-29-12-41-52-164.png|width=1134,height=213!
>
> nifi-registry-app.log (debug level)
> {code:java}
> 022-06-29 13:32:35,691 DEBUG [NiFi Registry Web Server-15]
> o.a.nifi.registry.db.DatabaseKeyService Deleting key with identity='myungwon'.
> 2022-06-29 13:32:35,697 INFO [NiFi Registry Web Server-15]
> o.a.n.r.w.s.a.jwt.JwtService Deleted token from database.
> 2022-06-29 13:32:35,797 DEBUG [NiFi Registry Web Server-21]
> o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using
> X509IdentityProvider
> 2022-06-29 13:32:35,797 DEBUG [NiFi Registry Web Server-21]
> o.a.n.r.w.s.a.x.X509CertificateExtractor No client certificate found in
> request.
> 2022-06-29 13:32:35,797 DEBUG [NiFi Registry Web Server-21]
> o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using
> JwtIdentityProvider
> 2022-06-29 13:32:35,797 DEBUG [NiFi Registry Web Server-21]
> o.a.n.r.s.a.BearerAuthIdentityProvider HTTP Bearer Auth credentials not
> present. Not attempting to extract credentials for authentication.
> 2022-06-29 13:32:35,797 DEBUG [NiFi Registry Web Server-21]
> o.a.n.r.w.s.a.AnonymousIdentityFilter Set SecurityContextHolder to anonymous
> SecurityContext
> 2022-06-29 13:32:35,797 DEBUG [NiFi Registry Web Server-21]
> o.a.n.r.w.s.a.ResourceAuthorizationFilter Request filter authorization check
> is not required for this HTTP Method on this resource. Allowing request to
> proceed. An additional authorization check might be performed downstream of
> this filter.
> 2022-06-29 13:32:35,799 INFO [NiFi Registry Web Server-21]
> o.a.n.r.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException:
> Kerberos service ticket login not supported by this NiFi Registry. Returning
> Conflict response.
> 2022-06-29 13:32:35,799 DEBUG [NiFi Registry Web Server-21]
> o.a.n.r.w.m.IllegalStateExceptionMapper
> java.lang.IllegalStateException: Kerberos service ticket login not supported
> by this NiFi Registry
> at
> org.apache.nifi.registry.web.api.AccessResource.createAccessTokenUsingKerberosTicket(AccessResource.java:348)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
> at
> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMe
[GitHub] [nifi] thenatog closed pull request #6637: NIFI-10177: implemented ID token logout and revoke access token logou…
thenatog closed pull request #6637: NIFI-10177: implemented ID token logout and revoke access token logou… URL: https://github.com/apache/nifi/pull/6637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #6637: NIFI-10177: implemented ID token logout and revoke access token logou…
thenatog commented on PR #6637: URL: https://github.com/apache/nifi/pull/6637#issuecomment-1332621055 +1 will merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10901) PublishKafka headers not sent in ProducerRecord
[ https://issues.apache.org/jira/browse/NIFI-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-10901: -- Resolution: Fixed Status: Resolved (was: Patch Available) > PublishKafka headers not sent in ProducerRecord > --- > > Key: NIFI-10901 > URL: https://issues.apache.org/jira/browse/NIFI-10901 > Project: Apache NiFi > Issue Type: Bug >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Major > Fix For: 2.0.0, 1.19.1 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10901) PublishKafka headers not sent in ProducerRecord
[ https://issues.apache.org/jira/browse/NIFI-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641485#comment-17641485 ] ASF subversion and git services commented on NIFI-10901: Commit 282c56b5ceca47caf711898c5929e35b964bb7fe in nifi's branch refs/heads/main from greyp9 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=282c56b5ce ] NIFI-10901 - PublishKafka headers not sent in ProducerRecord (#6731) > PublishKafka headers not sent in ProducerRecord > --- > > Key: NIFI-10901 > URL: https://issues.apache.org/jira/browse/NIFI-10901 > Project: Apache NiFi > Issue Type: Bug >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Major > Fix For: 2.0.0, 1.19.1 > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 commented on pull request #6731: NIFI-10901 - PublishKafka headers not sent in ProducerRecord
markap14 commented on PR #6731: URL: https://github.com/apache/nifi/pull/6731#issuecomment-1332565542 Thanks @greyp9 . Code changes look good. Was able to test & verify everything manually. Build looks good. +1 merged to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #6731: NIFI-10901 - PublishKafka headers not sent in ProducerRecord
markap14 merged PR #6731: URL: https://github.com/apache/nifi/pull/6731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10910) Update amqp-client to 5.16.0
[ https://issues.apache.org/jira/browse/NIFI-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641467#comment-17641467 ] ASF subversion and git services commented on NIFI-10910: Commit c79eca94a11a6c0224d40f816d9e5f93d8bf9bfb in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c79eca94a1 ] NIFI-10910 Upgraded amqp-client from 5.15.0 to 5.16.0 This closes #6734 Signed-off-by: David Handermann > Update amqp-client to 5.16.0 > > > Key: NIFI-10910 > URL: https://issues.apache.org/jira/browse/NIFI-10910 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update ampq-client from 5.15.0 to 5.16.0. [Releases · > rabbitmq/rabbitmq-java-client > (github.com)|https://github.com/rabbitmq/rabbitmq-java-client/releases] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10910) Update amqp-client to 5.16.0
[ https://issues.apache.org/jira/browse/NIFI-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10910: Labels: dependency-upgrade (was: ) > Update amqp-client to 5.16.0 > > > Key: NIFI-10910 > URL: https://issues.apache.org/jira/browse/NIFI-10910 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Labels: dependency-upgrade > Time Spent: 20m > Remaining Estimate: 0h > > Update ampq-client from 5.15.0 to 5.16.0. [Releases · > rabbitmq/rabbitmq-java-client > (github.com)|https://github.com/rabbitmq/rabbitmq-java-client/releases] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10910) Update amqp-client to 5.16.0
[ https://issues.apache.org/jira/browse/NIFI-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10910. - Fix Version/s: 1.20.0 Resolution: Fixed > Update amqp-client to 5.16.0 > > > Key: NIFI-10910 > URL: https://issues.apache.org/jira/browse/NIFI-10910 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.20.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update ampq-client from 5.15.0 to 5.16.0. [Releases · > rabbitmq/rabbitmq-java-client > (github.com)|https://github.com/rabbitmq/rabbitmq-java-client/releases] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10910) Update amqp-client to 5.16.0
[ https://issues.apache.org/jira/browse/NIFI-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10910: Priority: Minor (was: Major) > Update amqp-client to 5.16.0 > > > Key: NIFI-10910 > URL: https://issues.apache.org/jira/browse/NIFI-10910 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update ampq-client from 5.15.0 to 5.16.0. [Releases · > rabbitmq/rabbitmq-java-client > (github.com)|https://github.com/rabbitmq/rabbitmq-java-client/releases] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6734: NIFI-10910 Update amqp-client to 5.16.0
exceptionfactory closed pull request #6734: NIFI-10910 Update amqp-client to 5.16.0 URL: https://github.com/apache/nifi/pull/6734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10903) Build Registry Assembly directory with zip in default profile
[ https://issues.apache.org/jira/browse/NIFI-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10903: Summary: Build Registry Assembly directory with zip in default profile (was: Fix NiFi Registry assembly profiles) > Build Registry Assembly directory with zip in default profile > - > > Key: NIFI-10903 > URL: https://issues.apache.org/jira/browse/NIFI-10903 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nathan Gough >Assignee: Nathan Gough >Priority: Minor > Fix For: 1.20.0 > > Time Spent: 20m > Remaining Estimate: 0h > > NiFi Registry no longer builds the directory by default when compiling the > NiFi project, and only builds a zip of the compiled assembly. Can we update > this to build both zip and dir by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10903) Fix NiFi Registry assembly profiles
[ https://issues.apache.org/jira/browse/NIFI-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10903. - Fix Version/s: 1.20.0 Resolution: Fixed > Fix NiFi Registry assembly profiles > --- > > Key: NIFI-10903 > URL: https://issues.apache.org/jira/browse/NIFI-10903 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nathan Gough >Assignee: Nathan Gough >Priority: Minor > Fix For: 1.20.0 > > Time Spent: 20m > Remaining Estimate: 0h > > NiFi Registry no longer builds the directory by default when compiling the > NiFi project, and only builds a zip of the compiled assembly. Can we update > this to build both zip and dir by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10903) Fix NiFi Registry assembly profiles
[ https://issues.apache.org/jira/browse/NIFI-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641462#comment-17641462 ] ASF subversion and git services commented on NIFI-10903: Commit c2c6c1bcec0769f1dfcc6168de6d253018daa7ac in nifi's branch refs/heads/main from Nathan Gough [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c2c6c1bcec ] NIFI-10903 - Updated the registry pom with a fix for the build formats, and now builds dir by default alongside the zip. This closes #6732 Signed-off-by: David Handermann > Fix NiFi Registry assembly profiles > --- > > Key: NIFI-10903 > URL: https://issues.apache.org/jira/browse/NIFI-10903 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nathan Gough >Assignee: Nathan Gough >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > NiFi Registry no longer builds the directory by default when compiling the > NiFi project, and only builds a zip of the compiled assembly. Can we update > this to build both zip and dir by default? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6732: NIFI-10903 - Updated the registry pom with a fix for the build format…
exceptionfactory closed pull request #6732: NIFI-10903 - Updated the registry pom with a fix for the build format… URL: https://github.com/apache/nifi/pull/6732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10908) Weird JDBC Connection Closed Error
[ https://issues.apache.org/jira/browse/NIFI-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cemre Mengu updated NIFI-10908: --- Attachment: image-2022-11-30-20-43-26-441.png > Weird JDBC Connection Closed Error > -- > > Key: NIFI-10908 > URL: https://issues.apache.org/jira/browse/NIFI-10908 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.3 >Reporter: Cemre Mengu >Priority: Critical > Attachments: image-2022-11-30-16-38-53-210.png, > image-2022-11-30-20-43-26-441.png > > > Using hikari connection pool, I am getting weird `connection is closed` error > for a flow during sql updates. There is nothing wrong with the data or query > as far as I can see so I am confused why I am getting this error. I have many > flows but the only one with the error is this one. > Any ideas what can be causing this error? > !image-2022-11-30-16-38-53-210.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10906) Release NiFi NAR Maven Plugin 1.4.0
[ https://issues.apache.org/jira/browse/NIFI-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10906: Assignee: Kevin Doran Status: Patch Available (was: Open) > Release NiFi NAR Maven Plugin 1.4.0 > --- > > Key: NIFI-10906 > URL: https://issues.apache.org/jira/browse/NIFI-10906 > Project: Apache NiFi > Issue Type: Task >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Major > Fix For: nifi-nar-maven-plugin-1.4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-maven] exceptionfactory closed pull request #25: NIFI-10906 Set version to 1.4.0-SNAPSHOT
exceptionfactory closed pull request #25: NIFI-10906 Set version to 1.4.0-SNAPSHOT URL: https://github.com/apache/nifi-maven/pull/25 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10905) Bump NAR Maven Plugin checkstyle version
[ https://issues.apache.org/jira/browse/NIFI-10905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10905: Resolution: Fixed Status: Resolved (was: Patch Available) > Bump NAR Maven Plugin checkstyle version > > > Key: NIFI-10905 > URL: https://issues.apache.org/jira/browse/NIFI-10905 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: nifi-nar-maven-plugin-1.4.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update checkstyle to match NiFi (v9.3, which is the last version that > supports Java 8) > also externalize checkstyle rules into standalone xml file -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-maven] exceptionfactory closed pull request #24: NIFI-10905 Update checkstyle to v9.3
exceptionfactory closed pull request #24: NIFI-10905 Update checkstyle to v9.3 URL: https://github.com/apache/nifi-maven/pull/24 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markobean commented on pull request #6685: NIFI-10608 Copying Process Group now includes non-processor referenced controller services
markobean commented on PR #6685: URL: https://github.com/apache/nifi/pull/6685#issuecomment-1332496182 First, I was able to reproduce the problem with NiFi 1.18.0. I created a process group with an unreferenced controller service. Then, I copy/pasted the PG. The new PG did not contain the unreferenced controller service. Next, I rebased this PR branch to current main because I was having build issues on my platform with source code from pre-1.19.0. Performed a full build with unit tests successfully. I installed NiFi and attempted to reproduce the copy/paste problem. The pasted PG correctly contained the unreferenced controller service. Also, I noted the same problem exists when creating a template in 1.18.0; an unreferenced CS is not in the template. Confirmed by usage and by inspecting exported template XML. With this bug fix, the template correctly includes the unreferenced CS. Thanks for the fix @NissimShiman ! +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10911) NiFi fails to start due to (likely) corrupted encrypted value(s) in flow.xml.gz
[
https://issues.apache.org/jira/browse/NIFI-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641455#comment-17641455
]
David Handermann commented on NIFI-10911:
-
Thanks for the configuration details [~john.wise], that is helpful.
It is not necessary to share the sensitive.props.key value, but {{aes/gcm/256}}
should be the value of the {{.protected}} property, not the key property
itself. The key value should be a random string of at least 12 characters.
That being said, the value should work as defined. The presence of
{{aes/gcm/256}} in the {{nifi.sensitive.props.key.protected}} property
indicates the AES-GCM encryption is being used for those property values, based
on the root key in bootstrap.conf, so that is helpful to know.
Is there any kind of extra initialization script being used to start NiFi?
After setting the value of {{nifi.sensitive.props.key}}, the value cannot be
changed without using the encrypt-config.sh command to change the values while
NiFi is stopped. If anything changes the value of that property, that will
result in the errors described.
The encrypted Base64-encoded string values will change on flow updates because
the encryptor uses a random initialization vector. That approach has been in
place for a number of versions, so that does not seem to be a problem.
> NiFi fails to start due to (likely) corrupted encrypted value(s) in
> flow.xml.gz
> ---
>
> Key: NIFI-10911
> URL: https://issues.apache.org/jira/browse/NIFI-10911
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: John Wise
>Priority: Major
> Labels: decrypt, failure, startup
>
> Over the past 2-3 weeks, a couple of our clusters have failed to start due to
> a decryption failure. nifi-app.log displays
> "{{{}o.a.n.c.serialization.FlowFromDOMFactory There was a problem decrypting
> a sensitive flow configuration value. Check that the
> nifi.sensitive.props.key value in nifi.properties matches the value used to
> encrypt the flow.xml.gz file{}}}".
> In both cases, none of the encryption key values in {{bootstrap.conf}} and
> {{nifi.properties}} have changed. The issue appears to be that one, or more,
> of the "{{{}enc{}}}{}" values in flow.xml.gz have become corrupted. The
> issue doesn't present itself until a node is restarted, at which point, NiFi
> continually fails to start due to the service being configured to
> auto-restart.
> Ideally, rather than just failing to start, NiFi would still complete the
> startup & alert the user to any decryption issues, so that they can be fixed.
> Also, the log should indicate *which* configuration value(s) it failed to
> decrypt, to help narrow down where the issue is occurring.
> In the interim, I've been removing the "{{{}enc{}}}{}" values from the
> flowfile, which allows NiFi to restart & give us the opportunity to manually
> re-enter the removed values. It's not ideal, but it does allow us to get our
> nodes back online.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10901) PublishKafka headers not sent in ProducerRecord
[ https://issues.apache.org/jira/browse/NIFI-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10901: Fix Version/s: 2.0.0 1.19.1 > PublishKafka headers not sent in ProducerRecord > --- > > Key: NIFI-10901 > URL: https://issues.apache.org/jira/browse/NIFI-10901 > Project: Apache NiFi > Issue Type: Bug >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Fix For: 2.0.0, 1.19.1 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10901) PublishKafka headers not sent in ProducerRecord
[ https://issues.apache.org/jira/browse/NIFI-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10901: Priority: Major (was: Minor) > PublishKafka headers not sent in ProducerRecord > --- > > Key: NIFI-10901 > URL: https://issues.apache.org/jira/browse/NIFI-10901 > Project: Apache NiFi > Issue Type: Bug >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Major > Fix For: 2.0.0, 1.19.1 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10914) Version information is not handled properly in case of clustered setup
[ https://issues.apache.org/jira/browse/NIFI-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-10914: Fix Version/s: 1.20.0 1.19.1 > Version information is not handled properly in case of clustered setup > -- > > Key: NIFI-10914 > URL: https://issues.apache.org/jira/browse/NIFI-10914 > Project: Apache NiFi > Issue Type: Bug >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Blocker > Fix For: 1.20.0, 1.19.1 > > > When the NiFi runs in clustered mode, the already filled "statusLocation" is > being removed from the version information after creation. > - During adding a flow from registry (creating a process group from the > standpoint of NiFi) we do enrich the version control information with > storageLocation in order to be able to be reload it from registry in case it > will be part of a nested structure > - This information is being added, but in case of multiple nodes, the > information loses afterward > It looks like, the storageLocation is empty at in thee flow.xml and flow.json > and also it is not stored in the internal representation. Single node > environments do not suffer from this. > It is suspicious that the information loses during mergin in > VersionControlInformationEntityMerger -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10914) Version information is not handled properly in case of clustered setup
Simon Bence created NIFI-10914: -- Summary: Version information is not handled properly in case of clustered setup Key: NIFI-10914 URL: https://issues.apache.org/jira/browse/NIFI-10914 Project: Apache NiFi Issue Type: Bug Reporter: Simon Bence Assignee: Simon Bence When the NiFi runs in clustered mode, the already filled "statusLocation" is being removed from the version information after creation. - During adding a flow from registry (creating a process group from the standpoint of NiFi) we do enrich the version control information with storageLocation in order to be able to be reload it from registry in case it will be part of a nested structure - This information is being added, but in case of multiple nodes, the information loses afterward It looks like, the storageLocation is empty at in thee flow.xml and flow.json and also it is not stored in the internal representation. Single node environments do not suffer from this. It is suspicious that the information loses during mergin in VersionControlInformationEntityMerger -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] thenatog commented on pull request #6637: NIFI-10177: implemented ID token logout and revoke access token logou…
thenatog commented on PR #6637: URL: https://github.com/apache/nifi/pull/6637#issuecomment-1332450878 So I've re-tested after the changes and things look good. I guess the only issue right now is that the original ticket requested that the logout feature works for Keycloak (self signed/privately signed certs) but this PR does not address that. I think we could still merge this in, but keep the ticket open, and submit a subsequent PR against the same ticket to address that part. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-10911) NiFi fails to start due to (likely) corrupted encrypted value(s) in flow.xml.gz
[
https://issues.apache.org/jira/browse/NIFI-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641432#comment-17641432
]
John Wise edited comment on NIFI-10911 at 11/30/22 4:22 PM:
[~exceptionfactory] - Unfortunately, due to our customer's secure environment,
I can't upload any of the logs or config files, so I'll replicate what I can
here.
NiFi version 1.15.3
{{{}nifi.sensitive.props.key=aes/gcm/256 ({}}}also for
{{nifi.security.keystorePasswd.protected &
nifi.security.truststorePasswd.protected)}}
{{nifi.sensitive.props.algorithm=NIFI_PBKDFS_AES_GCM_256}}
{{nifi.sensitive.props.provider=BC}}
We haven't yet figured out what causes the issue to occur, but it doesn't
affect any running NiFi nodes - all of the controller services and processors
using encrypted values work as expected, and data is processed without any
errors related to those.
I've compared multiple flowfiles in {{/nifi/conf/archive}} & it appears that
the "{{{}enc{}}}{}" values get updated every time the flowfile is updated. I'm
not sure if that's the expected behavior, but that's what I'm seeing here.
was (Author: john.wise):
[~exceptionfactory] - Unfortunately, due to our customer's secure environment,
I can't upload any of the logs or config files, so I'll replicate what I can
here.
NiFi version 1.15.0
{{{}nifi.sensitive.props.key=aes/gcm/256 ({}}}also for
{{nifi.security.keystorePasswd.protected &
nifi.security.truststorePasswd.protected)}}
{{nifi.sensitive.props.algorithm=NIFI_PBKDFS_AES_GCM_256}}
{{nifi.sensitive.props.provider=BC}}
We haven't yet figured out what causes the issue to occur, but it doesn't
affect any running NiFi nodes - all of the controller services and processors
using encrypted values work as expected, and data is processed without any
errors related to those.
I've compared multiple flowfiles in {{/nifi/conf/archive}} & it appears that
the "{{{}enc{}}}{}" values get updated every time the flowfile is updated. I'm
not sure if that's the expected behavior, but that's what I'm seeing here.
> NiFi fails to start due to (likely) corrupted encrypted value(s) in
> flow.xml.gz
> ---
>
> Key: NIFI-10911
> URL: https://issues.apache.org/jira/browse/NIFI-10911
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: John Wise
>Priority: Major
> Labels: decrypt, failure, startup
>
> Over the past 2-3 weeks, a couple of our clusters have failed to start due to
> a decryption failure. nifi-app.log displays
> "{{{}o.a.n.c.serialization.FlowFromDOMFactory There was a problem decrypting
> a sensitive flow configuration value. Check that the
> nifi.sensitive.props.key value in nifi.properties matches the value used to
> encrypt the flow.xml.gz file{}}}".
> In both cases, none of the encryption key values in {{bootstrap.conf}} and
> {{nifi.properties}} have changed. The issue appears to be that one, or more,
> of the "{{{}enc{}}}{}" values in flow.xml.gz have become corrupted. The
> issue doesn't present itself until a node is restarted, at which point, NiFi
> continually fails to start due to the service being configured to
> auto-restart.
> Ideally, rather than just failing to start, NiFi would still complete the
> startup & alert the user to any decryption issues, so that they can be fixed.
> Also, the log should indicate *which* configuration value(s) it failed to
> decrypt, to help narrow down where the issue is occurring.
> In the interim, I've been removing the "{{{}enc{}}}{}" values from the
> flowfile, which allows NiFi to restart & give us the opportunity to manually
> re-enter the removed values. It's not ideal, but it does allow us to get our
> nodes back online.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10913) Release NiFi 1.19.1
[ https://issues.apache.org/jira/browse/NIFI-10913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641433#comment-17641433 ] ASF subversion and git services commented on NIFI-10913: Commit ff9ae733d66815b05940e2efbfd8c8aec0d1c0b1 in nifi's branch refs/heads/support/nifi-1.19 from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ff9ae733d6 ] NIFI-10913 updating all to 1.19.1-snapshot. line builds and ready > Release NiFi 1.19.1 > --- > > Key: NIFI-10913 > URL: https://issues.apache.org/jira/browse/NIFI-10913 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.19.1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
