[GitHub] [nifi] jbalchan commented on a diff in pull request #7398: NIFI-11686 update elasticsearch client version to fix CVE
jbalchan commented on code in PR #7398: URL: https://github.com/apache/nifi/pull/7398#discussion_r1234738302 ## nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml: ## @@ -39,7 +39,7 @@ language governing permissions and limitations under the License. --> Note: the low-level elasticsearch-rest-client remains licensed with Apache 2.0 (https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_license.html) even after the move of the main Elasticsearch product and elasticsearch-rest-high-level-client to Elastic 2.0/SSPL 1.0 in v7.11.0+ --> -7.13.4 +7.17.10 Review Comment: Sure, I'll look into those OWASP first. Closing this PR for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jbalchan closed pull request #7398: NIFI-11686 update elasticsearch client version to fix CVE
jbalchan closed pull request #7398: NIFI-11686 update elasticsearch client version to fix CVE URL: https://github.com/apache/nifi/pull/7398 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] zhangzhonglai commented on pull request #7045: NIFI-11288 Add missing dependencies required by AWS AssumeRoleWithWebIdentity method
zhangzhonglai commented on PR #7045: URL: https://github.com/apache/nifi/pull/7045#issuecomment-1598037389 Yes, I think you are right @ChrisSamo632 . My requirements are the minimum dependency requirements for the default behavior of the AWS client. If a more customized solution is required, #5775 with my additional dependencies would be a good option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11729) Upgrade OWASP Dependency Check to 8.3.1
[ https://issues.apache.org/jira/browse/NIFI-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11729: Status: Patch Available (was: Open) > Upgrade OWASP Dependency Check to 8.3.1 > --- > > Key: NIFI-11729 > URL: https://issues.apache.org/jira/browse/NIFI-11729 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > Maven OWASP Dependency Check 8.3.1 includes several additional suppressions > of known false positives. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7411: NIFI-11729 Upgrade OWASP Dependency Check from 8.2.1 to 8.3.1
exceptionfactory opened a new pull request, #7411: URL: https://github.com/apache/nifi/pull/7411 # Summary [NIFI-11729](https://issues.apache.org/jira/browse/NIFI-11729) Upgrades the OWASP Dependency Check plugin from 8.2.1 to [8.3.1](https://github.com/jeremylong/DependencyCheck/releases) and adds several suppression notes for false positive findings. Upgrading 8.3.1 also obviates several suppressions as the plugin now incorporates corrections to avoid those findings. Additional changes include excluding the JUnit 4 dependency from RethinkDB and Hive 3 JDBC modules. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11729) Upgrade OWASP Dependency Check to 8.3.1
David Handermann created NIFI-11729: --- Summary: Upgrade OWASP Dependency Check to 8.3.1 Key: NIFI-11729 URL: https://issues.apache.org/jira/browse/NIFI-11729 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Reporter: David Handermann Assignee: David Handermann Fix For: 1.latest, 2.latest Maven OWASP Dependency Check 8.3.1 includes several additional suppressions of known false positives. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-6829) RethinkDB processors require Username and Password
[ https://issues.apache.org/jira/browse/NIFI-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734540#comment-17734540 ] David Handermann edited comment on NIFI-6829 at 6/19/23 10:22 PM: -- This issue could be addressed if there is sufficient community interest in new versions of the Processors based on RethinkDB 2.4 or later. was (Author: exceptionfactory): This issue could be address if there is sufficient community interest in new versions of the Processors based on RethinkDB 2.4 or later. > RethinkDB processors require Username and Password > -- > > Key: NIFI-6829 > URL: https://issues.apache.org/jira/browse/NIFI-6829 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.2 >Reporter: Nicholas Carenza >Priority: Major > > The RethinkDB suite of processor all require username and password. They are > marked as optional but if you don't provide them then the processor fails to > initialize. > It's probably because in the AbstractRethinkDBProcessor the `makeConnection` > method doesn't have a case for making a connection in the absence of username > and password. > When you run the processor in this case you get an unhelpful RuntimeError > with no message and a NullPointerException . > [https://github.com/apache/nifi/blob/ea9b0db2f620526c8dd0db595cf8b44c3ef835be/nifi-nar-bundles/nifi-rethinkdb-bundle/nifi-rethinkdb-processors/src/main/java/org/apache/nifi/processors/rethinkdb/AbstractRethinkDBProcessor.java#L199-L203] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-6829) RethinkDB processors require Username and Password
[ https://issues.apache.org/jira/browse/NIFI-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-6829. Resolution: Won't Fix This issue could be address if there is sufficient community interest in new versions of the Processors based on RethinkDB 2.4 or later. > RethinkDB processors require Username and Password > -- > > Key: NIFI-6829 > URL: https://issues.apache.org/jira/browse/NIFI-6829 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.2 >Reporter: Nicholas Carenza >Priority: Major > > The RethinkDB suite of processor all require username and password. They are > marked as optional but if you don't provide them then the processor fails to > initialize. > It's probably because in the AbstractRethinkDBProcessor the `makeConnection` > method doesn't have a case for making a connection in the absence of username > and password. > When you run the processor in this case you get an unhelpful RuntimeError > with no message and a NullPointerException . > [https://github.com/apache/nifi/blob/ea9b0db2f620526c8dd0db595cf8b44c3ef835be/nifi-nar-bundles/nifi-rethinkdb-bundle/nifi-rethinkdb-processors/src/main/java/org/apache/nifi/processors/rethinkdb/AbstractRethinkDBProcessor.java#L199-L203] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11728) Remove Deprecated RethinkDB Processors
[ https://issues.apache.org/jira/browse/NIFI-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11728: Status: Patch Available (was: Open) > Remove Deprecated RethinkDB Processors > -- > > Key: NIFI-11728 > URL: https://issues.apache.org/jira/browse/NIFI-11728 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The RethinkDB Processors should be removed as part of NiFi 2.0 release goals. > Current RethinkDB components are based on the unmaintained driver version > 2.3. RethinkDB Driver 2.4.0 introduced notable API changes, and the existing > components have a number of configuration and implementation issues related > to memory efficiency. The Processors have not been updated since initial > introduction and should be removed, opening the possibility of future > potential work on new versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7410: NIFI-11728 Remove RethinkDB Processors
exceptionfactory opened a new pull request, #7410: URL: https://github.com/apache/nifi/pull/7410 # Summary [NIFI-11728](https://issues.apache.org/jira/browse/NIFI-11728) Removes RethinkDB Processors from the main branch in preparation for NiFi 2.0 release. [NIFI-11727](https://issues.apache.org/jira/browse/NIFI-11727) deprecates these Processors noting RethinkDB 2.4 API changes and the lack of updates since the introduction of these components. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11727) Deprecate RethinkDB Processors for Removal
[ https://issues.apache.org/jira/browse/NIFI-11727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11727: Status: Patch Available (was: Open) > Deprecate RethinkDB Processors for Removal > -- > > Key: NIFI-11727 > URL: https://issues.apache.org/jira/browse/NIFI-11727 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 1.latest > > Time Spent: 10m > Remaining Estimate: 0h > > The RethinkDB Processors should be deprecated for removal in the next major > release version. > The Processors have not been updated since being introduced. All Processors > perform unnecessary memory-intensive operations for reading and writing JSON > objects to RethinkDB. > The [RethinkDB Java client version > 2.4.0|https://github.com/rethinkdb/rethinkdb-java] contains a number of > important changes from the earlier versions. Building new Processors based on > the current version of the driver presents an opportunity to follow better > approaches, such as making use of Records and streams instead of reading > entire FlowFiles into memory. > In light of the minimal updates to these Processors, they should be > deprecated for removal, clearing the way for potential new versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7409: NIFI-11727 Deprecate RethinkDB Processors for Removal
exceptionfactory opened a new pull request, #7409: URL: https://github.com/apache/nifi/pull/7409 # Summary [NIFI-11727](https://issues.apache.org/jira/browse/NIFI-11727) Deprecates RethinkDB Processors for subsequent NiFi 1 releases in preparation for removal from NiFi 2 releases. The current RethinkDB Processors depend on RethinkDB 2.3. Rethink 2.4.0 introduced notable API driver changes. These components have not been updated since initial introduction, and the implementations have memory usage issues related to buffering input and output FlowFile content. Deprecating these Processors for removal opens the potential for new versions based on the latest RethinkDB driver. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11728) Remove Deprecated RethinkDB Processors
David Handermann created NIFI-11728: --- Summary: Remove Deprecated RethinkDB Processors Key: NIFI-11728 URL: https://issues.apache.org/jira/browse/NIFI-11728 Project: Apache NiFi Issue Type: Task Components: Extensions Reporter: David Handermann Assignee: David Handermann The RethinkDB Processors should be removed as part of NiFi 2.0 release goals. Current RethinkDB components are based on the unmaintained driver version 2.3. RethinkDB Driver 2.4.0 introduced notable API changes, and the existing components have a number of configuration and implementation issues related to memory efficiency. The Processors have not been updated since initial introduction and should be removed, opening the possibility of future potential work on new versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11727) Deprecate RethinkDB Processors for Removal
[ https://issues.apache.org/jira/browse/NIFI-11727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11727: Description: The RethinkDB Processors should be deprecated for removal in the next major release version. The Processors have not been updated since being introduced. All Processors perform unnecessary memory-intensive operations for reading and writing JSON objects to RethinkDB. The [RethinkDB Java client version 2.4.0|https://github.com/rethinkdb/rethinkdb-java] contains a number of important changes from the earlier versions. Building new Processors based on the current version of the driver presents an opportunity to follow better approaches, such as making use of Records and streams instead of reading entire FlowFiles into memory. In light of the minimal updates to these Processors, they should be deprecated for removal, clearing the way for potential new versions. was: The RethinkDB Processors should be deprecated for removal in the next major release version. The Processors have not been updated since being introduced. All Processors perform unnecessary memory-intensive operations for reading and writing JSON objects to RethinkDB. The[ RethinkDB Java client version 2.4.0|https://github.com/rethinkdb/rethinkdb-java] contains a number of important changes from the earlier versions. Building new Processors based on the current version of the driver presents an opportunity to follow better approaches, such as making use of Records and streams instead of reading entire FlowFiles into memory. In light of the minimal updates to these Processors, they should be deprecated for removal, clearing the way for potential new versions. > Deprecate RethinkDB Processors for Removal > -- > > Key: NIFI-11727 > URL: https://issues.apache.org/jira/browse/NIFI-11727 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 1.latest > > > The RethinkDB Processors should be deprecated for removal in the next major > release version. > The Processors have not been updated since being introduced. All Processors > perform unnecessary memory-intensive operations for reading and writing JSON > objects to RethinkDB. > The [RethinkDB Java client version > 2.4.0|https://github.com/rethinkdb/rethinkdb-java] contains a number of > important changes from the earlier versions. Building new Processors based on > the current version of the driver presents an opportunity to follow better > approaches, such as making use of Records and streams instead of reading > entire FlowFiles into memory. > In light of the minimal updates to these Processors, they should be > deprecated for removal, clearing the way for potential new versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on pull request #7403: NIFI-11721 Update rethinkdb-driver to 2.4.4
exceptionfactory commented on PR #7403: URL: https://github.com/apache/nifi/pull/7403#issuecomment-1597820210 After reviewing the current RethinkDB Processor implementation and driver changes, there are enough issues that the current Processors should be deprecated for removal as described in [NIFI-11727](https://issues.apache.org/jira/browse/NIFI-11727). RethinkDB Java driver 2.4.0 included breaking changes to the query API, which would require code changes to support. The current Processors are very inefficient in terms of memory usage and JSON parsing, so they should be deprecated for removal. If there is community interest in new versions, they could start with the latest RethinkDB Driver. Closing this PR in light of the many additional changes that would be required even for this apparently minor version upgrade. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11727) Deprecate RethinkDB Processors for Removal
David Handermann created NIFI-11727: --- Summary: Deprecate RethinkDB Processors for Removal Key: NIFI-11727 URL: https://issues.apache.org/jira/browse/NIFI-11727 Project: Apache NiFi Issue Type: Task Components: Extensions Reporter: David Handermann Assignee: David Handermann The RethinkDB Processors should be deprecated for removal in the next major release version. The Processors have not been updated since being introduced. All Processors perform unnecessary memory-intensive operations for reading and writing JSON objects to RethinkDB. The[ RethinkDB Java client version 2.4.0|https://github.com/rethinkdb/rethinkdb-java] contains a number of important changes from the earlier versions. Building new Processors based on the current version of the driver presents an opportunity to follow better approaches, such as making use of Records and streams instead of reading entire FlowFiles into memory. In light of the minimal updates to these Processors, they should be deprecated for removal, clearing the way for potential new versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11724) Update parquet-avro to 1.13.1
[ https://issues.apache.org/jira/browse/NIFI-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11724. - Fix Version/s: 2.0.0 1.23.0 Resolution: Fixed > Update parquet-avro to 1.13.1 > - > > Key: NIFI-11724 > URL: https://issues.apache.org/jira/browse/NIFI-11724 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update parquet-avro to 1.13.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11725) Update woodstox-core to 6.5.1
[ https://issues.apache.org/jira/browse/NIFI-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11725: Priority: Minor (was: Major) > Update woodstox-core to 6.5.1 > - > > Key: NIFI-11725 > URL: https://issues.apache.org/jira/browse/NIFI-11725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update woodstox-core to 6.5.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11725) Update woodstox-core to 6.5.1
[ https://issues.apache.org/jira/browse/NIFI-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11725. - Fix Version/s: 2.0.0 1.23.0 Resolution: Fixed > Update woodstox-core to 6.5.1 > - > > Key: NIFI-11725 > URL: https://issues.apache.org/jira/browse/NIFI-11725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update woodstox-core to 6.5.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11724) Update parquet-avro to 1.13.1
[ https://issues.apache.org/jira/browse/NIFI-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11724: Priority: Minor (was: Major) > Update parquet-avro to 1.13.1 > - > > Key: NIFI-11724 > URL: https://issues.apache.org/jira/browse/NIFI-11724 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update parquet-avro to 1.13.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11723) Update jedis.version to 3.10.0
[ https://issues.apache.org/jira/browse/NIFI-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11723. - Fix Version/s: 2.0.0 1.23.0 Resolution: Fixed > Update jedis.version to 3.10.0 > -- > > Key: NIFI-11723 > URL: https://issues.apache.org/jira/browse/NIFI-11723 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update jedis.version to 3.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11723) Update jedis.version to 3.10.0
[ https://issues.apache.org/jira/browse/NIFI-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11723: Priority: Minor (was: Major) > Update jedis.version to 3.10.0 > -- > > Key: NIFI-11723 > URL: https://issues.apache.org/jira/browse/NIFI-11723 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update jedis.version to 3.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11722) Update spring.data.redis.version to 2.7.13
[ https://issues.apache.org/jira/browse/NIFI-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11722. - Fix Version/s: 2.0.0 1.23.0 Resolution: Fixed > Update spring.data.redis.version to 2.7.13 > -- > > Key: NIFI-11722 > URL: https://issues.apache.org/jira/browse/NIFI-11722 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update spring.data.redis.version to 2.7.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11720) Update hbase.version to 2.5.5-hadoop3
[ https://issues.apache.org/jira/browse/NIFI-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11720. - Fix Version/s: 2.0.0 1.23.0 Resolution: Fixed > Update hbase.version to 2.5.5-hadoop3 > - > > Key: NIFI-11720 > URL: https://issues.apache.org/jira/browse/NIFI-11720 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update hbase.version to 2.5.5-hadoop3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11722) Update spring.data.redis.version to 2.7.13
[ https://issues.apache.org/jira/browse/NIFI-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11722: Priority: Minor (was: Major) > Update spring.data.redis.version to 2.7.13 > -- > > Key: NIFI-11722 > URL: https://issues.apache.org/jira/browse/NIFI-11722 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update spring.data.redis.version to 2.7.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11723) Update jedis.version to 3.10.0
[ https://issues.apache.org/jira/browse/NIFI-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734502#comment-17734502 ] ASF subversion and git services commented on NIFI-11723: Commit 65899cb54732650513cca484d4a105e8bee9cc7b in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=65899cb547 ] NIFI-11723 Upgraded Jedis from 3.9.0 to 3.10.0 This closes #7405 Signed-off-by: David Handermann (cherry picked from commit 69ef6c69713c825ec9b6fec3dd666d42a798955d) > Update jedis.version to 3.10.0 > -- > > Key: NIFI-11723 > URL: https://issues.apache.org/jira/browse/NIFI-11723 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update jedis.version to 3.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11724) Update parquet-avro to 1.13.1
[ https://issues.apache.org/jira/browse/NIFI-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734503#comment-17734503 ] ASF subversion and git services commented on NIFI-11724: Commit 346bddfa780508720d30723b59814be78140aab6 in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=346bddfa78 ] NIFI-11724 Upgraded parquet-avro from 1.13.0 to 1.13.1 This closes #7406 Signed-off-by: David Handermann (cherry picked from commit b8c5bb661ed013f2ee1796e7ae2f4822b0439a6c) > Update parquet-avro to 1.13.1 > - > > Key: NIFI-11724 > URL: https://issues.apache.org/jira/browse/NIFI-11724 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update parquet-avro to 1.13.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11720) Update hbase.version to 2.5.5-hadoop3
[ https://issues.apache.org/jira/browse/NIFI-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734500#comment-17734500 ] ASF subversion and git services commented on NIFI-11720: Commit 00b868870ad36ac38dd0bc1473f912f82a0c1ae0 in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=00b868870a ] NIFI-11720 Upgraded HBase from 2.5.3 to 2.5.5 This closes #7402 Signed-off-by: David Handermann (cherry picked from commit 8be9b016771fbefaac60533f00dbe7dab63ca41e) > Update hbase.version to 2.5.5-hadoop3 > - > > Key: NIFI-11720 > URL: https://issues.apache.org/jira/browse/NIFI-11720 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update hbase.version to 2.5.5-hadoop3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11722) Update spring.data.redis.version to 2.7.13
[ https://issues.apache.org/jira/browse/NIFI-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734501#comment-17734501 ] ASF subversion and git services commented on NIFI-11722: Commit 16f73d90e23ab6c2f3af92f1382bcc470be5196c in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=16f73d90e2 ] NIFI-11722 Upgraded Spring Data Redis from 2.7.10 to 2.7.13 This closes #7404 Signed-off-by: David Handermann (cherry picked from commit d252ed5e1b3584b01f421c12c95d32e523a3cfbe) > Update spring.data.redis.version to 2.7.13 > -- > > Key: NIFI-11722 > URL: https://issues.apache.org/jira/browse/NIFI-11722 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update spring.data.redis.version to 2.7.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11725) Update woodstox-core to 6.5.1
[ https://issues.apache.org/jira/browse/NIFI-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734504#comment-17734504 ] ASF subversion and git services commented on NIFI-11725: Commit b5d6453522c0b0a84eb535042e4c4b9f513c1f8e in nifi's branch refs/heads/support/nifi-1.x from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b5d6453522 ] NIFI-11725 Upgraded Woodstox from 6.5.0 to 6.5.1 This closes #7407 Signed-off-by: David Handermann (cherry picked from commit a5ae1558398c3618e6738b1b9a9181f3015dcc74) > Update woodstox-core to 6.5.1 > - > > Key: NIFI-11725 > URL: https://issues.apache.org/jira/browse/NIFI-11725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update woodstox-core to 6.5.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 opened a new pull request, #7408: NIFI-11638 Refactored Groovy tests in nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors to Java (and JUnit 5)
dan-s1 opened a new pull request, #7408: URL: https://github.com/apache/nifi/pull/7408 …nifi-cdc-mysql-processors to Java (and JUnit 5) # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11725) Update woodstox-core to 6.5.1
[ https://issues.apache.org/jira/browse/NIFI-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734499#comment-17734499 ] ASF subversion and git services commented on NIFI-11725: Commit a5ae1558398c3618e6738b1b9a9181f3015dcc74 in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a5ae155839 ] NIFI-11725 Upgraded Woodstox from 6.5.0 to 6.5.1 This closes #7407 Signed-off-by: David Handermann > Update woodstox-core to 6.5.1 > - > > Key: NIFI-11725 > URL: https://issues.apache.org/jira/browse/NIFI-11725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update woodstox-core to 6.5.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11723) Update jedis.version to 3.10.0
[ https://issues.apache.org/jira/browse/NIFI-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734497#comment-17734497 ] ASF subversion and git services commented on NIFI-11723: Commit 69ef6c69713c825ec9b6fec3dd666d42a798955d in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=69ef6c6971 ] NIFI-11723 Upgraded Jedis from 3.9.0 to 3.10.0 This closes #7405 Signed-off-by: David Handermann > Update jedis.version to 3.10.0 > -- > > Key: NIFI-11723 > URL: https://issues.apache.org/jira/browse/NIFI-11723 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update jedis.version to 3.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11724) Update parquet-avro to 1.13.1
[ https://issues.apache.org/jira/browse/NIFI-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734498#comment-17734498 ] ASF subversion and git services commented on NIFI-11724: Commit b8c5bb661ed013f2ee1796e7ae2f4822b0439a6c in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b8c5bb661e ] NIFI-11724 Upgraded parquet-avro from 1.13.0 to 1.13.1 This closes #7406 Signed-off-by: David Handermann > Update parquet-avro to 1.13.1 > - > > Key: NIFI-11724 > URL: https://issues.apache.org/jira/browse/NIFI-11724 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update parquet-avro to 1.13.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7407: NIFI-11725 Update woodstox-core to 6.5.1
exceptionfactory closed pull request #7407: NIFI-11725 Update woodstox-core to 6.5.1 URL: https://github.com/apache/nifi/pull/7407 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11720) Update hbase.version to 2.5.5-hadoop3
[ https://issues.apache.org/jira/browse/NIFI-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734495#comment-17734495 ] ASF subversion and git services commented on NIFI-11720: Commit 8be9b016771fbefaac60533f00dbe7dab63ca41e in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8be9b01677 ] NIFI-11720 Upgraded HBase from 2.5.3 to 2.5.5 This closes #7402 Signed-off-by: David Handermann > Update hbase.version to 2.5.5-hadoop3 > - > > Key: NIFI-11720 > URL: https://issues.apache.org/jira/browse/NIFI-11720 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update hbase.version to 2.5.5-hadoop3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11722) Update spring.data.redis.version to 2.7.13
[ https://issues.apache.org/jira/browse/NIFI-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734496#comment-17734496 ] ASF subversion and git services commented on NIFI-11722: Commit d252ed5e1b3584b01f421c12c95d32e523a3cfbe in nifi's branch refs/heads/main from mr1716 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d252ed5e1b ] NIFI-11722 Upgraded Spring Data Redis from 2.7.10 to 2.7.13 This closes #7404 Signed-off-by: David Handermann > Update spring.data.redis.version to 2.7.13 > -- > > Key: NIFI-11722 > URL: https://issues.apache.org/jira/browse/NIFI-11722 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update spring.data.redis.version to 2.7.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7406: NIFI-11724 Update parquet-avro to 1.13.1
exceptionfactory closed pull request #7406: NIFI-11724 Update parquet-avro to 1.13.1 URL: https://github.com/apache/nifi/pull/7406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #7404: NIFI-11722 Update spring.data.redis.version to 2.7.13
exceptionfactory closed pull request #7404: NIFI-11722 Update spring.data.redis.version to 2.7.13 URL: https://github.com/apache/nifi/pull/7404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #7405: NIFI-11723 Update jedis.version to 3.10.0
exceptionfactory closed pull request #7405: NIFI-11723 Update jedis.version to 3.10.0 URL: https://github.com/apache/nifi/pull/7405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #7402: NIFI-11720 Update hbase.version to 2.5.5-hadoop3
exceptionfactory closed pull request #7402: NIFI-11720 Update hbase.version to 2.5.5-hadoop3 URL: https://github.com/apache/nifi/pull/7402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #7396: NIFI-11716 Backport nifi-schema-inference-utils from NIFI-11241
exceptionfactory commented on PR #7396: URL: https://github.com/apache/nifi/pull/7396#issuecomment-1597747265 Thanks for the merge @ChrisSamo632! Following this commit, I was able to cherry-pick and merge the initial ExcelReader to the support branch. https://github.com/apache/nifi/commit/4d1c71476eb688cb480200370ff2e209beb97eda -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11716) Backport nifi-schema-inference-utils to support branch
[ https://issues.apache.org/jira/browse/NIFI-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734483#comment-17734483 ] ASF subversion and git services commented on NIFI-11716: Commit 4d1c71476eb688cb480200370ff2e209beb97eda in nifi's branch refs/heads/support/nifi-1.x from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4d1c71476e ] NIFI-11167 Added ExcelReader to nifi-poi-nar - Moved shared schema inference components to nifi-record-serialization-services-shared - Backported from main branch following the addition of nifi-schema-inference-utils from NIFI-11716 This closes #7194 Signed-off-by: David Handermann (cherry picked from commit e4d1dab8f7f901acb62a896acd3d6533caefb7cc) > Backport nifi-schema-inference-utils to support branch > -- > > Key: NIFI-11716 > URL: https://issues.apache.org/jira/browse/NIFI-11716 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.23.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > NIFI-11241 Refactored shared record schema inference utilities from > nifi-record-serialization-services to nifi-schema-inference-utils. These > selected changes should be backported to the version 1 support branch so that > other modules can use the new utilities from the separate module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11167: Fix Version/s: 1.23.0 > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 10h 10m > Remaining Estimate: 0h > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734482#comment-17734482 ] ASF subversion and git services commented on NIFI-11167: Commit 4d1c71476eb688cb480200370ff2e209beb97eda in nifi's branch refs/heads/support/nifi-1.x from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4d1c71476e ] NIFI-11167 Added ExcelReader to nifi-poi-nar - Moved shared schema inference components to nifi-record-serialization-services-shared - Backported from main branch following the addition of nifi-schema-inference-utils from NIFI-11716 This closes #7194 Signed-off-by: David Handermann (cherry picked from commit e4d1dab8f7f901acb62a896acd3d6533caefb7cc) > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 10h 10m > Remaining Estimate: 0h > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 closed pull request #7396: NIFI-11716 Backport nifi-schema-inference-utils from NIFI-11241
ChrisSamo632 closed pull request #7396: NIFI-11716 Backport nifi-schema-inference-utils from NIFI-11241 URL: https://github.com/apache/nifi/pull/7396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] ChrisSamo632 commented on pull request #7396: NIFI-11716 Backport nifi-schema-inference-utils from NIFI-11241
ChrisSamo632 commented on PR #7396: URL: https://github.com/apache/nifi/pull/7396#issuecomment-1597735562 Closing after merge to `support/nifi-1.x` branch (this code is already in `main`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11716) Backport nifi-schema-inference-utils to support branch
[ https://issues.apache.org/jira/browse/NIFI-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Sampson updated NIFI-11716: - Fix Version/s: 1.23.0 (was: 1.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Backport nifi-schema-inference-utils to support branch > -- > > Key: NIFI-11716 > URL: https://issues.apache.org/jira/browse/NIFI-11716 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.23.0 > > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-11241 Refactored shared record schema inference utilities from > nifi-record-serialization-services to nifi-schema-inference-utils. These > selected changes should be backported to the version 1 support branch so that > other modules can use the new utilities from the separate module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11241) Introduce API for building Processors in Python
[ https://issues.apache.org/jira/browse/NIFI-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734451#comment-17734451 ] ASF subversion and git services commented on NIFI-11241: Commit b6bf9c57257e740965b73730c2a7935c5533d8d4 in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b6bf9c5725 ] NIFI-11716 Backported nifi-schema-inference-utils from NIFI-11241 Signed-off-by: Chris Sampson This closes #7396 > Introduce API for building Processors in Python > --- > > Key: NIFI-11241 > URL: https://issues.apache.org/jira/browse/NIFI-11241 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework, Documentation & Website, Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.0.0 > > Time Spent: 8.5h > Remaining Estimate: 0h > > Introduce an API for building Processors in Python. Use Py4J for > communication layer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11716) Backport nifi-schema-inference-utils to support branch
[ https://issues.apache.org/jira/browse/NIFI-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734450#comment-17734450 ] ASF subversion and git services commented on NIFI-11716: Commit b6bf9c57257e740965b73730c2a7935c5533d8d4 in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b6bf9c5725 ] NIFI-11716 Backported nifi-schema-inference-utils from NIFI-11241 Signed-off-by: Chris Sampson This closes #7396 > Backport nifi-schema-inference-utils to support branch > -- > > Key: NIFI-11716 > URL: https://issues.apache.org/jira/browse/NIFI-11716 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.latest > > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-11241 Refactored shared record schema inference utilities from > nifi-record-serialization-services to nifi-schema-inference-utils. These > selected changes should be backported to the version 1 support branch so that > other modules can use the new utilities from the separate module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 commented on pull request #7045: NIFI-11288 Add missing dependencies required by AWS AssumeRoleWithWebIdentity method
ChrisSamo632 commented on PR #7045: URL: https://github.com/apache/nifi/pull/7045#issuecomment-1597685949 Looking at #5775, the main difference there is that it tried to introduce the ability to let the user define where the token file was located on the NiFi server/in the NiFi container. This **can** be defined through the `AWS_WEB_IDENTITY_TOKEN_FILE` environment variable as you indicate @zhangzhonglai, but not everyone will want/be able to use that approach - so the use of the configurable setting within the `AWSCredentialsProviderControllerService` would be a sensible addition and more in-fitting with how NiFi tries to make things configurable. It might be worth considering rejuvinating @nuttybrewer's PR and include your additional dependenies to allow the use of the default env vars as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11724) Update parquet-avro to 1.13.1
[ https://issues.apache.org/jira/browse/NIFI-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-11724: - Assignee: Mike R > Update parquet-avro to 1.13.1 > - > > Key: NIFI-11724 > URL: https://issues.apache.org/jira/browse/NIFI-11724 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update parquet-avro to 1.13.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 commented on a diff in pull request #7398: NIFI-11686 update elasticsearch client version to fix CVE
ChrisSamo632 commented on code in PR #7398: URL: https://github.com/apache/nifi/pull/7398#discussion_r1234309246 ## nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml: ## @@ -39,7 +39,7 @@ language governing permissions and limitations under the License. --> Note: the low-level elasticsearch-rest-client remains licensed with Apache 2.0 (https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_license.html) even after the move of the main Elasticsearch product and elasticsearch-rest-high-level-client to Elastic 2.0/SSPL 1.0 in v7.11.0+ --> -7.13.4 +7.17.10 Review Comment: Suggest checking the suppressions (and their justifications) in the OWASP Dependency Check [suppression.xml](https://github.com/apache/nifi/blob/main/nifi-dependency-check-maven/suppressions.xml#L107) - there's are several for Elasticsearch, have you detected a known "false positive" that instead you can suppress/ignore in your own checks? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11686) Update elasticsearch.client.version to 7.17.10
[ https://issues.apache.org/jira/browse/NIFI-11686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734347#comment-17734347 ] Chris Sampson commented on NIFI-11686: -- As noted within the pom, such an upgrade will likely break connectivity to non-Elastic based Elasticsearch services (e.g. AWS OpenSearch). What CVE are you attempting to mitigate with this change and does it really impact this library (as opposed to the elasticsearch server, which nifi doesn't use)? If updating the the client library is necessary, why only to 7.17.10 instead of the latest 8.8.1? > Update elasticsearch.client.version to 7.17.10 > -- > > Key: NIFI-11686 > URL: https://issues.apache.org/jira/browse/NIFI-11686 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Jeyassri Balachandran >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Update elasticsearch.client.version to 7.17.10 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 commented on a diff in pull request #7398: NIFI-11686 update elasticsearch client version to fix CVE
ChrisSamo632 commented on code in PR #7398: URL: https://github.com/apache/nifi/pull/7398#discussion_r1234305758 ## nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml: ## @@ -39,7 +39,7 @@ language governing permissions and limitations under the License. --> Note: the low-level elasticsearch-rest-client remains licensed with Apache 2.0 (https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_license.html) even after the move of the main Elasticsearch product and elasticsearch-rest-high-level-client to Elastic 2.0/SSPL 1.0 in v7.11.0+ --> -7.13.4 +7.17.10 Review Comment: The problem with upgrading this version is documented in the comment above within the `pom.xml` - this is likely to break connectivity to any non-Elastic based services that use the Elasticsearch API (e.g. AWS OpenSearch) What CVE are you attempting to address and does it impact these Elasticsearch low-level REST API or (more likely) the Elasticsearch server at this version, which **is not** used by NiFi? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11718) Update NAR Maven Plugin to 1.5.1
[ https://issues.apache.org/jira/browse/NIFI-11718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11718: Fix Version/s: 2.0.0 1.23.0 (was: 1.latest) (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Update NAR Maven Plugin to 1.5.1 > > > Key: NIFI-11718 > URL: https://issues.apache.org/jira/browse/NIFI-11718 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11718) Update NAR Maven Plugin to 1.5.1
[ https://issues.apache.org/jira/browse/NIFI-11718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734300#comment-17734300 ] ASF subversion and git services commented on NIFI-11718: Commit b8973a077b0858acd1a2862a794f4ba322c71613 in nifi's branch refs/heads/support/nifi-1.x from Nandor Soma Abonyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b8973a077b ] NIFI-11718 Updated NAR Maven Plugin from 1.5.0 to 1.5.1 This closes #7400 Signed-off-by: David Handermann (cherry picked from commit a26de9e2339be70b28e75a75c03de47c8c5a81ce) > Update NAR Maven Plugin to 1.5.1 > > > Key: NIFI-11718 > URL: https://issues.apache.org/jira/browse/NIFI-11718 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Fix For: 1.latest, 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11718) Update NAR Maven Plugin to 1.5.1
[ https://issues.apache.org/jira/browse/NIFI-11718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734298#comment-17734298 ] ASF subversion and git services commented on NIFI-11718: Commit a26de9e2339be70b28e75a75c03de47c8c5a81ce in nifi's branch refs/heads/main from Nandor Soma Abonyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a26de9e233 ] NIFI-11718 Updated NAR Maven Plugin from 1.5.0 to 1.5.1 This closes #7400 Signed-off-by: David Handermann > Update NAR Maven Plugin to 1.5.1 > > > Key: NIFI-11718 > URL: https://issues.apache.org/jira/browse/NIFI-11718 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7400: NIFI-11718 Update NAR Maven Plugin to 1.5.1
exceptionfactory closed pull request #7400: NIFI-11718 Update NAR Maven Plugin to 1.5.1 URL: https://github.com/apache/nifi/pull/7400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
lordgamez commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1234249400
##
extensions/aws/processors/PutS3Object.cpp:
##
@@ -77,7 +78,31 @@ void PutS3Object::onSchedule(const
std::shared_ptr &contex
use_virtual_addressing_ = !*use_path_style_access;
}
+ context->getProperty(MultipartThreshold.getName(), multipart_threshold_);
+ if (multipart_threshold_ > getMaxUploadSize() || multipart_threshold_ <
getMinPartSize()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Multipart Threshold is not
between the valid 5MB and 5GB range!");
+ }
+ logger_->log_debug("PutS3Object: Multipart Threshold %" PRIu64,
multipart_threshold_);
+ context->getProperty(MultipartPartSize.getName(), multipart_size_);
+ if (multipart_size_ > getMaxUploadSize() || multipart_size_ <
getMinPartSize()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Multipart Part Size is not
between the valid 5MB and 5GB range!");
+ }
+ logger_->log_debug("PutS3Object: Multipart Size %" PRIu64, multipart_size_);
+
+
+ multipart_upload_ageoff_interval_ =
minifi::utils::getRequiredPropertyOrThrow(*context,
MultipartUploadAgeOffInterval.getName()).getMilliseconds();
+ logger_->log_debug("PutS3Object: Multipart Upload Ageoff Interval %" PRIu64
" ms", multipart_upload_ageoff_interval_.count());
+
+ multipart_upload_max_age_threshold_ =
minifi::utils::getRequiredPropertyOrThrow(*context,
MultipartUploadMaxAgeThreshold.getName()).getMilliseconds();
+ logger_->log_debug("PutS3Object: Multipart Upload Max Age Threshold %"
PRIu64 " ms", multipart_upload_max_age_threshold_.count());
+
fillUserMetadata(context);
+
+ std::string multipart_temp_dir;
+ context->getProperty(TemporaryDirectoryMultipartState.getName(),
multipart_temp_dir);
+
+
+ s3_wrapper_.initailizeMultipartUploadStateStorage(multipart_temp_dir,
getUUIDStr());
Review Comment:
Added a commit with this change: ea2548e618b539facccf226fbc417c6d46fe71ee
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-11694) SAML logout signature verification failed
[
https://issues.apache.org/jira/browse/NIFI-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-11694.
-
Resolution: Information Provided
You're welcome [~fube1]! I will close this issue for now, but feel free to
follow up if you run into other issues.
> SAML logout signature verification failed
> -
>
> Key: NIFI-11694
> URL: https://issues.apache.org/jira/browse/NIFI-11694
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Beat Fuellemann
>Assignee: David Handermann
>Priority: Major
>
> We activated SAML Authentication with the following configuration:
> {code:java}
> nifi.security.user.saml.request.signing.enabled=false
> nifi.security.user.saml.want.assertions.signed=true
> nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> nifi.security.user.saml.authentication.expiration=1 hours
> nifi.security.user.saml.single.logout.enabled=true
> nifi.security.user.saml.http.client.truststore.strategy=JDK
> nifi.security.user.saml.http.client.connect.timeout=30 secs
> nifi.security.user.saml.http.client.read.timeout=30 secs{code}
> Login works fine.
> But during logout, it looks that NIFI signs the request, even if we
> "request.signing.enabled=false". This causes the logout fail on the IdP.
> it gives us the following error:
> {code:java}
> 2023-06-15 06:38:35,629 INFO [NiFi Web Server-78]
> org.apache.nifi.web.api.AccessResource Logout Request
> [7b8370e8-752f-484e-8caa-5a8ce3f29caf] Identity [TX] started
> 2023-06-15 06:38:35,673 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256': supported
> 2023-06-15 06:38:35,674 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmlenc#sha256': supported
> 2023-06-15 06:38:35,676 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Resolved
> SignatureSigningParameters:
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signing credential with
> key algorithm: RSA
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature algorithm URI:
> http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature
> KeyInfoGenerator: present
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference digest method
> algorithm URI: http://www.w3.org/2001/04/xmlenc#sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference
> canonicalization algorithm URI: null
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Canonicalization
> algorithm URI: http://www.w3.org/2001/10/xml-exc-c14n#
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver HMAC output length: null
> 2023-06-15 06:38:35,678 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computing signature over input using
> private key of type RSA and JCA algorithm ID SHA256withRSA
> 2023-06-15 06:38:35,691 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computed signature: [3, e, 2, 0, d, 4,
> 0, 7, d, 8, 2, 6, 9, 7, a, f, c, 1, 0, 8, b, 9, 5, f, d, 0, a, 3, 2, 9, b, 9,
> 3, d, b, 5, 2, 4, 2, f, a, 9, 7, 1, 2, 3, d, 3, c, d, 9, 8, 1, 0, a, 5, 1, 8,
> 8, 6, 3, 3, 8, a, a, 7, f, 1, 8, 9, c, a, 3, 5, 7, b, 2, e, c, 2, 5, 3, 7, 1,
> 2, b, 2, 1, 4, 3, e, 6, f, 4, 8, 5, e, 1, d, 3, e, 1, a, 5, 1, a, f, 8, 2, f,
> a, 3, 8, a, 3, 2, f, 0, 6, d, e, 8, 7, b, 9, f, d, 2, 8, b, d, f, 8, 2, 7, 9,
> 3, 5, 1, d, c, 1, 2, e, 3, 4, 8, f, 3, 7, e, 6, 5, c, e, 3, 8, 3, 1, 2, a, 6,
> 5, 6, 1, 2, 8, c, 8, 3, 8, 3, a, a, 9, 6, 2, a, 8, 3, 2, 9, 2, 5, 9, 2, b, e,
> 6, d, 0, 0, e, 1, 8, 9, 2, 4, 0, 2, a, 5, c, b, 3, 1, b, 1, b, b, a, e, 0, f,
> 6, e, 8, 0, b, c, 9, 0, 0, f, c, 1, 7, 5, c, 4, d, b, 5, c, 1, 0, f, b, 3, d,
> 4, c, e, 5, 7, 4, 3, 8, f, b, 1, f, 1, d, a, a, 0, c, 8, e, d, b, 5, 0, 5, 9,
> 7, a, c, 8, 7, 9, 4, 4, d, f, 1, 3, 2, 9, 6, 6, 2, 4, 1, e, c, 8, 3, 7, 3, 2,
> 4, 9, a, 9, 4, 0, 3, c, 4, b, 2, f, 1, b, 9, b, 4, 3, 1, f, 6, d, 3, d, 4, 5,
> 0, f, 7, 8, d, 1, c, 1, 8, f, 2, 4, 8, 3, 3, 9, e, 3, 4, b, 5, 0, 9, 9, 1, 0,
> c, b, e, 3, 7, 9, 4, 4, d, 7, a, a, 4, 6, 6, 0, 1, b, c, 8, b, 4, c, 9, c, a,
> b, 2, b, e, d, 4, 4, 4, 0, a, b, 9, 4, 4, 4, 4, 9, e, a, b, 4, b, 0, 1, 4, 0,
> b, 7, 2, f, d, b, 8,
[jira] [Updated] (NIFI-11726) Nifi 1.20.0 Funnel Loop
[ https://issues.apache.org/jira/browse/NIFI-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elif Beyza Çeltik updated NIFI-11726: - Description: In Apache Nifi 1.20.0 there is a problem with deleting the funnel loop as it crushes the nodes. This issue wasn't relevant in the 1.15.0 version but with 1.20 and 1.21.0 it is not possible to delete the funnel loop. Nifi crashes while trying to delete the funnels. (was: In Apache Nifi 1.20.0 there is a problem with deleting the funnel loop as it crushes the nodes and makes it impossible to touch the funnel loop. This issue wasn't relevant in the 1.15.0 version i used to work with as I could easily delete the funnel loop but with 1.20 and 1.21.0 it is not possible to delete the funnel loop. Nifi crashes while I try to delete the funnels. ) > Nifi 1.20.0 Funnel Loop > --- > > Key: NIFI-11726 > URL: https://issues.apache.org/jira/browse/NIFI-11726 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0, 1.19.1, 1.21.0 > Environment: Apache Nifi >Reporter: Elif Beyza Çeltik >Priority: Minor > Labels: Funnel, Nifi, loop > Fix For: 1.20.0 > > Attachments: nifi_delete_loop.png, nifi_error.png, nifi_loop.png > > > In Apache Nifi 1.20.0 there is a problem with deleting the funnel loop as it > crushes the nodes. This issue wasn't relevant in the 1.15.0 version but with > 1.20 and 1.21.0 it is not possible to delete the funnel loop. Nifi crashes > while trying to delete the funnels. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11726) Nifi 1.20.0 Funnel Loop
Elif Beyza Çeltik created NIFI-11726: Summary: Nifi 1.20.0 Funnel Loop Key: NIFI-11726 URL: https://issues.apache.org/jira/browse/NIFI-11726 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.21.0, 1.19.1, 1.20.0 Environment: Apache Nifi Reporter: Elif Beyza Çeltik Fix For: 1.20.0 Attachments: nifi_delete_loop.png, nifi_error.png, nifi_loop.png In Apache Nifi 1.20.0 there is a problem with deleting the funnel loop as it crushes the nodes and makes it impossible to touch the funnel loop. This issue wasn't relevant in the 1.15.0 version i used to work with as I could easily delete the funnel loop but with 1.20 and 1.21.0 it is not possible to delete the funnel loop. Nifi crashes while I try to delete the funnels. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7388: NIFI-11705 Append Operating System section in NiFi diagnostic tool
exceptionfactory commented on code in PR #7388:
URL: https://github.com/apache/nifi/pull/7388#discussion_r1234107619
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/diagnostics/bootstrap/shell/ShellCommandExecutor.java:
##
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.diagnostics.bootstrap.shell;
+
+import
org.apache.nifi.diagnostics.bootstrap.shell.commands.AbstractShellCommand;
+
+import java.io.IOException;
+import java.util.Collection;
+
+public class ShellCommandExecutor {
+
+public static Collection execute(final AbstractShellCommand
command) {
Review Comment:
Thanks, moving the method sounds like a good approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7388: NIFI-11705 Append Operating System section in NiFi diagnostic tool
exceptionfactory commented on code in PR #7388:
URL: https://github.com/apache/nifi/pull/7388#discussion_r1234108361
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/diagnostics/bootstrap/shell/commands/GetDiskLayoutCommand.java:
##
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.diagnostics.bootstrap.shell.commands;
+
+import org.apache.nifi.diagnostics.bootstrap.shell.results.LineSplittingResult;
+
+import java.util.Arrays;
+import java.util.List;
+
+public class GetDiskLayoutCommand extends AbstractShellCommand {
+private static final String COMMAND_NAME = "GetDiskLayout";
+private static final List RESULT_LABELS =
Arrays.asList("FileSystem/DeviceId", "Total size", "Used", "Free");
+private static final String REGEX_FOR_SPLITTING = "\\s+";
+private static final String[] GET_DISK_LAYOUT_FOR_MAC = new String[]
{"/bin/sh", "-c", "df -Ph | sed 1d"};
+private static final String[] GET_DISK_LAYOUT_FOR_LINUX = new String[]
{"/bin/sh", "-c", "df -h --output=source,size,used,avail,target -x tmpfs -x
devtmpfs | sed 1d"};
+private static final String[] GET_DISK_LAYOUT_FOR_WINDOWS = new String[]
{"powershell.exe", "Get-CIMInstance Win32_LogicalDisk " +
Review Comment:
Thanks for pointing out that detail, that's good to know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
lordgamez commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1234106980
##
extensions/aws/s3/S3Wrapper.cpp:
##
@@ -297,4 +404,57 @@ FetchObjectResult S3Wrapper::fillFetchObjectResult(const
GetObjectRequestParamet
return result;
}
+void S3Wrapper::addListMultipartUploadResults(const
Aws::Vector& uploads,
std::optional max_upload_age,
+std::vector& filtered_uploads) {
+ const auto now = Aws::Utils::DateTime::Now();
+ for (const auto& upload : uploads) {
+if (max_upload_age && now - upload.GetInitiated() <= *max_upload_age) {
+ logger_->log_debug("Multipart upload with key '%s' and upload id '%s'
did not meet the age limit", upload.GetKey(), upload.GetUploadId());
+ continue;
+}
Review Comment:
Updated the variable names and added some comments in
5624af2c3ed7c36ecb55193c739d6845bf26d877
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
lordgamez commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1234106443
##
docker/test/integration/cluster/checkers/AwsChecker.py:
##
@@ -29,6 +29,18 @@ def check_s3_server_object_data(self, container_name,
test_data):
(code, file_data) =
self.container_communicator.execute_command(container_name, ["cat", s3_mock_dir
+ "/binaryData"])
return code == 0 and file_data == test_data
+@retry_check()
+def check_s3_server_object_hash(self, container_name: str,
expected_file_hash: str):
+(code, output) =
self.container_communicator.execute_command(container_name, ["find",
"/s3mockroot/test_bucket", "-mindepth", "1", "-maxdepth", "1", "-type", "d"])
+if code != 0:
+return False
+s3_mock_dir = output.strip()
+(code, md5_output) =
self.container_communicator.execute_command(container_name, ["md5sum",
s3_mock_dir + "/binaryData"])
+if code != 0:
+return False
+file_hash = md5_output.split(' ')[0].strip()
+return code == 0 and file_hash == expected_file_hash
Review Comment:
Removed in 5624af2c3ed7c36ecb55193c739d6845bf26d877
##
extensions/aws/s3/MultipartUploadStateStorage.cpp:
##
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "MultipartUploadStateStorage.h"
+
+#include
+
+#include "utils/StringUtils.h"
+
+namespace org::apache::nifi::minifi::aws::s3 {
+
+MultipartUploadStateStorage::MultipartUploadStateStorage(const std::string&
state_directory, const std::string& state_id) {
+ if (state_directory.empty()) {
+char format[] = "/var/tmp/nifi-minifi-cpp.s3-multipart-upload.XX";
+state_file_path_ =
minifi::utils::file::FileUtils::create_temp_directory(format);
+ } else {
+state_file_path_ = std::filesystem::path(state_directory) /
std::string(state_id + "-s3-multipart-upload-state.properties");
+if (!std::filesystem::exists(state_file_path_)) {
+ std::filesystem::create_directories(state_file_path_.parent_path());
+ std::ofstream ofs(state_file_path_);
Review Comment:
My mistake, removed in 5624af2c3ed7c36ecb55193c739d6845bf26d877
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
lordgamez commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1234094500
##
extensions/aws/s3/S3Wrapper.cpp:
##
@@ -297,4 +404,57 @@ FetchObjectResult S3Wrapper::fillFetchObjectResult(const
GetObjectRequestParamet
return result;
}
+void S3Wrapper::addListMultipartUploadResults(const
Aws::Vector& uploads,
std::optional max_upload_age,
+std::vector& filtered_uploads) {
+ const auto now = Aws::Utils::DateTime::Now();
+ for (const auto& upload : uploads) {
+if (max_upload_age && now - upload.GetInitiated() <= *max_upload_age) {
+ logger_->log_debug("Multipart upload with key '%s' and upload id '%s'
did not meet the age limit", upload.GetKey(), upload.GetUploadId());
+ continue;
+}
Review Comment:
The lister functions list all of the pending uploads by default if the age
limit is not set, which is an optional parameter. I wanted to reuse it for the
age off lister as well, because that's the only difference. I agree we that the
namings should be better in this case I can work on that a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
fgerlits commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1234048245
##
docker/test/integration/cluster/checkers/AwsChecker.py:
##
@@ -29,6 +29,18 @@ def check_s3_server_object_data(self, container_name,
test_data):
(code, file_data) =
self.container_communicator.execute_command(container_name, ["cat", s3_mock_dir
+ "/binaryData"])
return code == 0 and file_data == test_data
+@retry_check()
+def check_s3_server_object_hash(self, container_name: str,
expected_file_hash: str):
+(code, output) =
self.container_communicator.execute_command(container_name, ["find",
"/s3mockroot/test_bucket", "-mindepth", "1", "-maxdepth", "1", "-type", "d"])
+if code != 0:
+return False
+s3_mock_dir = output.strip()
+(code, md5_output) =
self.container_communicator.execute_command(container_name, ["md5sum",
s3_mock_dir + "/binaryData"])
+if code != 0:
+return False
+file_hash = md5_output.split(' ')[0].strip()
+return code == 0 and file_hash == expected_file_hash
Review Comment:
we could remove the second `code == 0` check now:
```suggestion
if code != 0:
return False
file_hash = md5_output.split(' ')[0].strip()
return file_hash == expected_file_hash
```
##
extensions/aws/s3/MultipartUploadStateStorage.cpp:
##
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "MultipartUploadStateStorage.h"
+
+#include
+
+#include "utils/StringUtils.h"
+
+namespace org::apache::nifi::minifi::aws::s3 {
+
+MultipartUploadStateStorage::MultipartUploadStateStorage(const std::string&
state_directory, const std::string& state_id) {
+ if (state_directory.empty()) {
+char format[] = "/var/tmp/nifi-minifi-cpp.s3-multipart-upload.XX";
+state_file_path_ =
minifi::utils::file::FileUtils::create_temp_directory(format);
+ } else {
+state_file_path_ = std::filesystem::path(state_directory) /
std::string(state_id + "-s3-multipart-upload-state.properties");
+if (!std::filesystem::exists(state_file_path_)) {
+ std::filesystem::create_directories(state_file_path_.parent_path());
+ std::ofstream ofs(state_file_path_);
Review Comment:
it's commented out now; it would be better to delete the line
##
extensions/aws/s3/S3Wrapper.cpp:
##
@@ -297,4 +404,57 @@ FetchObjectResult S3Wrapper::fillFetchObjectResult(const
GetObjectRequestParamet
return result;
}
+void S3Wrapper::addListMultipartUploadResults(const
Aws::Vector& uploads,
std::optional max_upload_age,
+std::vector& filtered_uploads) {
+ const auto now = Aws::Utils::DateTime::Now();
+ for (const auto& upload : uploads) {
+if (max_upload_age && now - upload.GetInitiated() <= *max_upload_age) {
+ logger_->log_debug("Multipart upload with key '%s' and upload id '%s'
did not meet the age limit", upload.GetKey(), upload.GetUploadId());
+ continue;
+}
Review Comment:
So `multipartUploadExistsInS3()`, `listMultipartUploads()` and
`addListMultipartUploadResults()` only list/check uploads which have aged off?
In that case, the names of the functions, and also local variables like
`result` and `filtered_upload` should reflect this, to make the logic clearer.
Also, it would help people reading the code if the log message was something
like "... has not aged off yet" instead of the vaguer ".. did not meet the age
limit".
##
extensions/aws/tests/PutS3ObjectTests.cpp:
##
@@ -71,7 +87,7 @@ TEST_CASE_METHOD(PutS3ObjectTestsFixture, "Test AWS
credential setting", "[awsCr
setCredentialsService();
}
- test_controller.runSession(plan, true);
+ test_controller.runSession(plan);
Review Comment:
that's weird, but not important
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mr1716 opened a new pull request, #7407: NIFI-11725 Update woodstox-core to 6.5.1
mr1716 opened a new pull request, #7407: URL: https://github.com/apache/nifi/pull/7407 # Summary [NIFI-11725](https://issues.apache.org/jira/browse/NIFI-11725) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-11725) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11725) Update woodstox-core to 6.5.1
[ https://issues.apache.org/jira/browse/NIFI-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-11725: - Assignee: Mike R > Update woodstox-core to 6.5.1 > - > > Key: NIFI-11725 > URL: https://issues.apache.org/jira/browse/NIFI-11725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update woodstox-core to 6.5.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11725) Update woodstox-core to 6.5.1
Mike R created NIFI-11725: - Summary: Update woodstox-core to 6.5.1 Key: NIFI-11725 URL: https://issues.apache.org/jira/browse/NIFI-11725 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.22.0 Reporter: Mike R Update woodstox-core to 6.5.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 opened a new pull request, #7406: NIFI-11724 Update parquet-avro to 1.13.1
mr1716 opened a new pull request, #7406: URL: https://github.com/apache/nifi/pull/7406 # Summary [NIFI-11724](https://issues.apache.org/jira/browse/NIFI-11724) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-11724) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11724) Update parquet-avro to 1.13.1
Mike R created NIFI-11724: - Summary: Update parquet-avro to 1.13.1 Key: NIFI-11724 URL: https://issues.apache.org/jira/browse/NIFI-11724 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.22.0 Reporter: Mike R Update parquet-avro to 1.13.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 opened a new pull request, #7405: NIFI-11723 Update jedis.version to 3.10.0
mr1716 opened a new pull request, #7405: URL: https://github.com/apache/nifi/pull/7405 # Summary [NIFI-11723](https://issues.apache.org/jira/browse/NIFI-11723) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-11723) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11723) Update jedis.version to 3.10.0
[ https://issues.apache.org/jira/browse/NIFI-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-11723: - Assignee: Mike R > Update jedis.version to 3.10.0 > -- > > Key: NIFI-11723 > URL: https://issues.apache.org/jira/browse/NIFI-11723 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update jedis.version to 3.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11723) Update jedis.version to 3.10.0
Mike R created NIFI-11723: - Summary: Update jedis.version to 3.10.0 Key: NIFI-11723 URL: https://issues.apache.org/jira/browse/NIFI-11723 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.22.0 Reporter: Mike R Update jedis.version to 3.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
lordgamez commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1234036346
##
extensions/aws/s3/S3Wrapper.cpp:
##
@@ -75,42 +69,155 @@ std::string
S3Wrapper::getEncryptionString(Aws::S3::Model::ServerSideEncryption
return "";
}
-std::optional S3Wrapper::putObject(const
PutObjectRequestParameters& put_object_params, const
std::shared_ptr& data_stream) {
- Aws::S3::Model::PutObjectRequest request;
- request.SetBucket(put_object_params.bucket);
- request.SetKey(put_object_params.object_key);
-
request.SetStorageClass(STORAGE_CLASS_MAP.at(put_object_params.storage_class));
-
request.SetServerSideEncryption(SERVER_SIDE_ENCRYPTION_MAP.at(put_object_params.server_side_encryption));
- request.SetContentType(put_object_params.content_type);
- request.SetMetadata(put_object_params.user_metadata_map);
+std::shared_ptr S3Wrapper::readFlowFileStream(const
std::shared_ptr& stream, uint64_t read_limit, uint64_t&
read_size_out) {
+ std::vector buffer;
+ buffer.resize(BUFFER_SIZE);
+ auto data_stream = std::make_shared();
+ uint64_t read_size = 0;
+ while (read_size < read_limit) {
+const auto next_read_size = (std::min)(read_limit - read_size,
BUFFER_SIZE);
+const auto read_ret = stream->read(gsl::make_span(buffer).subspan(0,
next_read_size));
+if (io::isError(read_ret)) {
+ throw StreamReadException("Reading flow file inputstream failed!");
+}
+if (read_ret > 0) {
+ data_stream->write(reinterpret_cast(buffer.data()),
gsl::narrow(next_read_size));
+ read_size += read_ret;
+} else {
+ break;
+}
+ }
+ read_size_out = read_size;
+ return data_stream;
+}
+
+std::optional S3Wrapper::putObject(const
PutObjectRequestParameters& put_object_params, const
std::shared_ptr& stream, uint64_t flow_size) {
+ uint64_t read_size{};
+ auto data_stream = readFlowFileStream(stream, flow_size, read_size);
+ auto request =
createPutObjectRequest(put_object_params);
request.SetBody(data_stream);
- request.SetGrantFullControl(put_object_params.fullcontrol_user_list);
- request.SetGrantRead(put_object_params.read_permission_user_list);
- request.SetGrantReadACP(put_object_params.read_acl_user_list);
- request.SetGrantWriteACP(put_object_params.write_acl_user_list);
- setCannedAcl(request, put_object_params.canned_acl);
auto aws_result = request_sender_->sendPutObjectRequest(request,
put_object_params.credentials, put_object_params.client_config,
put_object_params.use_virtual_addressing);
if (!aws_result) {
return std::nullopt;
}
- PutObjectResult result;
- // Etags are returned by AWS in quoted form that should be removed
- result.etag =
minifi::utils::StringUtils::removeFramingCharacters(aws_result->GetETag(), '"');
- result.version = aws_result->GetVersionId();
+ return createPutObjectResult(*aws_result);
+}
- // GetExpiration returns a string pair with a date and a ruleid in
'expiry-date=\"\", rule-id=\"\"' format
- // s3.expiration only needs the date member of this pair
- result.expiration =
getExpiration(aws_result->GetExpiration()).expiration_time;
- result.ssealgorithm =
getEncryptionString(aws_result->GetServerSideEncryption());
+std::optional S3Wrapper::uploadParts(const
PutObjectRequestParameters& put_object_params, const
std::shared_ptr& stream,
+MultipartUploadState upload_state) {
+ stream->seek(upload_state.uploaded_size);
+ S3Wrapper::UploadPartsResult result;
+ result.upload_id = upload_state.upload_id;
+ result.part_etags = upload_state.uploaded_etags;
+ const auto flow_size = upload_state.full_size - upload_state.uploaded_size;
+ const size_t part_count = flow_size % upload_state.part_size == 0 ?
flow_size / upload_state.part_size : flow_size / upload_state.part_size + 1;
+ size_t total_read = 0;
+ const size_t start_part = upload_state.uploaded_parts + 1;
+ const size_t last_part = start_part + part_count - 1;
+ for (size_t i = start_part; i <= last_part; ++i) {
+uint64_t read_size{};
+const auto remaining = flow_size - total_read;
+const auto next_read_size = remaining < upload_state.part_size ? remaining
: upload_state.part_size;
+auto stream_ptr = readFlowFileStream(stream, next_read_size, read_size);
+total_read += read_size;
+
+Aws::S3::Model::UploadPartRequest upload_part_request;
+upload_part_request.WithBucket(put_object_params.bucket)
+ .WithKey(put_object_params.object_key)
+ .WithPartNumber(i)
+ .WithUploadId(upload_state.upload_id);
+upload_part_request.SetBody(stream_ptr);
+
+Aws::Utils::ByteBuffer
part_md5(Aws::Utils::HashingUtils::CalculateMD5(*stream_ptr));
+
upload_part_request.SetContentMD5(Aws::Utils::HashingUtils::Base64Encode(part_md5));
+
+auto upload_part_result =
request_sender_->sendUploadPartRequest(upload_part_request,
put_object_params.credentials, put_object_params.client_config,
put_object_params.use_virtual_addressing);
+i
[GitHub] [nifi] mr1716 opened a new pull request, #7404: NIFI-11722 Update spring.data.redis.version to 2.7.13
mr1716 opened a new pull request, #7404: URL: https://github.com/apache/nifi/pull/7404 # Summary [NIFI-11722](https://issues.apache.org/jira/browse/NIFI-11722) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-11722) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
lordgamez commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1234035975
##
docker/test/integration/cluster/checkers/AwsChecker.py:
##
@@ -29,6 +29,16 @@ def check_s3_server_object_data(self, container_name,
test_data):
(code, file_data) =
self.container_communicator.execute_command(container_name, ["cat", s3_mock_dir
+ "/binaryData"])
return code == 0 and file_data == test_data
+@retry_check()
+def check_s3_server_object_hash(self, container_name: str,
expected_file_hash: str):
+(code, output) =
self.container_communicator.execute_command(container_name, ["find",
"/s3mockroot/test_bucket", "-mindepth", "1", "-maxdepth", "1", "-type", "d"])
+if code != 0:
+return False
+s3_mock_dir = output.strip()
+(code, md5_output) =
self.container_communicator.execute_command(container_name, ["md5sum",
s3_mock_dir + "/binaryData"])
+file_hash = md5_output.split(' ')[0].strip()
+return code == 0 and file_hash == expected_file_hash
Review Comment:
Updated in b45280b22c6f4e74f36cef4fd4b6305f177abe59
##
docker/test/integration/features/steps/steps.py:
##
@@ -275,6 +275,11 @@ def step_impl(context, content, path):
context.test.add_test_data(path, content)
+@given("a file with {size} content is present in \"{path}\"")
Review Comment:
Updated in b45280b22c6f4e74f36cef4fd4b6305f177abe59
##
extensions/aws/s3/MultipartUploadStateStorage.cpp:
##
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "MultipartUploadStateStorage.h"
+
+#include
+
+#include "utils/StringUtils.h"
+
+namespace org::apache::nifi::minifi::aws::s3 {
+
+MultipartUploadStateStorage::MultipartUploadStateStorage(const std::string&
state_directory, const std::string& state_id) {
+ if (state_directory.empty()) {
+char format[] = "/var/tmp/nifi-minifi-cpp.s3-multipart-upload.XX";
+state_file_path_ =
minifi::utils::file::FileUtils::create_temp_directory(format);
+ } else {
+state_file_path_ = std::filesystem::path(state_directory) /
std::string(state_id + "-s3-multipart-upload-state.properties");
+if (!std::filesystem::exists(state_file_path_)) {
+ std::filesystem::create_directories(state_file_path_.parent_path());
+ std::ofstream ofs(state_file_path_);
Review Comment:
Good catch, it shouldn't be needed, I also corrected a mistake in the
`state_directory.empty()` branch and added a test for it in
b45280b22c6f4e74f36cef4fd4b6305f177abe59
##
extensions/aws/s3/MultipartUploadStateStorage.cpp:
##
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "MultipartUploadStateStorage.h"
+
+#include
+
+#include "utils/StringUtils.h"
+
+namespace org::apache::nifi::minifi::aws::s3 {
+
+MultipartUploadStateStorage::MultipartUploadStateStorage(const std::string&
state_directory, const std::string& state_id) {
+ if (state_directory.empty()) {
+char format[] = "/var/tmp/nifi-minifi-cpp.s3-multipart-upload.XX";
+state_file_path_ =
minifi::utils::file::FileUtils::create_temp_directory(format);
+ } else {
+state_file_path_ = std::filesystem::path(state_directory) /
std::string(state_id + "-s3-multipart-upload-state.properties");
+if (!std::filesystem::exists(state_file_path_)) {
+ std::filesystem::create_directories(stat
[GitHub] [nifi] mr1716 opened a new pull request, #7403: NIFI-11721 Update rethinkdb-driver to 2.4.4
mr1716 opened a new pull request, #7403: URL: https://github.com/apache/nifi/pull/7403 # Summary [NIFI-11721](https://issues.apache.org/jira/browse/NIFI-11721) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-11721) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11722) Update spring.data.redis.version to 2.7.13
[ https://issues.apache.org/jira/browse/NIFI-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-11722: - Assignee: Mike R > Update spring.data.redis.version to 2.7.13 > -- > > Key: NIFI-11722 > URL: https://issues.apache.org/jira/browse/NIFI-11722 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update spring.data.redis.version to 2.7.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11721) Update rethinkdb-driver to 2.4.4
[ https://issues.apache.org/jira/browse/NIFI-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-11721: - Assignee: Mike R > Update rethinkdb-driver to 2.4.4 > > > Key: NIFI-11721 > URL: https://issues.apache.org/jira/browse/NIFI-11721 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update rethinkdb-driver to 2.4.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11722) Update spring.data.redis.version to 2.7.13
Mike R created NIFI-11722: - Summary: Update spring.data.redis.version to 2.7.13 Key: NIFI-11722 URL: https://issues.apache.org/jira/browse/NIFI-11722 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.22.0 Reporter: Mike R Update spring.data.redis.version to 2.7.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11721) Update rethinkdb-driver to 2.4.4
Mike R created NIFI-11721: - Summary: Update rethinkdb-driver to 2.4.4 Key: NIFI-11721 URL: https://issues.apache.org/jira/browse/NIFI-11721 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.22.0 Reporter: Mike R Update rethinkdb-driver to 2.4.4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 opened a new pull request, #7402: NIFI-11720 Update hbase.version to 2.5.5-hadoop3
mr1716 opened a new pull request, #7402: URL: https://github.com/apache/nifi/pull/7402 # Summary [NIFI-11720](https://issues.apache.org/jira/browse/NIFI-11720) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-11720) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11720) Update hbase.version to 2.5.5-hadoop3
[ https://issues.apache.org/jira/browse/NIFI-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R reassigned NIFI-11720: - Assignee: Mike R > Update hbase.version to 2.5.5-hadoop3 > - > > Key: NIFI-11720 > URL: https://issues.apache.org/jira/browse/NIFI-11720 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.22.0 >Reporter: Mike R >Assignee: Mike R >Priority: Major > > Update hbase.version to 2.5.5-hadoop3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11720) Update hbase.version to 2.5.5-hadoop3
Mike R created NIFI-11720: - Summary: Update hbase.version to 2.5.5-hadoop3 Key: NIFI-11720 URL: https://issues.apache.org/jira/browse/NIFI-11720 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.22.0 Reporter: Mike R Update hbase.version to 2.5.5-hadoop3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11706) Create dedicated Parameter Contexts for versioned flows when importing
[ https://issues.apache.org/jira/browse/NIFI-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Bence updated NIFI-11706: --- Status: Patch Available (was: In Progress) https://github.com/apache/nifi/pull/7401 > Create dedicated Parameter Contexts for versioned flows when importing > -- > > Key: NIFI-11706 > URL: https://issues.apache.org/jira/browse/NIFI-11706 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > As many Versioned Flows are imported multiple times but used with in somewhat > different circumstances (such as: different endpoints, databases, etc.), it > would make sense to give the user the possibility to import flows in a way > that their Parameter Context become independent from the other instances. In > order to achieve this a new checkbox will be added to the Import Version > window which makes it possible to decide for the user if she wants to keep > the existing Parameter Contexts (bounding the parameters to the potentially > existing instances) or she wants to create a new set of Parameter Contexts > making the flow parametrised independently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] simonbence opened a new pull request, #7401: NIFI-11706 Add capability to create dedicated Parameter Contexts for versioned flows when importing
simonbence opened a new pull request, #7401: URL: https://github.com/apache/nifi/pull/7401 # Summary [NIFI-11706](https://issues.apache.org/jira/browse/NIFI-11706) (Test scenarios helping to understand and review the changes are added as comment on the ticket) As many Versioned Flows are imported multiple times but used with in somewhat different circumstances (such as: different endpoints, databases, etc.), it would make sense to give the user the possibility to import flows in a way that their Parameter Context become independent from the other instances. In order to achieve this a new checkbox will be added to the Import Version window which makes it possible to decide for the user if she wants to keep the existing Parameter Contexts (bounding the parameters to the potentially existing instances) or she wants to create a new set of Parameter Contexts making the flow parametrised independently. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (MINIFICPP-2125) Yield time is not always respected
[ https://issues.apache.org/jira/browse/MINIFICPP-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734127#comment-17734127 ] Martin Zink commented on MINIFICPP-2125: https://github.com/martinzink/nifi-minifi-cpp/commit/dcbabbad04abceacce2a246a905589ac1f0e8fd6 > Yield time is not always respected > -- > > Key: MINIFICPP-2125 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2125 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > Running a simple EventDriven processor with debug logs enabled reveals that > we are continously checking if the processor has work to do (periodically > busy looping). > > On a quick glance this is due to using nanosec clock and then rounding the > results to milliseconds. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (MINIFICPP-2125) Yield time is not always respected
[ https://issues.apache.org/jira/browse/MINIFICPP-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734127#comment-17734127 ] Martin Zink edited comment on MINIFICPP-2125 at 6/19/23 11:11 AM: -- https://github.com/apache/nifi-minifi-cpp/commit/dcbabbad04abceacce2a246a905589ac1f0e8fd6 was (Author: mzink): https://github.com/martinzink/nifi-minifi-cpp/commit/dcbabbad04abceacce2a246a905589ac1f0e8fd6 > Yield time is not always respected > -- > > Key: MINIFICPP-2125 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2125 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > Running a simple EventDriven processor with debug logs enabled reveals that > we are continously checking if the processor has work to do (periodically > busy looping). > > On a quick glance this is due to using nanosec clock and then rounding the > results to milliseconds. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-11694) SAML logout signature verification failed
[
https://issues.apache.org/jira/browse/NIFI-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734122#comment-17734122
]
Beat Fuellemann edited comment on NIFI-11694 at 6/19/23 11:00 AM:
--
Thanks David for the fast response. I will clarify that with our Identity
Provider. So I will close that case.
was (Author: JIRAUSER299087):
Thanks David for the fast response. I will clarify that with our Identity
Provider.
> SAML logout signature verification failed
> -
>
> Key: NIFI-11694
> URL: https://issues.apache.org/jira/browse/NIFI-11694
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Beat Fuellemann
>Assignee: David Handermann
>Priority: Major
>
> We activated SAML Authentication with the following configuration:
> {code:java}
> nifi.security.user.saml.request.signing.enabled=false
> nifi.security.user.saml.want.assertions.signed=true
> nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> nifi.security.user.saml.authentication.expiration=1 hours
> nifi.security.user.saml.single.logout.enabled=true
> nifi.security.user.saml.http.client.truststore.strategy=JDK
> nifi.security.user.saml.http.client.connect.timeout=30 secs
> nifi.security.user.saml.http.client.read.timeout=30 secs{code}
> Login works fine.
> But during logout, it looks that NIFI signs the request, even if we
> "request.signing.enabled=false". This causes the logout fail on the IdP.
> it gives us the following error:
> {code:java}
> 2023-06-15 06:38:35,629 INFO [NiFi Web Server-78]
> org.apache.nifi.web.api.AccessResource Logout Request
> [7b8370e8-752f-484e-8caa-5a8ce3f29caf] Identity [TX] started
> 2023-06-15 06:38:35,673 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256': supported
> 2023-06-15 06:38:35,674 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmlenc#sha256': supported
> 2023-06-15 06:38:35,676 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Resolved
> SignatureSigningParameters:
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signing credential with
> key algorithm: RSA
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature algorithm URI:
> http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature
> KeyInfoGenerator: present
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference digest method
> algorithm URI: http://www.w3.org/2001/04/xmlenc#sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference
> canonicalization algorithm URI: null
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Canonicalization
> algorithm URI: http://www.w3.org/2001/10/xml-exc-c14n#
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver HMAC output length: null
> 2023-06-15 06:38:35,678 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computing signature over input using
> private key of type RSA and JCA algorithm ID SHA256withRSA
> 2023-06-15 06:38:35,691 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computed signature: [3, e, 2, 0, d, 4,
> 0, 7, d, 8, 2, 6, 9, 7, a, f, c, 1, 0, 8, b, 9, 5, f, d, 0, a, 3, 2, 9, b, 9,
> 3, d, b, 5, 2, 4, 2, f, a, 9, 7, 1, 2, 3, d, 3, c, d, 9, 8, 1, 0, a, 5, 1, 8,
> 8, 6, 3, 3, 8, a, a, 7, f, 1, 8, 9, c, a, 3, 5, 7, b, 2, e, c, 2, 5, 3, 7, 1,
> 2, b, 2, 1, 4, 3, e, 6, f, 4, 8, 5, e, 1, d, 3, e, 1, a, 5, 1, a, f, 8, 2, f,
> a, 3, 8, a, 3, 2, f, 0, 6, d, e, 8, 7, b, 9, f, d, 2, 8, b, d, f, 8, 2, 7, 9,
> 3, 5, 1, d, c, 1, 2, e, 3, 4, 8, f, 3, 7, e, 6, 5, c, e, 3, 8, 3, 1, 2, a, 6,
> 5, 6, 1, 2, 8, c, 8, 3, 8, 3, a, a, 9, 6, 2, a, 8, 3, 2, 9, 2, 5, 9, 2, b, e,
> 6, d, 0, 0, e, 1, 8, 9, 2, 4, 0, 2, a, 5, c, b, 3, 1, b, 1, b, b, a, e, 0, f,
> 6, e, 8, 0, b, c, 9, 0, 0, f, c, 1, 7, 5, c, 4, d, b, 5, c, 1, 0, f, b, 3, d,
> 4, c, e, 5, 7, 4, 3, 8, f, b, 1, f, 1, d, a, a, 0, c, 8, e, d, b, 5, 0, 5, 9,
> 7, a, c, 8, 7, 9, 4, 4, d, f, 1, 3, 2, 9, 6, 6, 2, 4, 1, e, c, 8, 3, 7, 3, 2,
> 4, 9, a, 9, 4, 0, 3, c, 4, b, 2, f, 1, b, 9, b, 4, 3, 1, f, 6, d, 3, d, 4, 5,
> 0, f, 7, 8, d, 1, c, 1, 8, f, 2, 4, 8, 3, 3, 9, e, 3, 4, b, 5, 0, 9, 9, 1, 0,
> c,
[jira] [Commented] (NIFI-11694) SAML logout signature verification failed
[
https://issues.apache.org/jira/browse/NIFI-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17734122#comment-17734122
]
Beat Fuellemann commented on NIFI-11694:
Thanks David for the fast response. I will clarify that with our Identity
Provider.
> SAML logout signature verification failed
> -
>
> Key: NIFI-11694
> URL: https://issues.apache.org/jira/browse/NIFI-11694
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Beat Fuellemann
>Assignee: David Handermann
>Priority: Major
>
> We activated SAML Authentication with the following configuration:
> {code:java}
> nifi.security.user.saml.request.signing.enabled=false
> nifi.security.user.saml.want.assertions.signed=true
> nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> nifi.security.user.saml.authentication.expiration=1 hours
> nifi.security.user.saml.single.logout.enabled=true
> nifi.security.user.saml.http.client.truststore.strategy=JDK
> nifi.security.user.saml.http.client.connect.timeout=30 secs
> nifi.security.user.saml.http.client.read.timeout=30 secs{code}
> Login works fine.
> But during logout, it looks that NIFI signs the request, even if we
> "request.signing.enabled=false". This causes the logout fail on the IdP.
> it gives us the following error:
> {code:java}
> 2023-06-15 06:38:35,629 INFO [NiFi Web Server-78]
> org.apache.nifi.web.api.AccessResource Logout Request
> [7b8370e8-752f-484e-8caa-5a8ce3f29caf] Identity [TX] started
> 2023-06-15 06:38:35,673 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256': supported
> 2023-06-15 06:38:35,674 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmlenc#sha256': supported
> 2023-06-15 06:38:35,676 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Resolved
> SignatureSigningParameters:
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signing credential with
> key algorithm: RSA
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature algorithm URI:
> http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature
> KeyInfoGenerator: present
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference digest method
> algorithm URI: http://www.w3.org/2001/04/xmlenc#sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference
> canonicalization algorithm URI: null
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Canonicalization
> algorithm URI: http://www.w3.org/2001/10/xml-exc-c14n#
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver HMAC output length: null
> 2023-06-15 06:38:35,678 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computing signature over input using
> private key of type RSA and JCA algorithm ID SHA256withRSA
> 2023-06-15 06:38:35,691 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computed signature: [3, e, 2, 0, d, 4,
> 0, 7, d, 8, 2, 6, 9, 7, a, f, c, 1, 0, 8, b, 9, 5, f, d, 0, a, 3, 2, 9, b, 9,
> 3, d, b, 5, 2, 4, 2, f, a, 9, 7, 1, 2, 3, d, 3, c, d, 9, 8, 1, 0, a, 5, 1, 8,
> 8, 6, 3, 3, 8, a, a, 7, f, 1, 8, 9, c, a, 3, 5, 7, b, 2, e, c, 2, 5, 3, 7, 1,
> 2, b, 2, 1, 4, 3, e, 6, f, 4, 8, 5, e, 1, d, 3, e, 1, a, 5, 1, a, f, 8, 2, f,
> a, 3, 8, a, 3, 2, f, 0, 6, d, e, 8, 7, b, 9, f, d, 2, 8, b, d, f, 8, 2, 7, 9,
> 3, 5, 1, d, c, 1, 2, e, 3, 4, 8, f, 3, 7, e, 6, 5, c, e, 3, 8, 3, 1, 2, a, 6,
> 5, 6, 1, 2, 8, c, 8, 3, 8, 3, a, a, 9, 6, 2, a, 8, 3, 2, 9, 2, 5, 9, 2, b, e,
> 6, d, 0, 0, e, 1, 8, 9, 2, 4, 0, 2, a, 5, c, b, 3, 1, b, 1, b, b, a, e, 0, f,
> 6, e, 8, 0, b, c, 9, 0, 0, f, c, 1, 7, 5, c, 4, d, b, 5, c, 1, 0, f, b, 3, d,
> 4, c, e, 5, 7, 4, 3, 8, f, b, 1, f, 1, d, a, a, 0, c, 8, e, d, b, 5, 0, 5, 9,
> 7, a, c, 8, 7, 9, 4, 4, d, f, 1, 3, 2, 9, 6, 6, 2, 4, 1, e, c, 8, 3, 7, 3, 2,
> 4, 9, a, 9, 4, 0, 3, c, 4, b, 2, f, 1, b, 9, b, 4, 3, 1, f, 6, d, 3, d, 4, 5,
> 0, f, 7, 8, d, 1, c, 1, 8, f, 2, 4, 8, 3, 3, 9, e, 3, 4, b, 5, 0, 9, 9, 1, 0,
> c, b, e, 3, 7, 9, 4, 4, d, 7, a, a, 4, 6, 6, 0, 1, b, c, 8, b, 4, c, 9, c, a,
> b, 2, b, e, d, 4, 4, 4, 0, a, b, 9, 4, 4, 4, 4, 9, e, a, b, 4, b, 0, 1, 4, 0,
> b, 7, 2, f, d, b, 8, a, a, 8, f, 8,
[GitHub] [nifi] jbalchan closed pull request #7399: NIFI-10752 Fix CVEs in couchbase.client by upgrading versions
jbalchan closed pull request #7399: NIFI-10752 Fix CVEs in couchbase.client by upgrading versions URL: https://github.com/apache/nifi/pull/7399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11719) RecordPath to include math function
[ https://issues.apache.org/jira/browse/NIFI-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Korniets updated NIFI-11719: Description: Currently [NiFi Record Path |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't have any mathematical functions exposed It would be great to have at least following functions: * multiply(/field1, /field2) * divide(/field1, /field2) * add(/field1, /field2) * subtract(/field1, /field2) Current solution is to use QueryRecord, but for simple operations like above - its quite heavy. Real world scenario: We have fields: - SharePrice in EUR - FXRate EUR/USD We need to get SharePrice in USD. was: Currently [NiFi Record Path |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't have any mathematical functions exposed It would be great to have following functions: * multiply(/field1, /field2) * divide(/field1, /field2) * add(/field1, /field2) * subtract(/field1, /field2) Current solution is to use QueryRecord, but for simple operations like above - its quite heavy. Real world scenario: We have fields: - SharePrice in EUR - FXRate EUR/USD We need to get SharePrice in USD. > RecordPath to include math function > --- > > Key: NIFI-11719 > URL: https://issues.apache.org/jira/browse/NIFI-11719 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Philipp Korniets >Priority: Major > > Currently [NiFi Record Path > |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't > have any mathematical functions exposed > It would be great to have at least following functions: > * multiply(/field1, /field2) > * divide(/field1, /field2) > * add(/field1, /field2) > * subtract(/field1, /field2) > Current solution is to use QueryRecord, but for simple operations like above > - its quite heavy. > Real world scenario: > We have fields: > - SharePrice in EUR > - FXRate EUR/USD > We need to get SharePrice in USD. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11719) Record Path to include math function
[ https://issues.apache.org/jira/browse/NIFI-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Korniets updated NIFI-11719: Description: Currently [NiFi Record Path |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't have any mathematical functions exposed It would be great to have following functions: * multiply(/field1, /field2) * divide(/field1, /field2) * add(/field1, /field2) * subtract(/field1, /field2) Current solution is to use QueryRecord, but for simple operations like above - its quite heavy. Real world scenario: We have fields: - SharePrice in EUR - FXRate EUR/USD We need to get SharePrice in USD. was: Currently [NiFi Record Path |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't have any mathematical functions exposed It would be great to have following functions: * multiply(/field1, /field2) * divide(/field1, /field2) * add(/field1, /field2) * subtract(/field1, /field2) Current workaround is to use QueryRecord, but for simple operations like above - its quite heavy. Real world scenario: We have fields: - SharePrice in EUR - FXRate EUR/USD We need to get SharePrice in USD. > Record Path to include math function > > > Key: NIFI-11719 > URL: https://issues.apache.org/jira/browse/NIFI-11719 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Philipp Korniets >Priority: Major > > Currently [NiFi Record Path > |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't > have any mathematical functions exposed > It would be great to have following functions: > * multiply(/field1, /field2) > * divide(/field1, /field2) > * add(/field1, /field2) > * subtract(/field1, /field2) > Current solution is to use QueryRecord, but for simple operations like above > - its quite heavy. > Real world scenario: > We have fields: > - SharePrice in EUR > - FXRate EUR/USD > We need to get SharePrice in USD. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11719) RecordPath to include math function
[ https://issues.apache.org/jira/browse/NIFI-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Korniets updated NIFI-11719: Summary: RecordPath to include math function (was: Record Path to include math function) > RecordPath to include math function > --- > > Key: NIFI-11719 > URL: https://issues.apache.org/jira/browse/NIFI-11719 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Philipp Korniets >Priority: Major > > Currently [NiFi Record Path > |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't > have any mathematical functions exposed > It would be great to have following functions: > * multiply(/field1, /field2) > * divide(/field1, /field2) > * add(/field1, /field2) > * subtract(/field1, /field2) > Current solution is to use QueryRecord, but for simple operations like above > - its quite heavy. > Real world scenario: > We have fields: > - SharePrice in EUR > - FXRate EUR/USD > We need to get SharePrice in USD. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11719) Record Path to include math function
[ https://issues.apache.org/jira/browse/NIFI-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Korniets updated NIFI-11719: Description: Currently [NiFi Record Path |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't have any mathematical functions exposed It would be great to have following functions: * multiply(/field1, /field2) * divide(/field1, /field2) * add(/field1, /field2) * subtract(/field1, /field2) Current workaround is to use QueryRecord, but for simple operations like above - its quite heavy. Real world scenario: We have fields: - SharePrice in EUR - FXRate EUR/USD We need to get SharePrice in USD. was: Currently [NiFi Record Path |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't have any mathematical functions exposed It would be great to have following functions: * multiply(/field1, /field2) * divide(/field1, /field2) * add(/field1, /field2) * subtract(/field1, /field2) Current workaround is to use QueryRecord, but for simple operations like above - its quite heavy. Real world scenario: We have SharePrice in EUR, FXRate EUR/USD - we need to get SharePrice in USD. > Record Path to include math function > > > Key: NIFI-11719 > URL: https://issues.apache.org/jira/browse/NIFI-11719 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Philipp Korniets >Priority: Major > > Currently [NiFi Record Path > |https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html] doesn't > have any mathematical functions exposed > It would be great to have following functions: > * multiply(/field1, /field2) > * divide(/field1, /field2) > * add(/field1, /field2) > * subtract(/field1, /field2) > Current workaround is to use QueryRecord, but for simple operations like > above - its quite heavy. > Real world scenario: > We have fields: > - SharePrice in EUR > - FXRate EUR/USD > We need to get SharePrice in USD. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2124) Kafka docker tests transiently fail in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-2124. --- Resolution: Fixed > Kafka docker tests transiently fail in CI > - > > Key: MINIFICPP-2124 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2124 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Major > Attachments: kafka_test_failure.log > > > Some Kafka tests randomly fail in the CI, see example in attached logs -- This message was sent by Atlassian Jira (v8.20.10#820010)
