[jira] [Updated] (NIFI-5302) Add Support for Client Credentials Flow with OIDC Access Tokens
[ https://issues.apache.org/jira/browse/NIFI-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-5302: --- Status: Patch Available (was: Open) > Add Support for Client Credentials Flow with OIDC Access Tokens > --- > > Key: NIFI-5302 > URL: https://issues.apache.org/jira/browse/NIFI-5302 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Federico Michele Facca >Assignee: David Handermann >Priority: Major > Labels: security > Time Spent: 10m > Remaining Estimate: 0h > > Currently OIDC is only supported via implicit grant flow. This is perfectly > fine for the UI, but it makes impossible to programmatically interact with > the APIs using OIDC. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12550) Support OIDC Device Authorization Grant for API
[ https://issues.apache.org/jira/browse/NIFI-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-12550: --- Assignee: David Handermann > Support OIDC Device Authorization Grant for API > --- > > Key: NIFI-12550 > URL: https://issues.apache.org/jira/browse/NIFI-12550 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Affects Versions: 1.23.2 > Environment: NiFi with OIDC provider configured >Reporter: Igor Milavec >Assignee: David Handermann >Priority: Major > > Please add support for OIDC Device Authorization Grant. This is useful for > running scripts that access the NiFi API from the CLI. At this time the > options are: > # Copy __Secure-Authorization-Bearer cookie from the browser session: not > really a good practice, work and error prone > # Enable MTLS: painful for the users as the browser starts to frequently > challenge for the client cert and even if it worked fine, client certificate > management process is typically lagging behind OIDC identity management > # Use passwords: insecure and prohibited by policy > Having an API endpoint in the Access group that would allow the caller to > exchange OIDC id or refresh token for a NiFi session token would be perfect > for this use case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-5302 Add Support for Client Credentials Flow with OIDC [nifi]
exceptionfactory opened a new pull request, #8532: URL: https://github.com/apache/nifi/pull/8532 # Summary [NIFI-5302](https://issues.apache.org/jira/browse/NIFI-5302) Adds supports for the [OAuth 2.0 Client Credentials Flow](https://www.oauth.com/oauth2-servers/access-tokens/client-credentials/) when NiFi is configured for [OpenID Connect](https://openid.net/specs/openid-connect-core-1_0.html) authentication. The implementation changes maintain existing support for the [Authorization Code Grant Flow](https://www.oauth.com/oauth2-servers/server-side-apps/authorization-code/) as part of standard browser-based access, but enable machine-to-machine authentication using Client Credentials. Most of the code changes involve reorganizing the OIDC and JWT configuration objects for reuse between configuration classes. The configuration changes support verifying Access Tokens based on the provided Issuer claim. When the Token Issuer matches the OIDC Client Registration Issuer, NiFi uses the Identity Provider signing key for verification. In all other cases, NiFi uses the existing application signing key for verification, maintaining current behavior. Enabling machine-to-machine access with the Client Credentials Flow may also require authorization configuration changes depending on the deployed environment. The OpenID Connect fallback claims identifying user property may also be necessary to use the `sub` claim as the identity instead of the default `email` claim. These configuration elements can be enabled using existing configuration properties and do not require additional code changes. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12920) Remove Cassandra 3 Components
[ https://issues.apache.org/jira/browse/NIFI-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12920: Status: Patch Available (was: Open) > Remove Cassandra 3 Components > - > > Key: NIFI-12920 > URL: https://issues.apache.org/jira/browse/NIFI-12920 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 10m > Remaining Estimate: 0h > > Based on the [mailing list discussion > thread|https://lists.apache.org/thread/gx8nsrqrfxd4m3s0mj0145dhhhswq2m9] on > regarding new Cassandra component support, the existing nifi-cassandra-bundle > and associated components should be removed from the main branch for NiFi > version 2. > Removing existing components will provide the opportunity for a new > implementation based on current driver versions. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12920 Remove nifi-cassandra-bundle [nifi]
exceptionfactory opened a new pull request, #8531: URL: https://github.com/apache/nifi/pull/8531 # Summary [NIFI-12920](https://issues.apache.org/jira/browse/NIFI-12920) Removes the `nifi-cassandra-bundle` and associated components marked for deprecation in [NIFI-12919](https://issues.apache.org/jira/browse/NIFI-12919). This removal follows the [mailing list discussion](https://lists.apache.org/thread/gx8nsrqrfxd4m3s0mj0145dhhhswq2m9) and provides a way forward for the development of new Cassandra components based on the latest driver version. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12912) BUG - snackbar theming
[ https://issues.apache.org/jira/browse/NIFI-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan resolved NIFI-12912. Resolution: Fixed > BUG - snackbar theming > -- > > Key: NIFI-12912 > URL: https://issues.apache.org/jira/browse/NIFI-12912 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Scott Aslan >Assignee: James Elliott >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > The snackbar theme seems to have some regressions since the update to using > semantic colors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12912) BUG - snackbar theming
[ https://issues.apache.org/jira/browse/NIFI-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828108#comment-17828108 ] ASF subversion and git services commented on NIFI-12912: Commit f5e44a99acc80fb25b7e6aa3f0c10d3f0d91d41c in nifi's branch refs/heads/main from James Mingardi-Elliott [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f5e44a99ac ] NIFI-12912 snackbar bug (#8528) Removed unnecessary rules and class. The snackbars will be styled by Angular Material. Improved get-color-on-surface helper function. This closes #8528 > BUG - snackbar theming > -- > > Key: NIFI-12912 > URL: https://issues.apache.org/jira/browse/NIFI-12912 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Scott Aslan >Assignee: James Elliott >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > The snackbar theme seems to have some regressions since the update to using > semantic colors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12912 snackbar bug [nifi]
scottyaslan merged PR #8528: URL: https://github.com/apache/nifi/pull/8528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12908 provide powershell versions of the nifi-env.cmd and nifi.cmd [nifi]
exceptionfactory commented on PR #8509: URL: https://github.com/apache/nifi/pull/8509#issuecomment-2005050642 > Not compatibility issues per se but powershell does have a stronger security model requiring changing the default security settings. So, maybe dropping this PR would be the best move. I originally make this port because there was a problem with the nifi.cmd and rather than debug nifi.cmd I did this port. I mostly use linux, so I am far from a powershell expert (but I really do not like bat/cmd). That's good to know. NiFi usage is much more common on Linux and macOS, but development on Windows is common enough. Based on the security requirements you mentioned, I agree that closing the current pull request is the best option for now. If you are able to track down issues with the `nifi.cmd`, that would be helpful to maintain. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-6379) Add SSL Context Service Property to SQS and PutSNS Processors
[ https://issues.apache.org/jira/browse/NIFI-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-6379: --- Summary: Add SSL Context Service Property to SQS and PutSNS Processors (was: Add SSL Context Service Property to GetSQS Processor) > Add SSL Context Service Property to SQS and PutSNS Processors > - > > Key: NIFI-6379 > URL: https://issues.apache.org/jira/browse/NIFI-6379 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.7.1, 1.9.2 > Environment: GetSQS Processor >Reporter: John Crooke >Assignee: Reynaldo Rea >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 50m > Remaining Estimate: 0h > > The GetSQS processor does NOT let you choose an SSL Context Service, but > other AWS-related processors such as "FetchS3Object" do have the "SSL Context > Service" property. > > This improvement is to enable the choice of SSL Context Service for the > certificate presented and/or the truststore that will be used to trust the > SQS endpoint's certificate. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-6379) Add SSL Context Service Property to GetSQS Processor
[ https://issues.apache.org/jira/browse/NIFI-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-6379. Fix Version/s: 2.0.0-M3 Resolution: Fixed > Add SSL Context Service Property to GetSQS Processor > > > Key: NIFI-6379 > URL: https://issues.apache.org/jira/browse/NIFI-6379 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.7.1, 1.9.2 > Environment: GetSQS Processor >Reporter: John Crooke >Assignee: Reynaldo Rea >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 50m > Remaining Estimate: 0h > > The GetSQS processor does NOT let you choose an SSL Context Service, but > other AWS-related processors such as "FetchS3Object" do have the "SSL Context > Service" property. > > This improvement is to enable the choice of SSL Context Service for the > certificate presented and/or the truststore that will be used to trust the > SQS endpoint's certificate. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-6379) Add SSL Context Service Property to GetSQS Processor
[ https://issues.apache.org/jira/browse/NIFI-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828105#comment-17828105 ] ASF subversion and git services commented on NIFI-6379: --- Commit 12fc2f8508e2ceda306420afd4a0fc83b7bf9406 in nifi's branch refs/heads/main from Freedom9339 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=12fc2f8508 ] NIFI-6379 Added SSL Context to PutSNS, DeleteSQS, GetSQS, and PutSQS This closes #8505 Signed-off-by: David Handermann > Add SSL Context Service Property to GetSQS Processor > > > Key: NIFI-6379 > URL: https://issues.apache.org/jira/browse/NIFI-6379 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.7.1, 1.9.2 > Environment: GetSQS Processor >Reporter: John Crooke >Assignee: Reynaldo Rea >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > The GetSQS processor does NOT let you choose an SSL Context Service, but > other AWS-related processors such as "FetchS3Object" do have the "SSL Context > Service" property. > > This improvement is to enable the choice of SSL Context Service for the > certificate presented and/or the truststore that will be used to trust the > SQS endpoint's certificate. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-6379: Added SSL context to PutSNS, DeleteSQS, GetSQS, and PutSQS [nifi]
exceptionfactory closed pull request #8505: NIFI-6379: Added SSL context to PutSNS, DeleteSQS, GetSQS, and PutSQS URL: https://github.com/apache/nifi/pull/8505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12919) Deprecate Cassandra 3 Components for Removal
[ https://issues.apache.org/jira/browse/NIFI-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12919: Status: Patch Available (was: Open) > Deprecate Cassandra 3 Components for Removal > > > Key: NIFI-12919 > URL: https://issues.apache.org/jira/browse/NIFI-12919 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 1.26.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The current nifi-cassandra-bundle contains several Controller Services and > Processors based on the Datastax 3 driver. These components do not provide > sufficient abstraction from the Datastax 3 driver, which is has been > obsoleted by newer driver versions. > Following a [mailing list discussion > thread|https://lists.apache.org/thread/gx8nsrqrfxd4m3s0mj0145dhhhswq2m9] on > next steps, the nifi-cassandra-bundle and associated components should be > deprecated on the support branch for the NiFi version 1 release line and > removed from the main branch for NiFi version 2. This approach will provide > the opportunity for a new implementation based on current driver versions > without implying direct compatibility and associated problems with the legacy > driver. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12919 Deprecate Cassandra 3 Components for Removal [nifi]
exceptionfactory opened a new pull request, #8530: URL: https://github.com/apache/nifi/pull/8530 # Summary [NIFI-12919](https://issues.apache.org/jira/browse/NIFI-12919) Deprecates Cassandra 3 components in `nifi-cassandra-bundle` for subsequent removal based on [mailing list thread](https://lists.apache.org/thread/gx8nsrqrfxd4m3s0mj0145dhhhswq2m9) discussion and and subsequent plans for new components. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12920) Remove Cassandra 3 Components
David Handermann created NIFI-12920: --- Summary: Remove Cassandra 3 Components Key: NIFI-12920 URL: https://issues.apache.org/jira/browse/NIFI-12920 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0-M3 Based on the [mailing list discussion thread|https://lists.apache.org/thread/gx8nsrqrfxd4m3s0mj0145dhhhswq2m9] on regarding new Cassandra component support, the existing nifi-cassandra-bundle and associated components should be removed from the main branch for NiFi version 2. Removing existing components will provide the opportunity for a new implementation based on current driver versions. h4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12897) Allow users to upload files/resources to nifi for use in the dataflow
[ https://issues.apache.org/jira/browse/NIFI-12897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828101#comment-17828101 ] Mark Payne commented on NIFI-12897: --- [~pvillard] probably best to use comments in the feature proposal. > Allow users to upload files/resources to nifi for use in the dataflow > - > > Key: NIFI-12897 > URL: https://issues.apache.org/jira/browse/NIFI-12897 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Core UI >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > A common feature request that we receive is to make it easier to upload a > resource file, such as JDBC Driver JAR to nifi so that all of the nodes > receive the file. This epic is meant to capture that request -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12919) Deprecate Cassandra 3 Components for Removal
David Handermann created NIFI-12919: --- Summary: Deprecate Cassandra 3 Components for Removal Key: NIFI-12919 URL: https://issues.apache.org/jira/browse/NIFI-12919 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 1.26.0 The current nifi-cassandra-bundle contains several Controller Services and Processors based on the Datastax 3 driver. These components do not provide sufficient abstraction from the Datastax 3 driver, which is has been obsoleted by newer driver versions. Following a [mailing list discussion thread|https://lists.apache.org/thread/gx8nsrqrfxd4m3s0mj0145dhhhswq2m9] on next steps, the nifi-cassandra-bundle and associated components should be deprecated on the support branch for the NiFi version 1 release line and removed from the main branch for NiFi version 2. This approach will provide the opportunity for a new implementation based on current driver versions without implying direct compatibility and associated problems with the legacy driver. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
sujkm commented on code in PR #8529: URL: https://github.com/apache/nifi/pull/8529#discussion_r1529226621 ## nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/sns/ITPutSNS.java: ## @@ -36,43 +45,75 @@ */ public class ITPutSNS { -private static final String CREDENTIALS_FILE = System.getProperty("user.home") + "/aws-credentials.properties"; -private final String TOPIC_ARN = "Add SNS ARN here"; +private static final DockerImageName localstackImage = DockerImageName.parse("localstack/localstack:latest"); + +private static final LocalStackContainer localstack = new LocalStackContainer(localstackImage) +.withServices(LocalStackContainer.Service.SNS); + +private static final String CREDENTIALS_FILE = "src/test/resources/mock-aws-credentials.properties"; +private static String topicARN; +private static SnsClient client; @BeforeAll -public static void assumeCredentialsFileExists() { -Assumptions.assumeTrue(new File(CREDENTIALS_FILE).exists()); +public static void setup() throws InterruptedException { +System.setProperty("software.amazon.awssdk.http.service.impl", "software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService"); +localstack.start(); + +client = SnsClient.builder() +.endpointOverride(localstack.getEndpoint()) +.credentialsProvider( +StaticCredentialsProvider.create( + AwsBasicCredentials.create(localstack.getAccessKey(), localstack.getSecretKey()) +) +) +.region(Region.of(localstack.getRegion())) +.build(); + +final CreateTopicResponse response = client.createTopic(CreateTopicRequest.builder() +.name("SnsSystemTest") +.build()); +assertTrue(response.sdkHttpResponse().isSuccessful()); +topicARN = response.topicArn(); +} + +@AfterAll +public static void shutdown() { +client.close(); +localstack.stop(); } @Test public void testPublish() throws IOException { -final TestRunner runner = TestRunners.newTestRunner(new PutSNS()); -AuthUtils.enableCredentialsFile(runner, CREDENTIALS_FILE); -runner.setProperty(PutSNS.ARN, TOPIC_ARN); +final TestRunner runner = initRunner(PutSNS.class); +AuthUtils.enableAccessKey(runner, localstack.getAccessKey(), localstack.getSecretKey()); Review Comment: The current changes are only applied to the integration tests in ITPutSNS. TestPutSNS does not use the endpoint override. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12892) Use Kerberos user principal as isolation key instead of service identifier in Iceberg processors
[ https://issues.apache.org/jira/browse/NIFI-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Bathori resolved NIFI-12892. - Resolution: Not A Bug > Use Kerberos user principal as isolation key instead of service identifier in > Iceberg processors > > > Key: NIFI-12892 > URL: https://issues.apache.org/jira/browse/NIFI-12892 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Change the class loader isolation to use the kerberos user principal as key > instead of the service identifier to align with other HDFS processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
dan-s1 commented on code in PR #8529: URL: https://github.com/apache/nifi/pull/8529#discussion_r1529221562 ## nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/sns/ITPutSNS.java: ## @@ -36,43 +45,75 @@ */ public class ITPutSNS { -private static final String CREDENTIALS_FILE = System.getProperty("user.home") + "/aws-credentials.properties"; -private final String TOPIC_ARN = "Add SNS ARN here"; +private static final DockerImageName localstackImage = DockerImageName.parse("localstack/localstack:latest"); + +private static final LocalStackContainer localstack = new LocalStackContainer(localstackImage) +.withServices(LocalStackContainer.Service.SNS); + +private static final String CREDENTIALS_FILE = "src/test/resources/mock-aws-credentials.properties"; +private static String topicARN; +private static SnsClient client; @BeforeAll -public static void assumeCredentialsFileExists() { -Assumptions.assumeTrue(new File(CREDENTIALS_FILE).exists()); +public static void setup() throws InterruptedException { +System.setProperty("software.amazon.awssdk.http.service.impl", "software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService"); +localstack.start(); + +client = SnsClient.builder() +.endpointOverride(localstack.getEndpoint()) +.credentialsProvider( +StaticCredentialsProvider.create( + AwsBasicCredentials.create(localstack.getAccessKey(), localstack.getSecretKey()) +) +) +.region(Region.of(localstack.getRegion())) +.build(); + +final CreateTopicResponse response = client.createTopic(CreateTopicRequest.builder() +.name("SnsSystemTest") +.build()); +assertTrue(response.sdkHttpResponse().isSuccessful()); +topicARN = response.topicArn(); +} + +@AfterAll +public static void shutdown() { +client.close(); +localstack.stop(); } @Test public void testPublish() throws IOException { -final TestRunner runner = TestRunners.newTestRunner(new PutSNS()); -AuthUtils.enableCredentialsFile(runner, CREDENTIALS_FILE); -runner.setProperty(PutSNS.ARN, TOPIC_ARN); +final TestRunner runner = initRunner(PutSNS.class); +AuthUtils.enableAccessKey(runner, localstack.getAccessKey(), localstack.getSecretKey()); Review Comment: Correct me if I am wrong, the way it is set up now with the latest changes, TestPutSNS only tests with the endpoint override. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12892: Use Kerberos user principal as isolation key instead of s… [nifi]
mark-bathori closed pull request #8497: NIFI-12892: Use Kerberos user principal as isolation key instead of s… URL: https://github.com/apache/nifi/pull/8497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12912 snackbar bug [nifi]
scottyaslan commented on PR #8528: URL: https://github.com/apache/nifi/pull/8528#issuecomment-2004862852 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
sujkm commented on code in PR #8529: URL: https://github.com/apache/nifi/pull/8529#discussion_r1529166113 ## nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/sns/ITPutSNS.java: ## @@ -36,43 +45,75 @@ */ public class ITPutSNS { -private static final String CREDENTIALS_FILE = System.getProperty("user.home") + "/aws-credentials.properties"; -private final String TOPIC_ARN = "Add SNS ARN here"; +private static final DockerImageName localstackImage = DockerImageName.parse("localstack/localstack:latest"); + +private static final LocalStackContainer localstack = new LocalStackContainer(localstackImage) +.withServices(LocalStackContainer.Service.SNS); + +private static final String CREDENTIALS_FILE = "src/test/resources/mock-aws-credentials.properties"; +private static String topicARN; +private static SnsClient client; @BeforeAll -public static void assumeCredentialsFileExists() { -Assumptions.assumeTrue(new File(CREDENTIALS_FILE).exists()); +public static void setup() throws InterruptedException { +System.setProperty("software.amazon.awssdk.http.service.impl", "software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService"); +localstack.start(); + +client = SnsClient.builder() +.endpointOverride(localstack.getEndpoint()) +.credentialsProvider( +StaticCredentialsProvider.create( + AwsBasicCredentials.create(localstack.getAccessKey(), localstack.getSecretKey()) +) +) +.region(Region.of(localstack.getRegion())) +.build(); + +final CreateTopicResponse response = client.createTopic(CreateTopicRequest.builder() +.name("SnsSystemTest") +.build()); +assertTrue(response.sdkHttpResponse().isSuccessful()); +topicARN = response.topicArn(); +} + +@AfterAll +public static void shutdown() { +client.close(); +localstack.stop(); } @Test public void testPublish() throws IOException { -final TestRunner runner = TestRunners.newTestRunner(new PutSNS()); -AuthUtils.enableCredentialsFile(runner, CREDENTIALS_FILE); -runner.setProperty(PutSNS.ARN, TOPIC_ARN); +final TestRunner runner = initRunner(PutSNS.class); +AuthUtils.enableAccessKey(runner, localstack.getAccessKey(), localstack.getSecretKey()); Review Comment: @dan-s1 Currently the tests were being ignored if there was no credentials file. Based on NIFI-12144 i thought it would be preferable to use test containers and TestPutSNS would still test without the endpoint override. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12916) Develop an ExecutePythonScript Processor
[ https://issues.apache.org/jira/browse/NIFI-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828083#comment-17828083 ] Fredrick Eisele commented on NIFI-12916: Development repository: https://github.com/phreed/nifi-nar-pythonx > Develop an ExecutePythonScript Processor > > > Key: NIFI-12916 > URL: https://issues.apache.org/jira/browse/NIFI-12916 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 2.0.0-M2 >Reporter: Fredrick Eisele >Priority: Major > Fix For: 2.0.0-M3 > > > Given the ability to create python processors in NiFi 2, and the presence of > ExecuteGroovyScript, development of an ExecutePythonScript processor seems > logical. > That said there will be differences due to differences between on the NiFi > Python and JVM support. > The first step is to outline what capabilities are or may be possible. > "The new native Python support in NiFi 2.0 provides a more streamlined API > for Processor implementation, as opposed to the full set of capabilities that > the Java Processor API provides. > This enables simplified Processor implementation, and streamlined > communication between Java and Python processes. > For that reason, a generic ExecutePythonScript would not provide all of the > same options as the generalized ExecuteGroovyScript." > Ref: > * > [https://github.com/apache/nifi/tree/main/nifi-nar-bundles/nifi-groovyx-bundle/nifi-groovyx-processors/src/main/java/org/apache/nifi/processors/groovyx] > * > [https://nifi.apache.org/documentation/nifi-2.0.0-M2/html/python-developer-guide.html] -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
dan-s1 commented on code in PR #8529: URL: https://github.com/apache/nifi/pull/8529#discussion_r1529082976 ## nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/sns/ITPutSNS.java: ## @@ -36,43 +45,75 @@ */ public class ITPutSNS { -private static final String CREDENTIALS_FILE = System.getProperty("user.home") + "/aws-credentials.properties"; -private final String TOPIC_ARN = "Add SNS ARN here"; +private static final DockerImageName localstackImage = DockerImageName.parse("localstack/localstack:latest"); + +private static final LocalStackContainer localstack = new LocalStackContainer(localstackImage) +.withServices(LocalStackContainer.Service.SNS); + +private static final String CREDENTIALS_FILE = "src/test/resources/mock-aws-credentials.properties"; +private static String topicARN; +private static SnsClient client; @BeforeAll -public static void assumeCredentialsFileExists() { -Assumptions.assumeTrue(new File(CREDENTIALS_FILE).exists()); +public static void setup() throws InterruptedException { +System.setProperty("software.amazon.awssdk.http.service.impl", "software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService"); +localstack.start(); + +client = SnsClient.builder() +.endpointOverride(localstack.getEndpoint()) +.credentialsProvider( +StaticCredentialsProvider.create( + AwsBasicCredentials.create(localstack.getAccessKey(), localstack.getSecretKey()) +) +) +.region(Region.of(localstack.getRegion())) +.build(); + +final CreateTopicResponse response = client.createTopic(CreateTopicRequest.builder() +.name("SnsSystemTest") +.build()); +assertTrue(response.sdkHttpResponse().isSuccessful()); +topicARN = response.topicArn(); +} + +@AfterAll +public static void shutdown() { +client.close(); +localstack.stop(); } @Test public void testPublish() throws IOException { -final TestRunner runner = TestRunners.newTestRunner(new PutSNS()); -AuthUtils.enableCredentialsFile(runner, CREDENTIALS_FILE); -runner.setProperty(PutSNS.ARN, TOPIC_ARN); +final TestRunner runner = initRunner(PutSNS.class); +AuthUtils.enableAccessKey(runner, localstack.getAccessKey(), localstack.getSecretKey()); Review Comment: @sujkm I cannot say I know all the details but I am not sure why this test and the other test were modified to always have `ENDPOINT_OVERRIDE`? Shouldn't there be two other tests with this setting and this test and the other test kept as they were? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-6379: Added SSL context to PutSNS, DeleteSQS, GetSQS, and PutSQS [nifi]
Freedom9339 commented on PR #8505: URL: https://github.com/apache/nifi/pull/8505#issuecomment-2004649846 @exceptionfactory Thank You! Do you mind setting the requested changes as approved? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12908 provide powershell versions of the nifi-env.cmd and nifi.cmd [nifi]
phreed commented on PR #8509: URL: https://github.com/apache/nifi/pull/8509#issuecomment-2004586333 Not compatibility issues per se but powershell does have a stronger security model requiring changing the default security settings. So, maybe dropping this PR would be the best move. I originally make this port because there was a problem with the nifi.cmd and rather than debug nifi.cmd I did this port. I mostly use linux, so I am far from a powershell expert (but I really do not like bat/cmd). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12897) Allow users to upload files/resources to nifi for use in the dataflow
[ https://issues.apache.org/jira/browse/NIFI-12897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828068#comment-17828068 ] Pierre Villard commented on NIFI-12897: --- Thanks Mark! Do you prefer to have a discussion here or via the comments of the feature proposal? > Allow users to upload files/resources to nifi for use in the dataflow > - > > Key: NIFI-12897 > URL: https://issues.apache.org/jira/browse/NIFI-12897 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Core UI >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > A common feature request that we receive is to make it easier to upload a > resource file, such as JDBC Driver JAR to nifi so that all of the nodes > receive the file. This epic is meant to capture that request -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12918) Stateless NiFi NullPointerException on versioned PGs
Stephanie Ambrose created NIFI-12918: Summary: Stateless NiFi NullPointerException on versioned PGs Key: NIFI-12918 URL: https://issues.apache.org/jira/browse/NIFI-12918 Project: Apache NiFi Issue Type: Bug Reporter: Stephanie Ambrose If you create a process group and commit that to version control, and then add a sub-process group under that one which is also under a separate version control, it will throw a NullPointerException. {code:java} 2024-03-15T12:25:38.565156967Z 2024-03-15 12:25:38,564 INFO [main] o.a.nifi.groups.StandardProcessGroup StandardProcessGroup[identifier=9688e64b-3f4a-3668-81b9-e25d9ea61454,name=Sub Group] added to StandardProcessGroup[identifier=stateless-flow,name=Test Flow] 2024-03-15T12:25:38.570693223Z Exception in thread "main" java.lang.NullPointerException: Registry ID must be specified at java.base/java.util.Objects.requireNonNull(Unknown Source) 2024-03-15T12:25:38.570722016Z at org.apache.nifi.registry.flow.StandardVersionControlInformation$Builder.build(StandardVersionControlInformation.java:134) 2024-03-15T12:25:38.570725111Z at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:354) 2024-03-15T12:25:38.570729455Z at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addProcessGroup(StandardVersionedComponentSynchronizer.java:1185) at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:528) 2024-03-15T12:25:38.573083029Z at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:426) 2024-03-15T12:25:38.573086447Z at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:265) 2024-03-15T12:25:38.573092747Z at org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:551) 2024-03-15T12:25:38.573095677Z at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:260) 2024-03-15T12:25:38.573098999Z at org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3977) at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3957) at org.apache.nifi.stateless.engine.StandardStatelessEngine.createFlow(StandardStatelessEngine.java:180) 2024-03-15T12:25:38.573108177Z at org.apache.nifi.stateless.flow.StandardStatelessDataflowFactory.createDataflow(StandardStatelessDataflowFactory.java:243) at org.apache.nifi.stateless.bootstrap.StatelessBootstrap.createDataflow(StatelessBootstrap.java:73) {code} When stepping through the debugger, it appears the "registryId" property is null. The JerseyClient call is supposed to map the response from registry to the proper class, but registryId is never set: {code:java} java.lang.Thread.State: RUNNABLE at org.apache.nifi.flow.VersionedProcessGroup.setVersionedFlowCoordinates(VersionedProcessGroup.java:150) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Unknown Source:-1) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source:-1) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source:-1) at java.lang.reflect.Method.invoke(Unknown Source:-1) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:141) at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:359) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129) at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129) at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314) at com.fasterxml.jackso
[jira] [Commented] (NIFI-12897) Allow users to upload files/resources to nifi for use in the dataflow
[ https://issues.apache.org/jira/browse/NIFI-12897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828053#comment-17828053 ] Mark Payne commented on NIFI-12897: --- [~pvillard] I pushed up a Feature Proposal: [https://cwiki.apache.org/confluence/display/NIFI/Asset+Management] > Allow users to upload files/resources to nifi for use in the dataflow > - > > Key: NIFI-12897 > URL: https://issues.apache.org/jira/browse/NIFI-12897 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Core UI >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > A common feature request that we receive is to make it easier to upload a > resource file, such as JDBC Driver JAR to nifi so that all of the nodes > receive the file. This epic is meant to capture that request -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
sujkm opened a new pull request, #8529: URL: https://github.com/apache/nifi/pull/8529 # Summary [NIFI-12896](https://issues.apache.org/jira/browse/NIFI-12896) Adds endpoint override URL property to PutSNS processor. Updated ITPutSNS to include endpoint override property (modeled after AbstractSQSIT). # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12916) Develop an ExecutePythonScript Processor
[ https://issues.apache.org/jira/browse/NIFI-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredrick Eisele updated NIFI-12916: --- Description: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor seems logical. That said there will be differences due to differences between on the NiFi Python and JVM support. The first step is to outline what capabilities are or may be possible. "The new native Python support in NiFi 2.0 provides a more streamlined API for Processor implementation, as opposed to the full set of capabilities that the Java Processor API provides. This enables simplified Processor implementation, and streamlined communication between Java and Python processes. For that reason, a generic ExecutePythonScript would not provide all of the same options as the generalized ExecuteGroovyScript." Ref: * [https://github.com/apache/nifi/tree/main/nifi-nar-bundles/nifi-groovyx-bundle/nifi-groovyx-processors/src/main/java/org/apache/nifi/processors/groovyx] * [https://nifi.apache.org/documentation/nifi-2.0.0-M2/html/python-developer-guide.html] was: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor seems logical. That said there will be differences due to differences between on the NiFi Python and JVM support. The first step is to outline what capabilities are or may be possible. "The new native Python support in NiFi 2.0 provides a more streamlined API for Processor implementation, as opposed to the full set of capabilities that the Java Processor API provides. This enables simplified Processor implementation, and streamlined communication between Java and Python processes. For that reason, a generic {{ExecutePythonScript}} would not provide all of the same options as the generalized {{{}ExecuteGroovyScript{}}}." > Develop an ExecutePythonScript Processor > > > Key: NIFI-12916 > URL: https://issues.apache.org/jira/browse/NIFI-12916 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 2.0.0-M2 >Reporter: Fredrick Eisele >Priority: Major > Fix For: 2.0.0-M3 > > > Given the ability to create python processors in NiFi 2, and the presence of > ExecuteGroovyScript, development of an ExecutePythonScript processor seems > logical. > That said there will be differences due to differences between on the NiFi > Python and JVM support. > The first step is to outline what capabilities are or may be possible. > "The new native Python support in NiFi 2.0 provides a more streamlined API > for Processor implementation, as opposed to the full set of capabilities that > the Java Processor API provides. > This enables simplified Processor implementation, and streamlined > communication between Java and Python processes. > For that reason, a generic ExecutePythonScript would not provide all of the > same options as the generalized ExecuteGroovyScript." > Ref: > * > [https://github.com/apache/nifi/tree/main/nifi-nar-bundles/nifi-groovyx-bundle/nifi-groovyx-processors/src/main/java/org/apache/nifi/processors/groovyx] > * > [https://nifi.apache.org/documentation/nifi-2.0.0-M2/html/python-developer-guide.html] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12917) Content repository claim is being zero'd out
Evan F. created NIFI-12917: -- Summary: Content repository claim is being zero'd out Key: NIFI-12917 URL: https://issues.apache.org/jira/browse/NIFI-12917 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.19.1 Environment: Red Hat 7 running Apache NiFi 1.19.1 with no custom NARs. Reporter: Evan F. Attachments: example_MergeContent.txt I've had rare but consistent instances of MergeContent processors merging files and losing the content claim/payload while merging. I've pasted an example log entry below where there's a size to the file pre-merge and the post-merge content claim becomes zero. I've attached a text file with the configuration of my MergeContent processor as well. This happens a fraction of a percent of the time but it happens daily, somewhere around 10 times a day. I scoured forums and previous tickets but I wasn't able to find a relevant issue. I'm hoping someone can tell me if this has already been addressed. INFO [Timer-Driven Process Thread-7] o.a.n.processors.standard.MergeContent Merge Content[id=e6f460da-018d-1000-1c97-a6bd946d4f61] Merged [StandardFlowFileRecord[uuid=0e5b7c30-021d-4bd9-9edb-1681c9c14d4e,claim=StandardContentClaim [resourceClaim=StandardRe sourceClaim[id=1710769372188-3700202, container=default, section=490], offset=0, length=52430102],offset=0,name=testFile,size=52430102]] into StandardFlowFileRecord[uuid=536bf8e4-915b-4dad-9e16-063b35e834c1,claim=,offset=0, name=testFile.pkg,size=0]. Reason for merging: Maximum number of entries reached -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12916) Develop an ExecutePythonScript Processor
[ https://issues.apache.org/jira/browse/NIFI-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredrick Eisele updated NIFI-12916: --- Description: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor seems logical. That said there will be differences due to differences between on the NiFi Python and JVM support. The first step is to outline what capabilities are or may be possible. "The new native Python support in NiFi 2.0 provides a more streamlined API for Processor implementation, as opposed to the full set of capabilities that the Java Processor API provides. This enables simplified Processor implementation, and streamlined communication between Java and Python processes. For that reason, a generic {{ExecutePythonScript}} would not provide all of the same options as the generalized {{{}ExecuteGroovyScript{}}}." was: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor seems logical. That said there will be differences due to differences between on the NiFi Python and JVM support. The first step is to outline what capabilities are or may be possible. > Develop an ExecutePythonScript Processor > > > Key: NIFI-12916 > URL: https://issues.apache.org/jira/browse/NIFI-12916 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 2.0.0-M2 >Reporter: Fredrick Eisele >Priority: Major > Fix For: 2.0.0-M3 > > > Given the ability to create python processors in NiFi 2, and the presence of > ExecuteGroovyScript, development of an ExecutePythonScript processor seems > logical. > That said there will be differences due to differences between on the NiFi > Python and JVM support. > The first step is to outline what capabilities are or may be possible. > "The new native Python support in NiFi 2.0 provides a more streamlined API > for Processor implementation, as opposed to the full set of capabilities that > the Java Processor API provides. > This enables simplified Processor implementation, and streamlined > communication between Java and Python processes. > For that reason, a generic {{ExecutePythonScript}} would not provide all of > the same options as the generalized {{{}ExecuteGroovyScript{}}}." -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12916) Develop an ExecutePythonScript Processor
[ https://issues.apache.org/jira/browse/NIFI-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredrick Eisele updated NIFI-12916: --- Description: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor seems logical. That said there will be differences due to differences between on the NiFi Python and JVM support. The first step is to outline what capabilities are or may be possible. was: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor seems logical. That said there will be differences based on the NiFi Python support. The first step is to outline what capabilities are or may be possible. > Develop an ExecutePythonScript Processor > > > Key: NIFI-12916 > URL: https://issues.apache.org/jira/browse/NIFI-12916 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 2.0.0-M2 >Reporter: Fredrick Eisele >Priority: Major > Fix For: 2.0.0-M3 > > > Given the ability to create python processors in NiFi 2, and the presence of > ExecuteGroovyScript, development of an ExecutePythonScript processor seems > logical. > That said there will be differences due to differences between on the NiFi > Python and JVM support. > The first step is to outline what capabilities are or may be possible. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12916) Develop an ExecutePythonScript Processor
[ https://issues.apache.org/jira/browse/NIFI-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredrick Eisele updated NIFI-12916: --- Description: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor seems logical. That said there will be differences based on the NiFi Python support. The first step is to outline what capabilities are or may be possible. was: Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor is analogous. That said there will be differences based on the NiFi Python support. The first step is to outline what capabilities are or may be possible. > Develop an ExecutePythonScript Processor > > > Key: NIFI-12916 > URL: https://issues.apache.org/jira/browse/NIFI-12916 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 2.0.0-M2 >Reporter: Fredrick Eisele >Priority: Major > Fix For: 2.0.0-M3 > > > Given the ability to create python processors in NiFi 2, and the presence of > ExecuteGroovyScript, development of an ExecutePythonScript processor seems > logical. > That said there will be differences based on the NiFi Python support. > The first step is to outline what capabilities are or may be possible. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12916) Develop an ExecutePythonScript Processor
Fredrick Eisele created NIFI-12916: -- Summary: Develop an ExecutePythonScript Processor Key: NIFI-12916 URL: https://issues.apache.org/jira/browse/NIFI-12916 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 2.0.0-M2 Reporter: Fredrick Eisele Fix For: 2.0.0-M3 Given the ability to create python processors in NiFi 2, and the presence of ExecuteGroovyScript, development of an ExecutePythonScript processor is analogous. That said there will be differences based on the NiFi Python support. The first step is to outline what capabilities are or may be possible. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-6379: Added SSL context to PutSNS, DeleteSQS, GetSQS, and PutSQS [nifi]
exceptionfactory commented on PR #8505: URL: https://github.com/apache/nifi/pull/8505#issuecomment-2004246853 Thanks for the reply and reference @Freedom9339. My apologies, I missed the fact the the class hierarchy now integrates the SSL Context Service when building the HTTP Client for the appropriate Client Builder. With that understanding, it looks like this is ready to go forward. Thanks again for the helpful reply! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-6379: Added SSL context to PutSNS, DeleteSQS, GetSQS, and PutSQS [nifi]
Freedom9339 commented on PR #8505: URL: https://github.com/apache/nifi/pull/8505#issuecomment-2004127985 @exceptionfactory Thank you for the review. The SSL_CONTEXT_SERVICE property lives in AbstractAWSProcessor and can be used by any processor that uses it. The SSL context is added to the client here: https://github.com/apache/nifi/blob/129130fc0966fee6dfcf8fa514841ebc07045579/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/v2/AbstractAwsProcessor.java#L318C9-L326C10 I have tested this using a custom truststore and keystore and the processors do use them when it is set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12908 provide powershell versions of the nifi-env.cmd and nifi.cmd [nifi]
exceptionfactory commented on PR #8509: URL: https://github.com/apache/nifi/pull/8509#issuecomment-2004018034 > > Can you provide some additional background on the reason for introducing a PowerShell alternative to the `nifi.cmd` script for Windows? It would be ideal to maintain a unified approach for Windows, as opposed to maintaining two separate scripts for the same OS. > > I am just starting out with NiFi, so I will have to speak in generalities. > > Making changes to _.cmd/_.bat files is fraught (for me). Every time I think I am going to make a small change to one it ends up being a time sink. > > I am planning on extensively using NiFi for a project I am working on. A lot of the work will involve dynamic deployment of NiFi components on a dynamic cluster. I anticipate building tooling to support this work. I would much prefer to use powershell over CMD/BAT. I plan on contributing this tooling back to Apache. > > It may be that the tooling would be better written in one of the JVM scripting languages (i.e. Groovy, Kotlin, Clojure). That said I expect that some of the tooling would still best be written as powershell modules on Windows. > > Obviously, the `nifi.ps1` is not (yet) a powershell module, I am getting a feel for how the NiFi project operates. I would like to see the `nifi.cmd` deprecated in favor of powershell modules. Thanks for additional background @phreed. The shell scripts are intended to provide a minimum wrapper around the environment settings necessary to start NiFi Java processes. For that reason, these scripts should be as minimal as possible, and provide the broadest base of compatibility. That is why the current scripts for Windows do not use PowerShell, and should avoid additional complexity. That is also the reason why they do not use any type of JVM scripting. Running NiFi on Windows is supported in general, but running scalable production deployments most often depends on Linux as the operating system. So with that background, any shell script changes should continue to follow the minimalist approach to maintain compatibility across modern distributions of Windows. With that in mind, it may be best to avoid introducing PowerShell scripts in favor of retaining the limited scope of command scripting. This does highlight the need for reviewing the general approach of Windows startup scripts, including the removal of the `BAT` scripts. If the PowerShell approach works without installing any additional Windows packages on current versions of both Home and Pro, then this seems like a potential option. Are you aware of any compatibility issues with PowerShell on recent Windows versions, particularly on non-server editions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12877) RestLookupService - supports Sensitive Dynamic Properties
[ https://issues.apache.org/jira/browse/NIFI-12877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12877: Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > RestLookupService - supports Sensitive Dynamic Properties > - > > Key: NIFI-12877 > URL: https://issues.apache.org/jira/browse/NIFI-12877 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Roman >Assignee: Lehel Boér >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The RestLookupService service does not provide the possibility to support > Sensitive Dynamic Properties: Yes. > When we add a header in InvokeHTTP processor we can set "Sensitive Value" > Yes/No, however during adding dynamic values (tokens) in RestLookupService > there is no way to make it sensitive it is grayed out. > Desired improvement: > RestLookupService > *Dynamic Properties:* > Supports Sensitive Dynamic Properties: *Yes* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12877) RestLookupService - supports Sensitive Dynamic Properties
[ https://issues.apache.org/jira/browse/NIFI-12877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827986#comment-17827986 ] ASF subversion and git services commented on NIFI-12877: Commit d1387c4773f4c76362cdca2d14b7f2328adecfc0 in nifi's branch refs/heads/support/nifi-1.x from lehelb [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d1387c4773 ] NIFI-12877 Added Sensitive Dynamic Property Support to RestLookUpService This closes #8525 Signed-off-by: David Handermann (cherry picked from commit 129130fc0966fee6dfcf8fa514841ebc07045579) > RestLookupService - supports Sensitive Dynamic Properties > - > > Key: NIFI-12877 > URL: https://issues.apache.org/jira/browse/NIFI-12877 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Roman >Assignee: Lehel Boér >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The RestLookupService service does not provide the possibility to support > Sensitive Dynamic Properties: Yes. > When we add a header in InvokeHTTP processor we can set "Sensitive Value" > Yes/No, however during adding dynamic values (tokens) in RestLookupService > there is no way to make it sensitive it is grayed out. > Desired improvement: > RestLookupService > *Dynamic Properties:* > Supports Sensitive Dynamic Properties: *Yes* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12877) RestLookupService - supports Sensitive Dynamic Properties
[ https://issues.apache.org/jira/browse/NIFI-12877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827985#comment-17827985 ] ASF subversion and git services commented on NIFI-12877: Commit 129130fc0966fee6dfcf8fa514841ebc07045579 in nifi's branch refs/heads/main from lehelb [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=129130fc09 ] NIFI-12877 Added Sensitive Dynamic Property Support to RestLookUpService This closes #8525 Signed-off-by: David Handermann > RestLookupService - supports Sensitive Dynamic Properties > - > > Key: NIFI-12877 > URL: https://issues.apache.org/jira/browse/NIFI-12877 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Roman >Assignee: Lehel Boér >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The RestLookupService service does not provide the possibility to support > Sensitive Dynamic Properties: Yes. > When we add a header in InvokeHTTP processor we can set "Sensitive Value" > Yes/No, however during adding dynamic values (tokens) in RestLookupService > there is no way to make it sensitive it is grayed out. > Desired improvement: > RestLookupService > *Dynamic Properties:* > Supports Sensitive Dynamic Properties: *Yes* -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12877: RestLookUpService support sensitive dynamic properties [nifi]
exceptionfactory closed pull request #8525: NIFI-12877: RestLookUpService support sensitive dynamic properties URL: https://github.com/apache/nifi/pull/8525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12914) Dependabot: follow-redirects
[ https://issues.apache.org/jira/browse/NIFI-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFI-12914. Fix Version/s: 2.0.0-M3 Resolution: Duplicate NIFI-12914 > Dependabot: follow-redirects > > > Key: NIFI-12914 > URL: https://issues.apache.org/jira/browse/NIFI-12914 > Project: Apache NiFi > Issue Type: Task > Components: Core UI >Reporter: Matt Gilman >Priority: Major > Fix For: 2.0.0-M3 > > > Bump follow-redirects from 1.15.5 to 1.15.6 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12674 Modified ValidateCSV to make the schema optional if a head… [nifi]
Freedom9339 commented on PR #8362: URL: https://github.com/apache/nifi/pull/8362#issuecomment-2003939680 Any updates on reviewing this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12801) Add local file upload option in PutHDFS processor
[ https://issues.apache.org/jira/browse/NIFI-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12801: Fix Version/s: (was: 2.0.0-M3) > Add local file upload option in PutHDFS processor > - > > Key: NIFI-12801 > URL: https://issues.apache.org/jira/browse/NIFI-12801 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Shubham Sharma >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > To add support for FileResourceService in PutHDFS similar to PutAzureStorage > processors, following improvement will enable option to upload file from the > local filesystem directly to HDFS storage by-passing the content repo. > The two available file resource transfer sources are: > * FlowFile's Content (default, current behaviour) > * Local File > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12913) Python venv process failure on Ubuntu does not throw proper exception
[ https://issues.apache.org/jira/browse/NIFI-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12913. - Fix Version/s: 2.0.0 Assignee: Bob Paulin Resolution: Fixed > Python venv process failure on Ubuntu does not throw proper exception > - > > Key: NIFI-12913 > URL: https://issues.apache.org/jira/browse/NIFI-12913 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M2 > Environment: Ubuntu >Reporter: Bob Paulin >Assignee: Bob Paulin >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Staring nifi with python extensions on is throwing a NPE in the shutdown > process that swallows the root issue > Relevant stack trace: > > > {code:java} > Caused by: java.lang.NullPointerException: Identifier required > at java.base/java.util.Objects.requireNonNull(Objects.java:259) > at > org.apache.nifi.py4j.logging.StandardLogLevelChangeHandler.removeListener(StandardLogLevelChangeHandler.java:78) > at > org.apache.nifi.py4j.PythonProcess.killProcess(PythonProcess.java:364) > at org.apache.nifi.py4j.PythonProcess.shutdown(PythonProcess.java:360) > at > org.apache.nifi.py4j.StandardPythonBridge.shutdown(StandardPythonBridge.java:297) > at > org.apache.nifi.py4j.StandardPythonBridge.start(StandardPythonBridge.java:85) > at > org.apache.nifi.components.ClassLoaderAwarePythonBridge.start(ClassLoaderAwarePythonBridge.java:52) > at > org.apache.nifi.controller.FlowController.(FlowController.java:579) > at > org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:411) > at > org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:83){code} > > > Suggest at a minimum we need a null check here to let the real exception > related to the venv install failure come through. > > ROOT ISSUE > The venv install command fails on Ubuntu without first running an install > command on ubuntu > EX > > > {code:java} > apt install python3.10-venv {code} > > > If you run venv without it on some Ubuntu/Deb systems you get: > {code:java} > python3 -m venv /home/bpaulin/nifi-2.0.0-SNAPSHOT/work/python/ > The virtual environment was not created successfully because ensurepip is not > available. On Debian/Ubuntu systems, you need to install the python3-venv > package using the following command. apt install python3.10-venv You may need > to use sudo with that command. After installing the python3-venv package, > recreate your virtual environment. Failing command: > /home/bpaulin/nifi-2.0.0-SNAPSHOT/work/python/bin/python3 {code} > > Suggest either reading this process output text as a part of the exception > handling, sending the process output to the logger for easier debugging. > The process is exited with a 1 status but the NPE at the top of the issue is > preventing it from getting logged leaving the user guessing what happened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12883) JsonTreeReader not able to infer schema from JSON of a ControllerServiceEntity
[ https://issues.apache.org/jira/browse/NIFI-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz resolved NIFI-12883. - Resolution: Not A Problem > JsonTreeReader not able to infer schema from JSON of a ControllerServiceEntity > -- > > Key: NIFI-12883 > URL: https://issues.apache.org/jira/browse/NIFI-12883 > Project: Apache NiFi > Issue Type: Bug >Reporter: Daniel Stieglitz >Priority: Major > Attachments: ControllerServiceEntity.json > > > When trying to use the ConvertRecord processor configured with a > JsonTreeReader whose Schema > Access Strategy is to Infer schema and configured with a JsonRecordSetWriter > to write the > schema out all in order to read and write a ControllerServiceEntity JSON > object returned by the NIFI Rest API and produce the Avro schema, I got the > following stacktrace > below. I have attached the ControllerServiceEntity JSON which I tried to > convert. > {code:java} > 2024-03-08 17:41:26,198 ERROR [Timer-Driven Process Thread-2] > o.a.n.processors.standard.ConvertRecord > ConvertRecord[id=1f213eb6-018e-1000-e76a-b9ac6041ed48] Failed to process > StandardFlowFileRecord[uuid=45aa31af-0850-42be-9f9e-05001acbf8f2,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1709914518257-1, container=default, > section=1], offset=34287, > length=14358],offset=0,name=sampleAfterDisablingStandardJsonSchemaRegistry.json,size=14358]; > will route to failure > org.apache.avro.SchemaParseException: Illegal character in: > component_descriptors_JSON Schema Version_allowableValues_allowableValueType > at org.apache.avro.Schema.validateName(Schema.java:1625) > at org.apache.avro.Schema.access$400(Schema.java:94) > at org.apache.avro.Schema$Name.(Schema.java:713) > at org.apache.avro.Schema.createRecord(Schema.java:226) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:287) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:211) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130) > at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:122) > at > org.apache.nifi.avro.AvroTypeUtil.extractAvroSchema(AvroTypeUtil.java:102) > at > org.apache.nifi.schema.access.WriteAvroSchemaAttributeStrategy.lambda$new$0(WriteAvroSchemaAttributeStrategy.java:36) > at > com.github.benmanes.caffeine.cache.LocalLoadingCache.lambda$newMappingFunction$2(LocalLoadingCache.java:145) > at > com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406) > at > java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1916) > at > com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2404) > at > com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2387) > at > com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:108) > at > com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:56) > at > org.apache.nifi.schema.access.WriteAvroSchemaAttributeStrategy.getAttributes(WriteAvroSchemaAttributeStrategy.java:53) > at > org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:151) > at > org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:568) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControll
[jira] [Commented] (NIFI-12913) Python venv process failure on Ubuntu does not throw proper exception
[ https://issues.apache.org/jira/browse/NIFI-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827969#comment-17827969 ] ASF subversion and git services commented on NIFI-12913: Commit 2303a15f8ecf5b5b48764fae3555f33103252ca6 in nifi's branch refs/heads/main from Bob Paulin [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2303a15f8e ] NIFI-12913 Corrected NPE for Python Log Listener ID This closes #8526 Signed-off-by: David Handermann > Python venv process failure on Ubuntu does not throw proper exception > - > > Key: NIFI-12913 > URL: https://issues.apache.org/jira/browse/NIFI-12913 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0-M2 > Environment: Ubuntu >Reporter: Bob Paulin >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Staring nifi with python extensions on is throwing a NPE in the shutdown > process that swallows the root issue > Relevant stack trace: > > > {code:java} > Caused by: java.lang.NullPointerException: Identifier required > at java.base/java.util.Objects.requireNonNull(Objects.java:259) > at > org.apache.nifi.py4j.logging.StandardLogLevelChangeHandler.removeListener(StandardLogLevelChangeHandler.java:78) > at > org.apache.nifi.py4j.PythonProcess.killProcess(PythonProcess.java:364) > at org.apache.nifi.py4j.PythonProcess.shutdown(PythonProcess.java:360) > at > org.apache.nifi.py4j.StandardPythonBridge.shutdown(StandardPythonBridge.java:297) > at > org.apache.nifi.py4j.StandardPythonBridge.start(StandardPythonBridge.java:85) > at > org.apache.nifi.components.ClassLoaderAwarePythonBridge.start(ClassLoaderAwarePythonBridge.java:52) > at > org.apache.nifi.controller.FlowController.(FlowController.java:579) > at > org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:411) > at > org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:83){code} > > > Suggest at a minimum we need a null check here to let the real exception > related to the venv install failure come through. > > ROOT ISSUE > The venv install command fails on Ubuntu without first running an install > command on ubuntu > EX > > > {code:java} > apt install python3.10-venv {code} > > > If you run venv without it on some Ubuntu/Deb systems you get: > {code:java} > python3 -m venv /home/bpaulin/nifi-2.0.0-SNAPSHOT/work/python/ > The virtual environment was not created successfully because ensurepip is not > available. On Debian/Ubuntu systems, you need to install the python3-venv > package using the following command. apt install python3.10-venv You may need > to use sudo with that command. After installing the python3-venv package, > recreate your virtual environment. Failing command: > /home/bpaulin/nifi-2.0.0-SNAPSHOT/work/python/bin/python3 {code} > > Suggest either reading this process output text as a part of the exception > handling, sending the process output to the logger for easier debugging. > The process is exited with a 1 status but the NPE at the top of the issue is > preventing it from getting logged leaving the user guessing what happened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12913 [nifi]
exceptionfactory closed pull request #8526: NIFI-12913 URL: https://github.com/apache/nifi/pull/8526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-12915) Upgrade follow-redirects to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12915. - Resolution: Fixed > Upgrade follow-redirects to 1.15.6 > -- > > Key: NIFI-12915 > URL: https://issues.apache.org/jira/browse/NIFI-12915 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > The [follow-redirects|https://github.com/follow-redirects/follow-redirects] > library for NiFi Registry UI and the new NiFi Web Frontend should be upgraded > to 1.15.6 to mitigate CVE-2024-28849 related to Proxy Authorization header > handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12915) Upgrade follow-redirects to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827963#comment-17827963 ] ASF subversion and git services commented on NIFI-12915: Commit 68b880c74df0499289cce8dce7ef6a53d6ca037f in nifi's branch refs/heads/main from dependabot[bot] [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=68b880c74d ] NIFI-12915 Upgraded follow-redirects from 1.15.5 to 1.15.6 This closes #8524 Signed-off-by: David Handermann > Upgrade follow-redirects to 1.15.6 > -- > > Key: NIFI-12915 > URL: https://issues.apache.org/jira/browse/NIFI-12915 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The [follow-redirects|https://github.com/follow-redirects/follow-redirects] > library for NiFi Registry UI and the new NiFi Web Frontend should be upgraded > to 1.15.6 to mitigate CVE-2024-28849 related to Proxy Authorization header > handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12915) Upgrade follow-redirects to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827964#comment-17827964 ] ASF subversion and git services commented on NIFI-12915: Commit 4d825e6b6504b90d840b4f5a9d07467fe67159ba in nifi's branch refs/heads/main from dependabot[bot] [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4d825e6b65 ] NIFI-12915 Upgraded follow-redirects from 1.15.5 to 1.15.6 This closes #8523 Signed-off-by: David Handermann > Upgrade follow-redirects to 1.15.6 > -- > > Key: NIFI-12915 > URL: https://issues.apache.org/jira/browse/NIFI-12915 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The [follow-redirects|https://github.com/follow-redirects/follow-redirects] > library for NiFi Registry UI and the new NiFi Web Frontend should be upgraded > to 1.15.6 to mitigate CVE-2024-28849 related to Proxy Authorization header > handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12915 Bump follow-redirects from 1.15.4 to 1.15.6 in /nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main [nifi]
dependabot[bot] commented on PR #8523: URL: https://github.com/apache/nifi/pull/8523#issuecomment-2003893393 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12915 Bump follow-redirects from 1.15.5 to 1.15.6 in /nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi [nifi]
dependabot[bot] commented on PR #8524: URL: https://github.com/apache/nifi/pull/8524#issuecomment-2003893373 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12915 Bump follow-redirects from 1.15.4 to 1.15.6 in /nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main [nifi]
exceptionfactory closed pull request #8523: NIFI-12915 Bump follow-redirects from 1.15.4 to 1.15.6 in /nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main URL: https://github.com/apache/nifi/pull/8523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12915 Bump follow-redirects from 1.15.5 to 1.15.6 in /nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi [nifi]
exceptionfactory closed pull request #8524: NIFI-12915 Bump follow-redirects from 1.15.5 to 1.15.6 in /nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi URL: https://github.com/apache/nifi/pull/8524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12915) Upgrade follow-redirects to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12915: Summary: Upgrade follow-redirects to 1.15.6 (was: Upgraded follow-redirects to 1.15.6) > Upgrade follow-redirects to 1.15.6 > -- > > Key: NIFI-12915 > URL: https://issues.apache.org/jira/browse/NIFI-12915 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > > The [follow-redirects|https://github.com/follow-redirects/follow-redirects] > library for NiFi Registry UI and the new NiFi Web Frontend should be upgraded > to 1.15.6 to mitigate CVE-2024-28849 related to Proxy Authorization header > handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12915) Upgraded follow-redirects to 1.15.6
David Handermann created NIFI-12915: --- Summary: Upgraded follow-redirects to 1.15.6 Key: NIFI-12915 URL: https://issues.apache.org/jira/browse/NIFI-12915 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0 The [follow-redirects|https://github.com/follow-redirects/follow-redirects] library for NiFi Registry UI and the new NiFi Web Frontend should be upgraded to 1.15.6 to mitigate CVE-2024-28849 related to Proxy Authorization header handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12915) Upgraded follow-redirects to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12915: Component/s: NiFi Registry > Upgraded follow-redirects to 1.15.6 > --- > > Key: NIFI-12915 > URL: https://issues.apache.org/jira/browse/NIFI-12915 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > > The [follow-redirects|https://github.com/follow-redirects/follow-redirects] > library for NiFi Registry UI and the new NiFi Web Frontend should be upgraded > to 1.15.6 to mitigate CVE-2024-28849 related to Proxy Authorization header > handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12914) Dependabot: follow-redirects
Matt Gilman created NIFI-12914: -- Summary: Dependabot: follow-redirects Key: NIFI-12914 URL: https://issues.apache.org/jira/browse/NIFI-12914 Project: Apache NiFi Issue Type: Task Components: Core UI Reporter: Matt Gilman Bump follow-redirects from 1.15.5 to 1.15.6 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12902) Migrate to @angular-devkit/build-angular:application
[ https://issues.apache.org/jira/browse/NIFI-12902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12902: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Migrate to @angular-devkit/build-angular:application > > > Key: NIFI-12902 > URL: https://issues.apache.org/jira/browse/NIFI-12902 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Migrate to the latest builder to try avoid sporadic webpack build issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12902) Migrate to @angular-devkit/build-angular:application
[ https://issues.apache.org/jira/browse/NIFI-12902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827956#comment-17827956 ] ASF subversion and git services commented on NIFI-12902: Commit 940904e1eabb499c33b7d8774742a48f34281990 in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=940904e1ea ] NIFI-12902: (#8521) - Updating packaging to account for new build output. > Migrate to @angular-devkit/build-angular:application > > > Key: NIFI-12902 > URL: https://issues.apache.org/jira/browse/NIFI-12902 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Migrate to the latest builder to try avoid sporadic webpack build issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12902: Updating packaging to account for new build output [nifi]
rfellows merged PR #8521: URL: https://github.com/apache/nifi/pull/8521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12837) Add DFS setting to smb processors
[ https://issues.apache.org/jira/browse/NIFI-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-12837: --- Affects Version/s: 2.0.0-M2 Status: Patch Available (was: In Progress) > Add DFS setting to smb processors > - > > Key: NIFI-12837 > URL: https://issues.apache.org/jira/browse/NIFI-12837 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2, 1.25.0 >Reporter: Anders >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The hierynomus/smbj library has a setting for enabling DFS which is disabled > by default: > https://github.com/hierynomus/smbj/blob/f25d5c5478a5b73e9ba4202dcfb365974e15367e/src/main/java/com/hierynomus/smbj/SmbConfig.java#L106C17-L106C39 > This appears to cause problems in some SMB configurations. > Patched > https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java > to test in my environment with: > {code} > $ git diff > nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java > diff --git > a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java > > b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java > index 0895abfae0..eac765 100644 > --- > a/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java > +++ > b/nifi-nar-bundles/nifi-smb-bundle/nifi-smb-smbj-common/src/main/java/org/apache/nifi/smb/common/SmbUtils.java > @@ -46,6 +46,8 @@ public final class SmbUtils { > } > } > +configBuilder.withDfsEnabled(true); > + > if (context.getProperty(USE_ENCRYPTION).isSet()) { > > configBuilder.withEncryptData(context.getProperty(USE_ENCRYPTION).asBoolean()); > } > {code} > This appeared to resolve the issue. > It would be very useful if this setting was available to toggle in the UI for > all SMB processors. > Without this setting, I get a *STATUS_PATH_NOT_COVERED* error. > Somewhat related hierynomus/smbj github issues: > https://github.com/hierynomus/smbj/issues/152 > https://github.com/hierynomus/smbj/issues/419 > This setting should be made available in the following processors and > services: > * GetSmbFile > * PutSmbFile > * SmbjClientProviderService > Edit: It might require some more changes to handle the connections and > sessions correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12837 Added DFS support in SMB processors [nifi]
turcsanyip opened a new pull request, #8527: URL: https://github.com/apache/nifi/pull/8527 # Summary [NIFI-12837](https://issues.apache.org/jira/browse/NIFI-12837) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org