[GitHub] [nifi] mattyb149 commented on pull request #6903: NIFI-11111 add option to output Elasticsearch error responses as FlowFile to PutElasticsearchJson and PutElasticsearchRecord
mattyb149 commented on PR #6903: URL: https://github.com/apache/nifi/pull/6903#issuecomment-1477245200 Sorry I lost track of this, @MikeThomsen I'm good if you are -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003: URL: https://github.com/apache/nifi/pull/7003#discussion_r1142750736 ## nifi-nar-bundles/nifi-py4j-bundle/nifi-python-extension-api/src/main/python/src/nifiapi/properties.py: ## @@ -0,0 +1,411 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from enum import Enum +from nifiapi.__jvm__ import JvmHolder +import re + +class ExpressionLanguageScope(Enum): +NONE = 1 +LIMITED = 2 +FLOWFILE_ATTRIBUTES = 3 + + +class StandardValidators: +__standard_validators__ = JvmHolder.jvm.org.apache.nifi.processor.util.StandardValidators + +ALWAYS_VALID = JvmHolder.jvm.org.apache.nifi.components.Validator.VALID +NON_EMPTY_VALIDATOR = __standard_validators__.NON_EMPTY_VALIDATOR +INTEGER_VALIDATOR = __standard_validators__.INTEGER_VALIDATOR +POSITIVE_INTEGER_VALIDATOR = __standard_validators__.POSITIVE_INTEGER_VALIDATOR +POSITIVE_LONG_VALIDATOR = __standard_validators__.POSITIVE_LONG_VALIDATOR +NON_NEGATIVE_INTEGER_VALIDATOR = __standard_validators__.NON_NEGATIVE_INTEGER_VALIDATOR +NUMBER_VALIDATOR = __standard_validators__.NUMBER_VALIDATOR +LONG_VALIDATOR = __standard_validators__.LONG_VALIDATOR +PORT_VALIDATOR = __standard_validators__.PORT_VALIDATOR +NON_EMPTY_EL_VALIDATOR = __standard_validators__.NON_EMPTY_EL_VALIDATOR +HOSTNAME_PORT_LIST_VALIDATOR = __standard_validators__.HOSTNAME_PORT_LIST_VALIDATOR +BOOLEAN_VALIDATOR = __standard_validators__.BOOLEAN_VALIDATOR +URL_VALIDATOR = __standard_validators__.URL_VALIDATOR +URI_VALIDATOR = __standard_validators__.URI_VALIDATOR +REGULAR_EXPRESSION_VALIDATOR = __standard_validators__.REGULAR_EXPRESSION_VALIDATOR +REGULAR_EXPRESSION_WITH_EL_VALIDATOR = __standard_validators__.REGULAR_EXPRESSION_WITH_EL_VALIDATOR +TIME_PERIOD_VALIDATOR = __standard_validators__.TIME_PERIOD_VALIDATOR +DATA_SIZE_VALIDATOR = __standard_validators__.DATA_SIZE_VALIDATOR +FILE_EXISTS_VALIDATOR = __standard_validators__.FILE_EXISTS_VALIDATOR + + + +class PropertyDependency: +def __init__(self, property_descriptor, dependent_values=None): +if dependent_values is None: +dependent_values = [] + +if isinstance(property_descriptor, str): +self.property_name = property_descriptor +else: +self.property_name = property_descriptor.getName() +self.dependentValues = dependent_values + +@staticmethod +def from_java_dependency(java_dependencies): +if java_dependencies is None or len(java_dependencies) == 0: +return None + +dependencies = [] +for dependency in java_dependencies: + dependencies.append(PropertyDependency(dependency.getPropertyName(), dependency.getDependentValues())) + +return dependencies + + +class ResourceDefinition: +def __init__(self, allow_multiple=False, allow_file=True, allow_url=False, allow_directory=False, allow_text=False): +self.allow_multiple = allow_multiple +self.allow_file = allow_file +self.allow_url = allow_url +self.allow_directory = allow_directory +self.allow_text = allow_text + +@staticmethod +def from_java_definition(java_definition): +if java_definition is None: +return None + +allow_multiple = java_definition.getCardinality().name() == "MULTIPLE" +resource_types = java_definition.getResourceTypes() +allow_file = False +allow_url = False +allow_directory = False +allow_text = False +for type in resource_types: +name = type.name() +if name == "FILE": +allow_file = True +elif name == "DIRECTORY": +allow_directory = True +elif name == "TEXT": +allow_text = True +elif name == "URL": +allow_url = True + +return ResourceDefinition(allow_multiple, allow_file, allow_url, allow_directory, allow_text) + + +class PropertyDescriptor: +def __init__(self, name, description, required=False, sensitive=False, + display_name=None, default_value=None, allowable_values=None, +
[jira] [Updated] (NIFI-11312) NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
[
https://issues.apache.org/jira/browse/NIFI-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-11312:
-
Fix Version/s: 1.latest
2.latest
> NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
> --
>
> Key: NIFI-11312
> URL: https://issues.apache.org/jira/browse/NIFI-11312
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Major
> Fix For: 1.latest, 2.latest
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The {{dockermaven}} and {{dockerhub}} image builds are not consistent, e.g.
> then have different startup scripts copied into the images (some of which
> contain issues like incorrect properties being used).
> See https://github.com/apache/nifi/pull/6995 (for NIFI-7060) for a more
> complete list of known inconsistences that should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] nandorsoma commented on pull request #7027: NIFI-11264 Improve logging in list processors and fix ListFile's description
nandorsoma commented on PR #7027: URL: https://github.com/apache/nifi/pull/7027#issuecomment-1477021310 Thank you for the review @Lehel44 and @exceptionfactory! I tend to agree that ending log messages with `.` is not a best practice. In my defense, I did that because it was a pattern in this file. I've changed the logs as requested and changed two more previously existing logs to be in sync. I found it reasonable in the other cases, so I didn't change them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11316) Remove Admin Toolkit Commands
[
https://issues.apache.org/jira/browse/NIFI-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11316:
Status: Patch Available (was: Open)
> Remove Admin Toolkit Commands
> -
>
> Key: NIFI-11316
> URL: https://issues.apache.org/jira/browse/NIFI-11316
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{nifi-toolkit-admin}} module includes several commands for administering
> NiFi deployments. The NiFi CLI toolkit and other nifi.sh commands have
> obviated features such as the Node Manager Tool, and other configuration
> migration components are not relevant to current configuration formats.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11283) Add GitHub Actions CI build to nifi-maven project
[ https://issues.apache.org/jira/browse/NIFI-11283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11283: Resolution: Fixed Status: Resolved (was: Patch Available) > Add GitHub Actions CI build to nifi-maven project > - > > Key: NIFI-11283 > URL: https://issues.apache.org/jira/browse/NIFI-11283 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Kevin Doran >Assignee: David Handermann >Priority: Major > Fix For: nifi-nar-maven-plugin-1.5.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > We should have CI builds with contrib-check for the NAR Maven Plugin repo. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7065: NIFI-11316 Remove Admin Toolkit Commands
exceptionfactory opened a new pull request, #7065: URL: https://github.com/apache/nifi/pull/7065 # Summary [NIFI-11316](https://issues.apache.org/jira/browse/NIFI-11316) Removes the `nifi-toolkit-admin` module with associated administrative commands. The NiFi CLI provides node management commands and other functions, serving as a unified replacement for some elements of the Admin Toolkit. Other configuration migration features are not relevant to NiFi 2.0, and new migration features will require a different approach. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11316) Remove Admin Toolkit Commands
David Handermann created NIFI-11316:
---
Summary: Remove Admin Toolkit Commands
Key: NIFI-11316
URL: https://issues.apache.org/jira/browse/NIFI-11316
Project: Apache NiFi
Issue Type: Improvement
Components: Tools and Build
Reporter: David Handermann
Assignee: David Handermann
Fix For: 2.latest
The {{nifi-toolkit-admin}} module includes several commands for administering
NiFi deployments. The NiFi CLI toolkit and other nifi.sh commands have obviated
features such as the Node Manager Tool, and other configuration migration
components are not relevant to current configuration formats.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 opened a new pull request, #7064: NIFI-11312 rationalise nifi-registry docker image startup scripts
ChrisSamo632 opened a new pull request, #7064: URL: https://github.com/apache/nifi/pull/7064 # Summary [NIFI-11312](https://issues.apache.org/jira/browse/NIFI-11312) rationalise nifi-registry docker image startup scripts # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - ~[ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)~ - ~[ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files~ ### Documentation - ~[ ] Documentation formatting appears as expected in rendered files~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11312) NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
[
https://issues.apache.org/jira/browse/NIFI-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-11312:
-
Status: Patch Available (was: In Progress)
> NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
> --
>
> Key: NIFI-11312
> URL: https://issues.apache.org/jira/browse/NIFI-11312
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{dockermaven}} and {{dockerhub}} image builds are not consistent, e.g.
> then have different startup scripts copied into the images (some of which
> contain issues like incorrect properties being used).
> See https://github.com/apache/nifi/pull/6995 (for NIFI-7060) for a more
> complete list of known inconsistences that should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 closed pull request #7063: NIFI-11312 rationalise nifi-registry docker image startup scripts
ChrisSamo632 closed pull request #7063: NIFI-11312 rationalise nifi-registry docker image startup scripts URL: https://github.com/apache/nifi/pull/7063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] ChrisSamo632 opened a new pull request, #7063: NIFI-11312 rationalise nifi-registry docker image startup scripts
ChrisSamo632 opened a new pull request, #7063: URL: https://github.com/apache/nifi/pull/7063 # Summary [NIFI-11312](https://issues.apache.org/jira/browse/NIFI-11312) rationalise nifi-registry docker image startup scripts # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - ~[ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)~ - ~[ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files~ ### Documentation - ~[ ] Documentation formatting appears as expected in rendered files~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11312) NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
[
https://issues.apache.org/jira/browse/NIFI-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702946#comment-17702946
]
ASF subversion and git services commented on NIFI-11312:
Commit de1c0e855eef8e21441ca04142bc13d44e8abd71 in nifi's branch
refs/heads/NIFI-11312 from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=de1c0e855e ]
NIFI-11312 rationalise nifi-registry docker image startup scripts
> NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
> --
>
> Key: NIFI-11312
> URL: https://issues.apache.org/jira/browse/NIFI-11312
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Chris Sampson
>Priority: Major
>
> The {{dockermaven}} and {{dockerhub}} image builds are not consistent, e.g.
> then have different startup scripts copied into the images (some of which
> contain issues like incorrect properties being used).
> See https://github.com/apache/nifi/pull/6995 (for NIFI-7060) for a more
> complete list of known inconsistences that should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-11312) NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
[
https://issues.apache.org/jira/browse/NIFI-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson reassigned NIFI-11312:
Assignee: Chris Sampson
> NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
> --
>
> Key: NIFI-11312
> URL: https://issues.apache.org/jira/browse/NIFI-11312
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Major
>
> The {{dockermaven}} and {{dockerhub}} image builds are not consistent, e.g.
> then have different startup scripts copied into the images (some of which
> contain issues like incorrect properties being used).
> See https://github.com/apache/nifi/pull/6995 (for NIFI-7060) for a more
> complete list of known inconsistences that should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003: URL: https://github.com/apache/nifi/pull/7003#discussion_r1142710823 ## nifi-nar-bundles/nifi-py4j-bundle/nifi-python-extension-api/src/main/python/src/nifiapi/flowfiletransform.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from abc import ABC, abstractmethod +from nifiapi.__jvm__ import JvmHolder +from nifiapi.properties import ProcessContext + + +class FlowFileTransform(ABC): Review Comment: This gets complicated when it comes to automatically detecting extensions. The method employed currently is probably not a great solution. But with my limited Python understanding, it works. It's using the abstract syntax tree from parsing the class. Which means that it's looking for this in particular. We could probably change the AST to instead look for inheriting from `FlowFileTransform` but that could then be aliased in the import, etc. so that complicates things. As noted, I expect all of this to evolve over time. Just want to get something in the hands of users for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11311) Upgrade maven-antrun-plugin version
[
https://issues.apache.org/jira/browse/NIFI-11311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-11311:
-
Status: Patch Available (was: In Progress)
> Upgrade maven-antrun-plugin version
> ---
>
> Key: NIFI-11311
> URL: https://issues.apache.org/jira/browse/NIFI-11311
> Project: Apache NiFi
> Issue Type: Task
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{maven-antrun-plugin}} is used in several module builds (e.g. to copy
> files for a Docker Image build), but the version is inconsistent across
> modules and is very old in some cases.
> Current latest is {{3.1.0}}, we should look to upgrade to that across all
> modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11311) Upgrade maven-antrun-plugin version
[
https://issues.apache.org/jira/browse/NIFI-11311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-11311:
-
Fix Version/s: 1.latest
2.latest
> Upgrade maven-antrun-plugin version
> ---
>
> Key: NIFI-11311
> URL: https://issues.apache.org/jira/browse/NIFI-11311
> Project: Apache NiFi
> Issue Type: Task
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{maven-antrun-plugin}} is used in several module builds (e.g. to copy
> files for a Docker Image build), but the version is inconsistent across
> modules and is very old in some cases.
> Current latest is {{3.1.0}}, we should look to upgrade to that across all
> modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] ChrisSamo632 opened a new pull request, #7062: NIFI-11311 update maven-antrun-plugin and rationalise version across all modules
ChrisSamo632 opened a new pull request, #7062: URL: https://github.com/apache/nifi/pull/7062 # Summary [NIFI-11311](https://issues.apache.org/jira/browse/NIFI-11311) update maven-antrun-plugin and rationalise version across all modules # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 11 - [ ] JDK 17 ### Licensing - ~[ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)~ - ~[ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files~ ### Documentation - ~[ ] Documentation formatting appears as expected in rendered files~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11311) Upgrade maven-antrun-plugin version
[
https://issues.apache.org/jira/browse/NIFI-11311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson reassigned NIFI-11311:
Assignee: Chris Sampson
> Upgrade maven-antrun-plugin version
> ---
>
> Key: NIFI-11311
> URL: https://issues.apache.org/jira/browse/NIFI-11311
> Project: Apache NiFi
> Issue Type: Task
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Minor
>
> The {{maven-antrun-plugin}} is used in several module builds (e.g. to copy
> files for a Docker Image build), but the version is inconsistent across
> modules and is very old in some cases.
> Current latest is {{3.1.0}}, we should look to upgrade to that across all
> modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1142707556
##
nifi-docs/src/main/asciidoc/python-developer-guide.adoc:
##
@@ -0,0 +1,543 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+= NiFi Python Developer's Guide
+Apache NiFi Team
+:homepage: http://nifi.apache.org
+:linkattrs:
+
+
+== Introduction
+
+This guide is intended to provide an introduction and some guidance to
developing extensions for Apache NiFi using Python.
+This guide is not intended to be an alternative to the
link:developer-guide.adoc[NiFi Developers Guide] document but rather
+a supplement to it. The normal Developer Guide is far more in depth and
discusses more topics. However, that guide is
+targeted toward Java developers. The philosophies and guidance offered in that
guide, generally, still hold for Python extensions, though.
+
+[[java_python_comms]]
+== Java/Python Communication
+
+While NiFi is a Java based application, we do allow for native Python based
processors. In order for this to work, it is essential
+that both the Java and Python processes be able to communicate with one
another. To facilitate this, when a Python process is launched,
+a server is started on both the Java and Python sides. This server is started
in such a way that it listens only on local network interfaces.
+That is, it is not possible to connect to either the Java or Python server
from another machine. Connections must be made from localhost.
+This provides an important security layer.
+
+There are objects on the Java side that must be made available to the Python
side. Likewise, the Python side must return information to the Java
+side. For example, the Java application is responsible for storing the flow
definition, such as the fact that some Processor exists, the configuration
+of that Processor, etc. It's also responsible for maintaining the FlowFiles
and their data. This information must be conveyed over the socket from
+the Java side to the Python side. Once a Python Processor performs its task
and wants to route a given FlowFile to some relationship, this information
+must also be conveyed back to the Java side.
+
+Fortunately, the NiFi API handles all of this and makes this seamless. This is
handled by means of object proxies.
+
+=== Object Proxies
+
+Any time that a Java object must be made available to the Python API, it is
made available via a proxy object. This means that in order to access a Java
+object from Python, we need to simply call the appropriate method on the
Python proxy. When this method is called, a message is generated by the Python
+object and sent over the socket. That message is essentially an encoding of
"Invoke method ABC on object XYZ, using arguments W, X, and Y."
+
+For example, if we have an `InputFlowFile` object named `flowFile` and we want
the `filename` attribute, we can do so by calling:
+
+filename = flowFile.getAttribute('filename')
+
+
+From the Python API perspective, this is all that is necessary. Behind the
scenes, a message is written to the local socket that is an encoding of the
+message "Invoke the getAttribute method on the object with ID 679212, with
String argument 'filename'".
+The Java process then receives this command, invokes the specified method on
the object with the given identifier, and writes back to the socket the result
+of that method invocation. As a result, the Python side receives the value of
the "filename" attribute.
+
+This is important to understand, because it means that any method invocation
that occurs on a Java object must be serialized, written over the socket,
+deserialized, and then the method can be invoked. The result must then be
serialized, written over the socket, and the result deserialized on the Python
side.
+While this is a fairly efficient process, it is not nearly as efficient as
simply invoking a method natively. As a result, it is important to consider the
overhead of
+method invocations on Java objects.
+
+=== Object age-off
+
+It is also important to understand that any time that an object is provided as
an argument to a Python Processor, that object can only be accessed on the
Python
+side
[GitHub] [nifi] markap14 commented on pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on PR #7003: URL: https://github.com/apache/nifi/pull/7003#issuecomment-1476971571 @dam4rus regarding the creation of the environment / importing of the libraries - the environment is created before importing the libraries. The Environment creation happens in `PythonProcess.java` in the `setupEnvironment()` method. It then runs `Controller.py` from the `launchPythonProcess` method. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on PR #7003: URL: https://github.com/apache/nifi/pull/7003#issuecomment-1476969983 Thanks @dam4rus for the thorough review. As you might have guessed, I am not a Python expert. A lot of the suggestions that you make probably make a lot of sense. But they are largely not something that we want to do in this PR. Specifically, the idea of this PR is to make something available so that others who are more familiar with Python can start to iterate on it, and users can begin to experiment with the API. I aimed to make it very clear that this is not production ready through the documentation, etc. But we don't want to iterate on all the minor findings that occur before introducing this. By bringing it into the 2.0 codebase early I'm trying to get others who are more experienced in the Python side excited and iterate quickly to get to a production-ready capability. So I think most of your comments make sense but should be done in subsequent follow-on PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003: URL: https://github.com/apache/nifi/pull/7003#discussion_r1142698350 ## nifi-nar-bundles/nifi-py4j-bundle/nifi-python-extension-api/src/main/python/src/nifiapi/properties.py: ## @@ -0,0 +1,411 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from enum import Enum +from nifiapi.__jvm__ import JvmHolder +import re + +class ExpressionLanguageScope(Enum): +NONE = 1 +LIMITED = 2 +FLOWFILE_ATTRIBUTES = 3 + + +class StandardValidators: +__standard_validators__ = JvmHolder.jvm.org.apache.nifi.processor.util.StandardValidators + +ALWAYS_VALID = JvmHolder.jvm.org.apache.nifi.components.Validator.VALID +NON_EMPTY_VALIDATOR = __standard_validators__.NON_EMPTY_VALIDATOR +INTEGER_VALIDATOR = __standard_validators__.INTEGER_VALIDATOR +POSITIVE_INTEGER_VALIDATOR = __standard_validators__.POSITIVE_INTEGER_VALIDATOR +POSITIVE_LONG_VALIDATOR = __standard_validators__.POSITIVE_LONG_VALIDATOR +NON_NEGATIVE_INTEGER_VALIDATOR = __standard_validators__.NON_NEGATIVE_INTEGER_VALIDATOR +NUMBER_VALIDATOR = __standard_validators__.NUMBER_VALIDATOR +LONG_VALIDATOR = __standard_validators__.LONG_VALIDATOR +PORT_VALIDATOR = __standard_validators__.PORT_VALIDATOR +NON_EMPTY_EL_VALIDATOR = __standard_validators__.NON_EMPTY_EL_VALIDATOR +HOSTNAME_PORT_LIST_VALIDATOR = __standard_validators__.HOSTNAME_PORT_LIST_VALIDATOR +BOOLEAN_VALIDATOR = __standard_validators__.BOOLEAN_VALIDATOR +URL_VALIDATOR = __standard_validators__.URL_VALIDATOR +URI_VALIDATOR = __standard_validators__.URI_VALIDATOR +REGULAR_EXPRESSION_VALIDATOR = __standard_validators__.REGULAR_EXPRESSION_VALIDATOR +REGULAR_EXPRESSION_WITH_EL_VALIDATOR = __standard_validators__.REGULAR_EXPRESSION_WITH_EL_VALIDATOR +TIME_PERIOD_VALIDATOR = __standard_validators__.TIME_PERIOD_VALIDATOR +DATA_SIZE_VALIDATOR = __standard_validators__.DATA_SIZE_VALIDATOR +FILE_EXISTS_VALIDATOR = __standard_validators__.FILE_EXISTS_VALIDATOR + + + +class PropertyDependency: +def __init__(self, property_descriptor, dependent_values=None): +if dependent_values is None: +dependent_values = [] + +if isinstance(property_descriptor, str): +self.property_name = property_descriptor +else: +self.property_name = property_descriptor.getName() +self.dependentValues = dependent_values + +@staticmethod +def from_java_dependency(java_dependencies): +if java_dependencies is None or len(java_dependencies) == 0: +return None + +dependencies = [] +for dependency in java_dependencies: + dependencies.append(PropertyDependency(dependency.getPropertyName(), dependency.getDependentValues())) + +return dependencies + + +class ResourceDefinition: +def __init__(self, allow_multiple=False, allow_file=True, allow_url=False, allow_directory=False, allow_text=False): +self.allow_multiple = allow_multiple +self.allow_file = allow_file +self.allow_url = allow_url +self.allow_directory = allow_directory +self.allow_text = allow_text + +@staticmethod +def from_java_definition(java_definition): +if java_definition is None: +return None + +allow_multiple = java_definition.getCardinality().name() == "MULTIPLE" +resource_types = java_definition.getResourceTypes() +allow_file = False +allow_url = False +allow_directory = False +allow_text = False +for type in resource_types: +name = type.name() +if name == "FILE": +allow_file = True +elif name == "DIRECTORY": +allow_directory = True +elif name == "TEXT": +allow_text = True +elif name == "URL": +allow_url = True + +return ResourceDefinition(allow_multiple, allow_file, allow_url, allow_directory, allow_text) + + +class PropertyDescriptor: +def __init__(self, name, description, required=False, sensitive=False, + display_name=None, default_value=None, allowable_values=None, +
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1142696544
##
nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/ExtensionManager.py:
##
@@ -0,0 +1,531 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import importlib
+import sys
+import importlib.util # Note requires Python 3.4+
+import inspect
+import logging
+import subprocess
+import ast
+import pkgutil
+from pathlib import Path
+
+logger = logging.getLogger("org.apache.nifi.py4j.ExtensionManager")
+
+# A simple wrapper class to encompass a processor type and its version
+class ExtensionId:
Review Comment:
I don't know what that means... But feel free to submit a follow-on PR. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1142695591
##
nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/Controller.py:
##
@@ -0,0 +1,101 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import logging
+import ExtensionManager
+from py4j.java_gateway import JavaGateway, CallbackServerParameters,
GatewayParameters
+
+import PythonProcessorAdapter
+
+# Initialize logging
+logger = logging.getLogger("org.apache.nifi.py4j.Controller")
+logger.setLevel(logging.INFO)
+
+logging.getLogger("py4j").setLevel(logging.WARN)
+
+logsDir = os.getenv('LOGS_DIR')
+logging.basicConfig(filename=logsDir + '/python.log',
+format='%(asctime)s %(levelname)s %(name)s %(message)s',
+encoding='utf-8',
+level=logging.INFO)
+
+
+class Controller:
+
+def ping(self):
+return "pong"
+
+def getProcessorTypes(self):
+types = self.extensionManager.getProcessorTypes()
+typesList = self.gateway.jvm.java.util.ArrayList()
+for type in types:
+typesList.add(type)
+return typesList
+
+def discoverExtensions(self, dirs, work_dir):
+self.extensionManager.discoverExtensions(dirs, work_dir)
+
+def createProcessor(self, processorType, version, work_dir):
+processorClass =
self.extensionManager.getProcessorClass(processorType, version, work_dir)
+processor = processorClass(jvm=self.gateway.jvm)
+adapter = PythonProcessorAdapter.PythonProcessorAdapter(self.gateway,
processor, self.extensionManager, self.controllerServiceTypeLookup)
+return adapter
+
+def reloadProcessor(self, processorType, version, workDirectory):
+self.extensionManager.reload_processor(processorType, version,
workDirectory)
+
+def getModuleFile(self, processorType, version):
+module_file = self.extensionManager.get_module_file(processorType,
version)
+return module_file
+
+def getProcessorDependencies(self, processorType, version):
+deps =
self.extensionManager.__get_dependencies_for_extension_type__(processorType,
version)
+dependencyList = self.gateway.jvm.java.util.ArrayList()
+for dep in deps:
+dependencyList.add(dep)
+
+return dependencyList
+
+def setGateway(self, gateway):
+self.gateway = gateway
+self.extensionManager = ExtensionManager.ExtensionManager(gateway)
+
+def setControllerServiceTypeLookup(self, typeLookup):
+self.controllerServiceTypeLookup = typeLookup
+
+class Java:
+implements = ["org.apache.nifi.py4j.PythonController"]
+
+
+# Create the Controller
Review Comment:
Yeah, that's a good call. This should never be imported outside of the
context in which it's used as an entry point. But it is a good practice. Will
update this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6995: NIFI-7060 populate NiFi and NiFi Registry properties files using env vars in Docker Containers
exceptionfactory commented on PR #6995: URL: https://github.com/apache/nifi/pull/6995#issuecomment-1476956461 Thanks @ChrisSamo632, sounds good! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1142691974
##
nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/Controller.py:
##
@@ -0,0 +1,101 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import logging
+import ExtensionManager
+from py4j.java_gateway import JavaGateway, CallbackServerParameters,
GatewayParameters
+
+import PythonProcessorAdapter
+
+# Initialize logging
+logger = logging.getLogger("org.apache.nifi.py4j.Controller")
+logger.setLevel(logging.INFO)
+
+logging.getLogger("py4j").setLevel(logging.WARN)
+
+logsDir = os.getenv('LOGS_DIR')
+logging.basicConfig(filename=logsDir + '/python.log',
+format='%(asctime)s %(levelname)s %(name)s %(message)s',
+encoding='utf-8',
+level=logging.INFO)
+
+
+class Controller:
+
+def ping(self):
+return "pong"
+
+def getProcessorTypes(self):
+types = self.extensionManager.getProcessorTypes()
+typesList = self.gateway.jvm.java.util.ArrayList()
+for type in types:
+typesList.add(type)
+return typesList
+
+def discoverExtensions(self, dirs, work_dir):
+self.extensionManager.discoverExtensions(dirs, work_dir)
+
+def createProcessor(self, processorType, version, work_dir):
+processorClass =
self.extensionManager.getProcessorClass(processorType, version, work_dir)
+processor = processorClass(jvm=self.gateway.jvm)
+adapter = PythonProcessorAdapter.PythonProcessorAdapter(self.gateway,
processor, self.extensionManager, self.controllerServiceTypeLookup)
+return adapter
+
+def reloadProcessor(self, processorType, version, workDirectory):
+self.extensionManager.reload_processor(processorType, version,
workDirectory)
+
+def getModuleFile(self, processorType, version):
+module_file = self.extensionManager.get_module_file(processorType,
version)
+return module_file
+
+def getProcessorDependencies(self, processorType, version):
+deps =
self.extensionManager.__get_dependencies_for_extension_type__(processorType,
version)
+dependencyList = self.gateway.jvm.java.util.ArrayList()
+for dep in deps:
+dependencyList.add(dep)
+
+return dependencyList
+
+def setGateway(self, gateway):
+self.gateway = gateway
+self.extensionManager = ExtensionManager.ExtensionManager(gateway)
+
+def setControllerServiceTypeLookup(self, typeLookup):
+self.controllerServiceTypeLookup = typeLookup
+
+class Java:
+implements = ["org.apache.nifi.py4j.PythonController"]
+
+
+# Create the Controller
+controller = Controller()
+
+# Create the Java Gateway for communicating with NiFi Java process
+java_port = int(os.getenv('JAVA_PORT'))
+gateway = JavaGateway(
+ callback_server_parameters=CallbackServerParameters(port=0),
+ gateway_parameters=GatewayParameters(port=java_port, read_timeout=None,
enable_memory_management=True),
Review Comment:
True. But they are important details, so want to remain explicit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003: URL: https://github.com/apache/nifi/pull/7003#discussion_r1142690626 ## nifi-nar-bundles/nifi-py4j-bundle/nifi-py4j-bridge/src/main/java/org/apache/nifi/py4j/PythonProcess.java: ## @@ -0,0 +1,293 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.py4j; + +import org.apache.nifi.py4j.client.JavaObjectBindings; +import org.apache.nifi.py4j.client.NiFiPythonGateway; +import org.apache.nifi.py4j.client.StandardPythonClient; +import org.apache.nifi.py4j.server.NiFiGatewayServer; +import org.apache.nifi.python.ControllerServiceTypeLookup; +import org.apache.nifi.python.PythonController; +import org.apache.nifi.python.PythonProcessConfig; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import py4j.CallbackClient; +import py4j.GatewayServer; + +import javax.net.ServerSocketFactory; +import javax.net.SocketFactory; +import java.io.File; +import java.io.IOException; +import java.nio.file.Files; +import java.util.Collections; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.TimeUnit; + +// TODO / Figure Out for MVP: +// MUST DO: +// - Documentation +// - Admin Guide +// - JavaDocs +// - Developer Guide +// - Explain how communication between Java & Python work. +// - Java is preferred, Python is slower and more expensive b/c of network +// - Different Extension Points (FlowFileTransform, RecordTransform) +// - What the API Looks like, Links to JavaDocs for ProcessContext, etc. +// - Example Code +// - Exposing properties +// - Relationships +// - Controller Services +// - Need to update docs to show the interfaces that are exposed, explain how to get these... +// - Design Doc +// - Setup proper logging on the Python side: https://docs.python.org/2/howto/logging-cookbook.html#using-file-rotation +// - For FlowFileTransform, allow the result to contain either a byte array or a String. If a String, just convert in the parent class. +// - Figure out how to deal with Python Packaging +// - Need to figure out how to deal with additionalDetails.html, docs directory in python project typically? +// - Understand how to deal with versioning +// - Look at performance improvements for Py4J - socket comms appear to be INCREDIBLY slow. +// - Create test that calls Python 1M times. Just returns 'hello'. See how long it takes +// - Create test that calls Python 1M times. Returns .toString() and see how long it takes. +// - Will help to understand if it's the call from Java to Python that's slow, Python to Java, or both. +// - Performance concern for TransformRecord +// - Currently, triggering the transform() method is pretty fast. But then the Result object comes back and we have to call into the Python side to call the getters +//over and over. Need to look into instead serializing the entire response as JSON and sending that back. +// - Also, since this is heavy JSON processing, might want to consider ORJSON or something like that instead of inbuilt JSON parser/generator +// - Test pip install nifi-my-proc, does nifi pick it up? +// - When ran DetectObjectInImage with multiple threads, Python died. Need to figure out why. +// - If Python Process dies, need to create a new process and need to then create all of the Processors that were in that Process and initialize them. +//- Milestone 2 or 3, not Milestone 1. +// - Remove test-pypi usage from ExtensionManager.py +// - Additional Interfaces beyond just FlowFileTransform +// - FlowFileSource +// - Restructure Maven projects +// - Should this all go under Framework? +// +// +// CONSIDER: +// - Clustering: Ensure component on all nodes? +// - Consider "pip freeze" type of thing to ensure that python dependencies are same across nodes when joining cluster. +// - Update python code
[GitHub] [nifi] ChrisSamo632 commented on pull request #6995: NIFI-7060 populate NiFi and NiFi Registry properties files using env vars in Docker Containers
ChrisSamo632 commented on PR #6995: URL: https://github.com/apache/nifi/pull/6995#issuecomment-1476951986 @exceptionfactory I've raised tickets: - [NIFI-11311](https://issues.apache.org/jira/browse/NIFI-11311) to upgrade `maven-antrun-plugin` - [NIFI-11312](https://issues.apache.org/jira/browse/NIFI-11312) to rationalise the `nifi-registry` Docker Images builds - [NIFI-11313](https://issues.apache.org/jira/browse/NIFI-11313) to address the `shellcheck` issues - [NIFI-11314](https://issues.apache.org/jira/browse/NIFI-11314) for loading sensitive properties from local files (referenced by environment variables) - [NIFI-11315](https://issues.apache.org/jira/browse/NIFI-11315) for general environment variable configuration of applications With these, I'll close this PR and (hopefully sometime soon) break out the changes for the first 3 simpler tickets (although they're open for others to do first if they are interested and beat me to them). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] ChrisSamo632 closed pull request #6995: NIFI-7060 populate NiFi and NiFi Registry properties files using env vars in Docker Containers
ChrisSamo632 closed pull request #6995: NIFI-7060 populate NiFi and NiFi Registry properties files using env vars in Docker Containers URL: https://github.com/apache/nifi/pull/6995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1142689413
##
nifi-nar-bundles/nifi-py4j-bundle/nifi-py4j-bridge/src/main/java/org/apache/nifi/py4j/StandardPythonBridge.java:
##
@@ -0,0 +1,315 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.py4j;
+
+import org.apache.nifi.python.BoundObjectCounts;
+import org.apache.nifi.python.ControllerServiceTypeLookup;
+import org.apache.nifi.python.PythonBridge;
+import org.apache.nifi.python.PythonBridgeInitializationContext;
+import org.apache.nifi.python.PythonController;
+import org.apache.nifi.python.PythonProcessConfig;
+import org.apache.nifi.python.PythonProcessorDetails;
+import org.apache.nifi.python.processor.PythonProcessorAdapter;
+import org.apache.nifi.python.processor.PythonProcessorBridge;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class StandardPythonBridge implements PythonBridge {
+private static final Logger logger =
LoggerFactory.getLogger(StandardPythonBridge.class);
+
+private volatile boolean running = false;
+
+private PythonProcessConfig processConfig;
+private ControllerServiceTypeLookup serviceTypeLookup;
+private PythonProcess controllerProcess;
+private final Map processorCountByType = new
ConcurrentHashMap<>();
+private final Map>
processesByProcessorType = new ConcurrentHashMap<>();
+
+
+@Override
+public void initialize(final PythonBridgeInitializationContext context) {
+this.processConfig = context.getPythonProcessConfig();
+this.serviceTypeLookup = context.getControllerServiceTypeLookup();
+}
+
+@Override
+public synchronized void start() throws IOException {
+if (running) {
+logger.debug("{} already started, will not start again", this);
+return;
+}
+
+logger.debug("{} launching Python Process", this);
+
+try {
+final File envHome = new
File(processConfig.getPythonWorkingDirectory(), "controller");
+controllerProcess = new PythonProcess(processConfig,
serviceTypeLookup, envHome);
+controllerProcess.start();
+running = true;
+} catch (final Exception e) {
+shutdown();
+throw e;
+}
+}
+
+
+@Override
+public void discoverExtensions() {
+ensureStarted();
+final List extensionsDirs =
processConfig.getPythonExtensionsDirectories().stream()
+.map(File::getAbsolutePath)
+.collect(Collectors.toList());
+final String workDirPath =
processConfig.getPythonWorkingDirectory().getAbsolutePath();
+controllerProcess.getController().discoverExtensions(extensionsDirs,
workDirPath);
+}
+
+@Override
+public PythonProcessorBridge createProcessor(final String identifier,
final String type, final String version, final boolean preferIsolatedProcess) {
+ensureStarted();
+
+logger.debug("Creating Python Processor of type {}", type);
+
+final PythonProcess pythonProcess = getProcessForNextComponent(type,
version, preferIsolatedProcess);
+final String workDirPath =
processConfig.getPythonWorkingDirectory().getAbsolutePath();
+
+final PythonController controller = pythonProcess.getController();
+final PythonProcessorAdapter processorAdapter =
controller.createProcessor(type, version, workDirPath);
+final PythonProcessorBridge processorBridge = new
StandardPythonProcessorBridge.Builder()
+.controller(controller)
+.processorAdapter(processorAdapter)
+.processorType(type)
+.processorVersion(version)
+.workingDirectory(processConfig.getPythonWorkingDirectory())
+.moduleFile(new File(controller.getModuleFile(type, version)))
+.build();
+
+
[jira] [Created] (NIFI-11315) Enable environment variable driven configuration of applications
Chris Sampson created NIFI-11315: Summary: Enable environment variable driven configuration of applications Key: NIFI-11315 URL: https://issues.apache.org/jira/browse/NIFI-11315 Project: Apache NiFi Issue Type: Improvement Reporter: Chris Sampson During https://github.com/apache/nifi/pull/6995 for NIFI-7060 (abandoned), it was discussed that NiFi (and Registry/MiNiFi) should be extended to allow general configuration via environment variables (not just when run as a Docker Container). The "Sensitive Property Provider" framework may be of use here, although we'd want to be able to specify all/any properties via environment variables not just those marked "sensitive". Consideration will be needed for the order of precedence (e.g. env var vs. nifi.properties vs. property provider, etc.), appropriate documentation and clear property defaults. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11314) Load sensitive properties from files at startup
[ https://issues.apache.org/jira/browse/NIFI-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Sampson updated NIFI-11314: - Description: As part of https://github.com/apache/nifi/pull/6995 for NIFI-7060 (abandoned), the idea of being able to provide sensitive values for properties at startup was introduced, e.g. the password for the NiFi Registry database could be provided in a file on the localhost. This approach should be considered for NiFi components, particularly for use cases where configuration may otherwise be provided via environment variables and/or the nifi.properties file cannot easily be re-configured, e.g. when running as Docker Containers using the convenience Docker Images from Docker Hub. Ability to provide such sensitive properties via file (and reference those files via environment variables) is something found in other well-known applications, e.g. Elasticsearch. Consideration should be made as to whether the "Sensitive Property Providers" approach could be used here (at least for NiFi, although this capability should also be made available for NiFi Registry and MiNiFi where appropriate). was: As part of https://github.com/apache/nifi/pull/6995 for NIFI-7060 (abandoned), the idea of being able to provide sensitive values for properties at startup was introduced, e.g. the password for the NiFi Registry database could be provided in a file on the localhost. This approach should be considered for NiFi components, particularly for use cases where configuration may otherwise be provided via environment variables and/or the nifi.properties file cannot easily be re-configured, e.g. when running as Docker Containers using the convenience Docker Images from Docker Hub. Ability to provide such sensitive properties via file (and reference those files via environment variables) is something found in other well-known applications, e.g. Elasticsearch. > Load sensitive properties from files at startup > --- > > Key: NIFI-11314 > URL: https://issues.apache.org/jira/browse/NIFI-11314 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Chris Sampson >Priority: Major > > As part of https://github.com/apache/nifi/pull/6995 for NIFI-7060 > (abandoned), the idea of being able to provide sensitive values for > properties at startup was introduced, e.g. the password for the NiFi Registry > database could be provided in a file on the localhost. > This approach should be considered for NiFi components, particularly for use > cases where configuration may otherwise be provided via environment variables > and/or the nifi.properties file cannot easily be re-configured, e.g. when > running as Docker Containers using the convenience Docker Images from Docker > Hub. > Ability to provide such sensitive properties via file (and reference those > files via environment variables) is something found in other well-known > applications, e.g. Elasticsearch. > Consideration should be made as to whether the "Sensitive Property Providers" > approach could be used here (at least for NiFi, although this capability > should also be made available for NiFi Registry and MiNiFi where appropriate). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11314) Load sensitive properties from files at startup
Chris Sampson created NIFI-11314: Summary: Load sensitive properties from files at startup Key: NIFI-11314 URL: https://issues.apache.org/jira/browse/NIFI-11314 Project: Apache NiFi Issue Type: Improvement Reporter: Chris Sampson As part of https://github.com/apache/nifi/pull/6995 for NIFI-7060 (abandoned), the idea of being able to provide sensitive values for properties at startup was introduced, e.g. the password for the NiFi Registry database could be provided in a file on the localhost. This approach should be considered for NiFi components, particularly for use cases where configuration may otherwise be provided via environment variables and/or the nifi.properties file cannot easily be re-configured, e.g. when running as Docker Containers using the convenience Docker Images from Docker Hub. Ability to provide such sensitive properties via file (and reference those files via environment variables) is something found in other well-known applications, e.g. Elasticsearch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] Lehel44 commented on a diff in pull request #7051: NIFI-11255 Allowable value for 'Use s3.region Attribute'
Lehel44 commented on code in PR #7051:
URL: https://github.com/apache/nifi/pull/7051#discussion_r1142676010
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java:
##
@@ -198,6 +218,88 @@ protected AmazonS3Client createClient(final ProcessContext
context, final AWSCre
return s3;
}
+protected Region resolveRegion(final ProcessContext context, final
Map attributes) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+
+if (ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue)) {
+regionValue = attributes.get(S3_REGION_ATTRIBUTE);
+}
+
+return parseRegionValue(regionValue);
+}
+
+private Region parseRegionValue(String regionValue) {
+if (regionValue == null) {
+throw new ProcessException(format("[%s] was selected as region
source but [%s] attribute does not exist", ATTRIBUTE_DRIVEN_REGION,
S3_REGION_ATTRIBUTE));
+}
+
+try {
+return Region.getRegion(Regions.fromName(regionValue));
+} catch (Exception e) {
+throw new ProcessException(format("The [%s] attribute contains an
invalid region value [%s]", S3_REGION_ATTRIBUTE, regionValue), e);
+}
+}
+
+protected void setClientBasedOnRegionAttribute(final ProcessContext
context, final Map attributes) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+
+if (ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue)) {
+regionValue = attributes.get(S3_REGION_ATTRIBUTE);
+Region region = parseRegionValue(regionValue);
+
+final AWSConfiguration awsConfiguration =
getConfiguration(context, region);
+this.client = awsConfiguration.getClient();
+this.region = awsConfiguration.getRegion();
+}
+}
+
+protected AWSConfiguration getConfiguration(final ProcessContext context,
final Map attributes) {
+final Region region = resolveRegion(context, attributes);
+return super.getConfiguration(context, region);
+}
+
+@Override
+public List verify(final ProcessContext context,
final ComponentLog verificationLogger, final Map attributes) {
+final List results = new ArrayList<>();
+
+try {
+getConfiguration(context, attributes);
+results.add(new ConfigVerificationResult.Builder()
+.outcome(Outcome.SUCCESSFUL)
+.verificationStepName("Create S3 Client")
+.explanation("Successfully created S3 Client")
+.build());
+} catch (final Exception e) {
+verificationLogger.error("Failed to create S3 Client", e);
+results.add(new ConfigVerificationResult.Builder()
+.outcome(Outcome.FAILED)
+.verificationStepName("Create S3 Client")
+.explanation("Failed to crete S3 Client: " +
e.getMessage())
+.build());
+}
+
+return results;
+}
+
+@OnScheduled
+public void onScheduled(final ProcessContext context) {
+if (!isAttributeDrivenRegion(context)) {
+setClientAndRegion(context);
+}
+}
+
+private boolean isAttributeDrivenRegion(final ProcessContext context) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+return ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue);
+}
+
+private static AllowableValue[] getCustomAvailableRegions() {
+final List values = new ArrayList<>();
+Collections.addAll(values,
AbstractAWSCredentialsProviderProcessor.getAvailableRegions());
+values.add(ATTRIBUTE_DRIVEN_REGION);
+return values.toArray(new AllowableValue[0]);
+}
Review Comment:
We can also use Apache Commons ArrayUtils::addAll
```suggestion
private static AllowableValue[] getCustomAvailableRegions() {
AllowableValue[] availableRegions =
AbstractAWSCredentialsProviderProcessor.getAvailableRegions();
AllowableValue[] customRegions = new
AllowableValue[]{ATTRIBUTE_DRIVEN_REGION};
return ArrayUtils.addAll(availableRegions, customRegions);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] Lehel44 commented on a diff in pull request #7051: NIFI-11255 Allowable value for 'Use s3.region Attribute'
Lehel44 commented on code in PR #7051:
URL: https://github.com/apache/nifi/pull/7051#discussion_r1142673497
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java:
##
@@ -198,6 +218,88 @@ protected AmazonS3Client createClient(final ProcessContext
context, final AWSCre
return s3;
}
+protected Region resolveRegion(final ProcessContext context, final
Map attributes) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+
+if (ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue)) {
+regionValue = attributes.get(S3_REGION_ATTRIBUTE);
+}
+
+return parseRegionValue(regionValue);
+}
+
+private Region parseRegionValue(String regionValue) {
+if (regionValue == null) {
+throw new ProcessException(format("[%s] was selected as region
source but [%s] attribute does not exist", ATTRIBUTE_DRIVEN_REGION,
S3_REGION_ATTRIBUTE));
+}
+
+try {
+return Region.getRegion(Regions.fromName(regionValue));
+} catch (Exception e) {
+throw new ProcessException(format("The [%s] attribute contains an
invalid region value [%s]", S3_REGION_ATTRIBUTE, regionValue), e);
+}
+}
+
+protected void setClientBasedOnRegionAttribute(final ProcessContext
context, final Map attributes) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+
+if (ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue)) {
+regionValue = attributes.get(S3_REGION_ATTRIBUTE);
+Region region = parseRegionValue(regionValue);
+
+final AWSConfiguration awsConfiguration =
getConfiguration(context, region);
+this.client = awsConfiguration.getClient();
+this.region = awsConfiguration.getRegion();
+}
+}
+
+protected AWSConfiguration getConfiguration(final ProcessContext context,
final Map attributes) {
+final Region region = resolveRegion(context, attributes);
+return super.getConfiguration(context, region);
+}
+
+@Override
+public List verify(final ProcessContext context,
final ComponentLog verificationLogger, final Map attributes) {
+final List results = new ArrayList<>();
+
+try {
+getConfiguration(context, attributes);
+results.add(new ConfigVerificationResult.Builder()
+.outcome(Outcome.SUCCESSFUL)
+.verificationStepName("Create S3 Client")
+.explanation("Successfully created S3 Client")
+.build());
+} catch (final Exception e) {
+verificationLogger.error("Failed to create S3 Client", e);
+results.add(new ConfigVerificationResult.Builder()
+.outcome(Outcome.FAILED)
+.verificationStepName("Create S3 Client")
+.explanation("Failed to crete S3 Client: " +
e.getMessage())
+.build());
+}
+
+return results;
+}
+
+@OnScheduled
+public void onScheduled(final ProcessContext context) {
+if (!isAttributeDrivenRegion(context)) {
+setClientAndRegion(context);
+}
+}
+
+private boolean isAttributeDrivenRegion(final ProcessContext context) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+return ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue);
+}
+
+private static AllowableValue[] getCustomAvailableRegions() {
+final List values = new ArrayList<>();
+Collections.addAll(values,
AbstractAWSCredentialsProviderProcessor.getAvailableRegions());
+values.add(ATTRIBUTE_DRIVEN_REGION);
+return values.toArray(new AllowableValue[0]);
+}
Review Comment:
```suggestion
private static AllowableValue[] getCustomAvailableRegions() {
List values = new
ArrayList<>(Arrays.asList(AbstractAWSCredentialsProviderProcessor.getAvailableRegions()));
values.add(ATTRIBUTE_DRIVEN_REGION);
return values.toArray(new AllowableValue[0]);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] Lehel44 commented on a diff in pull request #7051: NIFI-11255 Allowable value for 'Use s3.region Attribute'
Lehel44 commented on code in PR #7051:
URL: https://github.com/apache/nifi/pull/7051#discussion_r1142672636
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java:
##
@@ -198,6 +218,88 @@ protected AmazonS3Client createClient(final ProcessContext
context, final AWSCre
return s3;
}
+protected Region resolveRegion(final ProcessContext context, final
Map attributes) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+
+if (ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue)) {
+regionValue = attributes.get(S3_REGION_ATTRIBUTE);
+}
+
+return parseRegionValue(regionValue);
+}
+
+private Region parseRegionValue(String regionValue) {
+if (regionValue == null) {
+throw new ProcessException(format("[%s] was selected as region
source but [%s] attribute does not exist", ATTRIBUTE_DRIVEN_REGION,
S3_REGION_ATTRIBUTE));
+}
+
+try {
+return Region.getRegion(Regions.fromName(regionValue));
+} catch (Exception e) {
+throw new ProcessException(format("The [%s] attribute contains an
invalid region value [%s]", S3_REGION_ATTRIBUTE, regionValue), e);
+}
+}
+
+protected void setClientBasedOnRegionAttribute(final ProcessContext
context, final Map attributes) {
Review Comment:
I recommend renaming this method to setClientAndRegion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11313) Docker Image start scripts contain shellcheck errors/warnings
Chris Sampson created NIFI-11313: Summary: Docker Image start scripts contain shellcheck errors/warnings Key: NIFI-11313 URL: https://issues.apache.org/jira/browse/NIFI-11313 Project: Apache NiFi Issue Type: Bug Reporter: Chris Sampson The startup scrpts used by the Docker Images (NiFi, Registry, MiNiFi) contain numerous [shellcheck|https://www.shellcheck.net/] errors/warnings that should be addressed to ensure consistency across environments and reduced risk of bugs. See https://github.com/apache/nifi/pull/6995 (for NIFI-7060) for a more complete list of known issues that should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003: URL: https://github.com/apache/nifi/pull/7003#discussion_r1142669010 ## nifi-docs/src/main/asciidoc/administration-guide.adoc: ## @@ -231,6 +232,38 @@ The `name` attribute must start with `deprecation`, followed by the component cl +[[python_configuration]] +== Python Configuration + +NiFi if a Java-based application. However, in the 2.0 release, support was introduces for a Python-based Processor API. This capability is still +considered to be in "Beta" mode and should not be used in production. By default, support for Python-based Processors is disabled. In order to enable it, +Python 3.9+ must be installed on the NiFi node. + +The following properties may be used to configure the Python 3 installation and process management. These properties are all located under the +"Python Extensions" heading in the _nifi.properties_ file: + +[options="header,footer"] +|== +| Property Name | Default Value | Description +| nifi.python.command | python3 | The command used to launch Python. By default, this property is set to "python3" but commented out. In order to enable Python-based Processors, +uncomment this line and set it to the command that should be used to invoke Python 3. +| nifi.python.framework.source.directory | ./python/framework | The directory that contains the Python framework for communicating between the Python and Java processes. +| nifi.python.extensions.source.directory.default | ./python/extensions | The directory that NiFi should look in to find Python-based Processors. Note that this property is supplied Review Comment: The API directory is expected to be a sibling of the framework directory. Will note this in the documentation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11312) NiFi Registry Docker Image builds are inconsistent (dockermaven vs. dockerhub)
Chris Sampson created NIFI-11312:
Summary: NiFi Registry Docker Image builds are inconsistent
(dockermaven vs. dockerhub)
Key: NIFI-11312
URL: https://issues.apache.org/jira/browse/NIFI-11312
Project: Apache NiFi
Issue Type: Bug
Reporter: Chris Sampson
The {{dockermaven}} and {{dockerhub}} image builds are not consistent, e.g.
then have different startup scripts copied into the images (some of which
contain issues like incorrect properties being used).
See https://github.com/apache/nifi/pull/6995 (for NIFI-7060) for a more
complete list of known inconsistences that should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11311) Upgrade maven-antrun-plugin version
Chris Sampson created NIFI-11311:
Summary: Upgrade maven-antrun-plugin version
Key: NIFI-11311
URL: https://issues.apache.org/jira/browse/NIFI-11311
Project: Apache NiFi
Issue Type: Task
Reporter: Chris Sampson
The {{maven-antrun-plugin}} is used in several module builds (e.g. to copy
files for a Docker Image build), but the version is inconsistent across modules
and is very old in some cases.
Current latest is {{3.1.0}}, we should look to upgrade to that across all
modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] Lehel44 commented on a diff in pull request #6985: NIFI-11215: Add custom validation for KerberosUserService in PutIceberg
Lehel44 commented on code in PR #6985:
URL: https://github.com/apache/nifi/pull/6985#discussion_r1142662090
##
nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-processors/src/main/java/org/apache/nifi/processors/iceberg/PutIceberg.java:
##
@@ -148,6 +153,28 @@ public Set getRelationships() {
return RELATIONSHIPS;
}
+@Override
+protected Collection customValidate(ValidationContext
context) {
+final List problems = new ArrayList<>();
+final IcebergCatalogService catalogService =
context.getProperty(CATALOG).asControllerService(IcebergCatalogService.class);
+boolean catalogServiceEnabled =
context.getControllerServiceLookup().isControllerServiceEnabled(catalogService);
+
+if (catalogServiceEnabled) {
+final boolean kerberosUserServiceIsSet =
context.getProperty(KERBEROS_USER_SERVICE).isSet();
+final boolean securityEnabled =
SecurityUtil.isSecurityEnabled(catalogService.getConfiguration());
+
+if (securityEnabled && !kerberosUserServiceIsSet) {
+problems.add(new ValidationResult.Builder()
+.subject(KERBEROS_USER_SERVICE.getDisplayName())
+.valid(false)
+.explanation("Security authentication is set to
'kerberos' in the configuration files but no KerberosUserService is
configured.")
+.build());
+}
+}
Review Comment:
Optional: I recommend combining the if statements for better readability.
```suggestion
if (catalogServiceEnabled &&
SecurityUtil.isSecurityEnabled(catalogService.getConfiguration())) {
boolean kerberosUserServiceIsSet =
context.getProperty(KERBEROS_USER_SERVICE).isSet();
if (!kerberosUserServiceIsSet) {
problems.add(new ValidationResult.Builder()
.subject(KERBEROS_USER_SERVICE.getDisplayName())
.valid(false)
.explanation("Security authentication is set to
'kerberos' in the configuration files but no KerberosUserService is
configured.")
.build());
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] ChrisSamo632 commented on pull request #6903: NIFI-11111 add option to output Elasticsearch error responses as FlowFile to PutElasticsearchJson and PutElasticsearchRecord
ChrisSamo632 commented on PR #6903: URL: https://github.com/apache/nifi/pull/6903#issuecomment-1476912051 @mattyb149 I believe I addressed your comments @MikeThomsen do the updates help with clarity around the transfer of FlowFiles/assignment of attributes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #7057: NIFI-11302: add Registry version to About dialog fixing bug for Java 11
exceptionfactory commented on PR #7057: URL: https://github.com/apache/nifi/pull/7057#issuecomment-1476906113 Thanks for the reply @markobean. I recommend keeping the existing approach since uses standard Java package manifests as opposed to loading properties. There are some options for where to make the changes, but after some testing, adjusting the Maven JAR plugin configuration for `nifi-registry-data-model` seems like the best location. It did require moving the the `RegistryAbout` model class to a subpackage to avoid confusion with the standard `org.apache.nifi.registry` package, but the following commit preserves the same basic approach using `getPackage().getImplementationVersion()`. https://github.com/exceptionfactory/nifi/commit/6c4386ec654eeff8e11af8b40db946cac8cd9c8f -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] Lehel44 commented on a diff in pull request #7051: NIFI-11255 Allowable value for 'Use s3.region Attribute'
Lehel44 commented on code in PR #7051:
URL: https://github.com/apache/nifi/pull/7051#discussion_r1142586898
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java:
##
@@ -150,6 +160,16 @@ public abstract class AbstractS3Processor extends
AbstractAWSCredentialsProvider
.dependsOn(SIGNER_OVERRIDE, CUSTOM_SIGNER)
.build();
+public static final String S3_REGION_ATTRIBUTE = "s3.region" ;
+static final AllowableValue ATTRIBUTE_DRIVEN_REGION = new
AllowableValue("Use '" + S3_REGION_ATTRIBUTE + "' Attribute",
Review Comment:
What do you think of ATTRIBUTE_DEFINED_REGION? For the value the standard
seems to be lowercase with dashes, so it would be
"attribute-driven/defined-region". The description could be a bit simpler like
"Uses the S3 Region Attribute as region"
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java:
##
@@ -198,6 +218,88 @@ protected AmazonS3Client createClient(final ProcessContext
context, final AWSCre
return s3;
}
+protected Region resolveRegion(final ProcessContext context, final
Map attributes) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+
+if (ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue)) {
+regionValue = attributes.get(S3_REGION_ATTRIBUTE);
+}
+
+return parseRegionValue(regionValue);
+}
+
+private Region parseRegionValue(String regionValue) {
+if (regionValue == null) {
+throw new ProcessException(format("[%s] was selected as region
source but [%s] attribute does not exist", ATTRIBUTE_DRIVEN_REGION,
S3_REGION_ATTRIBUTE));
+}
+
+try {
+return Region.getRegion(Regions.fromName(regionValue));
+} catch (Exception e) {
+throw new ProcessException(format("The [%s] attribute contains an
invalid region value [%s]", S3_REGION_ATTRIBUTE, regionValue), e);
+}
+}
+
+protected void setClientBasedOnRegionAttribute(final ProcessContext
context, final Map attributes) {
+String regionValue = context.getProperty(S3_CUSTOM_REGION).getValue();
+
+if (ATTRIBUTE_DRIVEN_REGION.getValue().equals(regionValue)) {
+regionValue = attributes.get(S3_REGION_ATTRIBUTE);
+Region region = parseRegionValue(regionValue);
Review Comment:
I think resolveRegion should be called here.
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/s3/AbstractS3Processor.java:
##
@@ -150,6 +160,16 @@ public abstract class AbstractS3Processor extends
AbstractAWSCredentialsProvider
.dependsOn(SIGNER_OVERRIDE, CUSTOM_SIGNER)
.build();
+public static final String S3_REGION_ATTRIBUTE = "s3.region" ;
+static final AllowableValue ATTRIBUTE_DRIVEN_REGION = new
AllowableValue("Use '" + S3_REGION_ATTRIBUTE + "' Attribute",
+"Use '" + S3_REGION_ATTRIBUTE + "' Attribute",
+"Inspects the '" + S3_REGION_ATTRIBUTE + "' FlowFile attribute to
determine the used S3 region.");
+
+public static final PropertyDescriptor S3_CUSTOM_REGION = new
PropertyDescriptor.Builder()
Review Comment:
Should we call this 'custom' region? There is one option to define a custom
region but overall it includes the regular regions from aws.
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java:
##
@@ -312,21 +312,9 @@ public void onTrigger(final ProcessContext context, final
ProcessSessionFactory
*/
public abstract void onTrigger(final ProcessContext context, final
ProcessSession session) throws ProcessException;
-protected Region getRegionAndInitializeEndpoint(final ProcessContext
context, final AmazonWebServiceClient client) {
-final Region region;
-// if the processor supports REGION, get the configured region.
-if (getSupportedPropertyDescriptors().contains(REGION)) {
-final String regionValue = context.getProperty(REGION).getValue();
-if (regionValue != null) {
-region = Region.getRegion(Regions.fromName(regionValue));
-if (client != null) {
-client.setRegion(region);
-}
-} else {
-region = null;
-}
-} else {
-region = null;
+protected void initializeEndpoint(final Region region, final
ProcessContext context, final AmazonWebServiceClient client) {
Review Comment:
I think the Region parameter is not needed, you can get it in the method by
calling getRegion(context).
--
This is an automated message from the Apache Git Service.
To respond
[jira] [Updated] (NIFI-11305) CaptureChangeMySQL does not stop if the queue is not empty
[ https://issues.apache.org/jira/browse/NIFI-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-11305: -- Fix Version/s: 2.0.0 1.21.0 (was: 1.latest) (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > CaptureChangeMySQL does not stop if the queue is not empty > -- > > Key: NIFI-11305 > URL: https://issues.apache.org/jira/browse/NIFI-11305 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 2.0.0, 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > There is a logic bug in the handling of stopping the CaptureChangeMySQL > processor. This causes it to not stop while there are events in the queue. If > the processor isn't running fast enough to drain the queue, stopping the > processor will have no effect. > The logic was being handled in OnStopped, but that won't get called until the > onTrigger has finished. Instead the loop should be checking to see if the > processor is still scheduled using isScheduled(), and if not should break out > of the loop and finish the onTrigger processing, thereby allowing the > OnStopped logic to be executed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11305) CaptureChangeMySQL does not stop if the queue is not empty
[ https://issues.apache.org/jira/browse/NIFI-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702918#comment-17702918 ] ASF subversion and git services commented on NIFI-11305: Commit b554102f4aa5f3cd40ba505ad233d4032b4accff in nifi's branch refs/heads/support/nifi-1.x from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b554102f4a ] NIFI-11305: Fix logic to correctly stop CaptureChangeMySQL (#7059) > CaptureChangeMySQL does not stop if the queue is not empty > -- > > Key: NIFI-11305 > URL: https://issues.apache.org/jira/browse/NIFI-11305 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > There is a logic bug in the handling of stopping the CaptureChangeMySQL > processor. This causes it to not stop while there are events in the queue. If > the processor isn't running fast enough to drain the queue, stopping the > processor will have no effect. > The logic was being handled in OnStopped, but that won't get called until the > onTrigger has finished. Instead the loop should be checking to see if the > processor is still scheduled using isScheduled(), and if not should break out > of the loop and finish the onTrigger processing, thereby allowing the > OnStopped logic to be executed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 commented on pull request #7059: NIFI-11305: Fix logic to correctly stop CaptureChangeMySQL
markap14 commented on PR #7059: URL: https://github.com/apache/nifi/pull/7059#issuecomment-1476859059 Thanks for the fix @mattyb149. Looks good to me. +1 merged to main, and support/nifi-1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11305) CaptureChangeMySQL does not stop if the queue is not empty
[ https://issues.apache.org/jira/browse/NIFI-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702917#comment-17702917 ] ASF subversion and git services commented on NIFI-11305: Commit 0c7de2c7482f18507877750ea00f9e9b9134fac7 in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0c7de2c748 ] NIFI-11305: Fix logic to correctly stop CaptureChangeMySQL (#7059) > CaptureChangeMySQL does not stop if the queue is not empty > -- > > Key: NIFI-11305 > URL: https://issues.apache.org/jira/browse/NIFI-11305 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > There is a logic bug in the handling of stopping the CaptureChangeMySQL > processor. This causes it to not stop while there are events in the queue. If > the processor isn't running fast enough to drain the queue, stopping the > processor will have no effect. > The logic was being handled in OnStopped, but that won't get called until the > onTrigger has finished. Instead the loop should be checking to see if the > processor is still scheduled using isScheduled(), and if not should break out > of the loop and finish the onTrigger processing, thereby allowing the > OnStopped logic to be executed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 merged pull request #7059: NIFI-11305: Fix logic to correctly stop CaptureChangeMySQL
markap14 merged PR #7059: URL: https://github.com/apache/nifi/pull/7059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #7061: NIFI-11310: Provide appropriate classpath resources to the ReloadComp…
mattyb149 commented on PR #7061: URL: https://github.com/apache/nifi/pull/7061#issuecomment-1476849783 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11310) When Processor terminated, any resources that modify the classpath are not reloaded
[
https://issues.apache.org/jira/browse/NIFI-11310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-11310:
--
Status: Patch Available (was: Open)
> When Processor terminated, any resources that modify the classpath are not
> reloaded
> ---
>
> Key: NIFI-11310
> URL: https://issues.apache.org/jira/browse/NIFI-11310
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Processors often provide Property Descriptors that can be used to modify the
> classpath. For example, processors may expose a property for specifying .jar
> files to load, such as JDBC drivers.
> When a Processor is terminated, though, and restarted, the Processor no
> longer has access to those additional classpath resources. The processor must
> have its properties modified in some way in order to re-establish those
> classpath resources. Otherwise, we see errors such as
> {{ClassNotFoundException}} being thrown
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] markap14 opened a new pull request, #7061: NIFI-11310: Provide appropriate classpath resources to the ReloadComp…
markap14 opened a new pull request, #7061: URL: https://github.com/apache/nifi/pull/7061 …onent when a processor is terminated # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11310) When Processor terminated, any resources that modify the classpath are not reloaded
[
https://issues.apache.org/jira/browse/NIFI-11310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-11310:
--
Fix Version/s: 1.latest
2.latest
> When Processor terminated, any resources that modify the classpath are not
> reloaded
> ---
>
> Key: NIFI-11310
> URL: https://issues.apache.org/jira/browse/NIFI-11310
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.latest, 2.latest
>
>
> Processors often provide Property Descriptors that can be used to modify the
> classpath. For example, processors may expose a property for specifying .jar
> files to load, such as JDBC drivers.
> When a Processor is terminated, though, and restarted, the Processor no
> longer has access to those additional classpath resources. The processor must
> have its properties modified in some way in order to re-establish those
> classpath resources. Otherwise, we see errors such as
> {{ClassNotFoundException}} being thrown
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11310) When Processor terminated, any resources that modify the classpath are not reloaded
Mark Payne created NIFI-11310:
-
Summary: When Processor terminated, any resources that modify the
classpath are not reloaded
Key: NIFI-11310
URL: https://issues.apache.org/jira/browse/NIFI-11310
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Processors often provide Property Descriptors that can be used to modify the
classpath. For example, processors may expose a property for specifying .jar
files to load, such as JDBC drivers.
When a Processor is terminated, though, and restarted, the Processor no longer
has access to those additional classpath resources. The processor must have its
properties modified in some way in order to re-establish those classpath
resources. Otherwise, we see errors such as {{ClassNotFoundException}} being
thrown
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-7123) onPropertyModified() is called on nifi start up even when properties have never been modified
[
https://issues.apache.org/jira/browse/NIFI-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702906#comment-17702906
]
Mark Payne commented on NIFI-7123:
--
[~Nissim Shiman] I believe this has been encountered before. But we did not
change the way that it works, because we didn't want to break backward
compatibility. Often, the logic assumes that the {{onPropertyModified}} will be
called on startup. So, typically, the pattern that is followed is something
like this:
{code:java}
private volatile configurationRestored = false;
@OnConfigurationRestored
public void onConfigurationRestored() {
this.configurationRestored = true;
}
public void onPropertyModified(PropertyDescriptor descriptor, String oldValue,
String newValue) {
if (!configurationRestored) {
return;
}
}{code}
So effectively, ignore the call to {{onPropertyModified}} until the
configuration has been restored on startup.
> onPropertyModified() is called on nifi start up even when properties have
> never been modified
> -
>
> Key: NIFI-7123
> URL: https://issues.apache.org/jira/browse/NIFI-7123
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Nissim Shiman
>Priority: Major
>
> Processors and Controller Services inherit the onPropertyModified() method
> from ConfigurableComponent. java [1]
> This method is called when nifi starts for all processors and controller
> services, even for properties that are set to their defaults (i.e. have never
> been modified).
> [1]
> https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/components/ConfigurableComponent.java#L68
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11204) Add configurable retry logic for table commits in PutIceberg processor
[ https://issues.apache.org/jira/browse/NIFI-11204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-11204: --- Fix Version/s: 2.0.0 1.21.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add configurable retry logic for table commits in PutIceberg processor > -- > > Key: NIFI-11204 > URL: https://issues.apache.org/jira/browse/NIFI-11204 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 2.0.0, 1.21.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Iceberg provides commit retry configuration at table level , but it is not > directly configurable at processor level. To provide the possibility to adapt > the retry behaviour to the actual usage of the processor a property based > configuration is needed above the Iceberg table logic so it won’t affect > other components interaction with the table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11204) Add configurable retry logic for table commits in PutIceberg processor
[ https://issues.apache.org/jira/browse/NIFI-11204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702885#comment-17702885 ] ASF subversion and git services commented on NIFI-11204: Commit 835c95fca0d4d4aff69096e9683f73a05dc4c196 in nifi's branch refs/heads/support/nifi-1.x from Mark Bathori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=835c95fca0 ] NIFI-11204: Add configurable retry logic for table commits in PutIceberg processor This closes #6976. Signed-off-by: Peter Turcsanyi > Add configurable retry logic for table commits in PutIceberg processor > -- > > Key: NIFI-11204 > URL: https://issues.apache.org/jira/browse/NIFI-11204 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > Iceberg provides commit retry configuration at table level , but it is not > directly configurable at processor level. To provide the possibility to adapt > the retry behaviour to the actual usage of the processor a property based > configuration is needed above the Iceberg table logic so it won’t affect > other components interaction with the table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-7123) onPropertyModified() is called on nifi start up even when properties have never been modified
[ https://issues.apache.org/jira/browse/NIFI-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nissim Shiman reassigned NIFI-7123: --- Assignee: (was: Nissim Shiman) > onPropertyModified() is called on nifi start up even when properties have > never been modified > - > > Key: NIFI-7123 > URL: https://issues.apache.org/jira/browse/NIFI-7123 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.11.0 >Reporter: Nissim Shiman >Priority: Major > > Processors and Controller Services inherit the onPropertyModified() method > from ConfigurableComponent. java [1] > This method is called when nifi starts for all processors and controller > services, even for properties that are set to their defaults (i.e. have never > been modified). > [1] > https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/components/ConfigurableComponent.java#L68 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11204) Add configurable retry logic for table commits in PutIceberg processor
[ https://issues.apache.org/jira/browse/NIFI-11204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702882#comment-17702882 ] ASF subversion and git services commented on NIFI-11204: Commit e370292d7f3c295a1810532e9349a11b4104e82b in nifi's branch refs/heads/main from Mark Bathori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e370292d7f ] NIFI-11204: Add configurable retry logic for table commits in PutIceberg processor This closes #6976. Signed-off-by: Peter Turcsanyi > Add configurable retry logic for table commits in PutIceberg processor > -- > > Key: NIFI-11204 > URL: https://issues.apache.org/jira/browse/NIFI-11204 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Iceberg provides commit retry configuration at table level , but it is not > directly configurable at processor level. To provide the possibility to adapt > the retry behaviour to the actual usage of the processor a property based > configuration is needed above the Iceberg table logic so it won’t affect > other components interaction with the table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #6976: NIFI-11204: Add configurable retry logic for table commits in PutIceberg processor
asfgit closed pull request #6976: NIFI-11204: Add configurable retry logic for table commits in PutIceberg processor URL: https://github.com/apache/nifi/pull/6976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mark-bathori commented on a diff in pull request #6976: NIFI-11204: Add configurable retry logic for table commits in PutIceberg processor
mark-bathori commented on code in PR #6976:
URL: https://github.com/apache/nifi/pull/6976#discussion_r1142508295
##
nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-processors/src/main/java/org/apache/nifi/processors/iceberg/PutIceberg.java:
##
@@ -113,6 +116,42 @@ public class PutIceberg extends AbstractIcebergProcessor {
.addValidator(StandardValidators.LONG_VALIDATOR)
.build();
+static final PropertyDescriptor NUMBER_OF_COMMIT_RETRIES = new
PropertyDescriptor.Builder()
+.name("number-of-commit-retries")
+.displayName("Number of Commit Retries")
+.description("Number of times to retry a commit before failing.")
+.required(true)
+.defaultValue("3")
+.addValidator(StandardValidators.INTEGER_VALIDATOR)
+.build();
+
+static final PropertyDescriptor MINIMUM_COMMIT_WAIT_TIME = new
PropertyDescriptor.Builder()
+.name("minimum-commit-wait-time")
+.displayName("Minimum Commit Wait Time")
+.description("Minimum time to wait before retrying a commit.")
+.required(true)
+.defaultValue("100 ms")
+.addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+.build();
+
+static final PropertyDescriptor MAXIMUM_COMMIT_WAIT_TIME = new
PropertyDescriptor.Builder()
+.name("maximum-commit-wait-time")
+.displayName("Maximum Commit Wait Time")
+.description("Maximum time to wait before retrying a commit.")
+.required(true)
+.defaultValue("1 min")
+.addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+.build();
+
+static final PropertyDescriptor MAXIMUM_COMMIT_DURATION = new
PropertyDescriptor.Builder()
+.name("maximum-commit-duration")
+.displayName("Maximum Commit Duration")
+.description("Total retry timeout period for a commit.")
+.required(true)
+.defaultValue("30 min")
+.addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+.build();
Review Comment:
Thanks @turcsanyip for the review. I've modified the defaults to your
recommended values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11309) Add connection details for a cluster to the context menu of a connection
Mark Bean created NIFI-11309: Summary: Add connection details for a cluster to the context menu of a connection Key: NIFI-11309 URL: https://issues.apache.org/jira/browse/NIFI-11309 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.19.1 Reporter: Mark Bean In a cluster, the node-specific details of a connection can be observed by choosing Global Menu > Summary > Connections > View Connection Details (on the right-hand side). This information should be made available directly from a given connection on the graph. Add a new item to the context menu available when right-clicking on a connections. Aside: there are two separate "View Connection Details" options on the Global Menu > Summary > Connections line items. This is confusing and one or both should be renamed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702845#comment-17702845 ] David Handermann commented on NIFI-11167: - The issue with different size records sounds similar to what could happen with the CSV Reader. For a configured Record Schema, the row in question either has the fields or it does not, so it is fine if all of the field values are null. Inferring the Record Schema from an Excel Reader might be more challenging, but should be able to follow the same approach as the CSV Reader. > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11266) PutGoogleDrive, ListGoogleDrive, FetchGoogleDrive can't access a SharedDrive
[ https://issues.apache.org/jira/browse/NIFI-11266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsihovszki Krisztina updated NIFI-11266: Status: Patch Available (was: In Progress) > PutGoogleDrive, ListGoogleDrive, FetchGoogleDrive can't access a SharedDrive > > > Key: NIFI-11266 > URL: https://issues.apache.org/jira/browse/NIFI-11266 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Baptiste Moisson >Assignee: Zsihovszki Krisztina >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > It seems that Google Drive Put, Fetch and List processor are not able to > perform action on SharedDrive. > Regarding the Google Drive API > ([https://developers.google.com/drive/api/v3/reference/files/list?apix_params=%7B%22includeTeamDriveItems%22%3Afalse%2C%22supportsTeamDrives%22%3Afalse%7D)] > , it seems that options like corpora, driveId , includeItemsFromAllDrives, > supportsAllDrives should be use to perform an SharedDrive access. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (NIFI-11213) Changes in older versioned flows are not recognized when they are contained by other flow
[ https://issues.apache.org/jira/browse/NIFI-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Bence closed NIFI-11213. -- > Changes in older versioned flows are not recognized when they are contained > by other flow > - > > Key: NIFI-11213 > URL: https://issues.apache.org/jira/browse/NIFI-11213 > Project: Apache NiFi > Issue Type: Bug >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 2.0.0, 1.21.0 > > Attachments: CFM-3234-TC1-INNER-version-1.json, > CFM-3234-TC1-OUTER-version-1.json, CFM-3234-TC2-INNER-version-1.json, > CFM-3234-TC2-OUTER-version-1.json > > Time Spent: 50m > Remaining Estimate: 0h > > h3. Preparation steps: > Given process group "Inner" is created and added to version control > Given process group "Outer" is created and added to version control > containing Inner PG as well > h3. Actual scenario: > Add "Outer" to the canvas from Registry > Change something within Inner PG (rename a processor or such) > Commit the change > Expected behaviour: > As "Inner" has the new version now, I would expect NiFi to show local changes > in the Outer PG, namely the version change for Inner. > h3. Actual behaviour: > The actual behaviour depends on what version of NiFi was used for > preparation. If the version contains the Registry refactoring (so 1.18.0+) > and the snapshot is saved using storageLocation, the actual behaviour matches > with the expected. For older NiFis (using URI to locate contained versioned > flows) Outer will not show any changes, both PGs will be "green" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markobean commented on pull request #7057: NIFI-11302: add Registry version to About dialog fixing bug for Java 11
markobean commented on PR #7057: URL: https://github.com/apache/nifi/pull/7057#issuecomment-1476519858 > Thanks for working to correct this problem @markobean! > > The current approach based on [Package.getImplementationVersion()](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Package.html#getImplementationVersion()) does not seem to be inherently problematic on Java 11, but rather reflects the lack of package information in the JAR. > > Did you evaluate whether the existing approach could work with updates to the Maven configuration so that the necessary manifest information is generated? I did not consider updating the manifest information. I made the assumption the missing information was inherently part of default Java 11 packaging and implemented an approach that would be independent of the availability of such information. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11213) Changes in older versioned flows are not recognized when they are contained by other flow
[ https://issues.apache.org/jira/browse/NIFI-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702832#comment-17702832 ] ASF subversion and git services commented on NIFI-11213: Commit 4a6e6751d5478514ab6fe937685b057193365f65 in nifi's branch refs/heads/support/nifi-1.x from simonbence [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4a6e6751d5 ] NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly (#7017) > Changes in older versioned flows are not recognized when they are contained > by other flow > - > > Key: NIFI-11213 > URL: https://issues.apache.org/jira/browse/NIFI-11213 > Project: Apache NiFi > Issue Type: Bug >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 2.0.0, 1.21.0 > > Attachments: CFM-3234-TC1-INNER-version-1.json, > CFM-3234-TC1-OUTER-version-1.json, CFM-3234-TC2-INNER-version-1.json, > CFM-3234-TC2-OUTER-version-1.json > > Time Spent: 50m > Remaining Estimate: 0h > > h3. Preparation steps: > Given process group "Inner" is created and added to version control > Given process group "Outer" is created and added to version control > containing Inner PG as well > h3. Actual scenario: > Add "Outer" to the canvas from Registry > Change something within Inner PG (rename a processor or such) > Commit the change > Expected behaviour: > As "Inner" has the new version now, I would expect NiFi to show local changes > in the Outer PG, namely the version change for Inner. > h3. Actual behaviour: > The actual behaviour depends on what version of NiFi was used for > preparation. If the version contains the Registry refactoring (so 1.18.0+) > and the snapshot is saved using storageLocation, the actual behaviour matches > with the expected. For older NiFis (using URI to locate contained versioned > flows) Outer will not show any changes, both PGs will be "green" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11307) Remove S2S Toolkit Command
[
https://issues.apache.org/jira/browse/NIFI-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11307:
Fix Version/s: 2.latest
Status: Patch Available (was: Open)
> Remove S2S Toolkit Command
> --
>
> Key: NIFI-11307
> URL: https://issues.apache.org/jira/browse/NIFI-11307
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{nifi-toolkit-s2s}} module provides a limited command line tool for
> sending or receiving NiFi Site-to-Site data packets formatted as JSON. The
> JSON format does not stream FlowFile content, which could result in memory
> usage issues. The tool has not been actively maintained, and in light of
> limited uses, it should be removed as part of other deprecation removals for
> NiFi 2.0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7060: NIFI-11307 Remove S2S Toolkit
exceptionfactory opened a new pull request, #7060: URL: https://github.com/apache/nifi/pull/7060 # Summary [NIFI-11307](https://issues.apache.org/jira/browse/NIFI-11307) Removes the NiFi Site-to-Site Toolkit command. The command provides limited support for sending and receiving data packets, serialized as JSON, using the Site-to-Site protocol. The command is not actively maintained, and the JSON format does not stream data, which could result in memory usage issues. The command should be removed for NiFi 2.0. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702828#comment-17702828 ] Daniel Stieglitz commented on NIFI-11167: - [~exceptionfactory] Thanks for the clarification on that. Another I question I had is there any issue of the records from each line returned from the Excel Reader varying in size (i.e. in terms of number of columns and values)? Since I am streaming I really do not know where the largest line with data is hence there is no way for me to know how much to pad each record with null column values. > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11213) Changes in older versioned flows are not recognized when they are contained by other flow
[ https://issues.apache.org/jira/browse/NIFI-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702826#comment-17702826 ] ASF subversion and git services commented on NIFI-11213: Commit 7954ff355cbadefd070a3601a91128e380f5aa87 in nifi's branch refs/heads/main from simonbence [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7954ff355c ] NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly (#7017) > Changes in older versioned flows are not recognized when they are contained > by other flow > - > > Key: NIFI-11213 > URL: https://issues.apache.org/jira/browse/NIFI-11213 > Project: Apache NiFi > Issue Type: Bug >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 2.0.0, 1.21.0 > > Attachments: CFM-3234-TC1-INNER-version-1.json, > CFM-3234-TC1-OUTER-version-1.json, CFM-3234-TC2-INNER-version-1.json, > CFM-3234-TC2-OUTER-version-1.json > > Time Spent: 40m > Remaining Estimate: 0h > > h3. Preparation steps: > Given process group "Inner" is created and added to version control > Given process group "Outer" is created and added to version control > containing Inner PG as well > h3. Actual scenario: > Add "Outer" to the canvas from Registry > Change something within Inner PG (rename a processor or such) > Commit the change > Expected behaviour: > As "Inner" has the new version now, I would expect NiFi to show local changes > in the Outer PG, namely the version change for Inner. > h3. Actual behaviour: > The actual behaviour depends on what version of NiFi was used for > preparation. If the version contains the Registry refactoring (so 1.18.0+) > and the snapshot is saved using storageLocation, the actual behaviour matches > with the expected. For older NiFis (using URI to locate contained versioned > flows) Outer will not show any changes, both PGs will be "green" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11213) Changes in older versioned flows are not recognized when they are contained by other flow
[ https://issues.apache.org/jira/browse/NIFI-11213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-11213: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Changes in older versioned flows are not recognized when they are contained > by other flow > - > > Key: NIFI-11213 > URL: https://issues.apache.org/jira/browse/NIFI-11213 > Project: Apache NiFi > Issue Type: Bug >Reporter: Simon Bence >Assignee: Simon Bence >Priority: Major > Fix For: 2.0.0, 1.21.0 > > Attachments: CFM-3234-TC1-INNER-version-1.json, > CFM-3234-TC1-OUTER-version-1.json, CFM-3234-TC2-INNER-version-1.json, > CFM-3234-TC2-OUTER-version-1.json > > Time Spent: 50m > Remaining Estimate: 0h > > h3. Preparation steps: > Given process group "Inner" is created and added to version control > Given process group "Outer" is created and added to version control > containing Inner PG as well > h3. Actual scenario: > Add "Outer" to the canvas from Registry > Change something within Inner PG (rename a processor or such) > Commit the change > Expected behaviour: > As "Inner" has the new version now, I would expect NiFi to show local changes > in the Outer PG, namely the version change for Inner. > h3. Actual behaviour: > The actual behaviour depends on what version of NiFi was used for > preparation. If the version contains the Registry refactoring (so 1.18.0+) > and the snapshot is saved using storageLocation, the actual behaviour matches > with the expected. For older NiFis (using URI to locate contained versioned > flows) Outer will not show any changes, both PGs will be "green" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] markap14 commented on pull request #7017: NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly
markap14 commented on PR #7017: URL: https://github.com/apache/nifi/pull/7017#issuecomment-1476491227 Thanks @simonbence +1 merged to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #7017: NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly
markap14 merged PR #7017: URL: https://github.com/apache/nifi/pull/7017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11305) CaptureChangeMySQL does not stop if the queue is not empty
[ https://issues.apache.org/jira/browse/NIFI-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11305: Status: Patch Available (was: In Progress) > CaptureChangeMySQL does not stop if the queue is not empty > -- > > Key: NIFI-11305 > URL: https://issues.apache.org/jira/browse/NIFI-11305 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > There is a logic bug in the handling of stopping the CaptureChangeMySQL > processor. This causes it to not stop while there are events in the queue. If > the processor isn't running fast enough to drain the queue, stopping the > processor will have no effect. > The logic was being handled in OnStopped, but that won't get called until the > onTrigger has finished. Instead the loop should be checking to see if the > processor is still scheduled using isScheduled(), and if not should break out > of the loop and finish the onTrigger processing, thereby allowing the > OnStopped logic to be executed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11308) Create an Expression Language function to return NiFi version
Mark Bean created NIFI-11308: Summary: Create an Expression Language function to return NiFi version Key: NIFI-11308 URL: https://issues.apache.org/jira/browse/NIFI-11308 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.19.1 Reporter: Mark Bean Create a new Expression Language function which returns the version of NiFi being run. This allows for programmatic access to version information within the flow. Suggest the function be subjectless as it applies to the instance making the EL call and be called "nifiVersion()" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 opened a new pull request, #7059: NIFI-11305: Fix logic to correctly stop CaptureChangeMySQL
mattyb149 opened a new pull request, #7059: URL: https://github.com/apache/nifi/pull/7059 # Summary [NIFI-11305](https://issues.apache.org/jira/browse/NIFI-11305) This PR removes the unused logic to stop the CaptureChangeMySQL processor when it has been unscheduled, and instead checks to see if it has been unscheduled as a sign to stop the onTrigger binlog event processing loop. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-11305` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-11305` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11305) CaptureChangeMySQL does not stop if the queue is not empty
[ https://issues.apache.org/jira/browse/NIFI-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11305: Fix Version/s: 1.latest 2.latest > CaptureChangeMySQL does not stop if the queue is not empty > -- > > Key: NIFI-11305 > URL: https://issues.apache.org/jira/browse/NIFI-11305 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > > There is a logic bug in the handling of stopping the CaptureChangeMySQL > processor. This causes it to not stop while there are events in the queue. If > the processor isn't running fast enough to drain the queue, stopping the > processor will have no effect. > The logic was being handled in OnStopped, but that won't get called until the > onTrigger has finished. Instead the loop should be checking to see if the > processor is still scheduled using isScheduled(), and if not should break out > of the loop and finish the onTrigger processing, thereby allowing the > OnStopped logic to be executed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11307) Remove S2S Toolkit Command
David Handermann created NIFI-11307:
---
Summary: Remove S2S Toolkit Command
Key: NIFI-11307
URL: https://issues.apache.org/jira/browse/NIFI-11307
Project: Apache NiFi
Issue Type: Improvement
Components: Tools and Build
Reporter: David Handermann
Assignee: David Handermann
The {{nifi-toolkit-s2s}} module provides a limited command line tool for
sending or receiving NiFi Site-to-Site data packets formatted as JSON. The JSON
format does not stream FlowFile content, which could result in memory usage
issues. The tool has not been actively maintained, and in light of limited
uses, it should be removed as part of other deprecation removals for NiFi 2.0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11306) Add Done button to Advanced UI of UpdateAttribute processor
Mark Bean created NIFI-11306: Summary: Add Done button to Advanced UI of UpdateAttribute processor Key: NIFI-11306 URL: https://issues.apache.org/jira/browse/NIFI-11306 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.19.1 Reporter: Mark Bean Once entering the Advanced UI of an UpdateAttribute processor, the only way to exit the dialog is to close it completely returning the user to the main graph. Recommend adding a "Done" button so that the UI returns to the Configure dialog of the UpdateAttribute processor. This is desirable for several reasons not the least of which is to quickly return to the Properties tab to enter additional properties and/or default values for attributes not matching any advanced rules. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mh013370 commented on pull request #7057: NIFI-11302: add Registry version to About dialog fixing bug for Java 11
mh013370 commented on PR #7057: URL: https://github.com/apache/nifi/pull/7057#issuecomment-1476445447 LGTM! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11305) CaptureChangeMySQL does not stop if the queue is not empty
Matt Burgess created NIFI-11305: --- Summary: CaptureChangeMySQL does not stop if the queue is not empty Key: NIFI-11305 URL: https://issues.apache.org/jira/browse/NIFI-11305 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Matt Burgess There is a logic bug in the handling of stopping the CaptureChangeMySQL processor. This causes it to not stop while there are events in the queue. If the processor isn't running fast enough to drain the queue, stopping the processor will have no effect. The logic was being handled in OnStopped, but that won't get called until the onTrigger has finished. Instead the loop should be checking to see if the processor is still scheduled using isScheduled(), and if not should break out of the loop and finish the onTrigger processing, thereby allowing the OnStopped logic to be executed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11305) CaptureChangeMySQL does not stop if the queue is not empty
[ https://issues.apache.org/jira/browse/NIFI-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-11305: --- Assignee: Matt Burgess > CaptureChangeMySQL does not stop if the queue is not empty > -- > > Key: NIFI-11305 > URL: https://issues.apache.org/jira/browse/NIFI-11305 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > There is a logic bug in the handling of stopping the CaptureChangeMySQL > processor. This causes it to not stop while there are events in the queue. If > the processor isn't running fast enough to drain the queue, stopping the > processor will have no effect. > The logic was being handled in OnStopped, but that won't get called until the > onTrigger has finished. Instead the loop should be checking to see if the > processor is still scheduled using isScheduled(), and if not should break out > of the loop and finish the onTrigger processing, thereby allowing the > OnStopped logic to be executed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11304) Add Bulletin indicator for Processors on the Summary page
[ https://issues.apache.org/jira/browse/NIFI-11304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Bean updated NIFI-11304: - Description: It would be useful to have a Bulletin indicator column on the Processors, Remote Process Group and Process Group tabs of the Summary page. This would allow the user to sort on this column to quickly find and navigate to components which have active bulletins. The additional space requirements are minimal, and some current columns could be reduced in size to accommodate the additional column, e.g. Run Status. was: It would be useful to have a Bulletin indicator column on the Processors tab of the Summary page. This would allow the user to sort on this column to quickly find all processors which have active bulletins. The additional space requirements are minimal, and some current columns could be reduced in size to accommodate the additional column, e.g. Run Status. > Add Bulletin indicator for Processors on the Summary page > - > > Key: NIFI-11304 > URL: https://issues.apache.org/jira/browse/NIFI-11304 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.19.1 >Reporter: Mark Bean >Priority: Major > > It would be useful to have a Bulletin indicator column on the Processors, > Remote Process Group and Process Group tabs of the Summary page. This would > allow the user to sort on this column to quickly find and navigate to > components which have active bulletins. The additional space requirements are > minimal, and some current columns could be reduced in size to accommodate the > additional column, e.g. Run Status. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[ https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702798#comment-17702798 ] David Handermann commented on NIFI-11167: - The Format Cell Values property in ConvertExcelToCSV controls the "Read Styles" property on the Workbook StreamingReader, which why naming the property "Read Styles" might be helpful in a new Excel Reader. > Add Excel Record Reader > --- > > Key: NIFI-11167 > URL: https://issues.apache.org/jira/browse/NIFI-11167 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: David Handermann >Assignee: Daniel Stieglitz >Priority: Minor > > A new Excel Record Reader should be implemented to support reading XSLX > spreadsheet rows as NiFi Records. This Reader will enable integration with > various record-oriented components, obviating the need for the narrowly > focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader > should not support the legacy binary XLS format. > The ExcelReader should use a library that supports reading from a stream of > rows to avoid consuming large amounts of heap memory during processing. > The ExcelReader should support configurable properties to read selected > sheets. With Excel supporting typed field values, some amount of field type > mapping will be required. Additional input filtering properties should not be > implemented as existing Processors like QueryRecord support a wide variety of > filtering and projection use cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11304) Add Bulletin indicator for Processors on the Summary page
Mark Bean created NIFI-11304: Summary: Add Bulletin indicator for Processors on the Summary page Key: NIFI-11304 URL: https://issues.apache.org/jira/browse/NIFI-11304 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.19.1 Reporter: Mark Bean It would be useful to have a Bulletin indicator column on the Processors tab of the Summary page. This would allow the user to sort on this column to quickly find all processors which have active bulletins. The additional space requirements are minimal, and some current columns could be reduced in size to accommodate the additional column, e.g. Run Status. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11167) Add Excel Record Reader
[
https://issues.apache.org/jira/browse/NIFI-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702792#comment-17702792
]
Daniel Stieglitz commented on NIFI-11167:
-
[~exceptionfactory] What did you mean by
{quote}
Read Styles for parsing files
{quote} ?
> Add Excel Record Reader
> ---
>
> Key: NIFI-11167
> URL: https://issues.apache.org/jira/browse/NIFI-11167
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: David Handermann
>Assignee: Daniel Stieglitz
>Priority: Minor
>
> A new Excel Record Reader should be implemented to support reading XSLX
> spreadsheet rows as NiFi Records. This Reader will enable integration with
> various record-oriented components, obviating the need for the narrowly
> focused ConvertExcelToCSVProcessor. The initial version of the Excel Reader
> should not support the legacy binary XLS format.
> The ExcelReader should use a library that supports reading from a stream of
> rows to avoid consuming large amounts of heap memory during processing.
> The ExcelReader should support configurable properties to read selected
> sheets. With Excel supporting typed field values, some amount of field type
> mapping will be required. Additional input filtering properties should not be
> implemented as existing Processors like QueryRecord support a wide variety of
> filtering and projection use cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] tpalfy commented on a diff in pull request #6987: NIFI-11137 Add record support to Consume/PublishJMS
tpalfy commented on code in PR #6987:
URL: https://github.com/apache/nifi/pull/6987#discussion_r1142228008
##
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/strategy/consumer/record/RecordUtils.java:
##
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.jms.processors.strategy.consumer.record;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.util.Tuple;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class RecordUtils {
+
+public static Record append(final Record originalRecord, final Map decoratorValues, final String decoratorPrefix) {
Review Comment:
```suggestion
public static Record merge(final Record originalRecord, final
Map decoratorValues, final String decoratorPrefix) {
```
##
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/strategy/consumer/record/RecordWriter.java:
##
@@ -0,0 +1,241 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.jms.processors.strategy.consumer.record;
+
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.jms.processors.strategy.consumer.AttributeSupplier;
+import org.apache.nifi.jms.processors.strategy.consumer.FlowFileWriter;
+import
org.apache.nifi.jms.processors.strategy.consumer.MessageConsumerCallback;
+import org.apache.nifi.jms.processors.strategy.consumer.Serializer;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.SchemaValidationException;
+import org.apache.nifi.serialization.WriteResult;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static
org.apache.nifi.jms.processors.strategy.consumer.record.OutputStrategy.USE_APPENDER;
+import static
org.apache.nifi.jms.processors.strategy.consumer.record.OutputStrategy.USE_VALUE;
+import static
org.apache.nifi.jms.processors.strategy.consumer.record.OutputStrategy.USE_WRAPPER;
+
+public class RecordWriter implements FlowFileWriter {
Review Comment:
I see a discrepancy in the naming between `RecordBasedFlowFileReader` vs
`RecordWriter`.
Maybe this should be `RecordBasedFlowFileWriter`...
[jira] [Created] (NIFI-11303) Create direct go-to option from Provenance lineage
Mark Bean created NIFI-11303: Summary: Create direct go-to option from Provenance lineage Key: NIFI-11303 URL: https://issues.apache.org/jira/browse/NIFI-11303 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.19.1 Reporter: Mark Bean The process to navigate from a Provenance event in the lineage diagram to the component which generated the event is a little cumbersome. It requires right-clicking on the event, choose 'View details', locating the Component Id, returning to the main graph, entering the Component Id in the Search bar and finally selecting the component. It would be useful and more direct to have a "go to" link option available on the context menu when right-clicking on the event. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
exceptionfactory commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1142173520
##
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java:
##
@@ -2061,6 +2074,25 @@ public Path getQuestDbStatusRepositoryPath() {
return
Paths.get(getProperty(STATUS_REPOSITORY_QUESTDB_PERSIST_LOCATION,
DEFAULT_COMPONENT_STATUS_REPOSITORY_PERSIST_LOCATION));
}
+/**
+ * @return the directory in which the Python framework source code is
located
+ */
+public File getPythonFrameworkSourceDirectory() {
Review Comment:
That's a good point. Considering NiFiProperties does not currently use
Optional, leaving this as-is sounds good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003:
URL: https://github.com/apache/nifi/pull/7003#discussion_r1142171807
##
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java:
##
@@ -2061,6 +2074,25 @@ public Path getQuestDbStatusRepositoryPath() {
return
Paths.get(getProperty(STATUS_REPOSITORY_QUESTDB_PERSIST_LOCATION,
DEFAULT_COMPONENT_STATUS_REPOSITORY_PERSIST_LOCATION));
}
+/**
+ * @return the directory in which the Python framework source code is
located
+ */
+public File getPythonFrameworkSourceDirectory() {
Review Comment:
In general, that's a good approach. But I would be very hesitant to update
NiFiProperties to start using Optionals. There are hundreds of methods already,
and they generally do not use Optionals. Introducing it here would be a bad
smell - we should either use Optionals in the class or not. Mixing them would
lead to more confusion, as it would seem as if the lack of an Optional always
implied non-null return values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a diff in pull request #7003: NIFI-11241: Initial implementation of Python-based Processor API with…
markap14 commented on code in PR #7003: URL: https://github.com/apache/nifi/pull/7003#discussion_r1142167979 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/pom.xml: ## @@ -204,6 +209,11 @@ org.xerial.snappy snappy-java + +org.apache.nifi +c2-client-service Review Comment: No, definitely not. Not sure where that came from. Perhaps my IDE was being clever, or (possibly more likely) I messed that up during a rebase at some point will remove. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11301) Fix doc typos in ISPEnrichIP and DistributeLoad
[ https://issues.apache.org/jira/browse/NIFI-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11301: Fix Version/s: 2.0.0 1.21.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Fix doc typos in ISPEnrichIP and DistributeLoad > --- > > Key: NIFI-11301 > URL: https://issues.apache.org/jira/browse/NIFI-11301 > Project: Apache NiFi > Issue Type: Task >Reporter: Dustin Rodrigues >Assignee: Dustin Rodrigues >Priority: Trivial > Fix For: 2.0.0, 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Fix multi-line documentation strings where needed spaces are omitted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11301) Fix doc typos in ISPEnrichIP and DistributeLoad
[ https://issues.apache.org/jira/browse/NIFI-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11301: Priority: Trivial (was: Minor) > Fix doc typos in ISPEnrichIP and DistributeLoad > --- > > Key: NIFI-11301 > URL: https://issues.apache.org/jira/browse/NIFI-11301 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Dustin Rodrigues >Assignee: Dustin Rodrigues >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > Fix multi-line documentation strings where needed spaces are omitted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11301) Fix doc typos in ISPEnrichIP and DistributeLoad
[ https://issues.apache.org/jira/browse/NIFI-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11301: Issue Type: Task (was: Improvement) > Fix doc typos in ISPEnrichIP and DistributeLoad > --- > > Key: NIFI-11301 > URL: https://issues.apache.org/jira/browse/NIFI-11301 > Project: Apache NiFi > Issue Type: Task >Reporter: Dustin Rodrigues >Assignee: Dustin Rodrigues >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > Fix multi-line documentation strings where needed spaces are omitted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11301) Fix doc typos in ISPEnrichIP and DistributeLoad
[ https://issues.apache.org/jira/browse/NIFI-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11301: Summary: Fix doc typos in ISPEnrichIP and DistributeLoad (was: Fix doc typos) > Fix doc typos in ISPEnrichIP and DistributeLoad > --- > > Key: NIFI-11301 > URL: https://issues.apache.org/jira/browse/NIFI-11301 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Dustin Rodrigues >Assignee: Dustin Rodrigues >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Fix multi-line documentation strings where needed spaces are omitted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11300) Upgrade json-smart to 2.4.10
[ https://issues.apache.org/jira/browse/NIFI-11300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11300: Fix Version/s: 2.0.0 1.21.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade json-smart to 2.4.10 > > > Key: NIFI-11300 > URL: https://issues.apache.org/jira/browse/NIFI-11300 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 2.0.0, 1.21.0, 1.latest, 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade json-smart from 2.4.8 to 2.4.10 > https://github.com/netplex/json-smart-v2/releases -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11299) Upgrade SLF4J to 2.0.7
[ https://issues.apache.org/jira/browse/NIFI-11299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11299: Fix Version/s: 2.0.0 1.21.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade SLF4J to 2.0.7 > -- > > Key: NIFI-11299 > URL: https://issues.apache.org/jira/browse/NIFI-11299 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 2.0.0, 1.21.0, 1.latest, 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade SLF4J from 2.0.6 to 2.0.7. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11300) Upgrade json-smart to 2.4.10
[ https://issues.apache.org/jira/browse/NIFI-11300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702726#comment-17702726 ] ASF subversion and git services commented on NIFI-11300: Commit 073d107f6eaeabd585f84002472eb8ed94080382 in nifi's branch refs/heads/support/nifi-1.x from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=073d107f6e ] NIFI-11300 Upgraded json-smart from 2.4.8 to 2.4.10 This closes #7055 Signed-off-by: David Handermann > Upgrade json-smart to 2.4.10 > > > Key: NIFI-11300 > URL: https://issues.apache.org/jira/browse/NIFI-11300 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade json-smart from 2.4.8 to 2.4.10 > https://github.com/netplex/json-smart-v2/releases -- This message was sent by Atlassian Jira (v8.20.10#820010)
