This is an automated email from the ASF dual-hosted git repository.
joewitt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 2d9943e2d3 NIFI-13042 Support Python 3.12 for Python Processors This
closes #8644.
2d9943e2d3 is described below
commit 2d9943e2d30561bc28969c1c3f626f4754d671ab
Author: exceptionfactory <exceptionfact...@apache.org>
AuthorDate: Fri Apr 12 17:41:15 2024 -0500
NIFI-13042 Support Python 3.12 for Python Processors
This closes #8644.
- Updated Controller.py main function to join non-daemon threads avoiding
RuntimeError on Python 3.12
- Replaced deprecated find_module method with find_spec
- Updated documentation to include support for Python 3.12
Signed-off-by: Joseph Witt <joew...@apache.org>
---
nifi-docs/src/main/asciidoc/administration-guide.adoc | 4 ++--
nifi-docs/src/main/asciidoc/python-developer-guide.adoc | 2 +-
.../src/main/python/framework/Controller.py | 9 ++++++++-
.../src/main/python/framework/ExtensionManager.py | 4 ++--
4 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/nifi-docs/src/main/asciidoc/administration-guide.adoc
b/nifi-docs/src/main/asciidoc/administration-guide.adoc
index fd5248948f..b2fbeb50c4 100644
--- a/nifi-docs/src/main/asciidoc/administration-guide.adoc
+++ b/nifi-docs/src/main/asciidoc/administration-guide.adoc
@@ -24,7 +24,7 @@ Apache NiFi Team <d...@nifi.apache.org>
Apache NiFi can run on something as simple as a laptop, but it can also be
clustered across many enterprise-class servers. Therefore, the amount of
hardware and memory needed will depend on the size and nature of the dataflow
involved. The data is stored on disk while NiFi is processing it. So NiFi needs
to have sufficient disk space allocated for its various repositories,
particularly the content repository, flowfile repository, and provenance
repository (see the <<system_properties>> s [...]
* Requires Java 21
-* Use of Python-based Processors (beta feature) requires Python 3.9, 3.10 or
3.11
+* Use of Python-based Processors (beta feature) requires Python 3.9, 3.10,
3.11, or 3.12
* Supported Operating Systems:
** Linux
** Unix
@@ -231,7 +231,7 @@ The `name` attribute must start with `deprecation`,
followed by the component cl
NiFi is a Java-based application. NiFi 2.0 introduces support for a
Python-based Processor API. This capability is still
considered to be in "Beta" mode and should not be used in production. By
default, support for Python-based Processors is disabled. In order to enable it,
-Python 3.9, 3.10 or 3.11 must be installed on the NiFi node (Python 3.12 is
not supported yet).
+Python 3.9, 3.10, 3.11, or 3.12 must be installed on the NiFi node.
The following properties may be used to configure the Python 3 installation
and process management. These properties are all located under the
"Python Extensions" heading in the _nifi.properties_ file:
diff --git a/nifi-docs/src/main/asciidoc/python-developer-guide.adoc
b/nifi-docs/src/main/asciidoc/python-developer-guide.adoc
index b798fb6d0f..fc85521ecb 100644
--- a/nifi-docs/src/main/asciidoc/python-developer-guide.adoc
+++ b/nifi-docs/src/main/asciidoc/python-developer-guide.adoc
@@ -509,7 +509,7 @@ because we know that the ListFile Processor will produce
these attributes for us
[[requirements]]
== Requirements
-The Python API requires that Python 3.9, 3.10 or 3.11 is available on the
machine hosting NiFi (Python 3.12 is not supported yet).
+The Python API requires that Python 3.9, 3.10, 3.11, or 3.12 is available on
the machine hosting NiFi.
Each Processor may have its own list of requirements / dependencies. These are
made available to the Processor by creating a separate
environment for each Processor implementation (not for each instance of a
Processor on the canvas). PyPI is then used to install these
diff --git
a/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/Controller.py
b/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/Controller.py
index b40a444d28..038911271c 100644
---
a/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/Controller.py
+++
b/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/Controller.py
@@ -127,4 +127,11 @@ if __name__ == "__main__":
# Notify the Java side of the port that Python is listening on
gateway.java_gateway_server.resetCallbackClient(
gateway.java_gateway_server.getCallbackClient().getAddress(),
- python_port)
\ No newline at end of file
+ python_port)
+
+ # Join main thread to non-daemon threads in order to avoid RuntimeError on
Python 3.12 blocking new thread creation in Py4J
+ import threading
+ for thread in threading.enumerate():
+ if thread.daemon or thread is threading.current_thread():
+ continue
+ thread.join()
diff --git
a/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/ExtensionManager.py
b/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/ExtensionManager.py
index 3bc6d25cd9..ee2e28ef7d 100644
---
a/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/ExtensionManager.py
+++
b/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework/ExtensionManager.py
@@ -188,8 +188,8 @@ class ExtensionManager:
if not require_nifi_prefix or name.startswith('nifi_'):
module_file = '<Unknown Module File>'
try:
- module = finder.find_module(name)
- module_file = module.path
+ module = finder.find_spec(name)
+ module_file = module.origin
# Ignore any packaged dependencies
if 'NAR-INF/bundled-dependencies' in module_file:
