[
https://issues.apache.org/jira/browse/NIFI-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17880115#comment-17880115
]
Lewis J commented on NIFI-12196:
--------------------------------
Suggest this may be more of a documentation fix, as opposed to a code fix.
@Janis Ax
For Stand-Alone installations on RHEL / Debian or other architectures. NiFi
should adhere to the Global pip.conf file (if present) or the user pip.conf
file, and then finally the virtualenv pip.conf
Since 2.0 creates a virtual environment for each Python processor, when the
virtualenv is created, it should inherit any settings.
e.g for example... In my dockerised NiFi-2.0.0-M4 container i mounted the
example pip conf file in {{/home/nifi/.config/pip/pip.conf}}
{code:java}
[global]
find-links = http://download.example.com
[install]
find-links = http://mirror1.example.com http://mirror2.example.com
trusted-host = mirror1.example.com {code}
Then i create a new Python Processor and attempt to import pandas.
{code:java}
from nifiapi.flowfiletransform import FlowFileTransform,
FlowFileTransformResultclass
import pandas as pd
WriteHelloWorld(FlowFileTransform):
class Java:
implements = ['org.apache.nifi.python.processor.FlowFileTransform']
class ProcessorDetails:
version = '0.0.1-SNAPSHOT'
dependencies = ['pandas']
def __init__(self, **kwargs):
pass
def transform(self, context, flowfile):
return FlowFileTransformResult(relationship = "success", contents =
"Hello World", attributes = {"greeting": "hello"}) {code}
The Pip install should fail, as my pip.conf in the user space should override
the VRITUAL_ENV pip config. As below.
{code:java}
2024-09-08 21:13:59 2024-09-08 11:13:59,065 INFO [python-log-225]
python.ExtensionManager Importing dependencies ['pandas'] for WriteHelloWorld
to
/opt/nifi/nifi-current/./work/python/extensions/WriteHelloWorld/0.0.1-SNAPSHOT
using command
['/opt/nifi/nifi-current/./work/python/extensions/WriteHelloWorld/0.0.1-SNAPSHOT/bin/python3',
'-m', 'pip', 'install', '--no-cache-dir', '--target',
'/opt/nifi/nifi-current/./work/python/extensions/WriteHelloWorld/0.0.1-SNAPSHOT',
'pandas']
2024-09-08 21:13:59 2024-09-08 11:13:59,683 INFO [python-log-225]
org.apache.nifi.py4j.ProcessLog Looking in links: http://mirror1.example.com,
http://mirror2.example.com
2024-09-08 21:13:59 2024-09-08 11:13:59,683 INFO [python-log-225]
org.apache.nifi.py4j.ProcessLog WARNING: The repository located at
mirror2.example.com is not a trusted or secure host and is being ignored. If
this repository is available via HTTPS we recommend you use HTTPS instead,
otherwise you may silence this warning and allow it anyway with '--trusted-host
mirror2.example.com'. {code}
Providing Global / or User space pip.conf files will give NiFi the ability to
specify private repo's / trusted hosts, ca-bundle paths etc...
> Allow usage of pip config file (proxy, package registry)
> --------------------------------------------------------
>
> Key: NIFI-12196
> URL: https://issues.apache.org/jira/browse/NIFI-12196
> Project: Apache NiFi
> Issue Type: Sub-task
> Reporter: Janis Ax
> Priority: Major
>
> Many companies have a proxy, or use a own package registry like a nexus.
> In a pip config file, we could configure this.
> More infos; [Configuration - pip documentation v23.2.1
> (pypa.io)|https://pip.pypa.io/en/stable/topics/configuration/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)