dam4rus commented on PR #7003: URL: https://github.com/apache/nifi/pull/7003#issuecomment-1471969300
Thanks for your hard work on this @markap14 . I mainly looked at the code and left comments there but there are some questions/observations left on my mind, mostly about dependency management - Why are we installing the dependencies for each Python extension on NiFi startup? This massively slows down NiFi startup. I believe installing them when creating the first processor of an extension would improve the user experience because there's a high chance the user will never use these extensions, yet they pay the price for them existing. - Related to the previous and one of my comments left: There's currently no UI indication of dependency installation, whether it failed or not and virtual env creation. Currently adding a Python processor takes around 10-20 on my machine and it feels like that the UI is unresponsive during that time. - I'm wondering why the dependency installation and virtual environment creation is ordered like this? It's kinda backwards of how most virtual environments are created for Python. The environment should be created first, then dependencies are installed afterward. Currently we are installing the dependencies, then creating the environment. - Related to the previous question: I don't really get how dependencies are "seen" by each interpreter. Dependencies should go to `$ENVDIR/lib/$PYTHONVERSION/site-packages`, yet dependencies are installed to `$ENVDIR`. I have tested this approach on my machine with my virtual environments and couldn't import the modules under `$ENVDIR` unless I've set `PYTHONPATH`. I do not see `PYTHONPATH` being set to `$ENVDIR`. Can you shed some lights on how this works? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
