Two reasons why see debug output you do. 1. Apache httpd starts as root if starting it from systemd and then drops privileges. Even if you use mod_wsgi daemon mode and tell it to run daemon processes as a specific user, that users login environment is never sourced and so will not pick up PATH changes in users local .bashrc/.zshrc/.profile/etc.
2. When Python is run in an embedded system, the "python" executable is never run and instead in this case the mod_wsgi module links to the Python library. In this case sys.executable has no meaning and can't be relied on for anything. As to the missing nghttp2 symbol, it is likely due to Anaconda Python supplying its own copy of that library, but some other Apache module is linking to the system variant of that library and they are different versions. Any Python module installed in Anaconda Python will though have been compiled against the Anaconda version. Because though the system version gets loaded into the process first by way of the separate Apache module (or even Apache itself), the system one wins out and the Anaconda version of the library is never used, so if Python modules were built with expectation of a certain symbol existing, but that does not exist in system version of that library, then you will get that error. If you want to test this theory, then disable mod_http2 in Apache since it seems to link to nghttp2. https://httpd.apache.org/docs/2.4/howto/http2.html Make sure you do a full stop/start of Apache to ensure that system nghttp2 library is unloaded. Graham > On 16 Nov 2024, at 11:14 am, Molly Foley <[email protected]> wrote: > > I'm having issues with a python package we have installed and I'm trying to > determine if it's a problem with our mod_wsgi install or the python package > itself. I am suspecting our mod_wsgi setup because the application works fine > when debugging it locally with the exact same directory acting as the python > environment that mod_wsgi is supposed to be using as python-home in our > Apache config. > > Part of my debugging process is having the WSGI application (Flask app) log a > bunch of info: > User running the app > $PATH > sys.path > sys.prefix > sys.executable > sys.base_prefix > > Output: > DEBUG [11/15/2024 06:37:38 PM] Running as user: hydro > > DEBUG [11/15/2024 06:37:38 PM] $PATH: > /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin > > > DEBUG [11/15/2024 06:37:38 PM] sys.path: > ['/var/www/vhosts/climate-hydro-analytics-platform-v2/api', > '/var/www/vhosts/climate-hydro-analytics-platform-v2', > '/opt/miniforge3/envs/chap/lib/python312.zip', > '/opt/miniforge3/envs/chap/lib/python3.12', > '/opt/miniforge3/envs/chap/lib/python3.12/lib-dynload', > '/opt/miniforge3/envs/chap/lib/python3.12/site-packages', > '/var/www/vhosts/climate-hydro-analytics-platform-v2/api/models'] > > DEBUG [11/15/2024 06:37:45 PM] Prefix: /opt/miniforge3/envs/chap > > DEBUG [11/15/2024 06:37:45 PM] Exec: /usr/bin/python3 > > > DEBUG [11/15/2024 06:37:45 PM] Base Prefix: /opt/miniforge3/envs/chap > > > I am spotting two potential problems: > I see no /opt/miniforge3/bin or /opt/miniforge3/envs/chap/bin in the $PATH > The sys.executable seems to be pointing to the system installation of python > instead of the virtual environment set as python-home > What's confusing me is that it's getting the prefix right, and the > application does actually run and doesn't experience import errors, so I > think it is still somehow finding the the right python packages (except for > one which is responsible for the error). > > First question is is are these values expected for $PATH and sys.executable? > > If I login as the user above (hydro) and echo $PATH, I get a very different > value where I do see our python env: > /opt/miniforge3/envs/chap/bin:/opt/miniforge3/condabin:/home/hydro/.local/bin:/home/hydro/bin:/opt/miniforge3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/var/cfengine/bin > > I have attempted to SetEnv in Apache virtual hosts .conf file as well as > export it as a new file in /etc/sysconfig, but the printed output above is > always the same. > > Here's our LoadModule directive: > > <IfModule !wsgi_module> > > LoadModule wsgi_module > /opt/miniforge3/lib/python3.12/site-packages/mod_wsgi/server/mod_wsgi-py312.cpython-312-x86_64-linux-gnu.so > > WSGIPythonHome "/opt/miniforge3" > > WSGIRestrictEmbedded On > > > </IfModule> > > > And our WSGI config in vhosts (there are two hosts, each using their own > python env at python-home): > > <VirtualHost *:443> > > ServerName myServer > > ## Vhost docroot > > DocumentRoot "/var/www/vhosts/climate-hydro-analytics-platform-v2" > > ## Alias declarations for resources outside the DocumentRoot > > Alias /jobs "/output/jobs" > > ## Logging > > ErrorLog "/var/log/httpd/chap-error.log" > > LogLevel info > > ServerSignature Off > > CustomLog "/var/log/httpd/chap-access.log" combined > > ## SSL directives > > SSLEngine on > > SSLCertificateFile "/etc/pki/tls/certs/myServer.crt" > > SSLCertificateKeyFile "/etc/pki/tls/private/myKey.key" > > SSLVerifyClient none > > SSLCACertificatePath "/etc/pki/tls/certs" > > SSLCACertificateFile "/etc/pki/tls/certs/sectigo.ca.crt" > > ## WSGI configuration > > WSGIApplicationGroup %{GLOBAL} > > WSGIDaemonProcess chap display-name=%{GROUP} > home=/var/www/vhosts/climate-hydro-analytics-platform-v2 > python-home=/opt/miniforge3/envs/chap processes=10 threads=1 user=hydro > > WSGIProcessGroup chap > > WSGIScriptAlias / > "/var/www/vhosts/climate-hydro-analytics-platform-v2/wsgi.py" > > #...more stuff > > > </VirtualHost> > > <VirtualHost *:443> > > ServerName myServer2 > > ## Vhost docroot > > DocumentRoot "/var/www/vhosts/hydrosource-2" > > ## Logging > > ErrorLog "/var/log/httpd/dataexplorer-error.log" > > LogLevel warn > > ServerSignature Off > > CustomLog "/var/log/httpd/dataexplorer-access.log" combined > > ## SSL directives > > SSLEngine on > > SSLCertificateFile "/etc/pki/tls/certs/myServer.crt" > > SSLCertificateKeyFile "/etc/pki/tls/private/myKey.key" > > SSLVerifyClient none > > SSLCACertificatePath "/etc/pki/tls/certs" > > SSLCACertificateFile "/etc/pki/tls/certs/sectigo.ca.crt" > > ## WSGI configuration > > WSGIApplicationGroup %{GLOBAL} > > WSGIDaemonProcess dataexplorer display-name=%{GROUP} > home=/var/www/vhosts/hydrosource-2 > python-home=/opt/miniforge3/envs/dataexplorer processes=10 threads=1 > user=hydro > > WSGIProcessGroup dataexplorer > > WSGIScriptAlias / "/var/www/vhosts/hydrosource-2/wsgi.py" > > #...more stuff > > > > </VirtualHost> > > I realize that conda isn't recommend and pip is, but we're using conda due to > difficulties with managing scientific package dependencies (lots of C > libraries) with pip. Given the error we're receiving, it is likely it's some > C-library stuff conflicting somehow with whatever mod_wsgi is doing under the > hood, so we may have to abandon mod_wsgi due to that. But I am very curious > about the answer to my above question. > > If you are curious about the error, we have a python package called xarray > where we call the method xarray.open_dataset(...). This prompts the xarray > package to import a package called netCDF4. This causes an import error like > so: > > DEBUG [11/15/2024 06:37:45 PM] Traceback (most recent call last): > > File > "/var/www/vhosts/climate-hydro-analytics-platform-v2/api/blueprints/climate_viewer.py", > line 69, in get_chart_data > > ctrl_ds = xarr.open_dataset(control, decode_times=False) > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > File > "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/xarray/backends/api.py", > line 671, in open_dataset > > backend_ds = backend.open_dataset( > > ^^^^^^^^^^^^^^^^^^^^^ > > File > "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", > line 666, in open_dataset > > store = NetCDF4DataStore.open( > > ^^^^^^^^^^^^^^^^^^^^^^ > > File > "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", > line 415, in open > > import netCDF4 > > File > "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/netCDF4/__init__.py", > line 3, in <module> > > from ._netCDF4 import * > > > ImportError: > /opt/miniforge3/envs/chap/lib/python3.12/site-packages/netCDF4/../../.././libcurl.so.4: > undefined symbol: > nghttp2_option_set_no_rfc9113_leading_and_trailing_ws_validation > > If I login as the hydro user, activate the /opt/miniforge3/envs/chap > environment, enter python and run the same piece of code > (xarray.open_dataset(...)), it works without error. So there's something > getting confused internally when going through mod_wsgi and Apache, but we > have yet to figure out what after days of trying different things. You told > me in my last question > <https://groups.google.com/g/modwsgi/c/-Dcai15uiZk>that PHP could be a > problem, but we do not have PHP installed (and we are not using Anaconda > official distribution either, but rather the open source distribution that > utilizes conda and mamba called Miniforge3). > > Thanks for your continued support! I appreciate it. > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion visit > https://groups.google.com/d/msgid/modwsgi/3fda8fd8-5847-4ee1-8849-80248a9ca006n%40googlegroups.com > > <https://groups.google.com/d/msgid/modwsgi/3fda8fd8-5847-4ee1-8849-80248a9ca006n%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/modwsgi/53A671DA-388D-4EEA-A2F3-F2F268DE7601%40gmail.com.
