bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hello! Ludovic Courtès skribis: >>From Marek’s message, it would seem that somewhere file names are > constructed by appending “..”. Given Unix’s weird dot-dot semantics, > relative symlinks could be interfering: Some news! Here’s a recap of the IRC discussion earlier today: ‘sys.prefix’ looks like /tmp/pack2/bin/../../zmv1alp7dzjd6hkc7l6fjz817n2mfzzw-python-3.9.9R. ‘sys.normpath’ does lexical dot-dot resolution (which is incorrect on POSIX): https://docs.python.org/3.11/library/os.path.html#os.path.normpath Thus, passing the prefix above to ‘sys.normpath’ leads “pack2” to be erased. Josselin mentioned “path_hooks” as a possible solution, though I’m not sure what that means; could you explain? :-) Thanks! Ludo’.
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hello everyone, After the IRC investigation, it turned out the use of abspath in Lib/site.py was in fact the culprit. Replacing all its uses with realpath and doing `./pre-inst-env guix pack -RR python python-numpy -S /bin=bin -S /etc=etc` did in fact resolve the issue! I've raised the issue upstream at [1]. The fix itself is pretty simple, but it's theoretically a breaking change, so we'll see how Python maintainers want to move forward with it. We could backport this fix for the next core-updates cycle, whether or not it gets accepted upstream, and we will also need to modify sitecustomize.py as well to call os.path.realpath on python_site. Best, -- Josselin Poiret
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Silly me forgetting the link, here it is: [1] https://bugs.python.org/issue46653 Apologies for the noise, -- Josselin Poiret
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hi Josselin, Josselin Poiret skribis: > After the IRC investigation, it turned out the use of abspath in > Lib/site.py was in fact the culprit. Replacing all its uses with > realpath and doing `./pre-inst-env guix pack -RR python python-numpy -S > /bin=bin -S /etc=etc` did in fact resolve the issue! > > I've raised the issue upstream at [1]. The fix itself is pretty simple, > but it's theoretically a breaking change, so we'll see how Python > maintainers want to move forward with it. We could backport this fix > for the next core-updates cycle, whether or not it gets accepted > upstream, and we will also need to modify sitecustomize.py as well to > call os.path.realpath on python_site. > [1] https://bugs.python.org/issue46653 Thanks for the investigation and for reporting it upstream! Is there a workaround we could apply locally, ideally one that does not lead to a world rebuild? Ludo’.
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hi Ludo, Ludovic Courtès writes: > Thanks for the investigation and for reporting it upstream! > > Is there a workaround we could apply locally, ideally one that does not > lead to a world rebuild? So, upstream hasn't replied at all to the above proposal. How do we move forward? I have a patch that still applies on Python 3.9.9 that we can put into core-updates. Would that be fine? Best, -- Josselin Poiret
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hi Josselin, Josselin Poiret writes: > Hi Ludo, > > Ludovic Courtès writes: > >> Thanks for the investigation and for reporting it upstream! >> >> Is there a workaround we could apply locally, ideally one that does not >> lead to a world rebuild? > > So, upstream hasn't replied at all to the above proposal. How do we move > forward? I have a patch that still applies on Python 3.9.9 that we can > put into core-updates. Would that be fine? I haven't revisited that patch, but if you are confident it wouldn't break anything in the Python world, core-updates would be a fine place, yes. -- Thanks, Maxim
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hi, On jeu., 02 févr. 2023 at 10:40, Maxim Cournoyer wrote: >> So, upstream hasn't replied at all to the above proposal. How do we move >> forward? I have a patch that still applies on Python 3.9.9 that we can >> put into core-updates. Would that be fine? > > I haven't revisited that patch, but if you are confident it wouldn't > break anything in the Python world, core-updates would be a fine place, > yes. Maybe for testing this change, you could create one Python interpreter variant containing the patch. Then using ’package-with-explicit-python’ package transformation from (guix build-system python) we could build Python packages and check if it correctly fixes and also incrementally check if nothing break. It would avoid a full world rebuild. Cheers, simon
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hi everyone, I found this issue while investigating a related one (the error message). It looks like the core problem discussed here has been solved in the meantime: Singularity> python3 -m numpy Error in sitecustomize; set PYTHONVERBOSE for traceback: ValueError: '/gnu/store/kx98dz2vc3gdpjficy4nbd7rscnkvz4s-profile/bin/../../h855kddqbay0pcbwr8a7i8m6ilz67cfn-python-3.10.7/lib/python3.10/site-packages' is not in list /gnu/store/kx98dz2vc3gdpjficy4nbd7rscnkvz4s-profile/bin/python3: No module named numpy.__main__; 'numpy' is a package and cannot be directly executed This error message is to be expected, and NumPy can be imported: Singularity> python3 Error in sitecustomize; set PYTHONVERBOSE for traceback: ValueError: '/gnu/store/kx98dz2vc3gdpjficy4nbd7rscnkvz4s-profile/bin/../../h855kddqbay0pcbwr8a7i8m6ilz67cfn-python-3.10.7/lib/python3.10/site-packages' is not in list Python 3.10.7 (main, Jan 1 1970, 00:00:01) [GCC 11.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> What remains is the error message, which comes from a bug in Guix' sitecustomize.py. It compares a raw path (sys.prefix) with paths that have been normalized (os.path.normpath). That comparison fails if sys.prefix contains "../", as it does when run from a Singularity container. I will submit a patch for this. Cheers, Konrad
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Konrad Hinsen writes: > I will submit a patch for this. Patch at https://issues.guix.gnu.org/68241 Cheers, Konrad
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Konrad Hinsen writes: > Patch at https://issues.guix.gnu.org/68241 If you want to test it without rebuilding tons of packages, here is a version that grafts a patched Python onto the existing ones (as substitutes): https://codeberg.org/khinsen/guix/src/branch/graft-fix-python-sitecustomize Cheers, Konrad
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hi Konrad, Konrad Hinsen writes: > Konrad Hinsen writes: > >> Patch at https://issues.guix.gnu.org/68241 > > If you want to test it without rebuilding tons of packages, here is a > version that grafts a patched Python onto the existing ones (as substitutes): > > https://codeberg.org/khinsen/guix/src/branch/graft-fix-python-sitecustomize I've applied your patch to core-updates. I trust that it fixed this issue based on your own testing. Closing; we can always reopen if we find it didn't fix every use cases or create a fresh issue. -- Thanks, Maxim
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hi Marek, Marek Felšöci skribis: > Recently, I have come around this issue. I created a Singularity > container using the following `guix pack` command: > > `guix pack -f squashfs bash coreutils python python-numpy` > > Then, I run a bash shell within the container: > > `singularity exec test.gz.squashfs bash` > > Now, if I want to use numpy, > > `python3 -m numpy` > > it gives me the following error and the numpy module IS NOT found: > > Error in sitecustomize; set PYTHONVERBOSE for traceback: > ValueError: > '/gnu/store/2dyd6rlcc9m68k3mvkw86k8dygkbg025-profile/bin/../../p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python3.9/site-packages' > > is not in list > /gnu/store/2dyd6rlcc9m68k3mvkw86k8dygkbg025-profile/bin/python3: No > module named numpy I can kinda reproduce via a relocatable pack (relocatable packs and Singularity packs have in common that they use relative symlinks in their profile, to allow for relocation): guix pack -RR python python-numpy -S /bin=bin -S /etc=etc Then I unpack the thing: mkdir /tmp/pack; cd /tmp/pack; tar xf /gnu/store/…-pack.tar.gz And from there I use the same trick as in ‘tests/guix-pack-relocatable.sh’ to “hide” /gnu/store: --8<---cut here---start->8--- $ unshare -mrf sh -c 'mount -t tmpfs -o ro none /gnu/store; . /tmp/pack/etc/profile; echo $GUIX_PYTHONPATH ; /tmp/pack/bin/python3 ' /gnu/store/w5cz9gbqbja99x0636zsg2fzksmfig4n-profile/lib/python3.9/site-packages Python 3.9.6 (default, Jan 1 1970, 00:00:01) [GCC 10.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'numpy' >>> import os >>> os.listdir("/gnu/store/w5cz9gbqbja99x0636zsg2fzksmfig4n-profile/lib/python3.9/site-packages") ['setuptools-56.0.0.dist-info', 'distutils-precedence.pth', '_distutils_hack', 'setuptools', 'pip-21.1.3.dist-info', 'sitecustomize.py', 'README.txt', 'pip', 'numpy-1.20.3-py3.9.egg-info', 'numpy', 'pkg_resources', '__pycache__'] >>> os.listdir("/gnu/store/w5cz9gbqbja99x0636zsg2fzksmfig4n-profile/lib/python3.9/site-packages/numpy") ['__init__.pyi', 'doc', 'matlib.py', 'ctypeslib.pyi', 'linalg', 'ctypeslib.py', 'compat', 'typing', '__init__.pxd', 'lib', 'char.pyi', 'version.py', '__config__.py', 'core', '_distributor_init.py', 'distutils', 'tests', 'emath.pyi', '_pytesttester.py', 'fft', '__init__.cython-30.pxd', 'conftest.py', '__init__.py', 'py.typed', '_globals.py', 'rec.pyi', 'testing', 'LICENSE.txt', 'polynomial', 'f2py', 'matrixlib', 'random', 'ma', 'dual.py', 'setup.py', '__pycache__'] --8<---cut here---end--->8--- So in this case ‘GUIX_PYTHONPATH’ is correctly set, but it looks as though ‘sitecustomize.py’ wasn’t loaded. (I don’t get the “Error in sitecustomize” message that Marek sees though, weird!) >From Marek’s message, it would seem that somewhere file names are constructed by appending “..”. Given Unix’s weird dot-dot semantics, relative symlinks could be interfering: --8<---cut here---start->8--- $ ls -l /tmp/pack/gnu/store/w5cz9gbqbja99x0636zsg2fzksmfig4n-profile/lib/python3.9/site-packages/ totalo 48 lrwxrwxrwx 1 ludo users 102 Jan 1 1970 _distutils_hack -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/_distutils_hack lrwxrwxrwx 1 ludo users 111 Jan 1 1970 distutils-precedence.pth -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/distutils-precedence.pth lrwxrwxrwx 1 ludo users 99 Jan 1 1970 numpy -> ../../../../mlccgh05bf8cdinq0ilpvpdmsspq36pv-python-numpy-1.20.3R/lib/python3.9/site-packages/numpy lrwxrwxrwx 1 ludo users 121 Jan 1 1970 numpy-1.20.3-py3.9.egg-info -> ../../../../mlccgh05bf8cdinq0ilpvpdmsspq36pv-python-numpy-1.20.3R/lib/python3.9/site-packages/numpy-1.20.3-py3.9.egg-info lrwxrwxrwx 1 ludo users 90 Jan 1 1970 pip -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/pip lrwxrwxrwx 1 ludo users 107 Jan 1 1970 pip-21.1.3.dist-info -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/pip-21.1.3.dist-info lrwxrwxrwx 1 ludo users 100 Jan 1 1970 pkg_resources -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/pkg_resources lrwxrwxrwx 1 ludo users 98 Jan 1 1970 __pycache__ -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/__pycache__ lrwxrwxrwx 1 ludo users 97 Jan 1 1970 README.txt -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/README.txt lrwxrwxrwx 1 ludo users 97 Jan 1 1970 setuptools -> ../../../../niddlj0qbk5vzk1mdn5im9y4x6l8a2k9-python-3.9.6R/lib/python3.9/site-packages/setuptools lrwxrwxrwx 1 ludo users 114 Jan 1 1970 setuptools-56.0.0.dist-info -> ../../../../niddlj0qb
bug#53258: Python unable to find modules within a Singularity container created with guix pack
Hello folks, Recently, I have come around this issue. I created a Singularity container using the following `guix pack` command: `guix pack -f squashfs bash coreutils python python-numpy` Then, I run a bash shell within the container: `singularity exec test.gz.squashfs bash` Now, if I want to use numpy, `python3 -m numpy` it gives me the following error and the numpy module IS NOT found: Error in sitecustomize; set PYTHONVERBOSE for traceback: ValueError: '/gnu/store/2dyd6rlcc9m68k3mvkw86k8dygkbg025-profile/bin/../../p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python3.9/site-packages' is not in list /gnu/store/2dyd6rlcc9m68k3mvkw86k8dygkbg025-profile/bin/python3: No module named numpy However, if I do, `export PYTHONPATH=$GUIX_PYTHONPATH` the numpy module IS found: Error in sitecustomize; set PYTHONVERBOSE for traceback: ValueError: '/gnu/store/2dyd6rlcc9m68k3mvkw86k8dygkbg025-profile/bin/../../p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python3.9/site-packages' is not in list /gnu/store/2dyd6rlcc9m68k3mvkw86k8dygkbg025-profile/bin/python3: No module named numpy.__main__; 'numpy' is a package and cannot be directly executed Still, the sitecustomize error persists. See the output of `guix describe` below (only the guix channel): Pokolenie 3 14. január 2022 14:33:06 (súčasné) guix 5c5d9e5 zdroj repozitára: https://git.savannah.gnu.org/git/guix.git vetva: master úprava: 5c5d9e5a3208fa51351c6ad8267eba3d96fa2232 Thanks, Marek