Bug#998739: python3.9: sysconfig should return /dist-packages as platlib
On Tue, Jun 21, 2022 at 07:05:17AM +0100, Julian Gilbey wrote:
> Here's a data point:
>
> $ python3.9
> Python 3.9.13 (main, Jun 8 2022, 09:45:57)
> [GCC 11.3.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sysconfig
> >>> sysconfig.get_path('purelib')
> '/usr/lib/python3.9/site-packages'
> >>> sysconfig.get_path('platlib')
> '/usr/lib/python3.9/site-packages'
> >>>
>
> $ python3.10
> Python 3.10.5 (main, Jun 8 2022, 09:26:22) [GCC 11.3.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sysconfig
> >>> sysconfig.get_path('purelib')
> '/usr/local/lib/python3.10/dist-packages'
> >>> sysconfig.get_path('platlib')
> '/usr/local/lib/python3.10/dist-packages'
> >>>
> [...]
And I should have added: in python3.9, the directories given do not
actually exist on a standard Debian system, whereas on python3.10 they
do.
There's a separate question of whether the directory given should
instead be /usr/lib/python3/site-packages, but that's far more
debatable.
Best wishes,
Julian
Bug#998739:
On Mon, Apr 11, 2022 at 11:00:24PM +0300, Ievgen Popovych wrote:
> [...]
> I was just playing a bit more and it turns out that
> `/usr/lib/python3.9/sysconfig.py` does not include changes from
> `patches/sysconfig-debian-schemes.diff` at all!
> I've verified I do have Debian Bullseye :), file is proided by
> `libpython3.9-minimal:amd64:`. sysconfig-debian-schemes.diff is also
> in `patches/series`.
> I do not know what to think at this point..
Here's a data point:
$ python3.9
Python 3.9.13 (main, Jun 8 2022, 09:45:57)
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_path('purelib')
'/usr/lib/python3.9/site-packages'
>>> sysconfig.get_path('platlib')
'/usr/lib/python3.9/site-packages'
>>>
$ python3.10
Python 3.10.5 (main, Jun 8 2022, 09:26:22) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_path('purelib')
'/usr/local/lib/python3.10/dist-packages'
>>> sysconfig.get_path('platlib')
'/usr/local/lib/python3.10/dist-packages'
>>>
So this issue has been resolved in the Debian python3.10 package but
not in python3.9.
(I ran into this issue as it broke a package test...)
Best wishes,
Julian
Bug#998739:
On Mon, 11 Apr 2022 22:18:26 +0300 Ievgen Popovych wrote: > Hello, > > I had the same issue when building a package with a system interpreter: > sysconfig module reports incorrect paths (pointing to `site-packages` > instead of `dist-packages`). > The culprit seems to be that it thinks that > > Current installation scheme: "posix_prefix" > instead of `deb_system`. > That, in turn, looks to be an issue with `_get_default_scheme()` > implementation in `patches/sysconfig-debian-schemes.diff` which > depends on environment variable `DEB_PYTHON_INSTALL_LAYOUT`. > The only reference I found is a changelog entry stating that this is > indeed a desired behavior. > > But I wonder why? > It is inconsistent with distutils behavior. > It does not refer to (quoting sysconfig module docstring) > > Python’s configuration information like the list of installation paths and > > the configuration variables relevant for the current platform > In current form it is essentially irrelevant. > I was just playing a bit more and it turns out that `/usr/lib/python3.9/sysconfig.py` does not include changes from `patches/sysconfig-debian-schemes.diff` at all! I've verified I do have Debian Bullseye :), file is proided by `libpython3.9-minimal:amd64:`. sysconfig-debian-schemes.diff is also in `patches/series`. I do not know what to think at this point..
Bug#998739:
Hello, I had the same issue when building a package with a system interpreter: sysconfig module reports incorrect paths (pointing to `site-packages` instead of `dist-packages`). The culprit seems to be that it thinks that > Current installation scheme: "posix_prefix" instead of `deb_system`. That, in turn, looks to be an issue with `_get_default_scheme()` implementation in `patches/sysconfig-debian-schemes.diff` which depends on environment variable `DEB_PYTHON_INSTALL_LAYOUT`. The only reference I found is a changelog entry stating that this is indeed a desired behavior. But I wonder why? It is inconsistent with distutils behavior. It does not refer to (quoting sysconfig module docstring) > Python’s configuration information like the list of installation paths and > the configuration variables relevant for the current platform In current form it is essentially irrelevant. Thanks. -- Sincerely, Ievgen Popovych
Bug#998739:
I was just going to report a similar bug to this, after attempting to
migrate a package from using distutils.sysconfig to sysconfig (due to 3.10
now screaming about distutils being removed Soon(tm)), and discovering that
distutils.sysconfig does report dist-packages, while sysconfig reports
site-packages, just as described in [1].
>>> distutils.sysconfig.get_python_lib(0,0)
/usr/lib/python3/dist-packages
>>> sysconfig.get_path('platlib')
'/usr/lib/python3.9/site-packages'
So I'd recommend changing _something_ so that these become at least
consistent, or people are just going to start having to add Debian-specific
hacks everywhere or just ignoring Debian's intended locations, and neither
strikes me as great for anyone.
- Rich
[1] -
https://ffy00.github.io/blog/02-python-debian-and-the-install-locations/
Bug#998739: python3.9: In sys.path, /usr/local should use site-packages, not dist-packages
control: retitle -1 python3.9: sysconfig should return /dist-packages as platlib
Hello,
Matthias Klose, le mer. 10 nov. 2021 14:53:38 +0100, a ecrit:
> On 11/7/21 13:53, Samuel Thibault wrote:
> sys.path
> > ['', '/usr/lib/python39.zip', '/usr/lib/python3.9',
> > '/usr/lib/python3.9/lib-dynload',
> > '/home/samy/.local/lib/python3.9/site-packages',
> > '/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages',
> > '/usr/lib/python3.9/dist-packages']
> >
> > We can see /usr/local/lib/python3.9/dist-packages, but AIUI,
> > 'dist-packages' is meant only for distribution-provided python packages,
> > which /usr/local is definitely not about. And indeed, when installing
> > some python library with the usual python3 setup.py build / install
> > process (e.g. from upstream speech-dispatcher source), the python
> > modules are going to /usr/local/lib/python3.9/site-packages, and not
> > dist-packages.
> >
> > And thus the just-installed python modules are not getting found, and
> > people are confused since they did run python3 setup.py properly.
> >
> >
> > So it seems that sys.path should be fixed into:
> >
> sys.path
> > ['', '/usr/lib/python39.zip', '/usr/lib/python3.9',
> > '/usr/lib/python3.9/lib-dynload',
> > '/home/samy/.local/lib/python3.9/site-packages',
> > '/usr/local/lib/python3.9/site-packages', '/usr/lib/python3/dist-packages',
> > '/usr/lib/python3.9/dist-packages']
>
> The argument against /usr/local/lib/python3.9/site-packages is, that the very
> same location is used for a local python build with the default configure
> flags,
> e.g. ./configure ...
>
> That allows way too easy mixing of locally installed python modules.
> Having a /usr/local/lib/python3.9/dist-packages avoids that.
Ok, but the set of people who build their own python interpreter with
./configure is *way* smaller than the set of people who build some
$random_package which happens to use python. And people who build their
own python interpreter are the same who know a bit about such kind of
issues.
That being said, it's Debian which tells applications to
install modules built with the Debian python interpreter into
/usr/local/lib/python3.9/site-packages. The python.m4 snippet shipped by
automake uses:
am_cv_python_pyexecdir=`$PYTHON -c "
$am_python_setup_sysconfig
if can_use_sysconfig:
sitedir = sysconfig.get_path('platlib',
vars={'platbase':'$am_py_exec_prefix'})
else:
i.e.
import sys
import sysconfig
sitedir = sysconfig.get_path('platlib', vars={'platbase':'/usr/local'})
which results to '/usr/local/lib/python3.9/site-packages'
Following your reasoning, sysconfig should return
/usr/local/lib/python3.9/dist-packages
instead of
/usr/local/lib/python3.9/site-packages
Samuel
Bug#998739: python3.9: In sys.path, /usr/local should use site-packages, not dist-packages
On 11/7/21 13:53, Samuel Thibault wrote: > Package: python3.9 > Version: 3.9.7-4 > Severity: normal > > Hello, > > In sys.path: > sys.path > ['', '/usr/lib/python39.zip', '/usr/lib/python3.9', > '/usr/lib/python3.9/lib-dynload', > '/home/samy/.local/lib/python3.9/site-packages', > '/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages', > '/usr/lib/python3.9/dist-packages'] > > We can see /usr/local/lib/python3.9/dist-packages, but AIUI, > 'dist-packages' is meant only for distribution-provided python packages, > which /usr/local is definitely not about. And indeed, when installing > some python library with the usual python3 setup.py build / install > process (e.g. from upstream speech-dispatcher source), the python > modules are going to /usr/local/lib/python3.9/site-packages, and not > dist-packages. > > And thus the just-installed python modules are not getting found, and > people are confused since they did run python3 setup.py properly. > > > So it seems that sys.path should be fixed into: > sys.path > ['', '/usr/lib/python39.zip', '/usr/lib/python3.9', > '/usr/lib/python3.9/lib-dynload', > '/home/samy/.local/lib/python3.9/site-packages', > '/usr/local/lib/python3.9/site-packages', '/usr/lib/python3/dist-packages', > '/usr/lib/python3.9/dist-packages'] > > ? The argument against /usr/local/lib/python3.9/site-packages is, that the very same location is used for a local python build with the default configure flags, e.g. ./configure ... That allows way too easy mixing of locally installed python modules. Having a /usr/local/lib/python3.9/dist-packages avoids that.
Bug#998739: python3.9: In sys.path, /usr/local should use site-packages, not dist-packages
Package: python3.9 Version: 3.9.7-4 Severity: normal Hello, In sys.path: >>> sys.path ['', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/samy/.local/lib/python3.9/site-packages', '/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.9/dist-packages'] We can see /usr/local/lib/python3.9/dist-packages, but AIUI, 'dist-packages' is meant only for distribution-provided python packages, which /usr/local is definitely not about. And indeed, when installing some python library with the usual python3 setup.py build / install process (e.g. from upstream speech-dispatcher source), the python modules are going to /usr/local/lib/python3.9/site-packages, and not dist-packages. And thus the just-installed python modules are not getting found, and people are confused since they did run python3 setup.py properly. So it seems that sys.path should be fixed into: >>> sys.path ['', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/samy/.local/lib/python3.9/site-packages', '/usr/local/lib/python3.9/site-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.9/dist-packages'] ? Samuel -- System Information: Debian Release: bookworm/sid APT prefers testing APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (500, 'stable-security'), (500, 'stable-debug'), (500, 'proposed-updates-debug'), (500, 'proposed-updates'), (500, 'oldstable-proposed-updates-debug'), (500, 'oldstable-proposed-updates'), (500, 'oldoldstable'), (500, 'buildd-unstable'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.14.0-2-amd64 (SMP w/8 CPU threads) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages python3.9 depends on: ii libpython3.9-stdlib 3.9.7-4 ii media-types 4.0.0 ii mime-support 3.66 ii python3.9-minimal3.9.7-4 python3.9 recommends no packages. Versions of packages python3.9 suggests: ii binutils2.37-7 pn python3.9-doc ii python3.9-venv 3.9.7-4 -- no debconf information -- Samuel La fonction memfrob() crypte les n premiers octets de la zone de mémoire s en effectuant un OU-exclusif entre chaque octet et le nombre 42. (extrait de la page de man de memfrob -- Manuel du programmeur Linux)
