On Fri, 30 Nov 2018 at 18:12, Nathaniel Smith <n...@pobox.com> wrote:
>
> We could do the Windows thing, and have a plain "manylinux" tag that means
> "any recent-ish glibc-based Linux". Today it would be defined to be "any
> distro newer than CentOS 6". When CentOS 6 goes out of service, we could
> tweak the definition to be "any distro newer than CentOS 7". Most parts of
> the toolchain wouldn't need to be updated, though, because the tag wouldn't
> change, and by assumption, enforcement wouldn't really be needed, because the
> only people who could break would be ones running on unsupported platforms.
> Just like happens on Windows.
The reason this approach works for Windows is because *CPython*
defines the target Windows ABI version - if you don't use the right
target ABI, your extension module won't even link to the CPython DLL.
So here, we're taking advantage of Microsoft's strict ABI management
policy, by way of CPython.
>
> We could do the macOS thing, and have a "manylinux_${glibc version}" tag that
> means "this package works on any Linux using glibc newer than ${glibc
> version}". We're already using this as our heuristic to handle the current
> manylinux profiles, so e.g. manylinux1 is effectively equivalent to
> manylinux_2_5, and manylinux2010 will be equivalent to manylinux_2_12. That
> way we'd define the manylinux tags once, get support into pip and warehouse
> and auditwheel once, and then in the future the only thing that would have to
> change to support new distro releases or new architectures would be to set up
> a proper build environment.
This approach works for Mac OS X because Apple have just as strict an
approach to ABI management as Microsoft do, and the OS version
specifies a lot of details about a broad range of operating system
interfaces, which are then given strict compatibility guarantees.
> What do y'all think?
I don't think we can get away from *something* specifying exactly what
can be assumed to be present in a given manylinux variant, since the
distros don't define any useful form of cross-distro ABI
compatibility, and CPython doesn't nominate a target Linux ABI either.
However, I like the design concept of making it so that auditwheel is
the only project that has to change in order to define a new revision
of manylinux, which means encoding the heuristic check that installers
should use directly into the platform compatibility tag rather than
defining it as a lookup table that needs to be updated in every
affected tool whenever a new version is defined.
>From my view, the most promising path towards achieving that would be
to go with Brett's suggestion of
"manylinux_{libc_variant}_{libc_version}", such that we get
"manylinux_glibc_2_17" as the next edition rather than "manylinux2014"
(assuming the trend of using RHEL/CentOS releases as the baseline
continues). If folks prove keen to start using the new cheaper aarch64
cloud systems, there may also be demand for a "manylinux_glibc_2_27"
(compatible with Ubuntu 18.04 and RHEL/CentOS 8).
While having both "many" and "glibc" in the name may seem redundant
initially, I think it's worth having them both for two reasons:
1. Even if new manylinux tags are being defined as part of the
auditwheel documentation rather than in a PEP, I'd still expect them
to allow the generated wheel archives to dynamically link against more
than just glibc. Accordingly, there will be some platforms that pass
the installer's heuristic check, but nevertheless still have
compatibility problems with prebuilt wheel archives.
2. PEP 513 defined a ctypes based algorithm for checking the glibc
version, and a similar heuristic could be defined for other libc
implementations (most notably musl, thanks to Docker/Alpine). If we're
going to go through the process of switching to a different naming
scheme, we may as well provide a path towards resolving that issue as
well, rather than further entrenching manylinux as being glibc-only.
Cheers,
Nick.
P.S. Paul asked how we can have manylinux tags without updating PEP
425 to include them, and the answer is that the actual compatibility
tag spec is at
https://packaging.python.org/specifications/platform-compatibility-tags/
and that references PEP 513 (manylinux1) and PEP 571 (manylinux2010)
in addition to PEP 425.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at
https://mail.python.org/archives/list/distutils-sig@python.org/message/BFHBWB7ZO3L55V5JXGKN3FBNQQUN3END/
