On Sunday, April 21, 2024 at 9:42:24 AM UTC-5 Nathan Dunfield wrote: For the statements in this thread, I don't see any contradictions about the definition of the "normal Python way of doing things". My understanding of that term is to post *self-contained* binary wheels to PyPI for all supported platforms that install in a minute or two with no compilation (as Dima wrote), as well as a source-code only package that serves as a rarely-used backup if no suitable binary is available. (The self-contained bit is important; for example, on Linux here are the only external libraries <https://peps.python.org/pep-0513/#the-manylinux1-policy> a binary wheel is allowed to link to.)
I hope it will help to acknowledge what might be regarded as the elephant in this room. That elephant consists of two facts: 1) *Users*, as opposed to *developers*, are very unlikely to be either willing to or capable of building a complex source-code only package which depends on external libraries, because doing that requires finding and installing (or building from source), all of the external libraries needed by the python package. 2) The use of a *self-contained *binary wheel on linux violates a core principle of linux packaging, namely that any library which is a runtime dependency of a package should be installed from the appropriate distro package. Note that 2) also means that a pypi package developer must support the full range of different versions of these libraries which are provided by different linux distributions. Sage initially used the same principle in its design, but avoided the maintenance burden caused by trying to support all of these different library versions by providing its own libraries in SAGE_LOCAL. It has since gradually accepted more and more of the burden by allowing "system versions" of libraries wherever it seemed feasible. In the "Python ecosystem" these issues have been worked around in a different way in order to make *self-contained* binary wheels a possibility, and thereby accommodate *users*. This is usually done with delocate on macOS or with auditwheel on linux. Those tools embed all of the required libraries inside the python package and adjust the rpaths in the python extension modules to make them use the embedded libraries instead of libraries which might or might not be installed on the host system. In the case of linux this also means compiling the libraries against a version of glibc which is older than the version found on any of the linux distributions supported by the package. The job of building libraries against old versions of glibc is done by building them on a manylinux docker image, which can be done on a CI runner. (On macOS one can specify a minimum target version of macOS, usually 10.12 these days.) As a typical example (which recently created issues for Sage related to the jpeg library) you can look at the binary wheel for Pillow. If you unzip that wheel you will find a subdirectory named pillow.libs which contains: libbrotlicommon-3ecfe81c.so.1, libpng16-58efbb84.so.16.43.0, libbrotlidec-ba690955.so.1, libsharpyuv-20f78091.so.0.0.1, libfreetype-1f2cdfbb.so.6.20.1, libtiff-4da6744b.so.6.0.2, libharfbuzz-8b0b6641.so.0.60840.0,libwebp-850e2bec.so.7.1.8, libjpeg-f391b078.so.62.4.0, libwebpdemux-df9b36c7.so.2.0.14, liblcms2-e69eef39.so.2.0.16,libwebpmux-9fe05867.so.3.0.13, liblzma-13fa198c.so.5.4.5,libXau-154567c4.so.6.0.0, libopenjp2-05423b53.so , and libxcb-80c5a837.so.1.1.0. A linux packager would provide all of these by adding as dependencies all of the distro packages needed to provide these libraries. In the "Python ecosystem" these libraries are all built on manylinux docker images and then embedded in the binary wheel. I don't see how Sage will be able to join the Python ecosystem without addressing this. - Marc -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/e96d4c08-25d7-466b-b9fd-6e76b6b740e5n%40googlegroups.com.
