On 21.01.2016 17:13, Nathaniel Smith wrote: > On Jan 21, 2016 2:07 AM, "M.-A. Lemburg" <m...@egenix.com> wrote: >> >> On 21.01.2016 10:31, Nick Coghlan wrote: >>> On 21 January 2016 at 19:03, M.-A. Lemburg <m...@egenix.com> wrote: >>>> By using the version based approach, we'd not run into this >>>> problem and gain a lot more. >>> >>> I think it's better to start with a small core that we *know* works, >>> then expand later, rather than trying to make the first iteration too >>> wide. The "manylinux1" tag itself is versioned (hence the "1" at the >>> end), so "manylinux2" may simply have *more* libraries defined, rather >>> than newer ones. >> >> My argument is that the file based approach taken by the PEP >> is too limiting to actually make things work for a large >> set of Python packages. >> >> It will basically only work for packages that do not interface >> to other external libraries (except for the few cases listed in >> the PEP, e.g. X11, GL, which aren't always installed or >> available either). > > The list of allowed libraries is exactly the same list of libraries as are > required by the Anaconda python distribution, so we *know* that it works > for about a hundred different python packages, including lots of tricky > ones (the whole scientific stack), and had been tested by tens or hundreds > of thousands of users. (I posted a link above to some actual working, > compliant pyside and numpy packages, which we picked for testing because > they're particular interesting/difficult packages that need to interface to > external libraries.) Yes, various extra tricks are needed to get things > working on top of this base, including strategies for shipping libraries > that are not in the baseline set, but these are problems that can be solved > on a project-by-project basis, and don't need a PEP.
And that's the problem: The set is limited to the needs of the scientific community and there to the users of one or two distributions only. It doesn't address needs of others that e.g. use Qt or GTK as basis for GUIs, people using OpenSSL for networking, people using ImageMagick for processing images, or type libs for type setting, or sound libs for doing sound processing, codec libs for video processing, etc. The idea to include the needed share libs in the wheel goes completely against the idea of relying on a system vendor to provide updates and security fixes. In some cases, this may be reasonable, but as design approach, it's not a good idea. > [...] >> Another detail we have found when external dependencies >> is that some platforms use different names for the libraries, >> e.g. RedHat has a tendency to use non-standard OpenSSL >> library names (/lib64/libssl.so.10 instead of the more >> common libssl.so.1.0.0). >> >>> The key is that we only have one chance to make a good first >>> impression with binary Linux wheel support on PyPI, and we want that >>> to be positive for everyone: >> >> Sure, but if we get the concept wrong, it'll be difficult >> to switch later on and since there will always be libs not in the >> set, we'll need to address this in some way. > > There's no lock-in here -- any alternative approach just needs its own > platform tag. Pypi and pip can easily support multiple such tags at the > same time, if more sophisticated proposals arise in the future. In the mean > time, for packagers targeting manylinux is at least as easy as targeting > windows (which also provides very few libraries "for free"). Yes, there's no lock-in, there's lock-out :-) We'd simply not allow people who have other requirements to upload Linux wheels to PyPI and that's not really acceptable. Right now, no one can upload Linux wheels, so that a fair setup. Using an approach where every single group first has to write a PEP, get it accepted and have PyPI and pip patched before they can upload wheels to PyPI does not read like a community friendly approach. I also can't imagine that we really want proliferation of "linux" tags for various purposes or even for various versions of library catalogs. What we need is a system that provides a few dimensions for various system specific differences (e.g. bitness, architecture) and a recommendation for library versions of a few very basic libraries to use when compiling for those systems. I believe the PEP is almost there, it just needs to use a slightly different concept, since limiting the set of allowed libraries does not provide the basis of an open system which PyPI/pip/wheels need to be. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jan 21 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
