Part of the challenge here from my perspective is that even though I can envision what the end solution looks like, it's not clear how to actually get there. How do we designate someone to make decisions about what to include? How do we protect ourselves against special-case logic being kept in individual project repos as it is now, but instead pushed back up to the main repo? From our perspective (pipenv/various other projects I maintain at least), the implementation in pip is the minimum viable standard to which all other tooling _must_ conform, at least from a UX standpoint. Since package installation is step 0 of any developer's work, even minor workflow changes can be jarring and seriously disruptive, which is why we rely on the pip implementations a lot
It reminds me of the old adage about starting off with one problem... you don't want a new developer to have a clever idea (problem one) and go to set up their new environment and the first thing that happens is they experience some initial error before they've even managed to install a package (now they have two problems and one of them has nothing to do with their actual task). As a consequence even though there are other libraries that may provide some of this functionality, pip has the reference implementation and that contains some significant additional logic. I don't imagine that pip is going to simply adopt some new library without significant review... The substantial effort required to actually get people to review the code involved in standardizing the functionality people are 'borrowing' from pip is probably going to be a challenge, and that's before we ever consider that it will be difficult getting people to agree on what should be standardized and extracted. People are basically taking a path-of-least-resistance approach right now which means everyone is building their own tooling as needed or simply using the tooling in pip because they know that will always provide the same UX that the user is accustomed to (see my first point). I don't see anyone sinking significant effort into a shared library until we agree on what needs to be shared and can be confident that it will land in the various tools at some point. That said, there aren't too many of us that can put all the pieces together so it shouldn't be that difficult to outline what we need -- anyone who wants to have a look at what most of our projects are using from pip can shoot me a message off-list and I can send you a link to the repository Dan Ryan gh: @techalchemy // e: d...@danryan.co > -----Original Message----- > From: Paul Moore [mailto:p.f.mo...@gmail.com] > Sent: Tuesday, September 18, 2018 3:25 PM > To: Thomas Kluyver > Cc: Distutils > Subject: [Distutils] Re: disable building wheel for a package > > On Tue, 18 Sep 2018 at 19:54, Thomas Kluyver <tho...@kluyver.me.uk> > wrote: > > > > On Fri, Sep 14, 2018, at 4:26 PM, Paul Moore wrote: > > > I don't think it's "hard to predict". I *do* think it's badly documented/not > standardised. > > > ... > > > Better would be to have a supported library that exposes the logic pip > > > uses (or as I said above, the standard-defined logic) to determine > > > such paths. See https://github.com/pypa/pip/issues/5191 > > > > There is an official standard library API in the sysconfig module to find > installation locations: > > https://docs.python.org/3/library/sysconfig.html#installation-paths > > Yes, if it weren't for the stuff mentioned below, that would be the > one obvious way of doing things. However... > > > Unfortunately, distutils has a copy of this logic rather than using the > sysconfig module, from what I remember. Some Linux distros have patched > distutils to put installed files in different locations, but have not necessarily > patched sysconfig, presumably because they didn't think about it. > > (Technically, I think the history is that sysconfig was created by > pulling the distutils logic out into a standalone module, but no-one > modified distutils to use sysconfig - probably the usual "fear of > changing anything in distutils" problem :-() Even more unfortunately, > pip further layers its own logic on this > (https://github.com/pypa/pip/blob/master/src/pip/_internal/locations.py#L > 136) > - basically a "sledgehammer to crack a nut" approach of initialising a > distutils install command object, and then introspecting it to find > out where it'll install stuff - then hacking some special cases on top > of that. I assume it's doing that to patch over the incomplete Linux > distro patching. > > > Even if sysconfig were patched, distros may have a different location for > files installed by the distro package manager and files installed by other > means (Debian based distros use /usr and /usr/local for these). So there's no > one data directory where you can find all files related to importable > packages. (Of course, we advise against 'sudo pip install', but people still do it > anyway). > > Yeah. We don't really have any control or influence over what distros > do, we just end up reacting to their changes. That's why I think we > need a centralised supported library - unless the distros start > working better with the Python stdlib APIs, someone needs to collect > together all the workarounds and hacks needed to interact with them, > and that's better done in one place than duplicating the work in > multiple projects (and the fact that people have asked for pip's > locations module to be exposed as an API confirms to me that there > *are* multiple projects that need this). > > > This may be somewhat outdated - it's been a while since I looked into this, > but I don't think the relevant pieces are changing rapidly. My conclusion at > the time was that the only reliable way to have data files findable at runtime > was to put them inside the importable package. > > Agreed. There's more engagement with the distros these days (I think) > but it hasn't resulted in much change in how they handle things yet. > > Paul > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at https://mail.python.org/mm3/archives/list/distutils- > s...@python.org/message/SOPHWWWXEPV4MJARUETPNWO5BVQA5RYB/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/7V3YIOBKJFXKCPQOWC4NS4AGY2QIGUEQ/
