2017-10-21 14:34 GMT+02:00 Paul Moore <p.f.mo...@gmail.com>: > On 21 October 2017 at 12:15, Nick Coghlan <ncogh...@gmail.com> wrote: >> (Note: this is entirely speculative, and I have no idea how hard it would >> be, so please read it as the question it's intended to be) > > No problem - I don't know myself how hard some of this would be, either ;-) > >> Do you know if there any key APIs (like installation) that could be turned >> into wrappers around pip CLI calls in order to mitigate some of the impact? > > The obvious one is pip.main(). That's the one a lot of people use, but > it's easily replaceable by a simple subprocess call. That's actually > one of the reasons this was so frustrating to us - the bug reports we > got were often from people doing things they didn't need to, that they > could handle trivially via a supported approach. > > Otherwise, no. We've had little or no feedback on the tracker from > people using more complex internals, so our working assumption has > been there's very little that can't be handled via the CLI or existing > packages. Feedback so far from this mail hints that maybe we were > wrong, but it's still hard to know if it's one or two key projects, or > a whole range of people that we've yet to hear from. I'm pretty sure, > for example, that pipenv uses internals, either directly or via one of > their dependencies, but we've not seen any feedback from them yet.
Another one that immediately comes to mind is pip-tools [1], which I think is quite widely used. But I just checked, and they filed a "check out how to deal with pip 10" issue two days ago [2] (I'm guessing in response to this thread). Elvis [1] https://github.com/jazzband/pip-tools [2] https://github.com/jazzband/pip-tools/issues/580 > >> The reason I ask is because it's unlikely anyone else is going to understand >> how to emulate the previous functionality better than the pip devs would, >> and if there's an API for those particular invocations, than they can be >> covered directly by pip's test suite. > > Understood. We understand *how*, but don't know what is needed. One of > the points of all this is to tease out such requirements. We'd hope to > get them addressed by including them into *other* packages like > distlib or packaging, but that's mostly just a matter of where the API > goes, not what is needed. It does help us avoid tying fixes to pip's > release cycle, though. > > It's also worth noting that the pip devs are possibly way too deep > into how pip does things, rather than what the standards say. Getting > others to implement libraries based on the published standards would > help immensely to ensure that we're not falling into the > "implementation defined" trap of the community being stuck with having > to use pip because no-one else knows how to do what pip does. > >> Plus if there are previous API capabilities that *can't* currently be >> emulated via the CLI, then the pip devs are the folks in the best position >> to add the necessary CLI enhancements (such as the ones Noah asked about for >> doing a more selective `pip list`). > > Oh, sure - apart from the aforementioned "pip.main()", most > capabilities of the internal API are *not* easily emulated by the CLI. > > But 3rd party libraries are just as much an option. Remember, the > issue here isn't so much about designing an API as about exposing (and > therefore locking in stone) internal implementation details of how pip > does things. And the pip devs are arguably in the *worst* position to > handle that option, precisely because they know so much about how pip > does things. > >> If that's an approach you might be amenable to, then a 10.0 pre-release >> could be a good time to solicit PRs from folks that were using particular >> APIs and would be prepared to invest time in defining comparable CLI >> wrappers for them. > > Well, I get your point here, but the implication of this is that we > have to design and build an API before we can release pip 10. Calling > it CLI wrappers (and implementing it that way) doesn't do anything to > ease the work needed on design, or the maintenance burden of providing > stability. We're already getting a lot of pressure to release pip 10, > and trying to do that would push it way further off. > > To be blunt, no-one on the pip team is really interested in trying to > provide or support a stable API at this point in time. We have our > hands full, as everyone is aware, and this is way down all our > priorities. Community-submitted PRs would help, but there's still work > in agreeing a design, reviewing those PRs, and maintaining them > (exposing details from the internal APIs limits how much we can change > those internals later, which is something we have to consider). > > What would be ideal would be for the community to build > standards-based libraries that satisfied the needs that people > currently handle by using pip internals. Heck, we could even consider > vendoring such libraries inside of pip and saving ourselves a bunch of > complexity ;-) The packaging, pkg_resources and distlib libraries are > examples of this happening. But it doesn't seem to be a popular route, > unfortunately. And those particular libraries are all maintained by > PyPA members, so don't really count as examples of the community > getting involved... > > The most likely alternative solution would be to revert the internal > API moves. If that happened, I'd still prefer to lose pip.main, as > that's the major pain point for us and the easiest for users to solve, > but we could put the rest back. But it would be naive to assume that > as a result people like Noah would be unaffected - they'd actually be > worse off, as the internal APIs they use would probably break anyway > (the installer/resolver stuff has been heavily modified for pip 10) > but not all would do so in obvious ways. > > Beyond this, I don't know. > > Paul > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig