Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs
FWIW we are using quite a bit of the internal api. My plan was to go over this once we cut over to the new warehouse uris. Of note might be the fact that pip-tools is a core dependency we bundle in pipenv and the current maintainer is a pipenv maintainer as well. For our specific case we have made sizeable changes to the dependency resolution stack and bundling allows us to patch freely. I don’t know that we are a good example though, we are doing significantly more with pip internals than the average project -dan Dan Ryan // pipenv maintainer gh: @techalchemy > On Apr 14, 2018, at 11:24 PM, Nick Coghlan wrote: > >> On 15 April 2018 at 07:31, Donald Stufft wrote: >> >> On Apr 14, 2018, at 4:57 PM, Matthew Brett wrote: >> >> Is the suggestion to use the `_internal` import, or carry a copy of >> the pep425tags code myself, that I have to keep up to date with the >> internal pip copy? That would also involve me copying the `glibc` >> part of the code. I see that the `wheel` package has an old copy of >> that code too, which doesn't deal with manylinux wheels.You >> probably saw that `pip-tools` ended up vendoring the whole of pip9 >> [1]. >> >> The best solution is to figure out what APIs people need, and either add >> them to packaging and have pip consume that as well as anyone else, or make >> a new library for the same. >> >> If that’s unacceptable, vendoring or version pinning is probably the best >> option. > > I think there are going to be at least two steps involved for most > projects affected by the API change: > > 1. The quick fix to add pip 10 compatibility (which is likely a matter > of "copy the code you need into the project that needs it") > 2. The technical debt reduction to reduce code duplication (which is > likely a matter of "add the required APIs to the 'packaging' project") > > Step 2 is the step that the pip internals refactoring is designed to > encourage, as we believe a lot of tool developers were just using > pip's internal APIs rather than filing RFEs and submitting PRs to help > guide the evolution of the stable APIs in packaging in a use-case > driven manner. > > FWIW, `pipenv`'s currently still on "Step 1" at the moment (and has a > lot of internal refactoring of its own ahead of it before it will > really move on to step 2). > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs
On 15 April 2018 at 07:31, Donald Stufft wrote: > > On Apr 14, 2018, at 4:57 PM, Matthew Brett wrote: > > Is the suggestion to use the `_internal` import, or carry a copy of > the pep425tags code myself, that I have to keep up to date with the > internal pip copy? That would also involve me copying the `glibc` > part of the code. I see that the `wheel` package has an old copy of > that code too, which doesn't deal with manylinux wheels.You > probably saw that `pip-tools` ended up vendoring the whole of pip9 > [1]. > > The best solution is to figure out what APIs people need, and either add > them to packaging and have pip consume that as well as anyone else, or make > a new library for the same. > > If that’s unacceptable, vendoring or version pinning is probably the best > option. I think there are going to be at least two steps involved for most projects affected by the API change: 1. The quick fix to add pip 10 compatibility (which is likely a matter of "copy the code you need into the project that needs it") 2. The technical debt reduction to reduce code duplication (which is likely a matter of "add the required APIs to the 'packaging' project") Step 2 is the step that the pip internals refactoring is designed to encourage, as we believe a lot of tool developers were just using pip's internal APIs rather than filing RFEs and submitting PRs to help guide the evolution of the stable APIs in packaging in a use-case driven manner. FWIW, `pipenv`'s currently still on "Step 1" at the moment (and has a lot of internal refactoring of its own ahead of it before it will really move on to step 2). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs
> On Apr 14, 2018, at 4:57 PM, Matthew Brett wrote: > > Is the suggestion to use the `_internal` import, or carry a copy of > the pep425tags code myself, that I have to keep up to date with the > internal pip copy? That would also involve me copying the `glibc` > part of the code. I see that the `wheel` package has an old copy of > that code too, which doesn't deal with manylinux wheels.You > probably saw that `pip-tools` ended up vendoring the whole of pip9 > [1]. The best solution is to figure out what APIs people need, and either add them to packaging and have pip consume that as well as anyone else, or make a new library for the same. If that’s unacceptable, vendoring or version pinning is probably the best option.
Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs
Hi, On Sun, Oct 22, 2017 at 8:52 AM, Elvis Stansvik wrote: > 2017-10-21 14:34 GMT+02:00 Paul Moore : >> On 21 October 2017 at 12:15, Nick Coghlan wrote: >>> (Note: this is entirely speculative, and I have no idea how hard it would >>> be, so please read it as the question it's intended to be) >> >> No problem - I don't know myself how hard some of this would be, either ;-) >> >>> Do you know if there any key APIs (like installation) that could be turned >>> into wrappers around pip CLI calls in order to mitigate some of the impact? >> >> The obvious one is pip.main(). That's the one a lot of people use, but >> it's easily replaceable by a simple subprocess call. That's actually >> one of the reasons this was so frustrating to us - the bug reports we >> got were often from people doing things they didn't need to, that they >> could handle trivially via a supported approach. >> >> Otherwise, no. We've had little or no feedback on the tracker from >> people using more complex internals, so our working assumption has >> been there's very little that can't be handled via the CLI or existing >> packages. Feedback so far from this mail hints that maybe we were >> wrong, but it's still hard to know if it's one or two key projects, or >> a whole range of people that we've yet to hear from. I'm pretty sure, >> for example, that pipenv uses internals, either directly or via one of >> their dependencies, but we've not seen any feedback from them yet. > > Another one that immediately comes to mind is pip-tools [1], which I > think is quite widely used. > > But I just checked, and they filed a "check out how to deal with pip > 10" issue two days ago [2] (I'm guessing in response to this thread). Now pip 10 is out, of course, I discover that I've lost the implementation of `get_supported` in pip.pep425tags. It's my fault for not remembering I had used it. Is the suggestion to use the `_internal` import, or carry a copy of the pep425tags code myself, that I have to keep up to date with the internal pip copy? That would also involve me copying the `glibc` part of the code. I see that the `wheel` package has an old copy of that code too, which doesn't deal with manylinux wheels.You probably saw that `pip-tools` ended up vendoring the whole of pip9 [1]. Cheers, Matthew [1] https://github.com/jazzband/pip-tools/tree/master/piptools/_vendored/pip
