On Tue, Sep 25, 2018 at 6:46 AM, Dan Ryan <d...@danryan.co> wrote: > As far as the pipenv roadmap, we are rewriting a good chunk of it but it > essentially the atomic things we do or would like to do with regards to > shared functionality at a high level: > > - parse user input/requirements > - retrieve package metadata > - resolve dependencies (not in pip right now obviously) — may involve > building wheels/sdists > - download / build packages > - install packages (into a virtualenv sometimes) > - uninstall packages (from a virtualenv sometimes) > - list packages etc (in a virtualenv sometimes) > > We know how these are accomplished in pip and we know how we are handling > them both in pipenv and in our alternative implementations, which I believe > still have some dependency on pip. For us the roadmap is basically to figure > out > > - where do the abstractions live > - how do we start > - ???
Thanks, Dan. I think Donald answered this in an email much earlier in this thread -- the one he sent on Sept. 19 beginning: """ My general recommendation if you want a Python implementation/interface for something pip does, is: - Open an issue on the pip repository to document your intent and to make sure that there is nobody there who is against having that functionality split out. ... ... """ In addition, to ensure you're taking the approach of splitting out parts of pip instead of creating yet another implementation separate from pip (leading to duplicated work as I said), I would recommend also doing the following: File PR's in pip to refactor out various functionality that makes sense into stand-alone bits with their own tests (e.g. unit tests, etc). This would live in pip but be disentangled from the rest of pip from a code dependency perspective. It could even be in a separate "future libraries" subdirectory if the pip committers are okay with it. (Optional: since you're already vendoring pip, you could perhaps use this factored out code at your own risk, like you're already doing with some of pip's code base anyway.) The above is something I started doing at a small scale -- not to break it out into a library, but for its own sake because it makes it easier to maintain and work on pip as a code base, fixing bugs, etc. Some of the nice things about this approach are that-- 1. It can be done even before the steps that Donald outlined (like I'm doing), e.g. so that you don't need to wait for a PEP to be finished. And it could even help in the later creation of such PEPs. 2. By doing so, as a side effect of the PR process, it automatically causes communication with pip maintainers and works towards the "splitting out parts of pip" goal -- all without duplicating any work because both pip and pipenv developers are aware and collaborating on these changes. 3. It is incremental as you go, ensuring that it is compatible with pip and that pip is using it, etc. 4. It improves pip in the process (as well as pipenv, if you're still vendoring in some form). In other words, the work is useful from the beginning, whether or not it's ever broken out into a separately maintained library. 5. It is forward progress towards the goal of a separate library. --Chris > > Dan Ryan // pipenv maintainer > gh: @techalchemy > >> On Sep 25, 2018, at 8:11 AM, Paul Moore <p.f.mo...@gmail.com> wrote: >> >>> On Tue, 25 Sep 2018 at 12:53, Chris Jerdonek <chris.jerdo...@gmail.com> >>> wrote: >>> >>>> On Tue, Sep 25, 2018 at 3:47 AM, Tzu-ping Chung <uranu...@gmail.com> wrote: >>>> We are using pip internals for things pip wasn’t implemented for. >>>> Specifically, >>>> Pipenv uses pip’s package-fetching functions to implement its >>>> platform-agnostic >>>> resolver. pip does not have this, so there’s no functional overlap here. >>>> Those >>>> utilities are used to build something that doesn’t exist in pip, so >>>> there’s no >>>> duplicated efforts. >>>> >>>> My recent focus on making sense of packaging implementations and splitting >>>> out >>>> parts of pip is exactly to prevent potential duplicated efforts. If we >>>> can’t use pip >>>> internals, let’s make things we want to use *not* internal! >>> >>> I don't see this practice of "splitting out parts of pip" actually >>> being followed so far though, and I want to tell you how it's leading >>> to duplicated efforts right now -- even for the PR's I happen to be >>> working on today. >> >> [...] >> >>> So by creating implementations of functions separate from pip instead >>> of splitting them out of pip, it's leading to duplicated work for >>> people working on pip's code base. A process of splitting something >>> out of pip would I think involve a process more like what I'm >>> following -- namely to refactor functionality in pip *first* so it can >>> be used separately, and then pull that out so you only have one >>> implementation at any point in time. Otherwise, you wind up creating >>> two versions of similar functionality that not only need to be created >>> twice, but also later reconciled if the plan is merge them back >>> together. >> >> This is a very good point, and thanks for the concrete example. For >> the record, I'm also a little annoyed that pipenv have been doing all >> this work in isolation. There's been very little communication between >> the pip and pipenv projects, and I think that's contributed to the >> problem. I, for one, wasn't even aware for a long time that they were >> embedding their own copy of pip. When we made pip's internals in >> private in pip 10, we got essentially no feedback from anyone saying >> that it would be a problem, and finding out that it had a sufficiently >> major impact on pipenv that they had to embed the old version of pip >> *after* the release was (to put it politely) suboptimal :-( >> >> Having said all this, this thread is the start of an attempt to work >> better together and I think we should be glad that it's happening and >> try to make it work. But I don't think we've got off to a particularly >> good start and it will need work. In particular, I'd like to see an >> attempt from *both* projects to clarify our goals - I've been speaking >> solely for myself so far, it would be good to get the other pip >> developers' views and set out a sort of roadmap, and for pipenv to do >> the same. >> >> Paul >> -- >> Distutils-SIG mailing list -- distutils-sig@python.org >> To unsubscribe send an email to distutils-sig-le...@python.org >> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ >> Message archived at >> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/OGXUNDRBLTNPOWLLFBVF45SW4SRGYSPP/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/BZLWZ7QXKJOILU2RI36CANKCPJHWPDWG/
