[Distutils] Re: Cannot upload distribution archives using twine using functioning username and login for pypi.org
According to the documentation[1], TestPyPI is a complete independent instance from PyPI, and they don’t share databases at all. You need to register a separate account there. [1]: https://packaging.python.org/guides/using-testpypi/ -- Tzu-ping Chung (@uranusjr) uranu...@gmail.com Sent from my iPhone > On 16 Aug 2018, at 23:55, t...@tombaker.org wrote: > > When I follow the instructions at > https://packaging.python.org/tutorials/packaging-projects/ > to upload my distribution archives with > >$ twine upload --repository-url https://test.pypi.org/legacy/ dist/*` > > I get prompted for my username and password, but the upload aborts with > `HTTPError: 403 Client Error` (see below) even though I can successfully log > into https://pypi.org using the same username and password. > > Can anyone advise? > > Tom > > ``` > Uploading distributions to https://test.pypi.org/legacy/ > Enter your username: tombaker > Enter your password: > Uploading mklists-0.1.1-py3-none-any.whl > 100%|*| > 10.6k/10.6k [00:00<00:00, 15.8kB/s] > HTTPError: 403 Client Error: Invalid or non-existent authentication > information. for url: https://test.pypi.org/legacy/ > ``` -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/RUD4LGHUEHCBERTHM5SE356GTPNOSKTX/
[Distutils] Re: Clarification on string arguments in PEP 517 hooks
On Mon, 20 Aug 2018 at 18:54, Thomas Kluyver wrote: > > On Mon, Aug 20, 2018, at 2:52 PM, Paul Moore wrote: > > The various hooks take directory paths as arguments, and typically > > return a filename (e.g., build_wheel). The returned filename is always > > explicitly noted as being *a unicode string*. However, argumnents > > (metadata_directory in build_wheel/prepare_metadata_for_build_wheel) > > are *not* explicitly mandated to be Unicode. > > > > My assumption is that the intent is that *all* strings, whether > > arguments or return values, must be Unicode. > > I'd say we should handle all paths the same way, i.e. unicode. If it's a > serious obstacle, we could alter the specification. But I would be surprised > if the required changes to the setuptools backend are so tricky that they > warrant a workaround in the specification. The required change to setuptools is pretty simple. What's tricky is getting pip (in this case the test suite) to pick up an unreleased version of setuptools so that I can continue work on pip's PEP 517 support. That's why I wanted to confirm that my assumption was correct, so I could get a change into setuptools quickly. Paul Paul -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/MHYCKEWIEU6GTQKHW5AKWEX5QZMAGZRC/
[Distutils] Re: pipenv and pip
On Mon, Aug 20, 2018 at 2:52 AM Wes Turner wrote: > What stable API would be worth maintaining in pip for others to use? > Just to be clear, nothing in my comments was meant to suggest maintaining a stable API. There are other kinds of things pip could do to make it easier for pipenv that don’t involve that. —Chris > "[Distutils] Announcement: Pip 10 is coming, and will move all internal > APIs" > https://groups.google.com/forum/m/#!topic/pypa-dev/JVTfS6ZdAuM > > > > On Monday, August 20, 2018, Chris Jerdonek > wrote: > >> Thanks. Is the state of affairs as you described them what you're >> planning for the future as well, or do you anticipate any changes >> worthy of note? >> >> Also, are any of the bugs filed in pipenv's tracker due to bugs or >> rough spots in pip -- is there a way to find those, like by using a >> label? It would be good to be able to know about those so pip can >> improve and become more useful. It doesn't seem like any bugs have >> been filed in pip's tracker in the past year by any of pipenv's top >> contributors. That seems a bit surprising to me given pipenv's heavy >> reliance on pip (together with the fact that I know pip has its share >> of issues), or is there another way you have of communicating >> regarding things that interconnect with pip? > > > Label ideas? > - 'stable-api' > - > > Who is offering to maintain a stable API in/with/for pip and the Python > community ad infinitum? > > >> Thanks, >> --Chris >> >> >> >> On Mon, Aug 20, 2018 at 12:51 AM, Dan Ryan wrote: >> > Sure I can grab that— we patch pip because we use some internals to >> handle resolution and we have some bugs around that currently. They aren’t >> upstreamed because they aren’t actually present in pip, only in pipenv. >> Pipenv crosses back and forth across the virtualenv boundary during the >> process. Pipenv relies on piptools and vendors a patched version of pip to >> ensure consistency as well as to provide a few hacks around querying the >> index. We do have a bit of reimplementation around some kinds of logic, >> with the largest overlap being in parsing of requirements. >> > >> > As we handle some resolution, which isn’t really something pip does, >> there is no cli interface to achieve this. I maintain a library (as of last >> week) which provides compatibility shims between pip versions 8-current. It >> is a good idea to use the cli, but we already spend enough resources >> forking subprocesses into the background that it is a lot more efficient to >> use the internals, which I track quite closely. The preference toward cli >> interaction is largely to allow internal api breakage which we don’t mind. > > > What is the URL of this library of which you are speaking? > > >> > >> > For the most part, we have open channels of communication as necessary. >> We rely as heavily as we can on pip, packaging, and setuptools to connect >> the dots, retrieve package info, etc. > > > An issue label and something like a PEP would likely survive the ravages > of 10 years of tools tooling around with community packaging commitments. > > >> > >> > Dan Ryan // pipenv maintainer >> > gh: @techalchemy >> > >> >> On Aug 20, 2018, at 2:41 AM, Chris Jerdonek >> wrote: >> >> >> >> Hi, >> >> >> >> Can someone explain to me the relationship between pipenv and pip, >> >> from the perspective of pipenv's maintainers? >> >> >> >> For example, does pipenv currently reimplement anything that pip tries >> >> to do, or does it simply call out to pip through the CLI or through >> >> its internal API's? Does it have any preferences or future plans in >> >> this regard? How about upstreaming to pip fixes or things that would >> >> help pipenv? >> >> >> >> I've been contributing to pip more lately, and I had a look at >> >> pipenv's repository for the first time today. >> >> https://github.com/pypa/pipenv >> >> >> >> Given that pip's code was recently made internal, I was a bit >> >> surprised to see that pipenv vendors and patches pip: >> >> https://github.com/pypa/pipenv/tree/master/pipenv/patched/notpip >> >> Before I had always assumed that pipenv used pip's CLI (because that's >> >> what pip says you should do). >> >> >> >> I also noticed that some bugs in pipenv's tracker seem closely related >> >> to pip's behavior, but I don't recall seeing any bugs or PR's in pip's >> >> tracker reported from pipenv maintainers. >> >> >> >> Without knowing a whole lot more than what I've stated, one concern I >> >> have is around fragmentation, duplication of work, and repeating >> >> mistakes (or introducing new ones) if a lot of work is going into >> >> pipenv without coordinating with pip. Is this in any way similar to >> >> the beginning of what happened with distutils, setuptools, and >> >> distribute that we are still recovering from? >> >> >> >> --Chris >> >> -- >> >> Distutils-SIG mailing list -- distutils-sig@python.org >> >> To unsubscribe send an email to distutils-sig-le...@python.org >> >>
[Distutils] Re: Clarification on string arguments in PEP 517 hooks
On Mon, Aug 20, 2018, at 2:52 PM, Paul Moore wrote: > The various hooks take directory paths as arguments, and typically > return a filename (e.g., build_wheel). The returned filename is always > explicitly noted as being *a unicode string*. However, argumnents > (metadata_directory in build_wheel/prepare_metadata_for_build_wheel) > are *not* explicitly mandated to be Unicode. > > My assumption is that the intent is that *all* strings, whether > arguments or return values, must be Unicode. I'd say we should handle all paths the same way, i.e. unicode. If it's a serious obstacle, we could alter the specification. But I would be surprised if the required changes to the setuptools backend are so tricky that they warrant a workaround in the specification. Thomas -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/Y4OHE2D24DHVCBZVJZ5OGXOFGBZKA3YQ/
[Distutils] Re: pipenv and pip
The truth is that it’s basically impossible to gauge bugs in pip vs bugs in our patches to it which are often a lot more likely — reproductions of edge cases can be impossible but there are specific things I know we broke (like parsing certain kinds of extras, previously) — mostly bugs land in pips issue tracker before we report them or we will direct people there. We have like 2 active maintainers and we are maintaining like 15 pipenv related projects so we normally just point people at pip rather than file an issue. I am usually on irc as well if needed, and I often ask for clarification there Dan Ryan // pipenv maintainer gh: @techalchemy > On Aug 20, 2018, at 4:32 AM, Chris Jerdonek wrote: > > Thanks. Is the state of affairs as you described them what you're > planning for the future as well, or do you anticipate any changes > worthy of note? > > Also, are any of the bugs filed in pipenv's tracker due to bugs or > rough spots in pip -- is there a way to find those, like by using a > label? It would be good to be able to know about those so pip can > improve and become more useful. It doesn't seem like any bugs have > been filed in pip's tracker in the past year by any of pipenv's top > contributors. That seems a bit surprising to me given pipenv's heavy > reliance on pip (together with the fact that I know pip has its share > of issues), or is there another way you have of communicating > regarding things that interconnect with pip? > > Thanks, > --Chris > > > >> On Mon, Aug 20, 2018 at 12:51 AM, Dan Ryan wrote: >> Sure I can grab that— we patch pip because we use some internals to handle >> resolution and we have some bugs around that currently. They aren’t >> upstreamed because they aren’t actually present in pip, only in pipenv. >> Pipenv crosses back and forth across the virtualenv boundary during the >> process. Pipenv relies on piptools and vendors a patched version of pip to >> ensure consistency as well as to provide a few hacks around querying the >> index. We do have a bit of reimplementation around some kinds of logic, >> with the largest overlap being in parsing of requirements. >> >> As we handle some resolution, which isn’t really something pip does, there >> is no cli interface to achieve this. I maintain a library (as of last week) >> which provides compatibility shims between pip versions 8-current. It is a >> good idea to use the cli, but we already spend enough resources forking >> subprocesses into the background that it is a lot more efficient to use the >> internals, which I track quite closely. The preference toward cli >> interaction is largely to allow internal api breakage which we don’t mind. >> >> For the most part, we have open channels of communication as necessary. We >> rely as heavily as we can on pip, packaging, and setuptools to connect the >> dots, retrieve package info, etc. >> >> Dan Ryan // pipenv maintainer >> gh: @techalchemy >> >>> On Aug 20, 2018, at 2:41 AM, Chris Jerdonek >>> wrote: >>> >>> Hi, >>> >>> Can someone explain to me the relationship between pipenv and pip, >>> from the perspective of pipenv's maintainers? >>> >>> For example, does pipenv currently reimplement anything that pip tries >>> to do, or does it simply call out to pip through the CLI or through >>> its internal API's? Does it have any preferences or future plans in >>> this regard? How about upstreaming to pip fixes or things that would >>> help pipenv? >>> >>> I've been contributing to pip more lately, and I had a look at >>> pipenv's repository for the first time today. >>> https://github.com/pypa/pipenv >>> >>> Given that pip's code was recently made internal, I was a bit >>> surprised to see that pipenv vendors and patches pip: >>> https://github.com/pypa/pipenv/tree/master/pipenv/patched/notpip >>> Before I had always assumed that pipenv used pip's CLI (because that's >>> what pip says you should do). >>> >>> I also noticed that some bugs in pipenv's tracker seem closely related >>> to pip's behavior, but I don't recall seeing any bugs or PR's in pip's >>> tracker reported from pipenv maintainers. >>> >>> Without knowing a whole lot more than what I've stated, one concern I >>> have is around fragmentation, duplication of work, and repeating >>> mistakes (or introducing new ones) if a lot of work is going into >>> pipenv without coordinating with pip. Is this in any way similar to >>> the beginning of what happened with distutils, setuptools, and >>> distribute that we are still recovering from? >>> >>> --Chris >>> -- >>> Distutils-SIG mailing list -- distutils-sig@python.org >>> To unsubscribe send an email to distutils-sig-le...@python.org >>> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ >>> Message archived at >>> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email
[Distutils] Re: pipenv and pip
I actually maintain a separate library for parsing requirements which relies mainly on packaging and which provides a backing implementation for moving between requirements files and Pipfile format. It relies on some pip internals (InstallRequirement specifically) for avoiding rework. As of recently it also includes some resolution functionality. I didn’t include a link to the pip shims library because Paul et. al. aren’t happy we are using this stuff to begin with, which I get. However it is also a good representation of things we have needed in multiple projects associated with pipenv (I reimplemented them in piptools and they were picked up in at least one additional project as well). In pipenvs case specifically we wind up installing packages from user input or from a structured file, in either case we want it to succeed. It would be kind of silly to start from point 0 and pretend there is no code behind pip as we construct dependency graphs. We run into some risk of rework on the resolver front. We just finished a stateful, forward looking, backtracking resolver (we checked in with Pradyun but he is quite busy). Dan Ryan // pipenv maintainer gh: @techalchemy > On Aug 20, 2018, at 9:16 AM, Wayne Werner wrote: > > > >> On Mon, Aug 20, 2018, 7:45 AM Paul Moore wrote: >> On Mon, 20 Aug 2018 at 13:21, Wes Turner wrote: >> > >> > Something as simple as reading a requirements.txt file into JSON must >> > either reimplement or wrongly import from pip._internal. >> >> Or copy pip's code and maintain it locally... >> >> > Anyways, >> > Tool authors reimplementing in particular the requirements.txt >> > functionality shouldn't be necessary. >> >> Agreed. Maybe someone should write a package to handle >> requirements.txt reading API. > > > Or as pipenv does, abort that issue entirely in favor of a toml file because > that's the future, right? :) > > -W > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/FJFQNINEKKUNFN5TGYROYFS7LPBF6XN5/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/5IWQZ4HAFKPMOYWDQYEPEYN72OGHRVBR/
[Distutils] Clarification on string arguments in PEP 517 hooks
I'm in the process of implementing PEP 517 for pip, and I've hit a question. I'm pretty sure I know the answer, but I want to be clear, as whatever the answer is will require some fixing up. The various hooks take directory paths as arguments, and typically return a filename (e.g., build_wheel). The returned filename is always explicitly noted as being *a unicode string*. However, argumnents (metadata_directory in build_wheel/prepare_metadata_for_build_wheel) are *not* explicitly mandated to be Unicode. My assumption is that the intent is that *all* strings, whether arguments or return values, must be Unicode. The reason that I've hit this is that the current setuptools PEP 517 backend passes the metadata_directory direct to distutils which, on Python 2, checks for a (non-Unicode) string type and errors if Unicode is supplied. Fixing that is going to be annoyingly tricky, so I want to be sure that's the requirement before I start making fixes to setuptools. Paul -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/WW44ZRHKD3VDDEWODBPRPZWWENVJNGW5/
[Distutils] Re: pipenv and pip
On Mon, Aug 20, 2018, 7:45 AM Paul Moore wrote: > On Mon, 20 Aug 2018 at 13:21, Wes Turner wrote: > > > > Something as simple as reading a requirements.txt file into JSON must > either reimplement or wrongly import from pip._internal. > > Or copy pip's code and maintain it locally... > > > Anyways, > > Tool authors reimplementing in particular the requirements.txt > functionality shouldn't be necessary. > > Agreed. Maybe someone should write a package to handle > requirements.txt reading API. > Or as pipenv does, abort that issue entirely in favor of a toml file because that's the future, right? :) -W > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/FJFQNINEKKUNFN5TGYROYFS7LPBF6XN5/
[Distutils] Re: pipenv and pip
On Mon, 20 Aug 2018 at 13:21, Wes Turner wrote: > > Something as simple as reading a requirements.txt file into JSON must either > reimplement or wrongly import from pip._internal. Or copy pip's code and maintain it locally... > Anyways, > Tool authors reimplementing in particular the requirements.txt functionality > shouldn't be necessary. Agreed. Maybe someone should write a package to handle requirements.txt reading API. They could copy the code from pip with our blessing, and we'd happily vendor them and use their code. As you can see, it's very easy to suggest that someone *else* should do something. The problem is finding someone who has both the interest in solving the problem, and the time to do so. The "why doesn't pip have a stable API" question is very much about the fact that we don't have the *resources* to support one, not about any dislike of programmatic APIs... Paul -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/6NDLQI4NJ2YF2HDTMAKVFFA2OTMLTBRD/
[Distutils] Re: pipenv and pip
On Mon, 20 Aug 2018 at 12:25, Wes Turner wrote: > > On Monday, August 20, 2018, Paul Moore wrote: >> I know "security by obscurity" doesn't work, but I'm happier if >> details of this library *aren't* widely known - it's not something I'd >> want to encourage people using, nor is it supported by pip, as it's >> basically a direct interface into pip's internal functions, papering >> over the name changes that we did in pip 10 specifically to dissuade >> people from doing this. > > > If someone was committing to identifying useful API methods, parameters, and > return values; > writing a ~PEP; > implementing said API; > and maintaining backwards compatible shims for some reason; > would something like `pip.api` be an appropriate namespace? > (now that we're on version 18 with a faster release cycle)? I'm not quite sure I know what you mean here. The key point is that pip 18.0 might have an internal function pip._internal.xxx, and in pip 18.1 there's no such function, and the functionality doesn't even exist any more. How would a 3rd party project maintain backwards compatible shims in the face of that? Agreed it's not likely in practice - but we're not going to guarantee it. To be honest I don't see the point of discussing pip's internal API. It's just that - internal. I'd rather discuss useful (general) packaging libraries, that tools can build on - pip can vendor those and act as (just) another consumer, rather than getting into debates about support and internal APIs. Paul -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/7CCBE53AM72JYQVEW3PH7ODVRFZV4WXA/
[Distutils] Re: pipenv and pip
On Monday, August 20, 2018, Paul Moore wrote: > On Mon, 20 Aug 2018 at 10:54, Wes Turner wrote: > > > > What stable API would be worth maintaining in pip for others to use? > > That's probably the sort of question that can only be usefully > answered by projects like pipenv identifying the functionality they > need and proposing something. Which is of course one of the reasons we > (the pip devs) advise against "just using pip's internals", because it > means we never get that information in any useful form. > > > Who is offering to maintain a stable API in/with/for pip and the Python > community ad infinitum? > > That's the crux of the problem - basically the answer is "no-one". > What we advocate is for generally useful functionality to be split out > into standalone libraries, and then pip, as well as other consumers, > can use those libraries. We already have that with the packaging > library and the script wrappers (part of distlib). The new resolver is > being developed as a standalone library (zazo) as is the PEP 517 hook > wrapper functionality (pep517). There's no reason this model couldn't > work in other areas. (But even then, the question "who's offering to > write these libraries" still applies :-() Thanks! > > >> > As we handle some resolution, which isn’t really something pip does, > there is no cli interface to achieve this. I maintain a library (as of last > week) which provides compatibility shims between pip versions 8-current. It > is a good idea to use the cli, but we already spend enough resources > forking subprocesses into the background that it is a lot more efficient to > use the internals, which I track quite closely. The preference toward cli > interaction is largely to allow internal api breakage which we don’t mind. > > > > What is the URL of this library of which you are speaking? > > I know "security by obscurity" doesn't work, but I'm happier if > details of this library *aren't* widely known - it's not something I'd > want to encourage people using, nor is it supported by pip, as it's > basically a direct interface into pip's internal functions, papering > over the name changes that we did in pip 10 specifically to dissuade > people from doing this. If someone was committing to identifying useful API methods, parameters, and return values; writing a ~PEP; implementing said API; and maintaining backwards compatible shims for some reason; would something like `pip.api` be an appropriate namespace? (now that we're on version 18 with a faster release cycle)? > > Paul > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/HTD4EF5WIQHMOZAJU4POMA6TI3WSEUMO/
[Distutils] Re: pipenv and pip
On Mon, 20 Aug 2018 at 10:54, Wes Turner wrote: > > What stable API would be worth maintaining in pip for others to use? That's probably the sort of question that can only be usefully answered by projects like pipenv identifying the functionality they need and proposing something. Which is of course one of the reasons we (the pip devs) advise against "just using pip's internals", because it means we never get that information in any useful form. > Who is offering to maintain a stable API in/with/for pip and the Python > community ad infinitum? That's the crux of the problem - basically the answer is "no-one". What we advocate is for generally useful functionality to be split out into standalone libraries, and then pip, as well as other consumers, can use those libraries. We already have that with the packaging library and the script wrappers (part of distlib). The new resolver is being developed as a standalone library (zazo) as is the PEP 517 hook wrapper functionality (pep517). There's no reason this model couldn't work in other areas. (But even then, the question "who's offering to write these libraries" still applies :-() >> > As we handle some resolution, which isn’t really something pip does, there >> > is no cli interface to achieve this. I maintain a library (as of last >> > week) which provides compatibility shims between pip versions 8-current. >> > It is a good idea to use the cli, but we already spend enough resources >> > forking subprocesses into the background that it is a lot more efficient >> > to use the internals, which I track quite closely. The preference toward >> > cli interaction is largely to allow internal api breakage which we don’t >> > mind. > > What is the URL of this library of which you are speaking? I know "security by obscurity" doesn't work, but I'm happier if details of this library *aren't* widely known - it's not something I'd want to encourage people using, nor is it supported by pip, as it's basically a direct interface into pip's internal functions, papering over the name changes that we did in pip 10 specifically to dissuade people from doing this. Paul -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/BHI36Q4WQENSMXT2YMPEGWGMX5BXSUOB/
[Distutils] Re: pipenv and pip
What stable API would be worth maintaining in pip for others to use? "[Distutils] Announcement: Pip 10 is coming, and will move all internal APIs" https://groups.google.com/forum/m/#!topic/pypa-dev/JVTfS6ZdAuM On Monday, August 20, 2018, Chris Jerdonek wrote: > Thanks. Is the state of affairs as you described them what you're > planning for the future as well, or do you anticipate any changes > worthy of note? > > Also, are any of the bugs filed in pipenv's tracker due to bugs or > rough spots in pip -- is there a way to find those, like by using a > label? It would be good to be able to know about those so pip can > improve and become more useful. It doesn't seem like any bugs have > been filed in pip's tracker in the past year by any of pipenv's top > contributors. That seems a bit surprising to me given pipenv's heavy > reliance on pip (together with the fact that I know pip has its share > of issues), or is there another way you have of communicating > regarding things that interconnect with pip? Label ideas? - 'stable-api' - Who is offering to maintain a stable API in/with/for pip and the Python community ad infinitum? > Thanks, > --Chris > > > > On Mon, Aug 20, 2018 at 12:51 AM, Dan Ryan wrote: > > Sure I can grab that— we patch pip because we use some internals to > handle resolution and we have some bugs around that currently. They aren’t > upstreamed because they aren’t actually present in pip, only in pipenv. > Pipenv crosses back and forth across the virtualenv boundary during the > process. Pipenv relies on piptools and vendors a patched version of pip to > ensure consistency as well as to provide a few hacks around querying the > index. We do have a bit of reimplementation around some kinds of logic, > with the largest overlap being in parsing of requirements. > > > > As we handle some resolution, which isn’t really something pip does, > there is no cli interface to achieve this. I maintain a library (as of last > week) which provides compatibility shims between pip versions 8-current. It > is a good idea to use the cli, but we already spend enough resources > forking subprocesses into the background that it is a lot more efficient to > use the internals, which I track quite closely. The preference toward cli > interaction is largely to allow internal api breakage which we don’t mind. What is the URL of this library of which you are speaking? > > > > For the most part, we have open channels of communication as necessary. > We rely as heavily as we can on pip, packaging, and setuptools to connect > the dots, retrieve package info, etc. An issue label and something like a PEP would likely survive the ravages of 10 years of tools tooling around with community packaging commitments. > > > > Dan Ryan // pipenv maintainer > > gh: @techalchemy > > > >> On Aug 20, 2018, at 2:41 AM, Chris Jerdonek > wrote: > >> > >> Hi, > >> > >> Can someone explain to me the relationship between pipenv and pip, > >> from the perspective of pipenv's maintainers? > >> > >> For example, does pipenv currently reimplement anything that pip tries > >> to do, or does it simply call out to pip through the CLI or through > >> its internal API's? Does it have any preferences or future plans in > >> this regard? How about upstreaming to pip fixes or things that would > >> help pipenv? > >> > >> I've been contributing to pip more lately, and I had a look at > >> pipenv's repository for the first time today. > >> https://github.com/pypa/pipenv > >> > >> Given that pip's code was recently made internal, I was a bit > >> surprised to see that pipenv vendors and patches pip: > >> https://github.com/pypa/pipenv/tree/master/pipenv/patched/notpip > >> Before I had always assumed that pipenv used pip's CLI (because that's > >> what pip says you should do). > >> > >> I also noticed that some bugs in pipenv's tracker seem closely related > >> to pip's behavior, but I don't recall seeing any bugs or PR's in pip's > >> tracker reported from pipenv maintainers. > >> > >> Without knowing a whole lot more than what I've stated, one concern I > >> have is around fragmentation, duplication of work, and repeating > >> mistakes (or introducing new ones) if a lot of work is going into > >> pipenv without coordinating with pip. Is this in any way similar to > >> the beginning of what happened with distutils, setuptools, and > >> distribute that we are still recovering from? > >> > >> --Chris > >> -- > >> Distutils-SIG mailing list -- distutils-sig@python.org > >> To unsubscribe send an email to distutils-sig-le...@python.org > >> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > >> Message archived at https://mail.python.org/mm3/ > archives/list/distutils-sig@python.org/message/ > 2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/ > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org >
[Distutils] Re: pipenv and pip
Thanks. Is the state of affairs as you described them what you're planning for the future as well, or do you anticipate any changes worthy of note? Also, are any of the bugs filed in pipenv's tracker due to bugs or rough spots in pip -- is there a way to find those, like by using a label? It would be good to be able to know about those so pip can improve and become more useful. It doesn't seem like any bugs have been filed in pip's tracker in the past year by any of pipenv's top contributors. That seems a bit surprising to me given pipenv's heavy reliance on pip (together with the fact that I know pip has its share of issues), or is there another way you have of communicating regarding things that interconnect with pip? Thanks, --Chris On Mon, Aug 20, 2018 at 12:51 AM, Dan Ryan wrote: > Sure I can grab that— we patch pip because we use some internals to handle > resolution and we have some bugs around that currently. They aren’t > upstreamed because they aren’t actually present in pip, only in pipenv. > Pipenv crosses back and forth across the virtualenv boundary during the > process. Pipenv relies on piptools and vendors a patched version of pip to > ensure consistency as well as to provide a few hacks around querying the > index. We do have a bit of reimplementation around some kinds of logic, with > the largest overlap being in parsing of requirements. > > As we handle some resolution, which isn’t really something pip does, there is > no cli interface to achieve this. I maintain a library (as of last week) > which provides compatibility shims between pip versions 8-current. It is a > good idea to use the cli, but we already spend enough resources forking > subprocesses into the background that it is a lot more efficient to use the > internals, which I track quite closely. The preference toward cli interaction > is largely to allow internal api breakage which we don’t mind. > > For the most part, we have open channels of communication as necessary. We > rely as heavily as we can on pip, packaging, and setuptools to connect the > dots, retrieve package info, etc. > > Dan Ryan // pipenv maintainer > gh: @techalchemy > >> On Aug 20, 2018, at 2:41 AM, Chris Jerdonek wrote: >> >> Hi, >> >> Can someone explain to me the relationship between pipenv and pip, >> from the perspective of pipenv's maintainers? >> >> For example, does pipenv currently reimplement anything that pip tries >> to do, or does it simply call out to pip through the CLI or through >> its internal API's? Does it have any preferences or future plans in >> this regard? How about upstreaming to pip fixes or things that would >> help pipenv? >> >> I've been contributing to pip more lately, and I had a look at >> pipenv's repository for the first time today. >> https://github.com/pypa/pipenv >> >> Given that pip's code was recently made internal, I was a bit >> surprised to see that pipenv vendors and patches pip: >> https://github.com/pypa/pipenv/tree/master/pipenv/patched/notpip >> Before I had always assumed that pipenv used pip's CLI (because that's >> what pip says you should do). >> >> I also noticed that some bugs in pipenv's tracker seem closely related >> to pip's behavior, but I don't recall seeing any bugs or PR's in pip's >> tracker reported from pipenv maintainers. >> >> Without knowing a whole lot more than what I've stated, one concern I >> have is around fragmentation, duplication of work, and repeating >> mistakes (or introducing new ones) if a lot of work is going into >> pipenv without coordinating with pip. Is this in any way similar to >> the beginning of what happened with distutils, setuptools, and >> distribute that we are still recovering from? >> >> --Chris >> -- >> Distutils-SIG mailing list -- distutils-sig@python.org >> To unsubscribe send an email to distutils-sig-le...@python.org >> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ >> Message archived at >> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/2AYIJ3KTB2QJRF3BGV446DXAJGCFVQ5R/
[Distutils] pipenv and pip
Hi, Can someone explain to me the relationship between pipenv and pip, from the perspective of pipenv's maintainers? For example, does pipenv currently reimplement anything that pip tries to do, or does it simply call out to pip through the CLI or through its internal API's? Does it have any preferences or future plans in this regard? How about upstreaming to pip fixes or things that would help pipenv? I've been contributing to pip more lately, and I had a look at pipenv's repository for the first time today. https://github.com/pypa/pipenv Given that pip's code was recently made internal, I was a bit surprised to see that pipenv vendors and patches pip: https://github.com/pypa/pipenv/tree/master/pipenv/patched/notpip Before I had always assumed that pipenv used pip's CLI (because that's what pip says you should do). I also noticed that some bugs in pipenv's tracker seem closely related to pip's behavior, but I don't recall seeing any bugs or PR's in pip's tracker reported from pipenv maintainers. Without knowing a whole lot more than what I've stated, one concern I have is around fragmentation, duplication of work, and repeating mistakes (or introducing new ones) if a lot of work is going into pipenv without coordinating with pip. Is this in any way similar to the beginning of what happened with distutils, setuptools, and distribute that we are still recovering from? --Chris -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/
