On 05/16/2014 02:15 PM, Donald Stufft wrote: >> I guess the key thing I don't understand yet is, why would we assume >> that any package that hasn't already switched to verified-external-links >> since PEP 438, given a one-year window so far to do so, is any more >> likely to populate this new discoverable-index metadata from PEP 470, >> given a six-month window? >> >> It seems like PEP 470 places a lot of weight on an assumption of active >> maintainers, when really the core problem is a significant chunk of >> packages that (from the evidence of PEP 438 uptake) don't have active >> maintainers. >> >> So if we conclude that the bulk of the problematic legacy packages will >> probably never populate this new discoverable-index metadata either, at >> what stage in the process would their users get any useful clue as to >> how to continue to install that package? > > Sometimes the answer is "it breaks".
Depends what you mean by that. I think "it breaks at some point" is a fine answer for these packages. But I think we can do better than "it breaks without the user of the package ever getting a clue what is happening, or how to continue to install that package in some other way". > To be specific I don't believe most of these packages are in active use. > It is my belief that a good chunk of them are vestigial appendages which a > fear > of breaking them is preventing forward progress. Given that PyPI is a web > service and is version-less that mandates that sometimes things break. The > alternative is trying to maintain a union of every feature (and in some cases > every bug) that has ever existed from now into perpetuity. > > The questions that need to be asked are: > > 1) Is this a feature that we want supported long term? > 2) How many projects is this going to affect? > 3) How many users is this going to affect? > 4) Is there a way we can maintain some semblance of compatibility? > 5) Is the answer to #4 worth it? > > For me it's: > > 1) Not in this form, in the form of PEP 470 yes. > 2) ~7% is the maximum possible impact, the real number will be lower. > 3) I'm attempting to get some sort of "activity" measurement for this now. > 4) Yesish > 5) See below. > >> >> One option is Holger's solution: scraping the current links and >> populating them as verified external links. We both don't like this >> because it involves PyPI misleading users about the status of those >> links, and you also don't like it because you want to deprecate verified >> external links too. > > Correct. > >> >> Another option is if the deprecation message for --allow-unverified also >> gives the user the exact --find-links URL(s) they need to install that >> same package/version (which I think is implementable in pip today >> without any changes in PyPI). The downside here is that it really >> doesn't improve the current UX for these legacy packages, it just >> replaces --allow-unverified with explicit --find-links: but at least the >> latter is more explicit and places the responsibility clearly on the >> external host, not PyPI. > > It also doesn’t allow us to stop the crawling on PyPI (in fact it makes > it worse because we have to crawl to discover these links, instead > of being able to skip crawling if —allow-unverified isn’t selected). I don't think it does make it worse (or I wouldn't have proposed it). Pip would only show a deprecation message for --allow-unverified if --allow-unverified was used, meaning the crawling is already being done anyway and pip already has those links. So I don't think this would introduce any crawling that isn't happening now. I'm not sure what you mean by "the crawling on PyPI" -- do you mean the link-scraping done by PyPI itself, or the crawling that pip does at install time? It's true that PyPI's link-scraping should continue to be supported (for legacy projects only) until good uptake of a version of pip that fully removed --allow-unverified, after the deprecation path. But that's true regardless. >> Or, thirdly, Paul's proposal could solve this, if PyPI automatically >> generated an "external legacy index" for any packages that haven't >> generated their own external index URL by a certain date. Really in a >> way this is similar to Holger's proposal, except it uses >> external-indexes instead of verified-external-URLs, and is again a bit >> more explicit about what's going on (at the cost of requiring more >> adjustment from users). > > It’s an interesting idea. I’d have to think about it. There is of course > nothing > stopping anyone from doing this and shoving it on pythonhosted.org. The part that not anyone could do would be auto-populating the discoverable external-index-url metadata with this auto-generated index url, for inactive projects. That would require PyPI admin intervention. That part is key, because it's the only way the user of such a package ever finds out about this new external index for it. >> Basically, I think some acknowledgment of this problem of packages >> without active maintainers (and ideally a proposed solution to it) >> should be in PEP 470. > > Right now the PEP's (and my) position is that it breaks because I believe that > the impact of this change is being overblown. I'm attempting to gather more > data now. You could be right. More data would certainly be good. Thanks for all your work on all this stuff! Carl
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
