On Wed, May 4, 2016 at 10:42 PM, Robert Collins <robe...@robertcollins.net> wrote: > On 5 May 2016 at 16:22, Nathaniel Smith <n...@pobox.com> wrote: > ... >> I'm sympathetic to the general approach, but on net I think I prefer a >> slightly different proposal. >> >> Downsides to just standardizing [setup_requires]: > > If I write a PEP, it won't be standardising setup_requires, it will be > standardising bootstrap requirements. The distinction is nuance but > critical: > > - we don't expect setuptools to ever need to honour it (of course, it > could, should it choose to, and ease of adoption may suggest that it > should) > - as a new feature, making it opt in allows folk to adopt it when > they are ready; if it was literally just a new home for > setup_requires, we may see build systems auto-populating it, and the > potential swamp you describe below would then apply.
The main argument I was making there was that it needs to be a new opt-in thing, so if we're agreed on that then great :-). >> - if projects have existing ones, that's actually kinda good / kinda >> bad -- pip has never been respecting these before, so if we suddenly >> start doing that then existing stuff could break. I don't know how >> likely this is, but experience suggests that *something* will break >> and make someone angry. (I'm still blinking at getting angry >> complaints arguing that uploading a wheel to pypi, where before there >> was only an sdist, should be treated as a compatibility-breaking >> change that requires a new version etc.) > > Yes, things will break: anyone using this will need a new pip, by > definition. Not everyone will be willing to wait 10 years before using > it :). Just to clarify (since we seem to agree): I meant that if pip starts interpreting an existing setup.cfg thing, then the new-pip/old-package situation could break, which would be bad. >> - IMO an extremely valuable aspect of this new declarative >> setup-requirements thing is that it gives us an opportunity to switch >> to enforcing the accuracy of this metadata. Right now we have a swamp >> we need to drain, where there's really no way to know what environment >> any given setup.py needs to run. Sometimes there are setup_requires, >> sometimes not; if there are setup_requires then sometimes they're > > Huh? I've not encountered any of this, ever. I'd love some examples to > go look at. The only issue I've ever had with setup_requires is the > easy_install stuff it ties into. I don't think I've ever seen a package that had accurate setup_requires (outside the trivial case of packages where setup_requires=[] is accurate). Scientific packages in particular universally have undeclared setup requirements. > ... >> instead of punishing them -- is if we make the rule be "projects that >> use the declarative setup-requirements feature also get isolated build >> environments". (Then the messaging is: "see, this helps you check that >> you actually set it up right! if your new metadata works for you in >> testing, it'll also work for your users!) But this again would mean we >> can't reuse the existing [setup_requires] section. > > I'm very much against forcing isolated build environments as part of > this effort. I get where you are coming from, but really it conflates > two entirely separate things, and you'll *utterly* break building > anything with dependencies that are e.g. SWIG based unless you > increase the size of the PEP by about 10-fold. (Thats not hyperbole, I > think). Okay, now's my turn to be baffled :-). I literally have no idea what you're talking about here. What would this 10x longer PEP be talking about? Why would this be needed? > Working around that will require a bunch of UX work, and its > transitive: you have to expose > how-to-workaround-the-fact-that-all-our-deps-are-not-installable all > the way up the chain. That's probably a good thing to do (e.g. see > bindep, or the aborted callout-to-system-packages we proposed after > PyCon AU last year), but tying these two things together is not > necessary, though I can certainly see the appeal; the main impact I > see is that it will just impede adoption. What are these things that aren't pip-installable and why isn't the solution to fix that? I definitely don't want to break things that work now, but providing new features that incentivize folks to clean up their stuff is a good thing, surely? Yeah, it means that the bootstrap-requirements stuff will take some time and cleanup to spread, but that's life. We've spent a huge amount of effort on reaching the point where pretty much everything *can* be made pip installable. Heck, *PyQt5*, which is my personal benchmark for a probably-totally-unpackageable package, announced last week that they now have binary wheels on pypi for all of Win/Mac/Linux: https://pypi.python.org/pypi/PyQt5/5.6 I want to work towards a world where this stuff just works, not keep holding the entire ecosystem back with compromise hacks to work around a minority of broken packages. > The reality, AFAICT, is that most projects with undeclared build deps > today get corrected fairly quickly: a bug is filed, folk fix it, and > we move on. A robotic system that isolates everything such that folk > *cannot* fix it is much less usable, and I'm very much in favour of > pragmatism here. Again, in my world ~100% of packages have undeclared build deps... >> - setup.cfg is kind of a terrible format for standardizing things >> because the only definition of the format is "read the ConfigParser >> source". You cannot parse setup.cfg from a non-Python language. And >> the *only* benefit is that it already exists; teaching pip to read >> pybuild.json or pybuild.toml instead would be completely trivial. So >> if we aren't going to try and grandfather in existing [setup_requires] >> sections, then we might as well switch to a better file format at the >> same time -- this is not the hard part. > > The patch to read a single list-valued key out of setup.cfg is trivial > and shallow. We've not managed to settle on consensus on a file format > choice in a year of debate. I hold zero confidence we will going > forward either. If the BDFL delegate makes a call - fine. I read > Nick's earlier email in the thread as such a thing TBH :). Oh sure, I think everyone agrees that the file format choice is not a make-or-break decision and that ultimately this is a good case for of BDFL-style pronouncement rather than endless consensus seeking. But I'm still allowed to make technical arguments for why I think the BDFL-delegate should pronounce one way or another :-). >> So I like the idea of splitting out the declarative setup-requirements >> PEP from the new build system hook interface PEP, but I think that the >> way we should do this is by defining a new pybuild.whatever file like >> the ones in PEP 516 / PEP 517 (they're identical in this regard) that >> *only* has schema-version and bootstrap-requirements, and then we can >> add the build backend key as a second step. > > I think the risk there is that that presumes the answer, and without > the abstract build effort actually moving forward we may be years > before we actually see the file being used for anything else. If ever. I see the risk, but I think it's minimal. We're not presuming anything about the answer except that somehow it will involve adding some kind of static metadata, which seems very safe. -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig