In my experience tools think an archive is an sdist if it is named properly, contains PKG-INFO with a minimum number of fields
Metadata-Version: 1.1Name: bobVersion: 1.0 and setup.py. setuptools sdists also contain .egg-info but it is unnecessary. On Wed, May 31, 2017 at 2:41 PM Donald Stufft <don...@stufft.io> wrote: > On May 31, 2017, at 2:01 PM, Paul Moore <p.f.mo...@gmail.com> wrote: > > On 31 May 2017 at 18:03, Donald Stufft <don...@stufft.io> wrote: > > No you’re correct, it currently just invokes ``setup.py sdist > bdist_wheel``. > The hook is needed so that Travis can have a singular tool to invoke > (likely > twine?) instead of needing to determine if it needs to invoke flit or > setuptools or mytotallyradbuildthing. The thing I’m trying to express (and > doing poorly it seems :( ) is that generating a sdist is an important thing > to have be possible, and it needs to be done in a way that it can be > invoked > generically. > > > I don't think that's either unclear or in dispute. The question here > is whether "produce a sdist" is in scope for this particular PEP. > > The problem is that you're proposing using a "build a sdist" hook as > the means for pip to do its "copy the source to an isolated build > directory" step. Currently, doing that fails because tools like > setuptools_scm work differently when run from a VCS checkout instead > of a sdist. The long term plan for pip is something we've always > described in terms of going via a sdist, but there's lots of details > we haven't thrashed out yet. I don't think we're at a point where we > can insist that in a post-PEP 517 world, we can switch straight to > building via a sdist. However, I agree with you that we want PEP 517 > to assist us with moving in that direction, and we *definitely* don't > want to get into a situation like we're in now, where a PEP 517 > compliant backend can leave us with no option better than "copy the > whole source tree”. > > > I don’t think we can start telling projects if they using a PEP 517 they > can delete their ``setup.py`` and live in the brave new world (assuming all > of their tools have been updated to PEP 517) when doing so is removing a > “standard” interface for producing a sdist. Either a replacement for > setup.py should support the things we want to keep from setup.py and > explicitly unsupport things we don’t want, or I don’t think that thing is > actually a replacement for setup.py and I don’t think we should support it. > > Taking pip completely off the table a second, let’s take a look at tox. > Tox’s default mode of operation is to produce a sdist. Now let’s say I’m > writing a project that I want to use PEP 517 and get rid of setup.py, > except now tox is broken with no path forward because PEP 517 doesn’t > define how to produce a sdist. > > The same story is true for TravisCI’s PyPI deployment pipeline, as soon as > any project starts depending on PEP 517, we completely break that feature > for them without a path for them to fix it (besides writing a PEP of > course). > > The same is true for Gem Fury’s private PyPI repositories where you can > ``git push fury`` and have them build a sdist automatically for you. > > This same pattern is over and over and over again, projects depend on the > ability to produce a sdist for any Python project. PEP 517 says that people > can delete their setup.py but doesn’t provide the mechanism for producing a > sdist, thus breaking parts of the ecosystem. Simply changing the PEP to say > “ok you can’t delete your setup.py yet” isn’t acceptable yet either, > because then you have two competing build systems both who think they > should be in charge, which makes the entire process *more* confusing for > the end users than just baking the concept of sdist generation into PEP 517. > > Now, independently of that, pip needs a way to take an arbitrary directory > that might contain a git clone with a bunch of extraneous files in it, or > it might also just be a sdist that was already unpacked. For a variety of > reasons we want to copy this directory into a temporary location, but doing > a blind copy of everything can trigger a bad path where a simple ``pip > install .`` can take a long time (up to minutes long have been reported in > the wild) trying to copy the entire directory, including files that we > don’t even need or want. We need some mechanism for copying these files > over, and it just so happens that the exact same process needs to occur > when computing what files going into a sdist, and since I believe that for > completely unrelated reasons, computing a sdist *must* be a part of any > attempt to replace setup.py, reusing that simplifies the process of > creating a PEP 517 backend (since having to only implement build_sdist is > simpler than having to implement build_sdist AND copy_files_for_build). > > In addition to all of the above, we currently have like 7 different > “paths” installation can go through on the process of going from a VCS > checkout/developer copy to a installed distribution, we have: > > 1) VCS Checkout -> Installed > 2) VCS Checkout -> Sdist -> Installed > 3) VCS Checkout -> Wheel -> Installed > 4) VCS Checkout -> Sdist -> Wheel -> Installed > 5) VCS Checkout -> Editable Install > 6) VCS Checkout -> Sdist -> Editable Install > > Unless you’re careful to have your packaging done exactly correct, each of > those 6 can end up having different (and often times surprising behavior) > that regular end users who are new to packaging (or hell, even old hands) > hit with some regularity. One of my long term goals is try and reduce the > number of those paths down, which will make it more likely that people are > not surprised by edge cases in how their own uses are calling ``pip > install`` and will ultimately provide a more enjoyable experience using > pip. We obviously cannot reduce the number of supported methods down to 1, > but we can reduce them down to: > > A) VCS Checkout -> Sdist -> Wheel -> Installed > B) VCS Checkout -> Editable Install > > Implementing build_sdist in PEP 517 and using that to handle copying the > files from what could either be a VCS checkout OR an unpacked sdist, means > that we eliminate (1) and (3) from the first list. Ensuring that we only > ever install a PEP 517 style project by always using build_wheel after > having used build_sdist then eliminates (2). We’re not dealing with > editable installs here (and it kind of pains me we aren’t, but they’re a > much bigger topic so I think it’s unavoidable) but preventing an editable > install of an sdist would eliminate (6) from above, leaving us with just > two paths (and the second path requiring an explicit flag to opt into, > rather than being implicit by nature of what you’re passing into pip and > what other libraries you have installed). > > In addition to all of the above, any part of building a sdist that is more > complicated than “copy some files”, these build backends are already going > to have to support by nature of the fact we’re expecting them to generate > wheel metadata. The wheel metadata has to include the version number, so if > someone wants to dynamically compute the version number from git, a PEP 517 > backend must already handle that or it simply won’t work. > > Finally, we’re should/are assuming that these build projects are going to > be capable of producing sdists. Thus they already have to implement 99% of > build_sdist anyways, and the only additional effort on their part is just > the glue code that wires up their internal mechanism for producing sdists > to the API that allows a standard mechanism for calling those mechanisms. > Hopefully it is not controversial that a build tool *must* be capable of > producing a sdist, since otherwise we’re throwing away support for any non > Windows/macOS/Linux platform. Implementing a custom “copy these files” is > *more* effort than exposing the mechanism that they should already have. > > So yes, one of the things I want to do with this hook is copy the source > files to an isolated directory, but that’s not the *only* thing I want to > do with that hook, and when I see single solution that can solve multiple > problems, I always vastly prefer that over a single solution that only > solves a single problem. > > > — > > Donald Stufft > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig >
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
