On 2 June 2017 at 06:45, Brett Cannon <[email protected]> wrote:
> And so I think in a very wordy way, I just said we need to stop saying "pip
> needs a standardized way to produce an sdist" and instead start saying
> "twine needs a way to produce an sdist". And that leads to the question
> about whether PEP 517 must cover sdist production for twine because we want
> to have that solved before we have the consumption side in pip in place. Or
> put another way, are we okay with pip consuming things that twine simply
> can't make from a pyproject.toml file (yet)? A "yes" means PEP 517 shouldn't
> be held up, while a "no" means we need a solution for twine.
I largely agree with this phrasing of the problem, but think it's
oversimplifying the underlying capabilities needed a bit:
1. sdist creation tools, whether tox, twine, a future "pip publish"
command, or something else, need to know how to:
1a. Generate sdist metadata (aka PKG-INFO)
1b. Generate the actual sdist contents
1c. Determine the dependencies needed to handle the above steps
2. Install tools, including pip, need to know how to:
2a. Generate a clean out-of-tree build directory
2b. Generate wheel metadata (including the METADATA file)
2c. Generate the actual wheel contents
2d. Determine the dependencies needed to handle the above steps
The agreed-to-be-missing piece that has been identified in PEP 517 is
"2a": the part that will let pip (and any other tools doing
out-of-tree builds) avoid copying entire VCS checkouts into the build
directory.
Donald's perspective is that instead of being an independent step,
generating an out-of-tree build directory should instead be defined in
terms of sdist generation on the following grounds:
* it better accounts for local publish-and-install tools like `tox`,
which relies on sdist generation to push the code under development
into the test venv
* defining the "export a build tree" and "generate an sdist tree"
steps independently creates the opportunity for future inconsistencies
where "VCS -> sdist -> build tree -> wheel" and "VCS -> build tree ->
wheel" give different results (just as "pip install project-dir/" and
"pip install project-sdist" can give different results today)
* `PKG-INFO` in an sdist is essentially the same file as PEP 427's
`METADATA`, so any wheel building backend is already required to be
able to generate it
While the first point doesn't really bother me (I'm OK with folks
needing to keep a "setup.py sdist" shim around for now), I have to
agree that I find the second point compelling, as it means that any
PEP 517 based project will inherently test the correctness of its
sdist generation *just by doing a local install*. That means we won't
be relying on opt-in pre-release testing with tox, or post-release bug
reports from users any more: if the sdist definition is broken, even
local installations from the VCS checkout into an active virtual
environment won't work.
The last point means that requiring an sdist export command shouldn't
impose an unreasonable burden on backend developers, as the only
differences between "generate an sdist tree" and "export a build tree"
would be:
- `pyproject.toml` MUST be included unmodified at the root of the sdist tree
- `PKG-INFO` MUST be generated at the root of the sdist tree
- a `setup.py` shim MAY be generated for backwards compatibility with
installation tools that are unaware of PEP 517
Beyond that, both approaches would have the same requirement of
"include everything needed to subsequently build a wheel file from the
sdist".
The mapping back to the originally identified activities would then be:
1. sdist creation tools:
1a. Generate sdist metadata: export the sdist and read PKG-INFO
1b. Generate the actual sdist contents: export the sdist
1c. Determine the dependencies needed to handle the above steps:
look at build-system.requires
2. Install tools:
2a. Generate a clean out-of-tree build directory: export the sdist
2b. Generate wheel metadata: export the wheel metadata directory
2c. Generate the actual wheel contents: build the wheel file
2d. Determine the dependencies needed to handle the above steps:
look at build-system.requires & get_build_requires()
So even though 1a, 1b, and 2a are conceptually different activities,
backends would only need to implement one operation to handle all 3:
sdist export.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
_______________________________________________
Distutils-SIG maillist - [email protected]
https://mail.python.org/mailman/listinfo/distutils-sig
