Re: [Distutils] Distutils question

[Daniel Holth]

Great, thanks! It looks like out of tree builds are supported fine even in
distutils. Properly written extensions will also find the build directory
as a property of the build command. There should be no barrier to using the
single build wheel function for the pep.

On Sun, Jul 9, 2017, 10:33 Jeremy Kloth  wrote:

> On Sat, Jul 8, 2017 at 9:09 PM, Daniel Holth  wrote:
>
>> Unrelated to pep 517, remind me whether when invoking setup.py build -b
>> dir bdist, if the -b argument passed to the build command is supposed to
>> affect the build command run as a subcommand of bdist? Asking for a friend
>> 
>>
>
> "setup.py build -b {dirname} bdist" will first run the build command (and
> its sub commands build_*) then invoke the bdist command.  When bdist
> requests build to be executed, distutils will see that it is already run
> and simply continue on.  Even if a reinit is requested (for build), the
> supplied options will be applied again when the build command is finalized
> prior to running.
>
> --
> Jeremy Kloth
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distutils question

[Jeremy Kloth]

On Sat, Jul 8, 2017 at 9:09 PM, Daniel Holth  wrote:

> Unrelated to pep 517, remind me whether when invoking setup.py build -b
> dir bdist, if the -b argument passed to the build command is supposed to
> affect the build command run as a subcommand of bdist? Asking for a friend
> 
>

"setup.py build -b {dirname} bdist" will first run the build command (and
its sub commands build_*) then invoke the bdist command.  When bdist
requests build to be executed, distutils will see that it is already run
and simply continue on.  Even if a reinit is requested (for build), the
supplied options will be applied again when the build command is finalized
prior to running.

-- 
Jeremy Kloth
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] A possible refactor/streamlining of PEP 517

[Steve Dower]

Big +1 for passing the option to the backend rather than making the frontend 
figure out how to do each backend’s job for them.

And add MSBuild to the list, which trivially supports separate source, 
intermediate and output directories, as well as incremental rebuilds. (And 
recently got full x-plat support, so could potentially be useful on all 
platforms.)

Cheers,
Steve

Top-posted from my Windows phone

From: Nick Coghlan
Sent: Saturday, July 8, 2017 7:59
To: Nathaniel Smith
Cc: distutils-sig
Subject: Re: [Distutils] A possible refactor/streamlining of PEP 517

On 8 July 2017 at 13:36, Nathaniel Smith  wrote:
> On Fri, Jul 7, 2017 at 6:23 AM, Daniel Holth  wrote:
>> FYI distutils supports out of tree builds too. It is the -b argument to
>> 'setup.py build'.
>
> In theory, yes, but in practice, there are lots of setup.py files out
> there that mutate the source directory. For example, every project
> using Cython.
>
> I don't think a distutils/setuptools backend could comply with
> Thomas's wording except by doing a copytree or sdist or something
> just-in-case.

Which is fine - the point of making it a parameter instead of a
separate hook is that the folks in the best position to know whether
or not a particular build system has native out-of-tree build support
are the developers of that particular PEP 517 backend.

Since we know Scons supports this natively (by way of the
"variant_dir" setting), I went and checked some of the other build
systems that folks may end up wanting to develop backends for.

Native build system support for out-of-tree builds, so developers of
these shims should be able to delegate handling "build_directory":

- Scons: set variant_dir appropriately
(http://scons.org/doc/production/HTML/scons-user.html#chap-separate)
- meson: *only* supports out-of-tree builds
(http://mesonbuild.com/Using-multiple-build-directories.html)
- waf: configure the output directory appropriately
(https://waf.io/book/#_custom_build_outputs)
- premake: set targetdir appropriately
(https://github.com/premake/premake-core/wiki/targetdir)
- CMake: simplest way seems to be to save the initial working
directory as the source directory, cd to the build directory, then run
"cmake "
- yotta; based on CMake, so same approach applies
- autotools/make: similar approach to CMake, but running
"/configure && make " from the build
directory
- maven: require that projects using the shim support a configurable
build dir 
(https://stackoverflow.com/questions/13173063/out-of-tree-build-with-maven-is-it-possible/13173140#13173140)
- ant: also uses pom.xml files, so same approach as maven applies
- cargo: set CARGO_TARGET_DIR
(http://doc.crates.io/environment-variables.html#environment-variables-cargo-reads)

Explicitly willing to add native out-of-tree wheel creation based on
PEP 517's requirements:

- flit

Mostly assume in-place builds, no generic support for out-of-tree
builds that I can find, so developers of these shims will need to work
out how to handle "build_directory" (probably by copying the relevant
input files into the specified build directory and then doing an
in-place build, similar to the way Scons handles "variant_dir" by
default):

- setuptools/distutils
- gyp/ninja (including node-gyp)
- gradle

So I think the folks saying "We don't think a separate hook for build
directory preparation is the right abstraction" have a strong point,
as that approach doesn't appear to map cleanly to the design of *any*
popular build system for precompiled languages.

By contrast, "build_directory" as an optional setting for the binary
build command *does* map cleanly as a concept in many cases, and for
the ones where it doesn't, I readily found questions from other folks
also wanting to know how to do it, and PEP 517 interface shims copying
the files they care about then doing an in-place build is a perfectly
acceptable technical solution.

And as an added bonus, if a backend starts out by blindly doing
"shutil.copytree(source_directory)" for out-of-tree builds, then the
potential performance implications of that approach will become more
clearly a discussion between the projects using the backend and the
developers of the backend, rather than being something that frontends
like pip will need to worry about handling in the general case.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] A possible refactor/streamlining of PEP 517

[Nick Coghlan]

On 9 July 2017 at 01:17, Jeremy Kloth  wrote:
> On Fri, Jul 7, 2017 at 9:45 PM, Nick Coghlan  wrote:
>> On 7 July 2017 at 23:23, Daniel Holth  wrote:
>>> FYI distutils supports out of tree builds too. It is the -b argument to
>>> 'setup.py build'.
>>
>> Sort of. That's short for "--bdist-dir" and tells distutils/setuptools
>> not to use the "dist/" subdirectory for either build trees or the
>> build artifacts. It doesn't say anything about where intermediate
>> artifacts generated by compilers etc should end up.
>
> No, Daniel is correct.  "setup.py *build* -b" is short for
> --build-base which is where all build artifacts go, even those from
> compilers.  It seems you are confusing it with "setup.py *bdist* -b"
> which is indeed short for "--bdist-dir".

Aye, I assumed he was referring to the latter, as I wasn't able to
find any documentation for "setup.py build" anywhere, and this is the
first time I've ever heard of "build" being available as a separate
setup.py command.

Even given that clarification, thought, Nathaniel's point about folks
doing local state manipulation directly in setup.py still stands.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig