Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Nick Coghlan
On 31 August 2016 at 07:04, Daniel Holth  wrote:
> In other systems I've worked on I sometimes have make-like rules that
> automatically rebuild static metadata depending on other files, like copying
> a version number between a .json and an .xml file - reprogramming the system
> that uses the .xml file is not an option. For example a rule could watch
> certain files in the .git directory to regenerate a version number
> automatically as part of the build if .git changed, and do nothing if the
> .git directory was absent per a tarball dist. This works pretty well. Once
> you have a system that's easy to customize with make-like rules there are
> all sorts of trivial build or housekeeping tasks you might decide to do
> which would never be considered in a more difficult to customize system.

CPython's own build process takes this to extremes by sometimes
bootstrapping a version of Python without an import system in order to
refreeze importlib before continuing with building the normal version.
Argument Clinic (which generates C function argument handling
preambles from specially formatted comments) similarly needs a
pre-existing Python build in order to run.

However, we successfully hide that complexity from folks that just
want to build their own Python from source by checking in the
generated files.

Similarly, it wouldn't astonish me if we eventually see an emergent
practice of people writing pyproject.toml.in files for complex
projects, in order to move some particular forms of complexity away
from build time and towards development time - this would be a similar
practice to folks using autoconf to generate a project's C Makefile.

Party of the beauty of the pyproject.toml format is that as
publication and installation system maintainers, we don't need to care
about this problem beyond ensuring that "maintain it by hand" is a
viable and fully supported option for the majority of projects - as
long as the resulting file gets checked in, folks are free to
autogenerate from another data source if they choose to do so.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Daniel Holth
On Tue, Aug 30, 2016 at 4:06 PM Donald Stufft  wrote:

>
> On Aug 30, 2016, at 2:32 PM, Daniel Holth  wrote:
>
> name, version, packages, install_requires, extras_require, description,
> license, classifiers, keywords, author, url, entry_points.
>
>
> Out of these, a number of them are regularly dynamic for people’s setup.py
> as is. The version number is often dynamically computed based on things
> like git tags, packages can be computed based on Python version,
> install_requires and extras_requires regularly get computed based on things
> like Python version, OS, etc (although environment markers eases some of
> this) but also things like “We support Numpy >= for building from source,
> but once you’ve built from source you only support Numpy >=
> VERSION_YOU_BUILT_AGAINST”.
>
> Outside of “name”, it’s not entirely unreasonable to find reasons why a
> lot of things need to be dynamic. Although there’s a difference in what
> needs to be dynamic when pulling from a VCS and when pulling from a sdist,
> and currently there’s not really a whole lot of difference in terms of
> capability or how they are handled.
>

Of course pip continues to call egg_info before trusting the metadata from
any sdist and 90k pypi projects say this isn't changing. Once you need
dynamic static metadata, madness.

In other systems I've worked on I sometimes have make-like rules that
automatically rebuild static metadata depending on other files, like
copying a version number between a .json and an .xml file - reprogramming
the system that uses the .xml file is not an option. For example a rule
could watch certain files in the .git directory to regenerate a version
number automatically as part of the build if .git changed, and do nothing
if the .git directory was absent per a tarball dist. This works pretty
well. Once you have a system that's easy to customize with make-like rules
there are all sorts of trivial build or housekeeping tasks you might decide
to do which would never be considered in a more difficult to customize
system.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Donald Stufft

> On Aug 30, 2016, at 2:32 PM, Daniel Holth  wrote:
> 
> name, version, packages, install_requires, extras_require, description, 
> license, classifiers, keywords, author, url, entry_points. 

Out of these, a number of them are regularly dynamic for people’s setup.py as 
is. The version number is often dynamically computed based on things like git 
tags, packages can be computed based on Python version, install_requires and 
extras_requires regularly get computed based on things like Python version, OS, 
etc (although environment markers eases some of this) but also things like “We 
support Numpy >= for building from source, but once you’ve built from source 
you only support Numpy >= VERSION_YOU_BUILT_AGAINST”.

Outside of “name”, it’s not entirely unreasonable to find reasons why a lot of 
things need to be dynamic. Although there’s a difference in what needs to be 
dynamic when pulling from a VCS and when pulling from a sdist, and currently 
there’s not really a whole lot of difference in terms of capability or how they 
are handled.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Daniel Holth
On Tue, Aug 30, 2016 at 1:13 PM Thomas Kluyver  wrote:

> On Tue, Aug 30, 2016, at 05:51 PM, Antony Lee wrote:
>
> I am not really a fan of PEP518 in general.  Basically, the idea of
> setup.py is that declarative languages are not sufficient to express a
> build system (and AFAICT this is always going to be the case for
> expressing, say, compiler flags for extensions), so I'd rather just accept
> that and stick everything in setup.py instead of adding more parameter
> files.  What if someone wants dynamic build dependencies?
>
>
> Dynamic build deps aren't precluded - the idea is that the build system
> can discover additional dependencies when it runs, while the static
> build-system field specifies just what's required to run the build system
> itself.  However, the build system interface was split out into separate
> PEPs (517 & 516 are alternatives) to allow 518 to go forwards.
>
> I take totally the opposite view: we should make as much metadata as
> possible declarative, even though we know we can't define a totally general
> build system with purely declarative information.
>

This comes up over and over again because we've been living with this
system for long enough that the build script and metadata are together both
in setup.py and in people's brains. But there is a whole lot of stuff that
makes perfect sense in a static file. For example: name, version, packages,
install_requires, extras_require, description, license, classifiers,
keywords, author, url, entry_points. The only thing that would cause
trouble is if the system had no available build script.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What is the official position on distutils?

2016-08-30 Thread Nick Coghlan
On 30 August 2016 at 23:44, Paul Moore  wrote:
> On 30 August 2016 at 13:38, Antoine Pitrou  wrote:
>> On Tue, 30 Aug 2016 13:34:22 +0100
>> Paul Moore  wrote:
>>>
>>> From what I know, anyone who is actually hacking into the internal
>>> APIs to that level tends to use only distutils (probably because as
>>> you say, setuptools is even worse) and therefore falls into my
>>> category of "people affected by distutils changes who won't mind if a
>>> change is made to setuptools".
>>
>> Except when, as you point out, pip injects setuptools nilly-willy when
>> running setup.py?
>
> I'm confused. People who hack into distutils internals can't cope with
> having setuptools injected (because their hacks conflict with
> setuptools' hacks). That's my understanding of the situation, but it's
> based mainly on what I've seen of people saying "Don't break my
> APIs!!!" when distutils changes have been proposed in the past. The
> projects involved are already outside of the current
> pip/setuptools/wheel infrastructure (they have their own install
> processes, they don't host on PyPI, they are not pip-installable or
> whatever). So injecting setuptools isn't relevant, because they are
> outside that particular ecosystem.

No, this isn't the case - if you want to reliably generate wheels (as
bdist_wheel doesn't work with plain distutils), and if you want the
installdb metadata to always be generated, then you need to use pip
and its forced setuptools injection in order to get plain distutils
projects to behave the way you want.

This means that other setup.py helper projects like numpy.distutils
need to be setuptools-tolerant so they can support folks using pip
rather than calling setup.py directly.

That consequence further means that there's isn't substantially lower
risk in only changing setuptools without changing distutils - in fact,
by introducing a further unhelpful discrepancy between the two, it
increases the risk of code that works with plain distutils on 3.6+
breaking when setuptools is injected.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Thomas Kluyver
On Tue, Aug 30, 2016, at 05:51 PM, Antony Lee wrote:
> I am not really a fan of PEP518 in general.  Basically, the idea of
> setup.py is that declarative languages are not sufficient to express a
> build system (and AFAICT this is always going to be the case for
> expressing, say, compiler flags for extensions), so I'd rather just
> accept that and stick everything in setup.py instead of adding more
> parameter files.  What if someone wants dynamic build dependencies?

Dynamic build deps aren't precluded - the idea is that the build system
can discover additional dependencies when it runs, while the static build-
system field specifies just what's required to run the build system
itself.  However, the build system interface was split out into separate
PEPs (517 & 516 are alternatives) to allow 518 to go forwards.

I take totally the opposite view: we should make as much metadata as
possible declarative, even though we know we can't define a totally
general build system with purely declarative information.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Antony Lee
2016-08-30 5:08 GMT-07:00 Erik Bray :

> I mean this sort of already exists but it's spelled:
>
> from setuptools import Distribution
> Distribution({'setup_requires': ['numpy'])
>
> Granted it's non-obvious and doesn't have the needed_for flag, which I
> like.  It's not entirely clear how needed_for would work though.  For
> example, what if the package you're requiring provides the command
> that you need that package to run?
>

needed_for just textually checks sys.argv[1], and does so before the call
to setup() itself happens, so that's not a problem.

The same can be done by subclassing commands, and there can be some
> corner cases where that gets extra tricky (Cython comes to mind).


I personally don't like subclassing commands at all (it's not very
composable -- what happens if two projects both attempt to subclass
build_ext?).  But even then, how is Cython an issue?

I'm glad you mentioned Daniel Holth's setup-requires hack.  Although I
> haven't used it myself directly I generally like the concept.
>

I am not really a fan of PEP518 in general.  Basically, the idea of
setup.py is that declarative languages are not sufficient to express a
build system (and AFAICT this is always going to be the case for
expressing, say, compiler flags for extensions), so I'd rather just accept
that and stick everything in setup.py instead of adding more parameter
files.  What if someone wants dynamic build dependencies?


> Yeah, setup_requires is a mess, but I'd be skeptical of solving the
> problem by depending on any new features in setuptools :/
>

This could also go into pip (perhaps a better solution now that pip is
de-facto stdlib (via ensurepip))... we'll need something new somewhere
anyways.


> Best,
> Erik
>

Best,

Antony
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Best way to handle packaging a project that only applies to certain versions of Python

2016-08-30 Thread Brett Cannon
All of this seems like too much worry for a project that only applies to
Python 2.6 and earlier or Python 3.0. :) Thanks for the suggestions
everyone, but I'm just going to let the folks stuck on legacy versions deal
with the lack of wheels.

On Mon, 29 Aug 2016 at 23:21 Thomas Kluyver  wrote:

> There is also the 'python requires' metadata, which can now be provided
> via setuptools, but I think pip 8.2 needs to be released before that is
> respected.
>
>
> On Tue, Aug 30, 2016, at 01:08 AM, Daniel Holth wrote:
>
> Let's say you have different code for python 3.3 and python 3.4+. Tag one
> wheel py33-none-any and the second py34-none-any. The second wheel is
> preferred on python 3.4 and above, but ignored by 3.3. The py3 tag wouldn't
> work well here.
>
> Order of preference on 3.5 is: ('py35', 'none', 'any'), ('py3', 'none',
> 'any'), ('py34', 'none', 'any'), ('py33', 'none', 'any'), ('py32', 'none',
> 'any'), ('py31', 'none', 'any'), ('py30', 'none', 'any')
>
> Enscons would let you control the wheel tag directly but doesn't yet let
> you build multiple wheels with a single invocation.
>
> On Mon, Aug 29, 2016, 19:46 Donald Stufft  wrote:
>
>
> On Aug 29, 2016, at 7:34 PM, Brett Cannon  wrote:
>
> Someone has asked that I do a new release of importlib that includes a
> LICENSE file on PyPI: https://pypi.org/project/importlib/. Historically I
> have had the setup.py simply not include any Python code when built on
> versions of Python that include importlib in the stdlib itself:
> https://github.com/brettcannon/importlib/blob/master/setup.py .
>
> But now I would like to do a wheel. Is there some way I'm not thinking of
> to have a wheel that will leave out code or not install itself if a certain
> version of Python is used? Or will the user have to specify a proper Python
> requirement in their requirements.txt to get that kind of experience?
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> If your setup.py produces different output on different versions of
> Python, you’ll need multiple wheels.
>
> —
>
> Donald Stufft
> ___
> Distutils-SIG maillist  - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
> *___*
> Distutils-SIG maillist  - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What is the official position on distutils?

2016-08-30 Thread Antoine Pitrou
On Tue, 30 Aug 2016 14:44:10 +0100
Paul Moore  wrote:
> On 30 August 2016 at 13:38, Antoine Pitrou  wrote:
> > On Tue, 30 Aug 2016 13:34:22 +0100
> > Paul Moore  wrote:  
> >>
> >> From what I know, anyone who is actually hacking into the internal
> >> APIs to that level tends to use only distutils (probably because as
> >> you say, setuptools is even worse) and therefore falls into my
> >> category of "people affected by distutils changes who won't mind if a
> >> change is made to setuptools".  
> >
> > Except when, as you point out, pip injects setuptools nilly-willy when
> > running setup.py?  
> 
> I'm confused. People who hack into distutils internals can't cope with
> having setuptools injected (because their hacks conflict with
> setuptools' hacks).

I'm not sure what you mean by that.  I'm sure there are situations
where it works (I can't remember for sure whether it did the last time
I did such "hacking").

The thing is, more and more problem expect packages to install with
"pip install ." and they report bugs for that (the PyPA's official
discours is of course partly responsible for this, since it's
insisting so heavily on using pip and discouraging direct calls to
setup.py).

And the day people expect all libraries to be available as wheel files,
it will probably become worse, because I don't know how to produce
wheel files using plain distutils.

Regards

Antoine.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What is the official position on distutils?

2016-08-30 Thread Paul Moore
On 30 August 2016 at 13:38, Antoine Pitrou  wrote:
> On Tue, 30 Aug 2016 13:34:22 +0100
> Paul Moore  wrote:
>>
>> From what I know, anyone who is actually hacking into the internal
>> APIs to that level tends to use only distutils (probably because as
>> you say, setuptools is even worse) and therefore falls into my
>> category of "people affected by distutils changes who won't mind if a
>> change is made to setuptools".
>
> Except when, as you point out, pip injects setuptools nilly-willy when
> running setup.py?

I'm confused. People who hack into distutils internals can't cope with
having setuptools injected (because their hacks conflict with
setuptools' hacks). That's my understanding of the situation, but it's
based mainly on what I've seen of people saying "Don't break my
APIs!!!" when distutils changes have been proposed in the past. The
projects involved are already outside of the current
pip/setuptools/wheel infrastructure (they have their own install
processes, they don't host on PyPI, they are not pip-installable or
whatever). So injecting setuptools isn't relevant, because they are
outside that particular ecosystem.

What I'm trying to say (badly, it seems - for which I apologise) is
that if we change setuptools, only people working in the current
ecosystem are affected, and those projects have typically been OK with
the level of change going on in setuptools. People outside of the
pip/setuptools ecosystem rely on distutils, and appear to be heavier
users of the internal distutils APIs. As a result they appear to be
typically much more conservative in their requirements (precisely
because they had to work with an undocumented internal API). So
there's a much higher barrier to changing distutils, and making
changes in setuptools instead is less likely to break existing
projects (at a cost of making such changes not accessible to
non-setuptools projects).

This is largely off-topic by now, though. I wasn't trying to derail
the discussion about policy on distutils, so I'll drop this subthread
now, with apologies to anyone who found it a distraction from the main
point.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Daniel Holth
On Tue, Aug 30, 2016 at 8:09 AM Erik Bray  wrote:

> On Mon, Aug 29, 2016 at 7:29 PM, Antony Lee  wrote:
> > Hi all,
> >
> > The `setup_requires` option to `setup()` is well-known to suffer from
> > multiple issues.  Most importantly, as it is a keyword argument to
> > `setup()`, it appears too late for modules that may need to be imported
> for
> > the build to occur (e.g., Cython, for which support must explicitly
> provided
> > by setuptools itself rather than by letting Cython hook into it);
> > additionally, there are various contorsions that people go to to avoid
> some
> > `setup_requires` when not building the package (such as checking the
> value
> > of `sys.argv`).  `setup_requires` also uses `easy_install` rather than
> > `pip`, but I do not see why this could not be fixed; let's focus on the
> > first issue instead.
> >
> > If `setup_requires` appears too late to be useful, the obvious(?) option
> is
> > to move it earlier: provide a function, say,
> `setuptools.setup_requires()`,
> > that should be called *before* `setup()` itself, e.g.:
> >
> > from setuptools import setup, setup_requires
> > setup_requires("numpy", needed_for=["build_ext"])
> > try:
> > import numpy as np
> > except ImportError:
> > np = None
> > setup(..., include_dirs=[np.get_include()] if np else [])
>
> I mean this sort of already exists but it's spelled:
>
> from setuptools import Distribution
> Distribution({'setup_requires': ['numpy'])
>
> Granted it's non-obvious and doesn't have the needed_for flag, which I
> like.  It's not entirely clear how needed_for would work though.  For
> example, what if the package you're requiring provides the command
> that you need that package to run?
>
> The same can be done by subclassing commands, and there can be some
> corner cases where that gets extra tricky (Cython comes to mind).
>
> > When `setup.py` is invoked, either directly or by pip, upon the call to
> > `setup_requires()`, if `sys.argv[0]` is in the `needed_for` kwarg, and at
> > least one requirement is missing, `setup_requires()` calls asks pip to
> > install the required packages (similarly to
> > `https://bitbucket.org/dholth/setup-requires`
> ) in a temporary directory,
> and
> > the whole Python process replaces itself (in the `os.execv()` sense) by a
> > new call to `python setup.py` with this temporary directory prepended to
> the
> > PYTHONPATH.  In this new process, the arguments to `setup_requires()` are
> > now available and we can proceed to the rest of `setup.py`.
> >
> > I feel like this idea is natural enough that someone must already have
> come
> > up with it... but I may be missing something :-)
>
> I'm glad you mentioned Daniel Holth's setup-requires hack.  Although I
> haven't used it myself directly I generally like the concept.
>
> Yeah, setup_requires is a mess, but I'd be skeptical of solving the
> problem by depending on any new features in setuptools :/


In June I updated my setup_requires hack to be a PEP 518 implementation. It
reads build-system.requires out of pyproject.toml and installs them into an
isolated directory if necessary. The basic feature is 14 lines of code not
counting blanks, and you use it by prepending the code to your setup.py.
Once pip implements PEP 518 that code becomes a no-op.

Antony is right that the main problem with setup_requires is that it
happens too late, or that you have to write a setuptools extension to use
it. Most people do not know that it is possible to write a setuptools
extension, let alone want to write one.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What is the official position on distutils?

2016-08-30 Thread Antoine Pitrou
On Tue, 30 Aug 2016 13:34:22 +0100
Paul Moore  wrote:
> 
> From what I know, anyone who is actually hacking into the internal
> APIs to that level tends to use only distutils (probably because as
> you say, setuptools is even worse) and therefore falls into my
> category of "people affected by distutils changes who won't mind if a
> change is made to setuptools".

Except when, as you point out, pip injects setuptools nilly-willy when
running setup.py?

Regards

Antoine.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What is the official position on distutils?

2016-08-30 Thread Paul Moore
On 30 August 2016 at 12:14, Antoine Pitrou  wrote:
> On Sun, 28 Aug 2016 23:36:43 +0100
> Paul Moore  wrote:
>> * as a result, there would have to be an *extremely* compelling reason
>> to make a change in distutils rather than in setuptools - sufficient
>> to justify the additional risk, the extra developer effort needed, and
>> the fact that any such change is only going to benefit users of newer
>> versions of Python
>
> There is a problem with this: distutils and setuptools don't document
> or specify their internal APIs.  Yet if you want to do something
> advanced, you have to hook into the command classes and either call
> their APIs or extend them.
>
> Using distutils, this is already tedious, since you have to go read its
> source code and try to figure out what happens (it's rarely obvious).
>
> Using setuptools, this is even worse, though, because setuptools builds
> on top of distutils by modifying the command classes, and so you have
> to figure out the effect that monkeypatching has on the overall
> behaviour (in addition to figure out what happens in distutils).
>
>
> I don't know if that's bothersome enough to compound the fact that
> changes made in setuptools can benefit all Python versions, but I think
> it's useful to keep it in mind.
>
>
> (that lack of programmability is really distutils' failure, of course,
> and setuptools inherited that failure by inheriting distutils'
> architecture.)

>From what I know, anyone who is actually hacking into the internal
APIs to that level tends to use only distutils (probably because as
you say, setuptools is even worse) and therefore falls into my
category of "people affected by distutils changes who won't mind if a
change is made to setuptools". Admittedly, that's a self-perpetuating
situation...

The fact remains that (as far as it's possible to tell) very few
people do dive into distutils to that level. At least, not many people
seem to have problems caused by the current "leave distutils alone,
make changes in setuptools" approach. Ideally, someone would be
working on alternative solutions to break the remaining few's tight
dependency on distutils, but with the effort they have invested in it,
there doesn't seem to be much interest in re-engineering something
that works.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] setup_requires: the obvious option(?)

2016-08-30 Thread Erik Bray
On Mon, Aug 29, 2016 at 7:29 PM, Antony Lee  wrote:
> Hi all,
>
> The `setup_requires` option to `setup()` is well-known to suffer from
> multiple issues.  Most importantly, as it is a keyword argument to
> `setup()`, it appears too late for modules that may need to be imported for
> the build to occur (e.g., Cython, for which support must explicitly provided
> by setuptools itself rather than by letting Cython hook into it);
> additionally, there are various contorsions that people go to to avoid some
> `setup_requires` when not building the package (such as checking the value
> of `sys.argv`).  `setup_requires` also uses `easy_install` rather than
> `pip`, but I do not see why this could not be fixed; let's focus on the
> first issue instead.
>
> If `setup_requires` appears too late to be useful, the obvious(?) option is
> to move it earlier: provide a function, say, `setuptools.setup_requires()`,
> that should be called *before* `setup()` itself, e.g.:
>
> from setuptools import setup, setup_requires
> setup_requires("numpy", needed_for=["build_ext"])
> try:
> import numpy as np
> except ImportError:
> np = None
> setup(..., include_dirs=[np.get_include()] if np else [])

I mean this sort of already exists but it's spelled:

from setuptools import Distribution
Distribution({'setup_requires': ['numpy'])

Granted it's non-obvious and doesn't have the needed_for flag, which I
like.  It's not entirely clear how needed_for would work though.  For
example, what if the package you're requiring provides the command
that you need that package to run?

The same can be done by subclassing commands, and there can be some
corner cases where that gets extra tricky (Cython comes to mind).

> When `setup.py` is invoked, either directly or by pip, upon the call to
> `setup_requires()`, if `sys.argv[0]` is in the `needed_for` kwarg, and at
> least one requirement is missing, `setup_requires()` calls asks pip to
> install the required packages (similarly to
> `https://bitbucket.org/dholth/setup-requires`) in a temporary directory, and
> the whole Python process replaces itself (in the `os.execv()` sense) by a
> new call to `python setup.py` with this temporary directory prepended to the
> PYTHONPATH.  In this new process, the arguments to `setup_requires()` are
> now available and we can proceed to the rest of `setup.py`.
>
> I feel like this idea is natural enough that someone must already have come
> up with it... but I may be missing something :-)

I'm glad you mentioned Daniel Holth's setup-requires hack.  Although I
haven't used it myself directly I generally like the concept.

Yeah, setup_requires is a mess, but I'd be skeptical of solving the
problem by depending on any new features in setuptools :/

Best,
Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] What is the official position on distutils?

2016-08-30 Thread Antoine Pitrou
On Sun, 28 Aug 2016 23:36:43 +0100
Paul Moore  wrote:
> * as a result, there would have to be an *extremely* compelling reason
> to make a change in distutils rather than in setuptools - sufficient
> to justify the additional risk, the extra developer effort needed, and
> the fact that any such change is only going to benefit users of newer
> versions of Python

There is a problem with this: distutils and setuptools don't document
or specify their internal APIs.  Yet if you want to do something
advanced, you have to hook into the command classes and either call
their APIs or extend them.

Using distutils, this is already tedious, since you have to go read its
source code and try to figure out what happens (it's rarely obvious).

Using setuptools, this is even worse, though, because setuptools builds
on top of distutils by modifying the command classes, and so you have
to figure out the effect that monkeypatching has on the overall
behaviour (in addition to figure out what happens in distutils).


I don't know if that's bothersome enough to compound the fact that
changes made in setuptools can benefit all Python versions, but I think
it's useful to keep it in mind.


(that lack of programmability is really distutils' failure, of course,
and setuptools inherited that failure by inheriting distutils'
architecture.)

Regards

Antoine.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig