Re: [Distutils] setup.py install using pip

[Robert Collins]

Thanks for digging that up.

-Rob

On 24 December 2015 at 06:35, Erik Bray  wrote:
> On Thu, Dec 10, 2015 at 5:12 PM, Robert Collins
>  wrote:
>> On 8 December 2015 at 09:14, Erik Bray  wrote:
>>> On Mon, Dec 7, 2015 at 2:40 PM, Paul Moore  wrote:
 On 7 December 2015 at 18:58, Erik Bray  wrote:
> I wasn't able to produce this problem.  Even with --no-binary
> specified pip installs (by default) with
> --single-version-externally-managed.  My prototype implicitly disables
> the --pip flag if --single-version-externally-managed was specified
> (true to the purpose of that flag).

 Ah - that was the bit I was missing, the
 --single-version-externally-managed flag can be used to trigger
 ignoring --pip.

> What *is* a problem is if --pip is in setup.cfg, and one invokes `pip
> install --egg .`.  I wasn't quite able to make that go into an
> infinite loop, but it did invoke pip.main recursively, and stuff broke
> on the second invocation for reasons not clear to me.

 Yeah, but honestly I don't think pip install --egg is that important a
 use case. I may be wrong (there's lots of ways people use pip that I
 know nothing of :-)) but as a starting point it might be OK just to
 say that at the same time as the --pip flag was introduced, "pip
 install --egg" was deprecated (and we explicitly document that pip
 install --egg is known to interact badly with setup.py --pip).
>>>
>>> I'd be fine with that too.  IIRC pip install --egg was introduced in
>>> part to work around problems with namespace packages.  This doesn't
>>> completely eliminate the need for that workaround, but it does reduce
>>> it.
>>
>> Huh? No, my understanding was that it was introduced solely to support
>> interop with folk using 'easy-install', and its considered deprecated
>> and delete-as-soon-as-practical.
>
> The original issue that motivated it did have to do with (lack of)
> interoperability of different ways namespace packages are implemented:
>
> https://github.com/pypa/pip/issues/3
>
> The fact that it introduced general backwards-compat for
> easy-install-like installation was a side "benefit", useful I'm sure
> to a few people.  But otherwise as you say, was intended to be deleted
> as soon as practical.
>
> Erik



-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Erik Bray]

On Thu, Dec 10, 2015 at 5:12 PM, Robert Collins
 wrote:
> On 8 December 2015 at 09:14, Erik Bray  wrote:
>> On Mon, Dec 7, 2015 at 2:40 PM, Paul Moore  wrote:
>>> On 7 December 2015 at 18:58, Erik Bray  wrote:
 I wasn't able to produce this problem.  Even with --no-binary
 specified pip installs (by default) with
 --single-version-externally-managed.  My prototype implicitly disables
 the --pip flag if --single-version-externally-managed was specified
 (true to the purpose of that flag).
>>>
>>> Ah - that was the bit I was missing, the
>>> --single-version-externally-managed flag can be used to trigger
>>> ignoring --pip.
>>>
 What *is* a problem is if --pip is in setup.cfg, and one invokes `pip
 install --egg .`.  I wasn't quite able to make that go into an
 infinite loop, but it did invoke pip.main recursively, and stuff broke
 on the second invocation for reasons not clear to me.
>>>
>>> Yeah, but honestly I don't think pip install --egg is that important a
>>> use case. I may be wrong (there's lots of ways people use pip that I
>>> know nothing of :-)) but as a starting point it might be OK just to
>>> say that at the same time as the --pip flag was introduced, "pip
>>> install --egg" was deprecated (and we explicitly document that pip
>>> install --egg is known to interact badly with setup.py --pip).
>>
>> I'd be fine with that too.  IIRC pip install --egg was introduced in
>> part to work around problems with namespace packages.  This doesn't
>> completely eliminate the need for that workaround, but it does reduce
>> it.
>
> Huh? No, my understanding was that it was introduced solely to support
> interop with folk using 'easy-install', and its considered deprecated
> and delete-as-soon-as-practical.

The original issue that motivated it did have to do with (lack of)
interoperability of different ways namespace packages are implemented:

https://github.com/pypa/pip/issues/3

The fact that it introduced general backwards-compat for
easy-install-like installation was a side "benefit", useful I'm sure
to a few people.  But otherwise as you say, was intended to be deleted
as soon as practical.

Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Robert Collins]

On 8 December 2015 at 09:14, Erik Bray  wrote:
> On Mon, Dec 7, 2015 at 2:40 PM, Paul Moore  wrote:
>> On 7 December 2015 at 18:58, Erik Bray  wrote:
>>> I wasn't able to produce this problem.  Even with --no-binary
>>> specified pip installs (by default) with
>>> --single-version-externally-managed.  My prototype implicitly disables
>>> the --pip flag if --single-version-externally-managed was specified
>>> (true to the purpose of that flag).
>>
>> Ah - that was the bit I was missing, the
>> --single-version-externally-managed flag can be used to trigger
>> ignoring --pip.
>>
>>> What *is* a problem is if --pip is in setup.cfg, and one invokes `pip
>>> install --egg .`.  I wasn't quite able to make that go into an
>>> infinite loop, but it did invoke pip.main recursively, and stuff broke
>>> on the second invocation for reasons not clear to me.
>>
>> Yeah, but honestly I don't think pip install --egg is that important a
>> use case. I may be wrong (there's lots of ways people use pip that I
>> know nothing of :-)) but as a starting point it might be OK just to
>> say that at the same time as the --pip flag was introduced, "pip
>> install --egg" was deprecated (and we explicitly document that pip
>> install --egg is known to interact badly with setup.py --pip).
>
> I'd be fine with that too.  IIRC pip install --egg was introduced in
> part to work around problems with namespace packages.  This doesn't
> completely eliminate the need for that workaround, but it does reduce
> it.

Huh? No, my understanding was that it was introduced solely to support
interop with folk using 'easy-install', and its considered deprecated
and delete-as-soon-as-practical.

-Rob
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Erik Bray]

On Mon, Dec 7, 2015 at 4:34 AM, Nick Coghlan  wrote:
> On 7 December 2015 at 17:07, Ronny Pfannschmidt
>  wrote:
>> That's a straw man, this has enough inconsistency potential to break many
>> edge cases in ugly ways,
>> So global setup is out.
>
> No, global set up *isn't* out - the inevitable edge cases won't matter
> to an application integrator if none of the components they're using
> hit them, and installation related problems have the virtue of being
> relatively straightforward to pick up in a continuous integration
> system.
>
> Using such a switch wouldn't be the right fit for everyone, but that's
> not the same as it being entirely useless.

Exactly--as both a library developer / maintainer and system
integrator I would find such a flag very useful (especially since I
can just set it in a config file and forget it).  It would be right
for me.  But wouldn't break anything for anyone else.

Ironically, the default behavior of `setup.py install`, on projects
that use setuptools, is to install an egg directory which is
*definitely* not for everybody, especially not anymore.  That's why
--single-version-externally-managed exists.  A --pip flag would be
very much like --single-version-externally-managed (sort of a
specialized extension of it) that also includes "do everything pip
does" which includes installing dependencies and copying
.egg-info/.dist-info to the appropriate location, which is what I want
to replace all instances of `setup.py install` with.  That includes
users running `setup.py install`, who have a hard enough time as it is
keeping up with Python build/installation "best practices" as it is.

Anyways, it has been frequently requested that setuptools change the
default behavior of the "install" command, so I wouldn't discount it
as a valid use case.  So far it hasn't been changed out of
backward-compat concerns, so making it loosely opt-in represents a
possible middle ground.

>> Projects themselves can really just switch to pip commands, same goes for
>> packagers and other tool makers
>>
>> Explicit is better than implicit, and in this case it also won't cost
>> additional maintenance on setuptools.
>> Please keep in mind, that setuptools is completely on volunteer time, and
>> the time given to it is scarce.
>
> Sure, that's why any decision on the desirability of this feature
> would be up to Jason as the setuptools lead. However, there's a
> trade-off to consider here, which is that offering this kind of global
> installer switch may help to lower the priority of some other
> easy_install enhancement requests.

Plus I've contributed to setuptools many times in the past (used to
have commit access on distribute too).  I'm offering to implement and
maintain this feature if it's decided desirable.  I think it *is*
desirable by definition--I desire it, and I suspect others would as
well.  I'd be more interested in technical reasons why it's a bad idea
but I haven't found any yet.

> That's a risk assessment trade-off on future bug reports against
> attempted pip support vs future RFEs against easy_install itself, as
> well as a priority assessment against other open changes proposed for
> setuptools. Those assessments may well come down on the side of "not
> worth the hassle", but the scope of the proposed change still falls a
> long way short of being a "maintenance horror".

I think the only maintenance horror right now is easy_install :)

Thanks,
Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Paul Moore]

On 7 December 2015 at 16:21, Erik Bray  wrote:
> Exactly--as both a library developer / maintainer and system
> integrator I would find such a flag very useful (especially since I
> can just set it in a config file and forget it).  It would be right
> for me.  But wouldn't break anything for anyone else.
>
> Ironically, the default behavior of `setup.py install`, on projects
> that use setuptools, is to install an egg directory which is
> *definitely* not for everybody, especially not anymore.  That's why
> --single-version-externally-managed exists.  A --pip flag would be
> very much like --single-version-externally-managed (sort of a
> specialized extension of it) that also includes "do everything pip
> does" which includes installing dependencies and copying
> .egg-info/.dist-info to the appropriate location, which is what I want
> to replace all instances of `setup.py install` with.  That includes
> users running `setup.py install`, who have a hard enough time as it is
> keeping up with Python build/installation "best practices" as it is.

One thing that bothers me about this proposal. If someone does "pip
install --no-binary" for your package, and you have the "--pip" flag
in your setup.cfg, pip will use "setup.py install" to do the install.
Which, if I understand this proposal correctly, will attempt to "fall
back" to pip because "--pip" is in setup.cfg. Which results in an
infinite loop of pip and setup.py invoking each other.

I'm not sure how pip could detect a situation like this, so there's a
risk of some *very* obscure corner cases, which I'm sure people will
end up hitting.

As a user level command line flag, "setup.py install --pip" isn't much
better than "pip install ."
As a project config, we get the issue noted above, and the user has to
edit the project code to fix it.
As a per-user or global config, we get the issue above, but we could
reasonably say it's the user's mistake and the user has the means to
fix it. But it's still not a great UX.

It's quite possible I'm missing something here (at a minimum, I'm
making huge assumptions about how the feature would be implemented)
but I think the behaviour needs to be thought through in a bit more
detail.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Nick Coghlan]

On 7 December 2015 at 17:07, Ronny Pfannschmidt
 wrote:
> That's a straw man, this has enough inconsistency potential to break many
> edge cases in ugly ways,
> So global setup is out.

No, global set up *isn't* out - the inevitable edge cases won't matter
to an application integrator if none of the components they're using
hit them, and installation related problems have the virtue of being
relatively straightforward to pick up in a continuous integration
system.

Using such a switch wouldn't be the right fit for everyone, but that's
not the same as it being entirely useless.

> Projects themselves can really just switch to pip commands, same goes for
> packagers and other tool makers
>
> Explicit is better than implicit, and in this case it also won't cost
> additional maintenance on setuptools.
> Please keep in mind, that setuptools is completely on volunteer time, and
> the time given to it is scarce.

Sure, that's why any decision on the desirability of this feature
would be up to Jason as the setuptools lead. However, there's a
trade-off to consider here, which is that offering this kind of global
installer switch may help to lower the priority of some other
easy_install enhancement requests.

That's a risk assessment trade-off on future bug reports against
attempted pip support vs future RFEs against easy_install itself, as
well as a priority assessment against other open changes proposed for
setuptools. Those assessments may well come down on the side of "not
worth the hassle", but the scope of the proposed change still falls a
long way short of being a "maintenance horror".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Paul Moore]

On 7 December 2015 at 18:58, Erik Bray  wrote:
> I wasn't able to produce this problem.  Even with --no-binary
> specified pip installs (by default) with
> --single-version-externally-managed.  My prototype implicitly disables
> the --pip flag if --single-version-externally-managed was specified
> (true to the purpose of that flag).

Ah - that was the bit I was missing, the
--single-version-externally-managed flag can be used to trigger
ignoring --pip.

> What *is* a problem is if --pip is in setup.cfg, and one invokes `pip
> install --egg .`.  I wasn't quite able to make that go into an
> infinite loop, but it did invoke pip.main recursively, and stuff broke
> on the second invocation for reasons not clear to me.

Yeah, but honestly I don't think pip install --egg is that important a
use case. I may be wrong (there's lots of ways people use pip that I
know nothing of :-)) but as a starting point it might be OK just to
say that at the same time as the --pip flag was introduced, "pip
install --egg" was deprecated (and we explicitly document that pip
install --egg is known to interact badly with setup.py --pip).

Anyway, thanks for the explanation, I see what you're intending now.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Erik Bray]

On Mon, Dec 7, 2015 at 2:40 PM, Paul Moore  wrote:
> On 7 December 2015 at 18:58, Erik Bray  wrote:
>> I wasn't able to produce this problem.  Even with --no-binary
>> specified pip installs (by default) with
>> --single-version-externally-managed.  My prototype implicitly disables
>> the --pip flag if --single-version-externally-managed was specified
>> (true to the purpose of that flag).
>
> Ah - that was the bit I was missing, the
> --single-version-externally-managed flag can be used to trigger
> ignoring --pip.
>
>> What *is* a problem is if --pip is in setup.cfg, and one invokes `pip
>> install --egg .`.  I wasn't quite able to make that go into an
>> infinite loop, but it did invoke pip.main recursively, and stuff broke
>> on the second invocation for reasons not clear to me.
>
> Yeah, but honestly I don't think pip install --egg is that important a
> use case. I may be wrong (there's lots of ways people use pip that I
> know nothing of :-)) but as a starting point it might be OK just to
> say that at the same time as the --pip flag was introduced, "pip
> install --egg" was deprecated (and we explicitly document that pip
> install --egg is known to interact badly with setup.py --pip).

I'd be fine with that too.  IIRC pip install --egg was introduced in
part to work around problems with namespace packages.  This doesn't
completely eliminate the need for that workaround, but it does reduce
it.

In either case, if the --pip feature is implemented in setuptools it
would still be good to at least have some workaround for installing
with pip install --egg, at least temporarily.  My goal here was to not
break any existing functionality when --pip is specified (either via
command line or setup.cfg).

> Anyway, thanks for the explanation, I see what you're intending now.

Great!
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Erik Bray]

On Mon, Dec 7, 2015 at 11:45 AM, Paul Moore  wrote:
> On 7 December 2015 at 16:21, Erik Bray  wrote:
>> Exactly--as both a library developer / maintainer and system
>> integrator I would find such a flag very useful (especially since I
>> can just set it in a config file and forget it).  It would be right
>> for me.  But wouldn't break anything for anyone else.
>>
>> Ironically, the default behavior of `setup.py install`, on projects
>> that use setuptools, is to install an egg directory which is
>> *definitely* not for everybody, especially not anymore.  That's why
>> --single-version-externally-managed exists.  A --pip flag would be
>> very much like --single-version-externally-managed (sort of a
>> specialized extension of it) that also includes "do everything pip
>> does" which includes installing dependencies and copying
>> .egg-info/.dist-info to the appropriate location, which is what I want
>> to replace all instances of `setup.py install` with.  That includes
>> users running `setup.py install`, who have a hard enough time as it is
>> keeping up with Python build/installation "best practices" as it is.
>
> One thing that bothers me about this proposal. If someone does "pip
> install --no-binary" for your package, and you have the "--pip" flag
> in your setup.cfg, pip will use "setup.py install" to do the install.
> Which, if I understand this proposal correctly, will attempt to "fall
> back" to pip because "--pip" is in setup.cfg. Which results in an
> infinite loop of pip and setup.py invoking each other.

I wasn't able to produce this problem.  Even with --no-binary
specified pip installs (by default) with
--single-version-externally-managed.  My prototype implicitly disables
the --pip flag if --single-version-externally-managed was specified
(true to the purpose of that flag).

What *is* a problem is if --pip is in setup.cfg, and one invokes `pip
install --egg .`.  I wasn't quite able to make that go into an
infinite loop, but it did invoke pip.main recursively, and stuff broke
on the second invocation for reasons not clear to me.

This is easily worked around, however, by detecting, from the install
command, if we're already using pip.  As a quick hack I added to
finalize_options:

if 'pip' in sys.modules:
self.pip = False

This did the trick.  pip ran fine and installed the package as an egg
using the standard setup.py install.  I don't think a more robust
solution would be hard.  pip could set an environment variable or even
a variable in the pip module itself to indicate that pip is already
being invoked to install this package.  There may even be something
like that already in pip that I'm not aware of.

> I'm not sure how pip could detect a situation like this, so there's a
> risk of some *very* obscure corner cases, which I'm sure people will
> end up hitting.

As mentioned above I don't think it should be pip's job to detect this
situation.  But if the setuptools install command can detect that
we're already in pip then the job is done.

> As a user level command line flag, "setup.py install --pip" isn't much
> better than "pip install ."
> As a project config, we get the issue noted above, and the user has to
> edit the project code to fix it.
> As a per-user or global config, we get the issue above, but we could
> reasonably say it's the user's mistake and the user has the means to
> fix it. But it's still not a great UX.

I don't think so either.  But it's also not great UX that we've told
users for decades that the way to install a Python package is to run
`python setup.py install`, but now the default behavior of that (for
packages using setuptools) which is to install a .egg, is old and bad.
I get confusion about this from users *frequently*.  It's only worse
that egg installs and flat installs are incompatible with each other
with respect to namespace packages.

If it would help, `setup.py install --pip` could also display a
warning that users should run `pip install .` instead.  To cut down on
noise I might only do this if the --pip option came from setup.cfg,
rather than when it's explicitly asked for via the command line.  This
would serve to inform users who don't know any better.

> It's quite possible I'm missing something here (at a minimum, I'm
> making huge assumptions about how the feature would be implemented)
> but I think the behaviour needs to be thought through in a bit more
> detail.

Completely agree!  I know there are corner cases I haven't thought of.
I'm also undecided on whether --pip should invoke pip.main() within
the same process, or run pip from a subprocess.  The former seems
sufficient but I don't know all the cases.

Thanks,
Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Nick Coghlan]

On 4 December 2015 at 17:55, Ronny Pfannschmidt
 wrote:
> But it still needs a change in command
> A direct switch to a real pip command comes with a free implementation and 
> zero additional features to maintain in setuptools

The key point here is that opting in to this new behaviour would just
mean setting a new configuration flag in pydistutils.cfg on the system
doing the build/install, or adding a new option to the list of options
passed to the install command. That's a significantly lower barrier to
entry than finding all of the occurrences of "./setup.py install" in
existing packaging and deployment scripts and converting them over to
use "pip install" instead, especially since all the *other* command
line flags would continue to be the setuptools flags rather than the
pip ones.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Ronny Pfannschmidt]

That's a straw man, this has enough inconsistency potential to break many edge 
cases in ugly ways,
So global setup is out.
Projects themselves can really just switch to pip commands, same goes for 
packagers and other tool makers

Explicit is better than implicit, and in this case it also won't cost 
additional maintenance on setuptools.
Please keep in mind, that setuptools is completely on volunteer time, and the 
time given to it is scarce.



Am 7. Dezember 2015 05:36:42 MEZ, schrieb Nick Coghlan :
>On 4 December 2015 at 17:55, Ronny Pfannschmidt
> wrote:
>> But it still needs a change in command
>> A direct switch to a real pip command comes with a free
>implementation and zero additional features to maintain in setuptools
>
>The key point here is that opting in to this new behaviour would just
>mean setting a new configuration flag in pydistutils.cfg on the system
>doing the build/install, or adding a new option to the list of options
>passed to the install command. That's a significantly lower barrier to
>entry than finding all of the occurrences of "./setup.py install" in
>existing packaging and deployment scripts and converting them over to
>use "pip install" instead, especially since all the *other* command
>line flags would continue to be the setuptools flags rather than the
>pip ones.
>
>Cheers,
>Nick.
>
>-- 
>Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia

-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Ronny Pfannschmidt]

But it still needs a change in command
A direct switch to a real pip command comes with a free implementation and zero 
additional features to maintain in setuptools

Am 3. Dezember 2015 23:01:21 MEZ, schrieb Erik Bray :
>On Thu, Dec 3, 2015 at 3:37 PM, Ronny Pfannschmidt
> wrote:
>> Lets avoid getting setuptools even more complex in that way
>
>It's not deeply complex--it's just bypassing the normal behavior of
>using easy_install.  In my example code I subclassed
>the existing command, but an easier approach would be to build it into
>setuptools.
>
>The way the 'install' command works in setuptools is to hand
>installation off to the easy_install command unless
>--single-version-externally-managed is specified (as well as
>--record).  Otherwise it hands installation off to the default
>base 'install' command of distutils.
>
>This proposal just adds a third option for what actual installer the
>'install' command should
>use.  But it's opt-in instead of forced on any package by default (as
>easy_install is forced on us by setuptools).
>
>> Putting pip-ish-ness on top of easy install is a maintenance horror
>and I don't think setuptools does has enough consistent developer
>resources to handle something like that
>
>It doesn't put anything "on top of" easy_install; it ignores
>easy_install.
>
>> Instead let's just give options to destroy the normal install command
>from setup.py so projects can phase out easy install forcefully making
>downstream require patches or pip usage
>
>That's the goal, yes.  This is just offering a transitional tool.
>
>
>Erik
>
>>
>> Am 3. Dezember 2015 21:06:06 MEZ, schrieb Erik Bray
>:
>>>Hi all,
>>>
>>>I've been on vacation for a bit in general, and on vacation from this
>>>mailing list even longer.  I'm not entirely caught up yet on the
>>>latest developments so apologies if something like this is entirely
>>>moot by now.
>>>
>>>But I have seen some discussions here and in other lists related to
>>>using pip for all installations, and phasing out the old distutils
>>>`./setup.py install` (eg. [1]).  This is not a new discussion, and
>>>there are many related discussions, for example, about changing
>>>setuptools not to default to egg installs anymore (see [2]).
>>>
>>>I'm definitely all for this in general.  Nowadays whenever I install
>a
>>>package from source I run `pip install .`  But of course there are a
>>>lot of existing tools, not to mention folk wisdom, assuming
>>>`./setup.py install`.  We also don't want to change the long-existing
>>>behavior in setuptools.
>>>
>>>I have a modest proposal for a small addition to setuptools that
>might
>>>be helpful in a transition away from using setuptools+distutils for
>>>installation.  This would be to add a `--pip` flag to setuptools'
>>>install command (or possibly straight in distutils too, but might as
>>>well start with setuptools).
>>>
>>>Therefore, running
>>>
>>>$ ./setup.py install --pip
>>>
>>>would be equivalent to running
>>>
>>>$ pip install .
>>>
>>>By extension, running
>>>
>>>$ ./setup.py install --pip --arg1 --arg2=foo
>>>
>>>would be equivalent to
>>>
>>>$ pip install --install-option="--arg1" --install-option="--arg2=foo"
>.
>>>
>>>and so on.
>>>
>>>By making `--pip` opt-in, it does not automatically break backward
>>>compatibility for users expecting `./setup.py install` to use
>>>easy_install.  However, individual users may opt into it globally by
>>>adding
>>>
>>>[install]
>>>pip = True
>>>
>>>to their .pydistutils.cfg.  Similarly, package authors who are
>>>confident that none of their users are ever going to care about egg
>>>installs (e.g. me) can add the same to their project's setup.cfg.
>>>
>>>Does something like this have any merit?  I hacked together a
>>>prototype which follows the sig line.  It's just a proof of concept,
>>>but seems to work in the most basic cases.  I'd like to add it to my
>>>own projects too, but would appreciate some peer review.
>>>
>>>Thanks,
>>>Erik
>>>
>>>
>>>
>>>[1]
>>>https://mail.scipy.org/pipermail/numpy-discussion/2015-November/074142.html
>>>[2]
>>>https://bitbucket.org/pypa/setuptools/issues/371/setuptools-and-state-of-pep-376
>>>
>>>
>>>$ cat pipinstall.py
>>>from distutils.errors import DistutilsArgError
>>>from setuptools.command.install import install as SetuptoolsInstall
>>>
>>>
>>>class PipInstall(SetuptoolsInstall):
>>>command_name = 'install'
>>>user_options = SetuptoolsInstall.user_options + [
>>>('pip', None, 'install using pip; ignored when also using '
>>>  '--single-version-externally-managed')
>>>]
>>>boolean_options = SetuptoolsInstall.boolean_options + ['pip']
>>>
>>>def initialize_options(self):
>>>SetuptoolsInstall.initialize_options(self)
>>>self.pip = False
>>>
>>>def finalize_options(self):
>>>SetuptoolsInstall.finalize_options(self)
>>>
>>>if self.single_version_externally_managed:

[Distutils] setup.py install using pip

[Erik Bray]

Hi all,

I've been on vacation for a bit in general, and on vacation from this
mailing list even longer.  I'm not entirely caught up yet on the
latest developments so apologies if something like this is entirely
moot by now.

But I have seen some discussions here and in other lists related to
using pip for all installations, and phasing out the old distutils
`./setup.py install` (eg. [1]).  This is not a new discussion, and
there are many related discussions, for example, about changing
setuptools not to default to egg installs anymore (see [2]).

I'm definitely all for this in general.  Nowadays whenever I install a
package from source I run `pip install .`  But of course there are a
lot of existing tools, not to mention folk wisdom, assuming
`./setup.py install`.  We also don't want to change the long-existing
behavior in setuptools.

I have a modest proposal for a small addition to setuptools that might
be helpful in a transition away from using setuptools+distutils for
installation.  This would be to add a `--pip` flag to setuptools'
install command (or possibly straight in distutils too, but might as
well start with setuptools).

Therefore, running

$ ./setup.py install --pip

would be equivalent to running

$ pip install .

By extension, running

$ ./setup.py install --pip --arg1 --arg2=foo

would be equivalent to

$ pip install --install-option="--arg1" --install-option="--arg2=foo" .

and so on.

By making `--pip` opt-in, it does not automatically break backward
compatibility for users expecting `./setup.py install` to use
easy_install.  However, individual users may opt into it globally by
adding

[install]
pip = True

to their .pydistutils.cfg.  Similarly, package authors who are
confident that none of their users are ever going to care about egg
installs (e.g. me) can add the same to their project's setup.cfg.

Does something like this have any merit?  I hacked together a
prototype which follows the sig line.  It's just a proof of concept,
but seems to work in the most basic cases.  I'd like to add it to my
own projects too, but would appreciate some peer review.

Thanks,
Erik



[1] https://mail.scipy.org/pipermail/numpy-discussion/2015-November/074142.html
[2] 
https://bitbucket.org/pypa/setuptools/issues/371/setuptools-and-state-of-pep-376


$ cat pipinstall.py
from distutils.errors import DistutilsArgError
from setuptools.command.install import install as SetuptoolsInstall


class PipInstall(SetuptoolsInstall):
command_name = 'install'
user_options = SetuptoolsInstall.user_options + [
('pip', None, 'install using pip; ignored when also using '
  '--single-version-externally-managed')
]
boolean_options = SetuptoolsInstall.boolean_options + ['pip']

def initialize_options(self):
SetuptoolsInstall.initialize_options(self)
self.pip = False

def finalize_options(self):
SetuptoolsInstall.finalize_options(self)

if self.single_version_externally_managed:
self.pip = False

if self.pip:
try:
import pip
except ImportError:
raise DistutilsArgError(
'pip must be installed in order to install with the '
'--pip option')

def run(self):
if self.pip:
import pip
opts = (['install', '--ignore-installed'] +
['--install-option="{0}"'.format(opt)
 for opt in self._get_command_line_opts()])
pip.main(opts + ['.'])
else:
SetuptoolsInstall.run(self)

def _get_command_line_opts(self):
# Generate a mapping from the attribute name associated with a
# command-line option to the name of the command-line option (including
# an = if the option takes an argument)
attr_to_opt = dict((opt[0].rstrip('=').replace('-', '_'), opt[0])
   for opt in self.user_options)

opt_dict = self.distribution.get_option_dict(self.get_command_name())
opts = []

for attr, value in opt_dict.items():
if value[0] != 'command line' or attr == 'pip':
# Only look at options passed in on the command line (ignoring
# the pip option itself)
continue

opt = attr_to_opt[attr]

if opt in self.boolean_options:
opts.append('--' + opt)
else:
opts.append('--{0}{1}'.format(opt, value[1]))

return opts

@staticmethod
def _called_from_setup(run_frame):
# A hack to work around a setuptools hack
return SetuptoolsInstall._called_from_setup(run_frame.f_back)
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.py install using pip

[Erik Bray]

On Thu, Dec 3, 2015 at 3:37 PM, Ronny Pfannschmidt
 wrote:
> Lets avoid getting setuptools even more complex in that way

It's not deeply complex--it's just bypassing the normal behavior of
using easy_install.  In my example code I subclassed
the existing command, but an easier approach would be to build it into
setuptools.

The way the 'install' command works in setuptools is to hand
installation off to the easy_install command unless
--single-version-externally-managed is specified (as well as
--record).  Otherwise it hands installation off to the default
base 'install' command of distutils.

This proposal just adds a third option for what actual installer the
'install' command should
use.  But it's opt-in instead of forced on any package by default (as
easy_install is forced on us by setuptools).

> Putting pip-ish-ness on top of easy install is a maintenance horror and I 
> don't think setuptools does has enough consistent developer resources to 
> handle something like that

It doesn't put anything "on top of" easy_install; it ignores easy_install.

> Instead let's just give options to destroy the normal install command from 
> setup.py so projects can phase out easy install forcefully making downstream 
> require patches or pip usage

That's the goal, yes.  This is just offering a transitional tool.


Erik

>
> Am 3. Dezember 2015 21:06:06 MEZ, schrieb Erik Bray :
>>Hi all,
>>
>>I've been on vacation for a bit in general, and on vacation from this
>>mailing list even longer.  I'm not entirely caught up yet on the
>>latest developments so apologies if something like this is entirely
>>moot by now.
>>
>>But I have seen some discussions here and in other lists related to
>>using pip for all installations, and phasing out the old distutils
>>`./setup.py install` (eg. [1]).  This is not a new discussion, and
>>there are many related discussions, for example, about changing
>>setuptools not to default to egg installs anymore (see [2]).
>>
>>I'm definitely all for this in general.  Nowadays whenever I install a
>>package from source I run `pip install .`  But of course there are a
>>lot of existing tools, not to mention folk wisdom, assuming
>>`./setup.py install`.  We also don't want to change the long-existing
>>behavior in setuptools.
>>
>>I have a modest proposal for a small addition to setuptools that might
>>be helpful in a transition away from using setuptools+distutils for
>>installation.  This would be to add a `--pip` flag to setuptools'
>>install command (or possibly straight in distutils too, but might as
>>well start with setuptools).
>>
>>Therefore, running
>>
>>$ ./setup.py install --pip
>>
>>would be equivalent to running
>>
>>$ pip install .
>>
>>By extension, running
>>
>>$ ./setup.py install --pip --arg1 --arg2=foo
>>
>>would be equivalent to
>>
>>$ pip install --install-option="--arg1" --install-option="--arg2=foo" .
>>
>>and so on.
>>
>>By making `--pip` opt-in, it does not automatically break backward
>>compatibility for users expecting `./setup.py install` to use
>>easy_install.  However, individual users may opt into it globally by
>>adding
>>
>>[install]
>>pip = True
>>
>>to their .pydistutils.cfg.  Similarly, package authors who are
>>confident that none of their users are ever going to care about egg
>>installs (e.g. me) can add the same to their project's setup.cfg.
>>
>>Does something like this have any merit?  I hacked together a
>>prototype which follows the sig line.  It's just a proof of concept,
>>but seems to work in the most basic cases.  I'd like to add it to my
>>own projects too, but would appreciate some peer review.
>>
>>Thanks,
>>Erik
>>
>>
>>
>>[1]
>>https://mail.scipy.org/pipermail/numpy-discussion/2015-November/074142.html
>>[2]
>>https://bitbucket.org/pypa/setuptools/issues/371/setuptools-and-state-of-pep-376
>>
>>
>>$ cat pipinstall.py
>>from distutils.errors import DistutilsArgError
>>from setuptools.command.install import install as SetuptoolsInstall
>>
>>
>>class PipInstall(SetuptoolsInstall):
>>command_name = 'install'
>>user_options = SetuptoolsInstall.user_options + [
>>('pip', None, 'install using pip; ignored when also using '
>>  '--single-version-externally-managed')
>>]
>>boolean_options = SetuptoolsInstall.boolean_options + ['pip']
>>
>>def initialize_options(self):
>>SetuptoolsInstall.initialize_options(self)
>>self.pip = False
>>
>>def finalize_options(self):
>>SetuptoolsInstall.finalize_options(self)
>>
>>if self.single_version_externally_managed:
>>self.pip = False
>>
>>if self.pip:
>>try:
>>import pip
>>except ImportError:
>>raise DistutilsArgError(
>>  'pip must be installed in order to install with the '
>>'--pip option')
>>
>>def run(self):
>>if self.pip:
>>import pip

Re: [Distutils] setup.py install using pip

[Ronny Pfannschmidt]

Lets avoid getting setuptools even more complex in that way 

Putting pip-ish-ness on top of easy install is a maintenance horror and I don't 
think setuptools does has enough consistent developer resources to handle 
something like that

Instead let's just give options to destroy the normal install command from 
setup.py so projects can phase out easy install forcefully making downstream 
require patches or pip usage



Am 3. Dezember 2015 21:06:06 MEZ, schrieb Erik Bray :
>Hi all,
>
>I've been on vacation for a bit in general, and on vacation from this
>mailing list even longer.  I'm not entirely caught up yet on the
>latest developments so apologies if something like this is entirely
>moot by now.
>
>But I have seen some discussions here and in other lists related to
>using pip for all installations, and phasing out the old distutils
>`./setup.py install` (eg. [1]).  This is not a new discussion, and
>there are many related discussions, for example, about changing
>setuptools not to default to egg installs anymore (see [2]).
>
>I'm definitely all for this in general.  Nowadays whenever I install a
>package from source I run `pip install .`  But of course there are a
>lot of existing tools, not to mention folk wisdom, assuming
>`./setup.py install`.  We also don't want to change the long-existing
>behavior in setuptools.
>
>I have a modest proposal for a small addition to setuptools that might
>be helpful in a transition away from using setuptools+distutils for
>installation.  This would be to add a `--pip` flag to setuptools'
>install command (or possibly straight in distutils too, but might as
>well start with setuptools).
>
>Therefore, running
>
>$ ./setup.py install --pip
>
>would be equivalent to running
>
>$ pip install .
>
>By extension, running
>
>$ ./setup.py install --pip --arg1 --arg2=foo
>
>would be equivalent to
>
>$ pip install --install-option="--arg1" --install-option="--arg2=foo" .
>
>and so on.
>
>By making `--pip` opt-in, it does not automatically break backward
>compatibility for users expecting `./setup.py install` to use
>easy_install.  However, individual users may opt into it globally by
>adding
>
>[install]
>pip = True
>
>to their .pydistutils.cfg.  Similarly, package authors who are
>confident that none of their users are ever going to care about egg
>installs (e.g. me) can add the same to their project's setup.cfg.
>
>Does something like this have any merit?  I hacked together a
>prototype which follows the sig line.  It's just a proof of concept,
>but seems to work in the most basic cases.  I'd like to add it to my
>own projects too, but would appreciate some peer review.
>
>Thanks,
>Erik
>
>
>
>[1]
>https://mail.scipy.org/pipermail/numpy-discussion/2015-November/074142.html
>[2]
>https://bitbucket.org/pypa/setuptools/issues/371/setuptools-and-state-of-pep-376
>
>
>$ cat pipinstall.py
>from distutils.errors import DistutilsArgError
>from setuptools.command.install import install as SetuptoolsInstall
>
>
>class PipInstall(SetuptoolsInstall):
>command_name = 'install'
>user_options = SetuptoolsInstall.user_options + [
>('pip', None, 'install using pip; ignored when also using '
>  '--single-version-externally-managed')
>]
>boolean_options = SetuptoolsInstall.boolean_options + ['pip']
>
>def initialize_options(self):
>SetuptoolsInstall.initialize_options(self)
>self.pip = False
>
>def finalize_options(self):
>SetuptoolsInstall.finalize_options(self)
>
>if self.single_version_externally_managed:
>self.pip = False
>
>if self.pip:
>try:
>import pip
>except ImportError:
>raise DistutilsArgError(
>  'pip must be installed in order to install with the '
>'--pip option')
>
>def run(self):
>if self.pip:
>import pip
>opts = (['install', '--ignore-installed'] +
>['--install-option="{0}"'.format(opt)
> for opt in self._get_command_line_opts()])
>pip.main(opts + ['.'])
>else:
>SetuptoolsInstall.run(self)
>
>def _get_command_line_opts(self):
># Generate a mapping from the attribute name associated with a
># command-line option to the name of the command-line option (including
># an = if the option takes an argument)
>  attr_to_opt = dict((opt[0].rstrip('=').replace('-', '_'), opt[0])
>   for opt in self.user_options)
>
>  opt_dict = self.distribution.get_option_dict(self.get_command_name())
>opts = []
>
>for attr, value in opt_dict.items():
>if value[0] != 'command line' or attr == 'pip':
> # Only look at options passed in on the command line (ignoring
># the pip option itself)
>continue
>
>opt = attr_to_opt[attr]
>
>if opt in