Re: [Distutils] pip behavior

[Robert Collins]

Its almost certainly the version of pip within each environment.

-Rob

On 27 June 2016 at 23:45, Steve Spicklemire  wrote:
> Hi Distutils-SIG Folks,
>
> If this is not the best place for a ‘pip’ question, please direct me to a 
> better destination.
>
> I have several virtual environments. As far as I know they should all be 
> about the same WRT the python binary that was used to create them (home-brew 
> python2.7.11/OSX). However they have different behavior WRT pip install. I’m 
> trying to track down the cues that pip uses to choose a binary wheel, or a 
> source/build install. I’ve been using ‘pip -v’ to get some hints, but it’s 
> still not clear. I’ve saved two log files that illustrate the issue:
>
> https://dl.dropboxusercontent.com/u/20562746/pydev_pip_log.txt
>
> https://dl.dropboxusercontent.com/u/20562746/pytest_pip_log.txt
>
> So my question is: How can I determine why pip is choosing ‘wheel’ in some 
> cases and ‘source’ in others when I have the same python build in both cases?
>
> thanks!
> -steve
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] pip behavior

[Steve Spicklemire]

Hi Distutils-SIG Folks,

If this is not the best place for a ‘pip’ question, please direct me to a 
better destination.

I have several virtual environments. As far as I know they should all be about 
the same WRT the python binary that was used to create them (home-brew 
python2.7.11/OSX). However they have different behavior WRT pip install. I’m 
trying to track down the cues that pip uses to choose a binary wheel, or a 
source/build install. I’ve been using ‘pip -v’ to get some hints, but it’s 
still not clear. I’ve saved two log files that illustrate the issue:

https://dl.dropboxusercontent.com/u/20562746/pydev_pip_log.txt

https://dl.dropboxusercontent.com/u/20562746/pytest_pip_log.txt

So my question is: How can I determine why pip is choosing ‘wheel’ in some 
cases and ‘source’ in others when I have the same python build in both cases?

thanks!
-steve


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

Indeed. Let's move the further discussion over to
https://github.com/pypa/pip/issues/3786.

On Mon, 27 Jun 2016 at 12:57 Robert Collins 
wrote:

> I've replied in the github issue; I don't really want to carry out
> *two* conversations, so perhaps we can pick one forum and stick there?
>
> -Rob
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

On Mon, 27 Jun 2016 at 12:21 Ralf Gommers  wrote:

> On Mon, Jun 27, 2016 at 8:36 AM, Pradyun Gedam 
> wrote:
>
>> Hello List!
>>
>> I feel it’s fine to hold back the other changes for later but the
>> upgrade-strategy change should get shipped out to the world as quickly as
>> possible. Even how the change is exposed the user can also be discussed
>> later.
>>
> What do you mean by "ship" if you say the behavior can still be changed
> later?
>

Sorry for the confusion and a useless-mail! Allow me to re-phrase that
statement...

I think it's fine if we hold back the changes to install-as-upstall and
--target for later. The
upgrade-strategy change should get released to the world as quickly as
possible. If no one has any outstanding issues with switching over to
non-eager upgrades by default, may we start a discussing on how the change
is to be exposed the user?


>
>> I request the list members to focus on *only* the change of the default
>> upgrade strategy to be non-eager.
>>
>> Does anyone have any concerns regarding the change of the default upgrade
>> strategy to be non-eager? If not, let’s get *just* that shipped out as
>> soon as possible.
>>
> The concerns were always with how to change it
>

There were security-related concerns raised by Robert which were addressed
[1] by Donald.


> one of:
> (1) add "pip upgrade"
> (2) change behavior of "pip install --upgrade"
> (3) change behavior of "pip install"
>
> Your sentence above suggests you're asking for agreement on (2), but I
> think you want agreement on (3) right? At least that was the conclusion of
> your PEP-style writeup.
>
>
As it stands, we currently have (3) implemented (as proposed in that
write-up) since that was what was extensively discussed and decided over at
Github. If no one has issues with it, let's go ahead with it. If there is
anyone has issues, I'm fine with going (2) as well. I do not like (1) due
to the weird model it creates providing 2 ways to do what most people would
see as one thing.

Basically, If we were voting, -1 for (1), +0.5 for (2), +1 for (3).

Personally I don't have a preference anymore, as long as a choice is made
> so we don't remain stuck where we are now.
>
> Ralf
>

[1] There's probably a better word for this than "addressed".

 --

Pradyun Gedam
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Robert Collins]

I've replied in the github issue; I don't really want to carry out
*two* conversations, so perhaps we can pick one forum and stick there?

-Rob
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

On Sun, 26 Jun 2016 at 23:02 Donald Stufft  wrote:

>
> On Jun 25, 2016, at 6:25 AM, Pradyun Gedam  wrote:
>
> There is currently a proposal to change the behaviour to pip install to
> upgrade a package that is passed even if it is already installed.
>
> This behaviour change is accompanied with a change in the upgrade strategy
> - pip would stop “eagerly” upgrading dependencies and would become more
> conservative, upgrading a dependency only when it doesn’t meet lower
> constraints of the newer version of a parent. Moreover, the behaviour of pip
> install --target would also be changed so that --upgrade no longer
> affects it.
>
> I think bundling these two changes (and I think I might have been the one
> that originally suggested it) is making this discussion harder than it
> needs to be as folks are having to fight on multiple different fronts at
> once. I think the change to the default behavior of pip install is
> dependent on the change to —upgrade, so I suggest we focus on the change to
> —upgrade first, changing from a “recursive” to a “conservative” strategy.
> Once we get that change figured out and landed then we can worry about what
> to do with pip install.
>

You were. In fact, the majority swayed in favour of changing the behaviour
of pip install post one of your comments on Github.

I'll be happier *only* seeing in change the behaviour of --upgrade and not
--target or pip install. It reduces the number of things that changes from
3 to 1. Much easier to discuss about.

I’m not going to repeat the entire post, but I just made a fairly lengthy
> comment at https://github.com/pypa/pip/issues/3786#issuecomment-228611906 but
> to try and boil it down to a few points:
>

Thanks for this.


> * ``pip install —upgrade`` is not a good security mechanism, relying on it
> is inconsistent at best. If we want to support trying to keep people on
> secure versions of software we need a better mechanism than this anyways,
> so we shouldn’t let it influence our choice here.
>

AFAIK, this was the only outstanding concern raised against having a
non-eager (conservative) upgrade strategy.

* For the general case, it’s not going to matter a lot which way we go, but
> not upgrading has the greatest chance of not breaking *already installed
> software*.
>

I strongly agree with this. Another thing worth a mention is that it's
easier to get the lower bounds of your requirements correct, rather than
upper bounds.


> * For the hard-to-upgrade case, the current behavior is so bad that people
> are outright attempting to subvert the way pip typically behaviors, *AND*
> advocating for other’s to do the same, in an attempt to escape that
> behavior. I think that this is not a good place to be in.
>

Ditto.

—
>
> Donald Stufft
>

Happy-to-see-Donald's-response-ly,
Pradyun Gedam
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Ralf Gommers]

On Mon, Jun 27, 2016 at 8:36 AM, Pradyun Gedam  wrote:

> Hello List!
>
> I feel it’s fine to hold back the other changes for later but the
> upgrade-strategy change should get shipped out to the world as quickly as
> possible. Even how the change is exposed the user can also be discussed
> later.
>
What do you mean by "ship" if you say the behavior can still be changed
later?


> I request the list members to focus on *only* the change of the default
> upgrade strategy to be non-eager.
>
> Does anyone have any concerns regarding the change of the default upgrade
> strategy to be non-eager? If not, let’s get *just* that shipped out as
> soon as possible.
>
The concerns were always with how to change it, one of:
(1) add "pip upgrade"
(2) change behavior of "pip install --upgrade"
(3) change behavior of "pip install"

Your sentence above suggests you're asking for agreement on (2), but I
think you want agreement on (3) right? At least that was the conclusion of
your PEP-style writeup.

Personally I don't have a preference anymore, as long as a choice is made
so we don't remain stuck where we are now.

Ralf




> Cheers,
> Pradyun Gedam
>
> On Mon, 27 Jun 2016 at 12:02 Pradyun Gedam pradyu...@gmail.com
>  wrote:
>
> On Sun, 26 Jun 2016 at 23:02 Donald Stufft  wrote:
>>
>>>
>>> On Jun 25, 2016, at 6:25 AM, Pradyun Gedam  wrote:
>>>
>>> There is currently a proposal to change the behaviour to pip install to
>>> upgrade a package that is passed even if it is already installed.
>>>
>>> This behaviour change is accompanied with a change in the upgrade
>>> strategy - pip would stop “eagerly” upgrading dependencies and would become
>>> more conservative, upgrading a dependency only when it doesn’t meet lower
>>> constraints of the newer version of a parent. Moreover, the behaviour of
>>>  pip install --target would also be changed so that --upgrade no longer
>>> affects it.
>>>
>>> I think bundling these two changes (and I think I might have been the
>>> one that originally suggested it) is making this discussion harder than it
>>> needs to be as folks are having to fight on multiple different fronts at
>>> once. I think the change to the default behavior of pip install is
>>> dependent on the change to —upgrade, so I suggest we focus on the change to
>>> —upgrade first, changing from a “recursive” to a “conservative” strategy.
>>> Once we get that change figured out and landed then we can worry about what
>>> to do with pip install.
>>>
>>
>> You were. In fact, the majority swayed in favour of changing the
>> behaviour of pip install post one of your comments on Github.
>>
>> I'll be happier *only* seeing in change the behaviour of --upgrade and
>> not --target or pip install. It reduces the number of things that changes
>> from 3 to 1. Much easier to discuss about.
>>
>> I’m not going to repeat the entire post, but I just made a fairly lengthy
>>> comment at
>>> https://github.com/pypa/pip/issues/3786#issuecomment-228611906 but to
>>> try and boil it down to a few points:
>>>
>>
>> Thanks for this.
>>
>>
>>> * ``pip install —upgrade`` is not a good security mechanism, relying on
>>> it is inconsistent at best. If we want to support trying to keep people on
>>> secure versions of software we need a better mechanism than this anyways,
>>> so we shouldn’t let it influence our choice here.
>>>
>>
>> AFAIK, this was the only outstanding concern raised against having a
>> non-eager (conservative) upgrade strategy.
>>
>> * For the general case, it’s not going to matter a lot which way we go,
>>> but not upgrading has the greatest chance of not breaking *already
>>> installed software*.
>>>
>>
>> I strongly agree with this. Another thing worth a mention is that it's
>> easier to get the lower bounds of your requirements correct, rather than
>> upper bounds.
>>
>>
>>> * For the hard-to-upgrade case, the current behavior is so bad that
>>> people are outright attempting to subvert the way pip typically behaviors,
>>> *AND* advocating for other’s to do the same, in an attempt to escape that
>>> behavior. I think that this is not a good place to be in.
>>>
>>
>> Ditto.
>>
>> —
>>>
>>> Donald Stufft
>>>
>>
>> Happy-to-see-Donald's-response-ly,
>> Pradyun Gedam
>>
> ​
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

I’ll be happier *only* seeing in change

s/*only* seeing/seeing *only*/​
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Request for comment: Proposal to change behaviour of pip install

[Pradyun Gedam]

Hello List!

I feel it’s fine to hold back the other changes for later but the
upgrade-strategy change should get shipped out to the world as quickly as
possible. Even how the change is exposed the user can also be discussed
later.

I request the list members to focus on *only* the change of the default
upgrade strategy to be non-eager.

Does anyone have any concerns regarding the change of the default upgrade
strategy to be non-eager? If not, let’s get *just* that shipped out as soon
as possible.

Cheers,
Pradyun Gedam

On Mon, 27 Jun 2016 at 12:02 Pradyun Gedam pradyu...@gmail.com
 wrote:

On Sun, 26 Jun 2016 at 23:02 Donald Stufft  wrote:
>
>>
>> On Jun 25, 2016, at 6:25 AM, Pradyun Gedam  wrote:
>>
>> There is currently a proposal to change the behaviour to pip install to
>> upgrade a package that is passed even if it is already installed.
>>
>> This behaviour change is accompanied with a change in the upgrade
>> strategy - pip would stop “eagerly” upgrading dependencies and would become
>> more conservative, upgrading a dependency only when it doesn’t meet lower
>> constraints of the newer version of a parent. Moreover, the behaviour of pip
>> install --target would also be changed so that --upgrade no longer
>> affects it.
>>
>> I think bundling these two changes (and I think I might have been the one
>> that originally suggested it) is making this discussion harder than it
>> needs to be as folks are having to fight on multiple different fronts at
>> once. I think the change to the default behavior of pip install is
>> dependent on the change to —upgrade, so I suggest we focus on the change to
>> —upgrade first, changing from a “recursive” to a “conservative” strategy.
>> Once we get that change figured out and landed then we can worry about what
>> to do with pip install.
>>
>
> You were. In fact, the majority swayed in favour of changing the behaviour
> of pip install post one of your comments on Github.
>
> I'll be happier *only* seeing in change the behaviour of --upgrade and not
> --target or pip install. It reduces the number of things that changes from
> 3 to 1. Much easier to discuss about.
>
> I’m not going to repeat the entire post, but I just made a fairly lengthy
>> comment at https://github.com/pypa/pip/issues/3786#issuecomment-228611906 but
>> to try and boil it down to a few points:
>>
>
> Thanks for this.
>
>
>> * ``pip install —upgrade`` is not a good security mechanism, relying on
>> it is inconsistent at best. If we want to support trying to keep people on
>> secure versions of software we need a better mechanism than this anyways,
>> so we shouldn’t let it influence our choice here.
>>
>
> AFAIK, this was the only outstanding concern raised against having a
> non-eager (conservative) upgrade strategy.
>
> * For the general case, it’s not going to matter a lot which way we go,
>> but not upgrading has the greatest chance of not breaking *already
>> installed software*.
>>
>
> I strongly agree with this. Another thing worth a mention is that it's
> easier to get the lower bounds of your requirements correct, rather than
> upper bounds.
>
>
>> * For the hard-to-upgrade case, the current behavior is so bad that
>> people are outright attempting to subvert the way pip typically behaviors,
>> *AND* advocating for other’s to do the same, in an attempt to escape that
>> behavior. I think that this is not a good place to be in.
>>
>
> Ditto.
>
> —
>>
>> Donald Stufft
>>
>
> Happy-to-see-Donald's-response-ly,
> Pradyun Gedam
>
​
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig