Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Marko Rauhamaa]

Steven D'Aprano :

> Perhaps I'm missing something, but aren't there easier ways to attack 
> os.system than the bash env vulnerability?

The main concern is the cases where you provide a service accessible
through an SSH login and try to sandbox the client with limited
functionality. SSH passes some environment variables on to the service
which can then be used as an XSS vector.

For example, if you wrote an SVN server's SSH front end with Python and
used subprocess.Popen(shell=True) to execute the SVN operations, you
could become a victim.


Marko
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Cameron Simpson]

On 25Sep2014 21:30, Tres Seaver  wrote:

On 09/25/2014 08:59 PM, Cameron Simpson wrote:

Your cable/adsl modem? Probably an embedded Linux box, possibly using
bash, and certainly a dhcp client of the ISP. Better still, for many
people that same comprimisable modem is the DHCP _server_ for their
home LAN...


Generally those devices are *not* using bash as '/bin/sh':  it is too
heavyweigh.  Most will use 'busybox' or some other Swiss-army command for
stuff which is separate commands on a "normal" linux system.


Fair point.

Cheers,
Cameron Simpson 

There is no reason anyone would want a computer in their home.
  --Ken Olson, president, chairman and founder of Digital
Equipment Corp.,  1977
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 09/25/2014 08:59 PM, Cameron Simpson wrote:
> Your cable/adsl modem? Probably an embedded Linux box, possibly using
> bash, and certainly a dhcp client of the ISP. Better still, for many
> people that same comprimisable modem is the DHCP _server_ for their
> home LAN...

Generally those devices are *not* using bash as '/bin/sh':  it is too
heavyweigh.  Most will use 'busybox' or some other Swiss-army command for
stuff which is separate commands on a "normal" linux system.


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlQkwaAACgkQ+gerLs4ltQ4mYwCguMEUfwXZTM4FRS80HPCZx8DY
hogAoNVIjWfn1R2obPH9LhRGFkzBR4Pw
=oXxP
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Cameron Simpson]

On 26Sep2014 09:40, Steven D'Aprano  wrote:

On Fri, Sep 26, 2014 at 12:17:46AM +0200, Antoine Pitrou wrote:

On Thu, 25 Sep 2014 13:00:16 -0700
Bob Hanson  wrote:
> Critical bash vulnerability CVE-2014-6271 may affect Python on
> *n*x and OSX:

[...]

See also:
http://adminlogs.info/2014/09/25/again-bash-cve-2014-7169/


Fortunately, Python's subprocess has its `shell` argument default to
False. However, `os.system` invokes the shell implicitly and is
therefore a possible attack vector.


Perhaps I'm missing something, but aren't there easier ways to attack
os.system than the bash env vulnerability? If I'm accepting and running
arbitrary strings from an untrusted user, there's no need for them to go
to the trouble of feeding me:

"env x='() { :;}; echo gotcha'  bash -c 'echo do something useful'"

when they can just feed me:

"echo gotcha"

In other words, os.system is *already* an attack vector, unless you only
use it with trusted strings. I don't think the bash env vulnerability
adds to the attack surface.

Have I missed something?


Yes. Although it is possible to craft safe things for os.system, the issue is 
delivery of the attacks: plenty of totally standard things expose the shell to 
outside-supplied strings. And on most of those things, "the shell" is bash.


The issue with the bash-imports-functions-badly issue is that shell functions 
are exported through the environment. Other things use the environment to pass 
info.


The loud examples on the net are CGI scripts (query parameters passed through 
the environment, along with other things) and hostile DHCP servers (DHCP 
settings passed by the _client_ dhcpd to action scripts, as root).


Both of these let someone outside your system deliver bash-exploit strings to 
bash scripts.


Your cable/adsl modem? Probably an embedded Linux box, possibly using bash, and 
certainly a dhcp client of the ISP. Better still, for many people that same 
comprimisable modem is the DHCP _server_ for their home LAN...


Cheers,
Cameron Simpson 

Rebel without a clue, Born to be mild.
- kevin.lo...@usask.ca, DoD #0975
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Donald Stufft]

> On Sep 25, 2014, at 6:44 PM, Chris Barker  wrote:
> 
> On Thu, Sep 25, 2014 at 9:00 AM, Donald Stufft  > wrote:
> 1) Just always default to —user and add a —system or similar flag, this
> is super easy to change but is a backwards incompatible change and
> would need to go through a deprecation window.
> 
> Maybe would have been the way to go to begin with, but I think backwards 
> compatibility should probably trump "better" -- even with a deprecation 
> window.
>  
> 2) Switch to —user based on if the user has permission to write to the
> site-packages or not.
> 
> ouch -- no. Why not a clear error message if pip can't write to site-packages 
> -- something like:
> 
> """
> pip doesn't have permissions to write to the system location. If you want to 
> install this package system-wide, you need to run pip with admin priviledges 
> (and example here if it's easy), if you want to install for this user only, 
> pass the "--user" flag to pip install
> """ 
> 
> "In the face of ambiguity, refuse the temptation to guess.”

I fairly strongly believe that the current default is doing a great disservice
to users. I believe that for *most* people --user is the correct option for
them to be using and the fact that it's not the default and requires opt in
is a historical artifact more than anything else.

Immediately switching to --user as a default (outside of a virtual environment)
would break two important use cases:

- A root/admin user attempting to install into a global site-packages
- A user who uses complete Python installations to isolate themselves such as
  those created by pyenv.

It would techincally break anyone relying on the fact that pip will currently
throw an error if you attempt to ``pip install`` something without sudo,
however I do not consider that to be an actual use case that is going to have
wide signifcance.

So we get down to how do we not break the two important use cases above. At
first I thought about detecting uid 0 (and something similar on Windows?) and
turning off the --user default in those cases. However that still left us
dealing with something for the second use case.

After thinking about it I realized there isn't a good way to determine if
something is a user controlled Python installation instead of a system
controlled one. This lead me to the realization that something that could be
used to "detect" this is whether or not the current user has the ability to
write to the site-packages as a means of determining "is my current user
allowed to manage that particular Python".

Either way I'm fairly commited to making --user the default, the only question
on my mind is what exactly does that look like (e.g. does root get --user by
default?) and how we get from where we are now to that point. I think that
raising an error here is fairly unfriendly when we can know with pretty good
certainity what the user wanted (and they can explicitly declare that if they
want too).

However this particular thing is somewhat offtopic for this list and the
particulars of pip moving to --user by default is best discussed on our issue
tracker here: https://github.com/pypa/pip/issues/1668.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Chris Angelico]

On Fri, Sep 26, 2014 at 10:29 AM, Devin Jeanpierre
 wrote:
> As I understand it, if the attacker can help specify the environment
> (e.g. this is a CGI script), and you run os.system('echo hi'), you can
> get pwned. Even safe uses of os.system are vulnerable unless you point
> /bin/sh at a secure shell (e.g. patched bash).

/bin/sh may well not point to bash anyway - it doesn't on any of my
systems. Debian provides dash instead, much faster than bash. But if
you're invoking a script that calls for bash, then it's vulnerable.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Devin Jeanpierre]

On Thu, Sep 25, 2014 at 4:53 PM, Antoine Pitrou  wrote:
>> In other words, os.system is *already* an attack vector, unless you only
>> use it with trusted strings. I don't think the bash env vulnerability
>> adds to the attack surface.
>>
>> Have I missed something?
>
> The part where the attack payload is passed through the environment, not
> through hypothetical user-injected command-line arguments.

As I understand it, if the attacker can help specify the environment
(e.g. this is a CGI script), and you run os.system('echo hi'), you can
get pwned. Even safe uses of os.system are vulnerable unless you point
/bin/sh at a secure shell (e.g. patched bash).

-- Devin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Chris Angelico]

On Fri, Sep 26, 2014 at 9:53 AM, Antoine Pitrou  wrote:
>> In other words, os.system is *already* an attack vector, unless you only
>> use it with trusted strings. I don't think the bash env vulnerability
>> adds to the attack surface.
>>
>> Have I missed something?
>
> The part where the attack payload is passed through the environment, not
> through hypothetical user-injected command-line arguments.

Which means this also affects anything that invokes shell scripts (if
they use bash, rather than sh), even if it doesn't use os.system().
I'm just in process of checking and patching my systems (most of them
are just 'apt-get update; apt-get upgrade' followed by a quick check),
and can confirm that it does happen in Python. All you have to do is
invoke bash, either explicitly or through your target's shebang.

>>> import os, subprocess
>>> os.environ["HAHA"]="() { :;}; echo This is crafted from the environment."
>>> subprocess.call(["./test.sh"])
This is crafted from the environment.
This is from my test script.
0
>>> open("./test.sh").read()
'#!/bin/bash\necho This is from my test script.\n\n'
>>> subprocess.call(["bash","-c","echo This is from the command line."])
This is crafted from the environment.
This is from the command line.
0

But this is a bash issue, not a Python one.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Wes Turner]

> The part where the attack payload is passed through the environment, not
through hypothetical user-injected command-line arguments.

So, best advice would be to:

1. Upgrade bash (and standby for an additional patch, according to reddit)
2. Upgrade to WSGI, if possible
3. Be careful about including things from os.environ?
3. Always avoid allowing user input with os.system, os.popen, and
subprocess.xyz(cmd, shell=True) [which doesn't need "#aftershock"]

Am I missing something?

seeAlso:

* http://cwe.mitre.org/top25/#CWE-78 "Improper Neutralization of
Special Elements used in an OS Command ('OS Command Injection')"
* http://cwe.mitre.org/data/definitions/78.html
* https://www.owasp.org/index.php/Top_10_2013-A1-Injection
* https://en.wikipedia.org/wiki/Code_injection#Shell_injection

On Thu, Sep 25, 2014 at 6:53 PM, Antoine Pitrou  wrote:
> On Fri, 26 Sep 2014 09:40:17 +1000
> Steven D'Aprano  wrote:
>> Perhaps I'm missing something, but aren't there easier ways to attack
>> os.system than the bash env vulnerability? If I'm accepting and running
>> arbitrary strings from an untrusted user, there's no need for them to go
>> to the trouble of feeding me:
>>
>> "env x='() { :;}; echo gotcha'  bash -c 'echo do something useful'"
>>
>> when they can just feed me:
>>
>> "echo gotcha"
>>
>> In other words, os.system is *already* an attack vector, unless you only
>> use it with trusted strings. I don't think the bash env vulnerability
>> adds to the attack surface.
>>
>> Have I missed something?
>
> The part where the attack payload is passed through the environment, not
> through hypothetical user-injected command-line arguments.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com



-- 
Wes Turner
https://westurner.github.io/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Antoine Pitrou]

On Fri, 26 Sep 2014 09:40:17 +1000
Steven D'Aprano  wrote:
> Perhaps I'm missing something, but aren't there easier ways to attack 
> os.system than the bash env vulnerability? If I'm accepting and running 
> arbitrary strings from an untrusted user, there's no need for them to go 
> to the trouble of feeding me:
> 
> "env x='() { :;}; echo gotcha'  bash -c 'echo do something useful'"
> 
> when they can just feed me:
> 
> "echo gotcha"
> 
> In other words, os.system is *already* an attack vector, unless you only 
> use it with trusted strings. I don't think the bash env vulnerability 
> adds to the attack surface.
> 
> Have I missed something?

The part where the attack payload is passed through the environment, not
through hypothetical user-injected command-line arguments.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Chris Barker]

On Thu, Sep 25, 2014 at 9:00 AM, Donald Stufft  wrote:

> 1) Just always default to —user and add a —system or similar flag, this
> is super easy to change but is a backwards incompatible change and
> would need to go through a deprecation window.
>

Maybe would have been the way to go to begin with, but I think backwards
compatibility should probably trump "better" -- even with a deprecation
window.


> 2) Switch to —user based on if the user has permission to write to the
> site-packages or not.
>

ouch -- no. Why not a clear error message if pip can't write to
site-packages -- something like:

"""
pip doesn't have permissions to write to the system location. If you want
to install this package system-wide, you need to run pip with admin
priviledges (and example here if it's easy), if you want to install for
this user only, pass the "--user" flag to pip install
"""

"In the face of ambiguity, refuse the temptation to guess."

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Steven D'Aprano]

On Fri, Sep 26, 2014 at 12:17:46AM +0200, Antoine Pitrou wrote:
> On Thu, 25 Sep 2014 13:00:16 -0700
> Bob Hanson  wrote:
> > Critical bash vulnerability CVE-2014-6271 may affect Python on
> > *n*x and OSX:
[...]

See also:

http://adminlogs.info/2014/09/25/again-bash-cve-2014-7169/


> Fortunately, Python's subprocess has its `shell` argument default to
> False. However, `os.system` invokes the shell implicitly and is
> therefore a possible attack vector.

Perhaps I'm missing something, but aren't there easier ways to attack 
os.system than the bash env vulnerability? If I'm accepting and running 
arbitrary strings from an untrusted user, there's no need for them to go 
to the trouble of feeding me:

"env x='() { :;}; echo gotcha'  bash -c 'echo do something useful'"

when they can just feed me:

"echo gotcha"

In other words, os.system is *already* an attack vector, unless you only 
use it with trusted strings. I don't think the bash env vulnerability 
adds to the attack surface.

Have I missed something?



-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Cameron Simpson]

On 26Sep2014 00:17, Antoine Pitrou  wrote:

On Thu, 25 Sep 2014 13:00:16 -0700
Bob Hanson  wrote:

Critical bash vulnerability CVE-2014-6271 may affect Python on
*n*x and OSX:


[...]

Fortunately, Python's subprocess has its `shell` argument default to
False. However, `os.system` invokes the shell implicitly and is
therefore a possible attack vector.


Only if /bin/sh is bash :-) Not always the case, fortunately.

Cheers,
Cameron Simpson 

Death is life's way of telling you you've been fired.   - R. Geis
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Wes Turner]

This was helpful:
http://sarge.readthedocs.org/en/latest/internals.html#how-shell-quoting-works
--
Wes Turner


On Thu, Sep 25, 2014 at 5:17 PM, Antoine Pitrou  wrote:
> On Thu, 25 Sep 2014 13:00:16 -0700
> Bob Hanson  wrote:
>> Critical bash vulnerability CVE-2014-6271 may affect Python on
>> *n*x and OSX:
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> Also see  for thread on
>> same being started today.
>
> Fortunately, Python's subprocess has its `shell` argument default to
> False. However, `os.system` invokes the shell implicitly and is
> therefore a possible attack vector.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Nick Coghlan]

On 26 Sep 2014 01:56, "Paul Moore"  wrote:
>
> Basically, I'd like to hold off moving to "Program Files" as a default
> until *after* we have enough confidence in user installs that we are
> willing to switch pip to --user as the default behaviour everywhere.
> And yes, I'm aware that the first "we" in that was "python-dev" and
> the second was "PyPA". And that expecting python-dev to wait for PyPA
> to approve the change may well be unacceptable.

This sounds like a reasonable plan to me (I've been keen to have pip do
this on the Linux side of things for a while).

If the pip team were willing to start down this path, changing the default
in 3.5 next year might still be cutting things a little fine. While I think
it's worth aiming for that, I also wouldn't see it as a big deal if the
change slipped to 3.6 in 2017.

Regards,
Nick.

>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Antoine Pitrou]

On Thu, 25 Sep 2014 13:00:16 -0700
Bob Hanson  wrote:
> Critical bash vulnerability CVE-2014-6271 may affect Python on
> *n*x and OSX:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Also see  for thread on
> same being started today.

Fortunately, Python's subprocess has its `shell` argument default to
False. However, `os.system` invokes the shell implicitly and is
therefore a possible attack vector.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX

[Bob Hanson]

Critical bash vulnerability CVE-2014-6271 may affect Python on
*n*x and OSX:









Also see  for thread on
same being started today.

--Bob Hanson

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Ethan Furman]

On 09/24/2014 09:11 PM, Larry Hastings wrote:


Therefore: if VC14 doesn't ship by 3.5 RC1, currently set at August 5, 2015, I 
decree we have to ship 3.5 with the
previous version.

Reasonable?


Seems reasonable to me.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Martin v. Löwis]

Am 24.09.14 14:34, schrieb Antoine Pitrou:
> On Wed, 24 Sep 2014 17:12:35 +1000
> Nick Coghlan  wrote:
>> On 24 Sep 2014 15:15, "Tim Golden"  wrote:
>>>
>>> On 23/09/2014 18:05, Steve Dower wrote:

 I'm also considering/experimenting with installing into "Program
 Files" by default, but I suspect that isn't going to work out yet.
>>>

>> It might be better to offer that as a supported option in 3.5, and then
>> make it the default in 3.6.
> 
> I would be surprised if this weren't already a supported option. Decent
> Windows installers generally allow you to override the installation
> path (it's been a while I haven't used the Windows Python installer,
> sorry).

I regularly test whether it installs properly into Program Files
(although regularly actually means "for every major release"). Also,
users regularly report if it doesn't work.

As it turns out, 3.4 indeed got a new problem when installing into
Program Files, namely that pip would not install. As discussed elsewhere
in this thread, installation into Program Files requires elevated
privileges, which meant that I had to reschedule the pip installation
procedure within the MSI to run in the system context.

> 
> I think making the move in 3.5 would be a good idea. Experts can
> override the installation path and choose C:\PythonXY. There's no
> actual breakage since the path changes for every major release, anyway
> (i.e. people would have had to change the path from "C:\Python34" to
> "C:\Python35"; instead, they will have to change it to "C:\Program
> Files\Python35").

I expect breakage elsewhere (as also discussed):
- some code will fail because of the space in the path
- some installations will fail because of the restricted access to
  Program Files (which, of course, is also the main reason why people
  want that install location)

It's of course now up to Steve to rule on the default install location;
if he changes it, he will get lots of both praise and blame, if he
doesn't change it, he will get a little blame for continuing the
tradition.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Paul Moore]

On 25 September 2014 18:13, Steve Dower  wrote:
> Again, this isn't trivial to get right. The design for the elevation model 
> seems to have focused mainly on GUI rather than console, probably assuming 
> that people who need to elevate from the console will elevate the shell 
> itself (this is a guess - I have no insight into how the Windows team 
> designed this), so there are limitations we have to work within.

Yeah, I think my view is that we don't really want to have to support
code that complex, so let's keep it simple. To use pip to install into
a protected location, pip must be run from an elevated shell. Given
that doing so is inconvenient (specifically, even more inconvenient
than sudo on Unix) let's ensure that for the normal case the user
doesn't need to do that.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Steve Dower]

> Paul Moore wrote:
> On 25 September 2014 17:05, Steve Dower  wrote:
>> So yes, pip can certainly do this, and if it's already running
>> elevated then it shouldn't reprompt, but it's not entirely trivial to
>> get this right ("are you denied write access to that directory because
>> you're not admin or because it's on read-only media?") and it's
>> considerably easier to try it, fail on access issues, but provide a
>> flag for the user to force elevation. "pip --sudo install ..." would
>> be fine by me :)
> 
> I thought one issue with running an elevated command line subprocess from a
> non-elevated one, was that the elevated one didn't have access to the
> non-elevated console, so it popped up its own independent console window, 
> which
> disappeared immediately the process completed (hence losing any error 
> messages).
> I definitely recall easy_install did that at one stage, and it was a real 
> pain.
> Or is that something the parent process can affect, and the cmd/easy_install
> pair just didn't do so?

I'm not sure you can do it automatically, but if you own both sides of the 
application you can set up a pipe between the two processes and let the 
unelevated side forward stdio.

Again, this isn't trivial to get right. The design for the elevation model 
seems to have focused mainly on GUI rather than console, probably assuming that 
people who need to elevate from the console will elevate the shell itself (this 
is a guess - I have no insight into how the Windows team designed this), so 
there are limitations we have to work within.

> Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Paul Moore]

On 25 September 2014 17:05, Steve Dower  wrote:
> So yes, pip can certainly do this, and if it's already running elevated then 
> it shouldn't reprompt, but it's not entirely trivial to get this right ("are 
> you denied write access to that directory because you're not admin or because 
> it's on read-only media?") and it's considerably easier to try it, fail on 
> access issues, but provide a flag for the user to force elevation. "pip 
> --sudo install ..." would be fine by me :)

I thought one issue with running an elevated command line subprocess
from a non-elevated one, was that the elevated one didn't have access
to the non-elevated console, so it popped up its own independent
console window, which disappeared immediately the process completed
(hence losing any error messages). I definitely recall easy_install
did that at one stage, and it was a real pain. Or is that something
the parent process can affect, and the cmd/easy_install pair just
didn't do so?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Steve Dower]

Chris Angelico wrote:
> On Thu, Sep 25, 2014 at 6:50 AM, Steve Dower  
> wrote:
>> Donald Stufft wrote:
>>> One thing about *nix is even though you can’t write to your normal
>>> Python install location without root, invoking pip with permissions
>>> (assuming you have
>>> them) is as easy as prefacing it with ``sudo`` in most cases. Does
>>> Windows have an equivalent or do you need to launch a whole new shell?
>>
>> Unfortunately not. The "easy way" is for the executable to declare that it
> needs administrative privileges, and then the OS will take over and let you
> approve/reject/sign-in/etc. according to your settings.
> 
> When you say the executable declares this, is that a static (compile/link 
> time)
> declaration, or a run-time one? That is, could pip defer the declaration until
> it's parsed its command line args and decided that it'll be installing into 
> the
> system directory, but NOT make that declaration if it's given --user, or if 
> it's
> running inside a venv, or anything else that means it doesn't need that power?
> If so, that could actually be a very powerful feature: a user without admin
> privs (or choosing not to exercise them) is still able to install into a venv,
> but if s/he forgets to activate the venv, the "hey, I want to modify system
> files" prompt will be the alarm that says the situation is not what was
> expected. That's what happens on my Linux system - I get errors back if I 
> don't
> use sudo, but in a venv, everything works without being root.

It's a compile time option (it's specified in the manifest you can embed into 
an executable) but it is possible to explicitly launch a separate process with 
the flag. Most programs that look like they can elevate on-demand are really 
launching a background process to do the work - you can't change the user for a 
process once it's started in Windows.

This process can obviously be the same one you started with (maybe with 
different command line args), and the main observable difference is that you 
get fewer access denied errors.

So yes, pip can certainly do this, and if it's already running elevated then it 
shouldn't reprompt, but it's not entirely trivial to get this right ("are you 
denied write access to that directory because you're not admin or because it's 
on read-only media?") and it's considerably easier to try it, fail on access 
issues, but provide a flag for the user to force elevation. "pip --sudo install 
..." would be fine by me :)

Cheers,
Steve
 
> ChrisA
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Donald Stufft]

> On Sep 25, 2014, at 11:54 AM, Paul Moore  wrote:
> 
> On 25 September 2014 16:43, Donald Stufft  wrote:
>> Basically people have Python in a ton of different configurations and it’s
>> hard to figure out if —user will work out of the box in all of them or not.
> 
> I guess that "Using the python.org Python installer on Windows" is a
> limited enough subset that we probably could check that --user worked
> in that situation.
> 
> The problem is, how do we implement it? A special case so that pip
> defaults to --user sometimes, but not others? (I'm strongly against
> that) Leave the default as not --user and document that Windows users
> with Python in "Program Files" should always specify --user? (I'm
> against that because it makes the documentation highly confusing, and
> we've just done a lot of work to simplify it).
> 
> Basically, I'd like to hold off moving to "Program Files" as a default
> until *after* we have enough confidence in user installs that we are
> willing to switch pip to --user as the default behaviour everywhere.
> And yes, I'm aware that the first "we" in that was "python-dev" and
> the second was "PyPA". And that expecting python-dev to wait for PyPA
> to approve the change may well be unacceptable.
> 
> Paul


My thoughts on the pip side has basically always been that pip should
either:

1) Just always default to —user and add a —system or similar flag, this
is super easy to change but is a backwards incompatible change and
would need to go through a deprecation window.
2) Switch to —user based on if the user has permission to write to the
site-packages or not.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Paul Moore]

On 25 September 2014 16:43, Donald Stufft  wrote:
> Basically people have Python in a ton of different configurations and it’s
> hard to figure out if —user will work out of the box in all of them or not.

I guess that "Using the python.org Python installer on Windows" is a
limited enough subset that we probably could check that --user worked
in that situation.

The problem is, how do we implement it? A special case so that pip
defaults to --user sometimes, but not others? (I'm strongly against
that) Leave the default as not --user and document that Windows users
with Python in "Program Files" should always specify --user? (I'm
against that because it makes the documentation highly confusing, and
we've just done a lot of work to simplify it).

Basically, I'd like to hold off moving to "Program Files" as a default
until *after* we have enough confidence in user installs that we are
willing to switch pip to --user as the default behaviour everywhere.
And yes, I'm aware that the first "we" in that was "python-dev" and
the second was "PyPA". And that expecting python-dev to wait for PyPA
to approve the change may well be unacceptable.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Donald Stufft]

> On Sep 25, 2014, at 4:54 AM, Antoine Pitrou  wrote:
> 
> On Thu, 25 Sep 2014 07:34:31 +0100
> Paul Moore  wrote:
>> On 25 September 2014 02:08, Antoine Pitrou  wrote:
 Indeed. Moving towards having --user as the norm is definitely
 something we want to look at for pip. One of the biggest concerns is
 how well-exercised the whole user site directory area is in practice.
>>> 
>>> What do you mean by well-exercised?
>> 
>> Basically, although --user is available in pip (and the underlying
>> facilities in Python have been around for some time), it's difficult
>> to gauge how many people are using them, and as a result what level of
>> testing has happened in real-life situations.
> 
> I'm using it often. I'm also unsure how broken it could be. The user
> site-packages is just another site-packages directory.
> 

Broken like the prefix problem :)

Basically people have Python in a ton of different configurations and it’s
hard to figure out if —user will work out of the box in all of them or not.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Nick Coghlan]

On 24 September 2014 23:16, Mike Miller  wrote:
> Hi all,
>
> ProgramFiles was the default in Python 1.X.
>
> It has been a supported option for just shy of 15 years on 2.X...  most if
> not all the bugs (setuptools) were fixed a decade ago, and right now
> thousands, if not millions of people are running it under Program Files
> right now.  I can vouch for several thousand because a company I work for
> distributes Python and pip there for its customers all around the world w/o
> issue.
>
> I've never once encountered a bug due to install to ProgramFiles, or heard
> of anyone who has, and have been using Python for everything since the year
> 2000. If any rare bugs happen to surface, they can likely be fixed or
> replaced with a line of code, or worked around by installing elsewhere.

Right, the problem I was thinking of was actually a transient one
during the 3.4 pre-release cycle (the ensurepip integration initially
didn't work properly when installing into Program Files).

So consider that a switch to +1 from me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Antoine Pitrou]

Le 25/09/2014 09:22, INADA Naoki a écrit :
> FYI, homebrew's Python uses prefix option, so I can't use `--user`.
> Is it a bug?
> 
> $ /usr/local/bin/pip -V
> pip 1.5.6 from 
> /usr/local/Cellar/python/2.7.8_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg
> (python 2.7)
> 
> $ /usr/local/bin/pip install --user tornado
> ...
> error: can't combine user with prefix, exec_prefix/home, or install_(plat)base

Yes, it is.
http://bugs.python.org/issue22269

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Antoine Pitrou]

On Thu, 25 Sep 2014 07:34:31 +0100
Paul Moore  wrote:
> On 25 September 2014 02:08, Antoine Pitrou  wrote:
> >> Indeed. Moving towards having --user as the norm is definitely
> >> something we want to look at for pip. One of the biggest concerns is
> >> how well-exercised the whole user site directory area is in practice.
> >
> > What do you mean by well-exercised?
> 
> Basically, although --user is available in pip (and the underlying
> facilities in Python have been around for some time), it's difficult
> to gauge how many people are using them, and as a result what level of
> testing has happened in real-life situations.

I'm using it often. I'm also unsure how broken it could be. The user
site-packages is just another site-packages directory.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[INADA Naoki]

FYI, homebrew's Python uses prefix option, so I can't use `--user`.
Is it a bug?

$ /usr/local/bin/pip -V
pip 1.5.6 from 
/usr/local/Cellar/python/2.7.8_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg
(python 2.7)

$ /usr/local/bin/pip install --user tornado
...
error: can't combine user with prefix, exec_prefix/home, or install_(plat)base


$ cat 
/usr/local/Cellar/python/2.7.8_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/distutils.cfg
[global]
verbose=1
[install]
force=1
prefix=/usr/local



On Thu, Sep 25, 2014 at 3:34 PM, Paul Moore  wrote:
> On 25 September 2014 02:08, Antoine Pitrou  wrote:
>>> Indeed. Moving towards having --user as the norm is definitely
>>> something we want to look at for pip. One of the biggest concerns is
>>> how well-exercised the whole user site directory area is in practice.
>>
>> What do you mean by well-exercised?
>
> Basically, although --user is available in pip (and the underlying
> facilities in Python have been around for some time), it's difficult
> to gauge how many people are using them, and as a result what level of
> testing has happened in real-life situations. There's probably no way
> to improve that much other than by making --user the default and
> waiting for reports of any issues, but feedback like Mike's adds a
> certain level of confidence that there are no significant problems.
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com



-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com