Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 31 August 2014 12:21, R. David Murray rdmur...@bitdance.com wrote: On Sun, 31 Aug 2014 03:25:25 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 31 Aug 2014 09:26:30 +1000 Nick Coghlan ncogh...@gmail.com wrote: In relation to changing the Python CLI API to offer some of the

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Donald Stufft
On Aug 31, 2014, at 2:09 AM, Nick Coghlan ncogh...@gmail.com wrote: At the same time, we need to account for the fact that most existing organisations still trust in perimeter defence for their internal network security, and hence tolerate (or even actively encourage) the use of unsecured

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 31 August 2014 12:21, R. David Murray rdmur...@bitdance.com wrote: Do those knobs allow one to instruct urllib to accept an invalid certificate without changing the program code? My first reply ended up being a context dump of the challenges created by legacy corporate intranets that may not

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 31 August 2014 16:16, Donald Stufft don...@stufft.io wrote: On Aug 31, 2014, at 2:09 AM, Nick Coghlan ncogh...@gmail.com wrote: At the same time, we need to account for the fact that most existing organisations still trust in perimeter defence for their internal network security, and

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Cory Benfield
On 31 August 2014 07:45, Nick Coghlan ncogh...@gmail.com wrote: There's also the fact that most corporate Python users are unlikely to know that PyPI exists, let alone that it contains a module called requests that does SSL certificate validation by default. Those of us in the corporate world

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 30.08.2014 00:22, Antoine Pitrou wrote: SSL_CERT_DIR and SSL_CERT_FILE are used, if set, when SSLContext.load_verify_locations() is called. Actually, come to think of it, this allows us to write a better test for that method. Patch welcome! The environment vars are used only when

[Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Victor Stinner
HTML version: http://legacy.python.org/dev/peps/pep-0475/ PEP: 475 Title: Retry system calls failing with EINTR Version: $Revision$ Last-Modified: $Date$ Author: Charles-François Natali cf.nat...@gmail.com, Victor Stinner victor.stin...@gmail.com Status: Draft Type: Standards Track Content-Type:

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread R. David Murray
On Sun, 31 Aug 2014 16:45:42 +1000, Nick Coghlan ncogh...@gmail.com wrote: On 31 August 2014 16:16, Donald Stufft don...@stufft.io wrote: On Aug 31, 2014, at 2:09 AM, Nick Coghlan ncogh...@gmail.com wrote: At the same time, we need to account for the fact that most existing

[Python-Dev] [libmpdec] mpdecimal-2.4.1 released

2014-08-31 Thread Stefan Krah
Hi, I've released mpdecimal-2.4.1: http://www.bytereef.org/mpdecimal/changelog.html da74d3cfab559971a4fbd4fb506e1b4498636eb77d0fd09e44f8e546d18ac068 mpdecimal-2.4.1.tar.gz Starting with Python 3.4.2, this version should be used for an external libmpdec. Stefan Krah

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Marko Rauhamaa
Victor Stinner victor.stin...@gmail.com: Proposition === If a system call fails with ``EINTR``, Python must call signal handlers: call ``PyErr_CheckSignals()``. If a signal handler raises an exception, the Python function fails with the exception. Otherwise, the system call is

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 16:16, R. David Murray wrote: Self -signed certificates are not crazy in an internal corporate environment even when properly playing the defense in depth game. Once you've acked the cert the first time, you will be warned if it changes (like an ssh host key). Sure, as Nick

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 17:27, Christian Heimes christ...@python.org wrote: It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store. Stuff it into the trust store is the hard bit, though. I have honestly no idea how to do that. Or if it's temporary

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 08:24, Nick Coghlan wrote: To answer David's specific question, the existing knobs at the OpenSSL level (SSL_CERT_DIR and SSL_CERT_FILE ) let people add an internal CA, opt out of the default CA system, and trust *specific* self-signed certs. This works only on Unix platforms

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 19:03, Paul Moore a écrit : On 31 August 2014 17:27, Christian Heimes christ...@python.org wrote: It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store. Stuff it into the trust store is the hard bit, though. I have honestly no

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 18:29, Antoine Pitrou anto...@python.org wrote: If an application has special needs that require trusting a self-signed certificate, then it should expose a configuration setting to let users specify the cert's location. I can't see how that would be something the application

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 20:28, Paul Moore a écrit : I can't see how that would be something the application would know. For example, pip allows me to specify an alternate cert bundle but not a single additional cert. So IIUC, I can't use my local index that serves https using a self-signed cert. I'd find

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 19:37, Antoine Pitrou anto...@python.org wrote: Well, it's certainly pip's responsibility more than Python's. What would Python do? Provide a setting that would blindly add a cert for all uses of httplib? That's more or less my point, pip doesn't have that much better idea

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 21:12, Paul Moore a écrit : On 31 August 2014 19:37, Antoine Pitrou anto...@python.org wrote: Well, it's certainly pip's responsibility more than Python's. What would Python do? Provide a setting that would blindly add a cert for all uses of httplib? That's more or less my

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 19:29, Antoine Pitrou wrote: You certainly shouldn't do so. If an application has special needs that require trusting a self-signed certificate, then it should expose a configuration setting to let users specify the cert's location. Stuffing self-signed certs into the system

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Paul Moore
On 31 August 2014 21:15, Antoine Pitrou anto...@python.org wrote: What do you call your local cert store? I was referring to Christian's comment It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store. From his recent response, I guess he meant

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Victor Stinner
Hi, Sorry but I don't understand your remark. What is your problem with retrying syscall on EINTR? Can you please elaborate? What do you mean by get wrong? Victor Le dimanche 31 août 2014, Marko Rauhamaa ma...@pacujo.net a écrit : Victor Stinner victor.stin...@gmail.com javascript:;:

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Marko Rauhamaa
Victor Stinner victor.stin...@gmail.com: Sorry but I don't understand your remark. What is your problem with retrying syscall on EINTR? The application will often want the EINTR return (exception) instead of having the function resume on its own. Can you please elaborate? What do you mean by

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Ethan Furman
On 08/31/2014 02:19 PM, Marko Rauhamaa wrote: Victor Stinner victor.stin...@gmail.com: Sorry but I don't understand your remark. What is your problem with retrying syscall on EINTR? The application will often want the EINTR return (exception) instead of having the function resume on its own.

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Victor Stinner
Le dimanche 31 août 2014, Marko Rauhamaa ma...@pacujo.net a écrit : Victor Stinner victor.stin...@gmail.com javascript:;: Sorry but I don't understand your remark. What is your problem with retrying syscall on EINTR? The application will often want the EINTR return (exception) instead of

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 1 Sep 2014 06:32, Paul Moore p.f.mo...@gmail.com wrote: On 31 August 2014 21:15, Antoine Pitrou anto...@python.org wrote: What do you call your local cert store? I was referring to Christian's comment It's very simple to trust a self-signed certificate: just download it and stuff it

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I think there may be something worthwhile we can do that's closer to the way browsers work, and has the significant benefit of being implementable as a

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Antoine Pitrou
Le 31/08/2014 23:41, Nick Coghlan a écrit : Right, this is why I came to the conclusion we need to follow the browser vendors lead here and support a per-user Python specific supplementary certificate cache before we can start validating certs by default at the *Python* level. There are still

[Python-Dev] PEP 477: selected ensurepip backports for Python 2.7

2014-08-31 Thread Nick Coghlan
Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7 maintenance release in addition to including it with Python 3.4. That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Christian Heimes
On 31.08.2014 22:30, Paul Moore wrote: On 31 August 2014 21:15, Antoine Pitrou anto...@python.org wrote: What do you call your local cert store? I was referring to Christian's comment It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store.

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Marko Rauhamaa
Victor Stinner victor.stin...@gmail.com: But I don't get you point. How does this PEP make the situation worse? Did I say it would? I just wanted to make sure the system call resumption doesn't become mandatory. Haven't thought through what the exception raising technique would entail. It

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 1 Sep 2014 07:43, Christian Heimes christ...@python.org wrote: On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I think there may be something worthwhile we can do that's closer to the way

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Marko Rauhamaa
Ethan Furman et...@stoneleaf.us: On 08/31/2014 02:19 PM, Marko Rauhamaa wrote: The application will often want the EINTR return (exception) instead of having the function resume on its own. Examples? As an ignorant person in this area, I do not know why I would ever want to have EINTR

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Donald Stufft
On Aug 31, 2014, at 5:43 PM, Christian Heimes christ...@python.org wrote: On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I think there may be something worthwhile we can do that's closer to the

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Victor Stinner
Le 1 sept. 2014 00:04, Marko Rauhamaa ma...@pacujo.net a écrit : Victor Stinner victor.stin...@gmail.com: But I don't get you point. How does this PEP make the situation worse? Did I say it would? I just wanted to make sure the system call resumption doesn't become mandatory. The syscall

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Antoine Pitrou
On Mon, 01 Sep 2014 01:15:12 +0300 Marko Rauhamaa ma...@pacujo.net wrote: If a signal is received when read() or write() has completed its task partially ( 0 bytes), no EINTR is returned but the partial count. Obviously, Python should take that possibility into account so that raising an

Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7

2014-08-31 Thread Terry Reedy
On 8/31/2014 6:00 PM, Nick Coghlan wrote: Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7 maintenance release in addition to including it with Python 3.4. That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Victor Stinner
Le 1 sept. 2014 00:17, Marko Rauhamaa ma...@pacujo.net a écrit : If a signal is received when read() or write() has completed its task partially ( 0 bytes), no EINTR is returned but the partial count. Obviously, Python should take that possibility into account so that raising an exception in

Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7

2014-08-31 Thread Antoine Pitrou
On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan ncogh...@gmail.com wrote: That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in

Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7

2014-08-31 Thread Benjamin Peterson
On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote: On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan ncogh...@gmail.com wrote: That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release

Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7

2014-08-31 Thread Nick Coghlan
On 1 Sep 2014 09:23, Benjamin Peterson benja...@python.org wrote: On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote: On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan ncogh...@gmail.com wrote: That part of the proposal proved to be controversial, so we dropped it from the original

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread Nick Coghlan
On 1 Sep 2014 08:15, Donald Stufft don...@stufft.io wrote: On Aug 31, 2014, at 5:43 PM, Christian Heimes christ...@python.org wrote: Companies could install them in a system global directory on their servers. Users could put them in their own user site directory and even each virtual env

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Greg Ewing
Victor Stinner wrote: As written in the PEP, if you want to be notified of the signal, set a signal handler which raises an exception. I'm not convinced that this covers all possible use cases. It might be all right if you have control over the signal handler, but what if you don't? I think

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

2014-08-31 Thread R. David Murray
On Mon, 01 Sep 2014 08:10:58 +1000, Nick Coghlan ncogh...@gmail.com wrote: On 1 Sep 2014 07:43, Christian Heimes christ...@python.org wrote: On 31.08.2014 08:09, Nick Coghlan wrote: As Antoine says here, I'm also opposed to adding more Python specific configuration options. However, I

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Dan Stromberg
On Sun, Aug 31, 2014 at 3:28 PM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Victor Stinner wrote: As written in the PEP, if you want to be notified of the signal, set a signal handler which raises an exception. I'm not convinced that this covers all possible use cases. It might be all

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread R. David Murray
On Sun, 31 Aug 2014 20:14:50 -0700, Dan Stromberg drsali...@gmail.com wrote: On Sun, Aug 31, 2014 at 3:28 PM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Victor Stinner wrote: As written in the PEP, if you want to be notified of the signal, set a signal handler which raises an

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-08-31 Thread Marko Rauhamaa
R. David Murray rdmur...@bitdance.com: PS: I recently switched from using selectors to using a timeout on a socket because in that particular application I could, and because reading a socket with a timeout handles EINTR (in recent python versions), whereas reading a non-blocking socket