Re: [Python-Dev] Proposed schedule for 3.4.2

[Antoine Pitrou]

On Tue, 9 Sep 2014 08:20:52 +1000
Nick Coghlan  wrote:
> On 9 Sep 2014 04:00, "Barry Warsaw"  wrote:
> > >
> > >This would need to be updated first, once it *did* take such an argument,
> > >this would be accomplished by:
> > >
> > >context = ssl.create_default_context()
> > >context.verify_mode = CERT_OPTIONACERT_NONE
> > >context.verify_hostname = False
> > >urllib.request.urlopen("
> https://something-i-apparently-dont-care-much-about";,
> > >context=context)
> >
> > There's probably an ugly hack possibility that uses unittest.mock.patch.
> ;)
> 
> We could actually make it an "official" hack:
> 
> import urllib.request
> urllib.request.urlopen = urllib.request._unverified_urlopen

-1. Instead of disabling cert verification for *one* urlopen() call
site, you're doing it for *every* urlopen call site in the program - or,
even worse, for every urlopen that's imported after the monkey-patching
(which makes the final effect potentially dependent on module import
order, and import style). It may affect third-party libraries that have
their own REST calls, or whatever.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Antoine Pitrou]

On Mon, 8 Sep 2014 10:44:51 -0700
Alex Gaynor  wrote:
> *Shifts uncomfortably* it looks like presently there's not a good way to
> change anything about the SSL configuration for urllib.request.urlopen. It
> does not take a `context` argument, as the http.client API does:
> https://docs.python.org/3/library/urllib.request.html#module-urllib.request
> and instead takes the cafile, capath, cadefault args.

The reason it doesn't take a context argument is that SSL contexts are
hard to understand and configure for the average urlopen() user (why
should people know about ssl module specifics to open an HTTPS URL?).

http.client is more low-level and therefore more amenable to such
powerful knobs.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Larry Hastings]

On 09/08/2014 05:08 AM, Nick Coghlan wrote:

On 8 September 2014 14:28, Ned Deily  wrote:

As I've already discussed with Larry, I think adding a week to the
scheduled dates would be preferable.  The original dates give pretty
short notice and there are a number of open issues that would be good to
resolve now in 3.4.2.

It would also be good to get Guido's official verdict on PEP 476 (the
switch to validating HTTPS by default) in time for 3.4.2. Based on the
previous discussion, Alex updated the PEP to suggest "just fix it" for
all of 3.5, 3.4.x and 2.7.x (including the httplib SSLContext support
backport in the latter case).

I think that would be feasible with an rc on the 20th, but challenging
if the rc is this coming weekend.



Okay, the thread has died down.  It seems like pushing back a week will 
possibly help.  So the new dates:


3.4.2 rc 1: September 22, 2014
3.4.2 final: October 6, 2014


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Glenn Linderman]

On 9/8/2014 8:41 PM, Stephen J. Turnbull wrote:

  > Why not provide _urlopen_with_scary_keyword_parameter as the
  > monkey-patch option?
  >
  > So after the (global to the module) monkeypatch, they would_still_  have
  > to add the keyword parameter.

I understand the hardline position, though I don't like it: "if you
don't know how to do it yourself, we won't help you do it at all."[1]

But this "defense in depth" suggestion really violates the "consenting
adults" principle.  One warning in the docs and another in the name
itself should be enough, and if it isn't, Mommy should take Jimmy's
RaspberryPi away.
I was assuming, because of the suggestion for a monkey patch at all, in 
response to Guido's suggestion of a keyword parameter, that there was a 
problem adding a keyword parameter to urlopen. If there is, then the 
combo above could be useful in making them track down and adjust the 
places that need it, without forcing them to adjust the places that 
don't need it? If there is not, then no need for the monkey patch at 
all, they can just change add the keyword parameter.


If the alternate function doesn't have an extra keyword parameter, the 
monkeypatch solution would be the "easy" way to apply the change 
globally, even to places that don't need it, or optionally don't need 
it... too big a hammer. Having the extra parameter also might make them 
not apply it globally, and think more about what and why they are doing 
what they are doing.


Whatever, the idea is out there. If no one likes it, let it die.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Stephen J. Turnbull]

Glenn Linderman writes:

 > Well, this thread seems to be top-posted so...

Not a good enough reason for me!

 > Why not provide _urlopen_with_scary_keyword_parameter as the 
 > monkey-patch option?
 > 
 > So after the (global to the module) monkeypatch, they would _still_ have 
 > to add the keyword parameter.

I understand the hardline position, though I don't like it: "if you
don't know how to do it yourself, we won't help you do it at all."[1]

But this "defense in depth" suggestion really violates the "consenting
adults" principle.  One warning in the docs and another in the name
itself should be enough, and if it isn't, Mommy should take Jimmy's
RaspberryPi away.

Footnotes: 
[1]  Personally, I think that taken seriously, this reasoning applies
to anybody who uses computers for anything other than programming,
though.  Should anybody be allowed to use computers, given that
they're going to put their personal data on Facebook for their
stalkers to see or inadvertently install botnet software with whatever
warez they are weak for?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

Replacing urllib.urlopen(url) with
urllib._unsafe_urlopen_without_secure_https(url) would be fine too (actual
name to be picked by whoever writes the code) but I don't see that it
offers much more of a barrier against abuse of this compatibility feature
compared to a keyword argument.

Requiring a monkeypatch feels unnecessarily mean -- I see no reason why the
code can't be in the standard library. It's a bit like the emergency hammer
on a train -- what keeps riders from misusing it is convention (and the
sign next to it), since locking it up would miss the point.

Do note that there are a couple of different common patterns for how this
is used in legacy code, e.g. urllib vs.urllib2, URLOpener vs
FancyURLOpener, urlopen vs. urlretrieve; there are also some internal
calls, e.g. in response to redirects. The ultimate form of the solution
(keyword argument of alternate function or whatever) may depend on the
needs of these various (ancient) architectures.

Regarding 3.4 and 3.5, there's presumably much less legacy code for 3.4,
but its expected lifetime is also much shorter than 2.7's (since we're
already close to releasing 3.5). So I'm still a bit torn -- in the end one
reason to do it in 3.4 is that 3.4 shouldn't have a weaker default than 2.7.

Onwards,

On Mon, Sep 8, 2014 at 7:46 PM, Glenn Linderman 
wrote:

>  Well, this thread seems to be top-posted so...
>
> Why not provide _urlopen_with_scary_keyword_parameter as the monkey-patch
> option?
>
> So after the (global to the module) monkeypatch, they would _still_ have
> to add the keyword parameter.
>
>
>
> On 9/8/2014 4:31 PM, Guido van Rossum wrote:
>
> I still prefer having a parameter on urlopen (or thereabouts) -- it feels
> wrong to make it easier to change this globally than on a per-call basis,
> and if you don't understand monkey-patching, it's impossible to debug if
> you put the patch in the wrong place.
>
> For the poor soul who has a script with many urlopen("https"//")
> calls, well, they probably don't mind the busywork of editing each and
> every one of them.
>
> I'm fine with giving the actual keyword parameter a scary-sounding ugly
> name.
>
> On Mon, Sep 8, 2014 at 3:48 PM, Donald Stufft  wrote:
>
>>
>>  On Sep 8, 2014, at 6:43 PM, Nick Coghlan  wrote:
>>
>>
>> On 9 Sep 2014 08:30, "Donald Stufft"  wrote:
>> >
>> > If someone wants to do this, can’t they write their own 6 line function?
>>
>> Unfortunately not, as the domain knowledge required to know what those
>> six lines should look like is significant.
>>
>> Keeping the old unsafe behaviour around with a more obviously dangerous
>> name is much simpler than explaining to people "Here, copy this chunk of
>> code you don't understand".
>>
>> If we were starting with a blank slate there's no way we'd offer such a
>> thing, but as Jim pointed out, we do want to make it relatively easy for
>> Standard Operating Environment maintainers to hack around it if necessary.
>>
>> Cheers,
>> Nick.
>>
>> >
>> > import ssl
>> > import urllib.request
>> > _real_urlopen = urllib.request.urlopen
>> > def _unverified(*args, **kwargs):
>> > if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
>> > ctx = ssl.create_default_context()
>> > ctx.verify_mode = CERT_NONE
>> > ctx.verify_hostname = False
>> > kwargs[“context”] = ctx
>> > return _real_urlopen(*args, **kwargs)
>> >
>> > ---
>> > Donald Stufft
>> > PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>> >
>>
>>
>>   Why isn’t documentation with appropriate red warnings a suitable place
>> if we really must have it? That sounds like a much better solution that
>> some weird function people monkeypatch. It gives them more control over
>> things (maybe they have a valid certificate chain, but an invalid host
>> name!), it’ll work across all Python implementations, and most importantly,
>> it gives us a place where there is some long form location to be like “yea
>> you really probably don’t want to be doing this” in big red letters.
>>
>>  Overall I’m -1 on either offering the function or documenting it at
>> all, but if we must do something then I think documentation is more than
>> enough.
>>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Glenn Linderman]

Well, this thread seems to be top-posted so...

Why not provide _urlopen_with_scary_keyword_parameter as the 
monkey-patch option?


So after the (global to the module) monkeypatch, they would _still_ have 
to add the keyword parameter.



On 9/8/2014 4:31 PM, Guido van Rossum wrote:
I still prefer having a parameter on urlopen (or thereabouts) -- it 
feels wrong to make it easier to change this globally than on a 
per-call basis, and if you don't understand monkey-patching, it's 
impossible to debug if you put the patch in the wrong place.


For the poor soul who has a script with many 
urlopen("https"//") calls, well, they probably don't mind 
the busywork of editing each and every one of them.


I'm fine with giving the actual keyword parameter a scary-sounding 
ugly name.


On Mon, Sep 8, 2014 at 3:48 PM, Donald Stufft > wrote:




On Sep 8, 2014, at 6:43 PM, Nick Coghlan mailto:ncogh...@gmail.com>> wrote:


On 9 Sep 2014 08:30, "Donald Stufft" mailto:don...@stufft.io>> wrote:
>
> If someone wants to do this, can’t they write their own 6 line
function?

Unfortunately not, as the domain knowledge required to know what
those six lines should look like is significant.

Keeping the old unsafe behaviour around with a more obviously
dangerous name is much simpler than explaining to people "Here,
copy this chunk of code you don't understand".

If we were starting with a blank slate there's no way we'd offer
such a thing, but as Jim pointed out, we do want to make it
relatively easy for Standard Operating Environment maintainers to
hack around it if necessary.

Cheers,
Nick.

>
> import ssl
> import urllib.request
> _real_urlopen = urllib.request.urlopen
> def _unverified(*args, **kwargs):
> if not kwargs.keys() & {“context”, “cafile”, “capath”,
“cadefault”}:
> ctx = ssl.create_default_context()
> ctx.verify_mode = CERT_NONE
> ctx.verify_hostname = False
> kwargs[“context”] = ctx
> return _real_urlopen(*args, **kwargs)
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>



Why isn’t documentation with appropriate red warnings a suitable
place if we really must have it? That sounds like a much better
solution that some weird function people monkeypatch. It gives
them more control over things (maybe they have a valid certificate
chain, but an invalid host name!), it’ll work across all Python
implementations, and most importantly, it gives us a place where
there is some long form location to be like “yea you really
probably don’t want to be doing this” in big red letters.

Overall I’m -1 on either offering the function or documenting it
at all, but if we must do something then I think documentation is
more than enough.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

I still prefer having a parameter on urlopen (or thereabouts) -- it feels
wrong to make it easier to change this globally than on a per-call basis,
and if you don't understand monkey-patching, it's impossible to debug if
you put the patch in the wrong place.

For the poor soul who has a script with many urlopen("https"//")
calls, well, they probably don't mind the busywork of editing each and
every one of them.

I'm fine with giving the actual keyword parameter a scary-sounding ugly
name.

On Mon, Sep 8, 2014 at 3:48 PM, Donald Stufft  wrote:

>
> On Sep 8, 2014, at 6:43 PM, Nick Coghlan  wrote:
>
>
> On 9 Sep 2014 08:30, "Donald Stufft"  wrote:
> >
> > If someone wants to do this, can’t they write their own 6 line function?
>
> Unfortunately not, as the domain knowledge required to know what those six
> lines should look like is significant.
>
> Keeping the old unsafe behaviour around with a more obviously dangerous
> name is much simpler than explaining to people "Here, copy this chunk of
> code you don't understand".
>
> If we were starting with a blank slate there's no way we'd offer such a
> thing, but as Jim pointed out, we do want to make it relatively easy for
> Standard Operating Environment maintainers to hack around it if necessary.
>
> Cheers,
> Nick.
>
> >
> > import ssl
> > import urllib.request
> > _real_urlopen = urllib.request.urlopen
> > def _unverified(*args, **kwargs):
> > if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
> > ctx = ssl.create_default_context()
> > ctx.verify_mode = CERT_NONE
> > ctx.verify_hostname = False
> > kwargs[“context”] = ctx
> > return _real_urlopen(*args, **kwargs)
> >
> > ---
> > Donald Stufft
> > PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> >
>
>
> Why isn’t documentation with appropriate red warnings a suitable place if
> we really must have it? That sounds like a much better solution that some
> weird function people monkeypatch. It gives them more control over things
> (maybe they have a valid certificate chain, but an invalid host name!),
> it’ll work across all Python implementations, and most importantly, it
> gives us a place where there is some long form location to be like “yea you
> really probably don’t want to be doing this” in big red letters.
>
> Overall I’m -1 on either offering the function or documenting it at all,
> but if we must do something then I think documentation is more than enough.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Donald Stufft]

> On Sep 8, 2014, at 6:43 PM, Nick Coghlan  wrote:
> 
> 
> On 9 Sep 2014 08:30, "Donald Stufft"  > wrote:
> >
> > If someone wants to do this, can’t they write their own 6 line function?
> 
> Unfortunately not, as the domain knowledge required to know what those six 
> lines should look like is significant.
> 
> Keeping the old unsafe behaviour around with a more obviously dangerous name 
> is much simpler than explaining to people "Here, copy this chunk of code you 
> don't understand".
> 
> If we were starting with a blank slate there's no way we'd offer such a 
> thing, but as Jim pointed out, we do want to make it relatively easy for 
> Standard Operating Environment maintainers to hack around it if necessary.
> 
> Cheers,
> Nick.
> 
> >
> > import ssl
> > import urllib.request
> > _real_urlopen = urllib.request.urlopen
> > def _unverified(*args, **kwargs):
> > if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
> > ctx = ssl.create_default_context()
> > ctx.verify_mode = CERT_NONE
> > ctx.verify_hostname = False
> > kwargs[“context”] = ctx
> > return _real_urlopen(*args, **kwargs)
> >
> > ---
> > Donald Stufft
> > PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> >


Why isn’t documentation with appropriate red warnings a suitable place if we 
really must have it? That sounds like a much better solution that some weird 
function people monkeypatch. It gives them more control over things (maybe they 
have a valid certificate chain, but an invalid host name!), it’ll work across 
all Python implementations, and most importantly, it gives us a place where 
there is some long form location to be like “yea you really probably don’t want 
to be doing this” in big red letters.

Overall I’m -1 on either offering the function or documenting it at all, but if 
we must do something then I think documentation is more than enough.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 9 Sep 2014 08:30, "Donald Stufft"  wrote:
>
> If someone wants to do this, can’t they write their own 6 line function?

Unfortunately not, as the domain knowledge required to know what those six
lines should look like is significant.

Keeping the old unsafe behaviour around with a more obviously dangerous
name is much simpler than explaining to people "Here, copy this chunk of
code you don't understand".

If we were starting with a blank slate there's no way we'd offer such a
thing, but as Jim pointed out, we do want to make it relatively easy for
Standard Operating Environment maintainers to hack around it if necessary.

Cheers,
Nick.

>
> import ssl
> import urllib.request
> _real_urlopen = urllib.request.urlopen
> def _unverified(*args, **kwargs):
> if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
> ctx = ssl.create_default_context()
> ctx.verify_mode = CERT_NONE
> ctx.verify_hostname = False
> kwargs[“context”] = ctx
> return _real_urlopen(*args, **kwargs)
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 9 Sep 2014 08:20, "Nick Coghlan"  wrote:
>
>
> On 9 Sep 2014 04:00, "Barry Warsaw"  wrote:
> > >
> > >This would need to be updated first, once it *did* take such an
argument,
> > >this would be accomplished by:
> > >
> > >context = ssl.create_default_context()
> > >context.verify_mode = CERT_OPTIONACERT_NONE
> > >context.verify_hostname = False
> > >urllib.request.urlopen("
https://something-i-apparently-dont-care-much-about";,
> > >context=context)
> >
> > There's probably an ugly hack possibility that uses
unittest.mock.patch. ;)
>
> We could actually make it an "official" hack:
>
> import urllib.request
> urllib.request.urlopen = urllib.request._unverified_urlopen

Thinking about it a bit more, I suspect httplib would be the right level
for such a hack.

Either way, I actually think a monkeypatching based solution is a
reasonable choice here. You can downgrade back to the old behaviour
selectively (calling the unverified version or monkeypatching the calling
module) or globally (monkeypatching the httplib module)

If folks go "Ewww, I'm going to fix my code or certs instead", that's a
good outcome :)

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Donald Stufft]

> On Sep 8, 2014, at 6:20 PM, Nick Coghlan  wrote:
> 
> 
> On 9 Sep 2014 04:00, "Barry Warsaw"  > wrote:
> > >
> > >This would need to be updated first, once it *did* take such an argument,
> > >this would be accomplished by:
> > >
> > >context = ssl.create_default_context()
> > >context.verify_mode = CERT_OPTIONACERT_NONE
> > >context.verify_hostname = False
> > >urllib.request.urlopen("https://something-i-apparently-dont-care-much-about
> > > ",
> > >context=context)
> >
> > There's probably an ugly hack possibility that uses unittest.mock.patch. ;)
> 
> We could actually make it an "official" hack:
> 
> import urllib.request
> urllib.request.urlopen = urllib.request._unverified_urlopen
> 
> Or else the user can just change the code to call the unverified one directly.
> 
> All we'd have to do is keep the existing version that doesn't validate certs 
> properly around under the name "_unverified_urlopen".
> 
> I like this for a few reasons:
> 
> 1. It doesn't get much easier than calling function A instead of function B
> 2. Monkeypatching lets you do a process global hack 
> 3. The name tells you exactly why this is a bad idea
> 4. It's easy to grep for later after you fix your certs
> 5. The leading underscore acts as a strong "keep away" signal
> 6. The leading underscore makes it clear this function may not always be 
> available (e.g. Jython, older versions of Python)
> 
> 

If someone wants to do this, can’t they write their own 6 line function? 

import ssl
import urllib.request
_real_urlopen = urllib.request.urlopen
def _unverified(*args, **kwargs):
if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
ctx = ssl.create_default_context()
ctx.verify_mode = CERT_NONE
ctx.verify_hostname = False
kwargs[“context”] = ctx
return _real_urlopen(*args, **kwargs)

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Barry Warsaw]

On Sep 09, 2014, at 08:20 AM, Nick Coghlan wrote:

>We could actually make it an "official" hack:
>
>import urllib.request
>urllib.request.urlopen = urllib.request._unverified_urlopen
>
>Or else the user can just change the code to call the unverified one
>directly.
>
>All we'd have to do is keep the existing version that doesn't validate
>certs properly around under the name "_unverified_urlopen".
>
>I like this for a few reasons:
>
>1. It doesn't get much easier than calling function A instead of function B
>2. Monkeypatching lets you do a process global hack
>3. The name tells you exactly why this is a bad idea
>4. It's easy to grep for later after you fix your certs
>5. The leading underscore acts as a strong "keep away" signal
>6. The leading underscore makes it clear this function may not always be
>available (e.g. Jython, older versions of Python)

+1.  This would also make it easy to mock in a context manager if you just
wanted to ignore certs for a small section of code.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 9 Sep 2014 04:00, "Barry Warsaw"  wrote:
> >
> >This would need to be updated first, once it *did* take such an argument,
> >this would be accomplished by:
> >
> >context = ssl.create_default_context()
> >context.verify_mode = CERT_OPTIONACERT_NONE
> >context.verify_hostname = False
> >urllib.request.urlopen("
https://something-i-apparently-dont-care-much-about";,
> >context=context)
>
> There's probably an ugly hack possibility that uses unittest.mock.patch.
;)

We could actually make it an "official" hack:

import urllib.request
urllib.request.urlopen = urllib.request._unverified_urlopen

Or else the user can just change the code to call the unverified one
directly.

All we'd have to do is keep the existing version that doesn't validate
certs properly around under the name "_unverified_urlopen".

I like this for a few reasons:

1. It doesn't get much easier than calling function A instead of function B
2. Monkeypatching lets you do a process global hack
3. The name tells you exactly why this is a bad idea
4. It's easy to grep for later after you fix your certs
5. The leading underscore acts as a strong "keep away" signal
6. The leading underscore makes it clear this function may not always be
available (e.g. Jython, older versions of Python)

Cheers,
Nick.

>
> -Barry
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Jim Baker]

+1 for the suggested change to 2.7.

Something I have put off in the work on SSL support in Jython 2.7 is what
to do about the possibility of adding a large security hole to support
standard Python behavior here with CERT_NONE. By default, we use the
standard trust database and corresponding manager from Java, as augmented
by any provided ca_certs. In practice, I don't think people are really
noticing that it's currently locked down in the latest beta, which added
this SSL support.

Although it's very easy to open such a hole on Jython/Java, it's much nicer
if we require the user has to do some work to do so, and on a per
connection basis.

- Jim



On Mon, Sep 8, 2014 at 10:58 AM, Guido van Rossum  wrote:

> On Mon, Sep 8, 2014 at 5:08 AM, Nick Coghlan  wrote:
>
>>
>> It would also be good to get Guido's official verdict on PEP 476 (the
>> switch to validating HTTPS by default) in time for 3.4.2. Based on the
>> previous discussion, Alex updated the PEP to suggest "just fix it" for
>> all of 3.5, 3.4.x and 2.7.x (including the httplib SSLContext support
>> backport in the latter case).
>>
>
> My opinion hasn't changed since the last time I opened my mouth
> prematurely. :-) I would very much like these to go in, but for 2.7 I am
> now worried about what we should tell people who have a script that uses an
> https URL to access a service that can only be accessed via SSL/TLS to a
> self-signed or otherwise mis-configured cert. I am not insisting on an
> environment variable to disable this (too easy) but I do think it must be
> possible to make a simple change to the code, on the order of tracking down
> the urlopen() call and adding a new keyword parameter. Such a band-aid
> needn't be backward compatible (we can introduce a new keyword parameter
> for this purpose) and it needn't be totally straightforward (we can assume
> some modicum of understanding of finding and editing .py files) but it
> should definitely not require a refactor of the script (e.g. swapping out
> urlopen and replacing it with httplib or requests would be too much of a
> burden). And we should have prominent documentation (perhaps in FAQ form?)
> with an example of how to do it.
>
>
>> I think that would be feasible with an rc on the 20th, but challenging
>> if the rc is this coming weekend.
>>
>> Note, as I stated in the previous thread, I'm now +1 on that PEP,
>> because I don't see any way to write an automated scan for a large
>> code base that ensures we're not relying on the default handling at
>> all. If the default behaviour is to validate HTTPS, the lack of such a
>> scanner isn't a problem - any failures to cope with self-signed or
>> invalid certs will be noisy, and we can either fix the certs, patch
>> the code to cope with them appropriately, or (for the self-signed cert
>> case) configure OpenSSL via environment variables. If dealing with
>> invalid certs is truly necessary, and the code can't be updated
>> either, then I'm OK with the answer being "keep using an older version
>> of Python, as that's going to be the least of your security concerns".
>>
>
> Yeah, I am not interested in helping out the case where the user is
> incapable (for whatever reason) of tracking down and changing a couple of
> lines of code. Such users are dependent on someone else with wizard powers
> anyway (who gave them the script?).
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Barry Warsaw]

On Sep 08, 2014, at 10:44 AM, Alex Gaynor wrote:

>*Shifts uncomfortably* it looks like presently there's not a good way to
>change anything about the SSL configuration for urllib.request.urlopen. It
>does not take a `context` argument, as the http.client API does:
>https://docs.python.org/3/library/urllib.request.html#module-urllib.request
>and instead takes the cafile, capath, cadefault args.
>
>This would need to be updated first, once it *did* take such an argument,
>this would be accomplished by:
>
>context = ssl.create_default_context()
>context.verify_mode = CERT_OPTIONACERT_NONE
>context.verify_hostname = False
>urllib.request.urlopen("https://something-i-apparently-dont-care-much-about";,
>context=context)

There's probably an ugly hack possibility that uses unittest.mock.patch. ;)

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

Well, get cracking then! :-)

On Mon, Sep 8, 2014 at 10:44 AM, Alex Gaynor  wrote:

> *Shifts uncomfortably* it looks like presently there's not a good way to
> change anything about the SSL configuration for urllib.request.urlopen. It
> does not take a `context` argument, as the http.client API does:
> https://docs.python.org/3/library/urllib.request.html#module-urllib.request
> and instead takes the cafile, capath, cadefault args.
>
> This would need to be updated first, once it *did* take such an argument,
> this would be accomplished by:
>
> context = ssl.create_default_context()
> context.verify_mode = CERT_OPTIONACERT_NONE
> context.verify_hostname = False
> urllib.request.urlopen("
> https://something-i-apparently-dont-care-much-about";, context=context)
>
> Alex
>
>
> On Mon, Sep 8, 2014 at 10:35 AM, Guido van Rossum 
> wrote:
>
>> I will pronounce for 3.4 once you point me to the documentation that
>> explains how to disable cert validation for an example program that
>> currently pulls down an https URL using urlopen. Without adding package
>> dependencies.
>>
>> On Mon, Sep 8, 2014 at 10:25 AM, Alex Gaynor 
>> wrote:
>>
>>> Guido van Rossum  python.org> writes:
>>>
>>> >
>>> >
>>>
>>> Would you be willing to officially pronounce on PEP-476 in the context
>>> of 3.4.x,
>>> so we can get it into the release, and then we can defer on officially
>>> approving
>>> it for 2.7.X until we figure out all the moving pieces?
>>>
>>> Cheers,
>>> Alex
>>>
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Alex Gaynor]

*Shifts uncomfortably* it looks like presently there's not a good way to
change anything about the SSL configuration for urllib.request.urlopen. It
does not take a `context` argument, as the http.client API does:
https://docs.python.org/3/library/urllib.request.html#module-urllib.request
and instead takes the cafile, capath, cadefault args.

This would need to be updated first, once it *did* take such an argument,
this would be accomplished by:

context = ssl.create_default_context()
context.verify_mode = CERT_OPTIONACERT_NONE
context.verify_hostname = False
urllib.request.urlopen("https://something-i-apparently-dont-care-much-about";,
context=context)

Alex


On Mon, Sep 8, 2014 at 10:35 AM, Guido van Rossum  wrote:

> I will pronounce for 3.4 once you point me to the documentation that
> explains how to disable cert validation for an example program that
> currently pulls down an https URL using urlopen. Without adding package
> dependencies.
>
> On Mon, Sep 8, 2014 at 10:25 AM, Alex Gaynor 
> wrote:
>
>> Guido van Rossum  python.org> writes:
>>
>> >
>> >
>>
>> Would you be willing to officially pronounce on PEP-476 in the context of
>> 3.4.x,
>> so we can get it into the release, and then we can defer on officially
>> approving
>> it for 2.7.X until we figure out all the moving pieces?
>>
>> Cheers,
>> Alex
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

I will pronounce for 3.4 once you point me to the documentation that
explains how to disable cert validation for an example program that
currently pulls down an https URL using urlopen. Without adding package
dependencies.

On Mon, Sep 8, 2014 at 10:25 AM, Alex Gaynor  wrote:

> Guido van Rossum  python.org> writes:
>
> >
> >
>
> Would you be willing to officially pronounce on PEP-476 in the context of
> 3.4.x,
> so we can get it into the release, and then we can defer on officially
> approving
> it for 2.7.X until we figure out all the moving pieces?
>
> Cheers,
> Alex
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Alex Gaynor]

Guido van Rossum  python.org> writes:

> 
> 

Would you be willing to officially pronounce on PEP-476 in the context of 3.4.x,
so we can get it into the release, and then we can defer on officially approving
it for 2.7.X until we figure out all the moving pieces?

Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

On Mon, Sep 8, 2014 at 5:08 AM, Nick Coghlan  wrote:

>
> It would also be good to get Guido's official verdict on PEP 476 (the
> switch to validating HTTPS by default) in time for 3.4.2. Based on the
> previous discussion, Alex updated the PEP to suggest "just fix it" for
> all of 3.5, 3.4.x and 2.7.x (including the httplib SSLContext support
> backport in the latter case).
>

My opinion hasn't changed since the last time I opened my mouth
prematurely. :-) I would very much like these to go in, but for 2.7 I am
now worried about what we should tell people who have a script that uses an
https URL to access a service that can only be accessed via SSL/TLS to a
self-signed or otherwise mis-configured cert. I am not insisting on an
environment variable to disable this (too easy) but I do think it must be
possible to make a simple change to the code, on the order of tracking down
the urlopen() call and adding a new keyword parameter. Such a band-aid
needn't be backward compatible (we can introduce a new keyword parameter
for this purpose) and it needn't be totally straightforward (we can assume
some modicum of understanding of finding and editing .py files) but it
should definitely not require a refactor of the script (e.g. swapping out
urlopen and replacing it with httplib or requests would be too much of a
burden). And we should have prominent documentation (perhaps in FAQ form?)
with an example of how to do it.


> I think that would be feasible with an rc on the 20th, but challenging
> if the rc is this coming weekend.
>
> Note, as I stated in the previous thread, I'm now +1 on that PEP,
> because I don't see any way to write an automated scan for a large
> code base that ensures we're not relying on the default handling at
> all. If the default behaviour is to validate HTTPS, the lack of such a
> scanner isn't a problem - any failures to cope with self-signed or
> invalid certs will be noisy, and we can either fix the certs, patch
> the code to cope with them appropriately, or (for the self-signed cert
> case) configure OpenSSL via environment variables. If dealing with
> invalid certs is truly necessary, and the code can't be updated
> either, then I'm OK with the answer being "keep using an older version
> of Python, as that's going to be the least of your security concerns".
>

Yeah, I am not interested in helping out the case where the user is
incapable (for whatever reason) of tracking down and changing a couple of
lines of code. Such users are dependent on someone else with wizard powers
anyway (who gave them the script?).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 8 September 2014 14:28, Ned Deily  wrote:
> In article <540c521c.7070...@hastings.org>,
>  Larry Hastings  wrote:
>> Matthias asked me when I was going to release 3.4.2.  I propose the
>> following schedule:
>>
>> Tag 3.4.2rc1 Friday Sep 12 2014
>> Release 3.4.2rc1 Saturday Sep 13 2014
>> Tag 3.4.2 final Saturday Sep 27 2014
>> Release 3.4.2 final Sunday Sep 28 2014
>>
>> Normally I want to tag on Saturdays and release on Sundays, however I
>> propose moving rc1 a day earlier because I'll be unavailable that Sunday.
>>
>> Any objections?  I will interpret silence as tacit approval.
>
> As I've already discussed with Larry, I think adding a week to the
> scheduled dates would be preferable.  The original dates give pretty
> short notice and there are a number of open issues that would be good to
> resolve now in 3.4.2.

It would also be good to get Guido's official verdict on PEP 476 (the
switch to validating HTTPS by default) in time for 3.4.2. Based on the
previous discussion, Alex updated the PEP to suggest "just fix it" for
all of 3.5, 3.4.x and 2.7.x (including the httplib SSLContext support
backport in the latter case).

I think that would be feasible with an rc on the 20th, but challenging
if the rc is this coming weekend.

Note, as I stated in the previous thread, I'm now +1 on that PEP,
because I don't see any way to write an automated scan for a large
code base that ensures we're not relying on the default handling at
all. If the default behaviour is to validate HTTPS, the lack of such a
scanner isn't a problem - any failures to cope with self-signed or
invalid certs will be noisy, and we can either fix the certs, patch
the code to cope with them appropriately, or (for the self-signed cert
case) configure OpenSSL via environment variables. If dealing with
invalid certs is truly necessary, and the code can't be updated
either, then I'm OK with the answer being "keep using an older version
of Python, as that's going to be the least of your security concerns".

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Ned Deily]

In article <540c521c.7070...@hastings.org>,
 Larry Hastings  wrote:
> Matthias asked me when I was going to release 3.4.2.  I propose the 
> following schedule:
> 
> Tag 3.4.2rc1 Friday Sep 12 2014
> Release 3.4.2rc1 Saturday Sep 13 2014
> Tag 3.4.2 final Saturday Sep 27 2014
> Release 3.4.2 final Sunday Sep 28 2014
> 
> Normally I want to tag on Saturdays and release on Sundays, however I 
> propose moving rc1 a day earlier because I'll be unavailable that Sunday.
> 
> Any objections?  I will interpret silence as tacit approval.

As I've already discussed with Larry, I think adding a week to the 
scheduled dates would be preferable.  The original dates give pretty 
short notice and there are a number of open issues that would be good to 
resolve now in 3.4.2.

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com