Re: [Python-Dev] Proposed schedule for 3.4.2

[Glenn Linderman]

On 9/8/2014 8:41 PM, Stephen J. Turnbull wrote:

  > Why not provide _urlopen_with_scary_keyword_parameter as the
  > monkey-patch option?
  >
  > So after the (global to the module) monkeypatch, they would_still_  have
  > to add the keyword parameter.

I understand the hardline position, though I don't like it: "if you
don't know how to do it yourself, we won't help you do it at all."[1]

But this "defense in depth" suggestion really violates the "consenting
adults" principle.  One warning in the docs and another in the name
itself should be enough, and if it isn't, Mommy should take Jimmy's
RaspberryPi away.
I was assuming, because of the suggestion for a monkey patch at all, in 
response to Guido's suggestion of a keyword parameter, that there was a 
problem adding a keyword parameter to urlopen. If there is, then the 
combo above could be useful in making them track down and adjust the 
places that need it, without forcing them to adjust the places that 
don't need it? If there is not, then no need for the monkey patch at 
all, they can just change add the keyword parameter.


If the alternate function doesn't have an extra keyword parameter, the 
monkeypatch solution would be the "easy" way to apply the change 
globally, even to places that don't need it, or optionally don't need 
it... too big a hammer. Having the extra parameter also might make them 
not apply it globally, and think more about what and why they are doing 
what they are doing.


Whatever, the idea is out there. If no one likes it, let it die.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backwards compatibility after certificate autovalidation

[Donald Stufft]

It’s create_default_context() -> 
https://docs.python.org/3.4/library/ssl.html#ssl.create_default_context


> On Sep 8, 2014, at 11:40 PM, Guido van Rossum  wrote:
> 
> The multiple threads going on are confusing (or maybe GMail makes them more 
> confusing), but the architecture you are sketching here sounds good. I can't 
> find get_default_context() in the repo, but perhaps I need to refresh, or 
> perhaps you're talking about a design in a PEP.
> 
> On Mon, Sep 8, 2014 at 8:03 PM, Nick Coghlan  > wrote:
> 
> On 9 Sep 2014 10:48, "Jim J. Jewett"  > wrote:
> > I assume that adding _unverified_urlopen or urlopen(context=...) do
> > provide incremental improvements compatible with the eventual full
> > opt-in.  If so, adding them is probably reasonable, but I think the
> > PEP should explicitly list all such approved half-measures as a guard
> > against API feature creep.
> 
> From Guido's and your feedback, I think we may need two things to approve 
> this for 3.4.2 (putting 2.7 aside for now):
> 
> 1. "context" parameter support in urllib.request (to opt out on a per-call 
> basis)
> 2. a documented way to restore the old behaviour via sitecustomize (which may 
> involve monkeypatching)
> 
> The former change seems non-controversial.
> 
> I think the more fine-grained solution for the latter can wait until 3.5 (and 
> will be an independent PEP), we just need an interim workaround for 3.4 that 
> could conceivably be backported to 2.7.
> 
> On that front, ssl currently has two context factories: get_default_context() 
> and _get_stdlib_context. One possible option would be to:
> 
> 1. Rename "_get_stdlib_context" to "_get_unverified_context"
> 2. Add "_get_https_context" as an alias for "get_default_context"
> 
> Opting out on a per-call basis means passing an unverified context.
> 
> Opting out globally would mean monkeypatching _get_https_context to refer to 
> _get_unverified_context instead.
> 
> These would both be documented as part of transition, but with clear security 
> warnings. The use of the leading underscores in the names is intended to 
> emphasise "you probably don't want to be using this".
> 
> Regards,
> Nick.
> 
> 
> 
> >
> > -jJ
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org 
> > https://mail.python.org/mailman/listinfo/python-dev 
> > 
> > Unsubscribe: 
> > https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com 
> > 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev 
> 
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/guido%40python.org 
> 
> 
> 
> 
> 
> -- 
> --Guido van Rossum (python.org/~guido )
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Stephen J. Turnbull]

Glenn Linderman writes:

 > Well, this thread seems to be top-posted so...

Not a good enough reason for me!

 > Why not provide _urlopen_with_scary_keyword_parameter as the 
 > monkey-patch option?
 > 
 > So after the (global to the module) monkeypatch, they would _still_ have 
 > to add the keyword parameter.

I understand the hardline position, though I don't like it: "if you
don't know how to do it yourself, we won't help you do it at all."[1]

But this "defense in depth" suggestion really violates the "consenting
adults" principle.  One warning in the docs and another in the name
itself should be enough, and if it isn't, Mommy should take Jimmy's
RaspberryPi away.

Footnotes: 
[1]  Personally, I think that taken seriously, this reasoning applies
to anybody who uses computers for anything other than programming,
though.  Should anybody be allowed to use computers, given that
they're going to put their personal data on Facebook for their
stalkers to see or inadvertently install botnet software with whatever
warez they are weak for?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backwards compatibility after certificate autovalidation

[Guido van Rossum]

The multiple threads going on are confusing (or maybe GMail makes them more
confusing), but the architecture you are sketching here sounds good. I
can't find get_default_context() in the repo, but perhaps I need to
refresh, or perhaps you're talking about a design in a PEP.

On Mon, Sep 8, 2014 at 8:03 PM, Nick Coghlan  wrote:

>
> On 9 Sep 2014 10:48, "Jim J. Jewett"  wrote:
> > I assume that adding _unverified_urlopen or urlopen(context=...) do
> > provide incremental improvements compatible with the eventual full
> > opt-in.  If so, adding them is probably reasonable, but I think the
> > PEP should explicitly list all such approved half-measures as a guard
> > against API feature creep.
>
> From Guido's and your feedback, I think we may need two things to approve
> this for 3.4.2 (putting 2.7 aside for now):
>
> 1. "context" parameter support in urllib.request (to opt out on a per-call
> basis)
> 2. a documented way to restore the old behaviour via sitecustomize (which
> may involve monkeypatching)
>
> The former change seems non-controversial.
>
> I think the more fine-grained solution for the latter can wait until 3.5
> (and will be an independent PEP), we just need an interim workaround for
> 3.4 that could conceivably be backported to 2.7.
>
> On that front, ssl currently has two context factories:
> get_default_context() and _get_stdlib_context. One possible option would be
> to:
>
> 1. Rename "_get_stdlib_context" to "_get_unverified_context"
> 2. Add "_get_https_context" as an alias for "get_default_context"
>
> Opting out on a per-call basis means passing an unverified context.
>
> Opting out globally would mean monkeypatching _get_https_context to refer
> to _get_unverified_context instead.
>
> These would both be documented as part of transition, but with clear
> security warnings. The use of the leading underscores in the names is
> intended to emphasise "you probably don't want to be using this".
>
> Regards,
> Nick.
>
>
> >
> > -jJ
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

Replacing urllib.urlopen(url) with
urllib._unsafe_urlopen_without_secure_https(url) would be fine too (actual
name to be picked by whoever writes the code) but I don't see that it
offers much more of a barrier against abuse of this compatibility feature
compared to a keyword argument.

Requiring a monkeypatch feels unnecessarily mean -- I see no reason why the
code can't be in the standard library. It's a bit like the emergency hammer
on a train -- what keeps riders from misusing it is convention (and the
sign next to it), since locking it up would miss the point.

Do note that there are a couple of different common patterns for how this
is used in legacy code, e.g. urllib vs.urllib2, URLOpener vs
FancyURLOpener, urlopen vs. urlretrieve; there are also some internal
calls, e.g. in response to redirects. The ultimate form of the solution
(keyword argument of alternate function or whatever) may depend on the
needs of these various (ancient) architectures.

Regarding 3.4 and 3.5, there's presumably much less legacy code for 3.4,
but its expected lifetime is also much shorter than 2.7's (since we're
already close to releasing 3.5). So I'm still a bit torn -- in the end one
reason to do it in 3.4 is that 3.4 shouldn't have a weaker default than 2.7.

Onwards,

On Mon, Sep 8, 2014 at 7:46 PM, Glenn Linderman 
wrote:

>  Well, this thread seems to be top-posted so...
>
> Why not provide _urlopen_with_scary_keyword_parameter as the monkey-patch
> option?
>
> So after the (global to the module) monkeypatch, they would _still_ have
> to add the keyword parameter.
>
>
>
> On 9/8/2014 4:31 PM, Guido van Rossum wrote:
>
> I still prefer having a parameter on urlopen (or thereabouts) -- it feels
> wrong to make it easier to change this globally than on a per-call basis,
> and if you don't understand monkey-patching, it's impossible to debug if
> you put the patch in the wrong place.
>
> For the poor soul who has a script with many urlopen("https"//")
> calls, well, they probably don't mind the busywork of editing each and
> every one of them.
>
> I'm fine with giving the actual keyword parameter a scary-sounding ugly
> name.
>
> On Mon, Sep 8, 2014 at 3:48 PM, Donald Stufft  wrote:
>
>>
>>  On Sep 8, 2014, at 6:43 PM, Nick Coghlan  wrote:
>>
>>
>> On 9 Sep 2014 08:30, "Donald Stufft"  wrote:
>> >
>> > If someone wants to do this, can’t they write their own 6 line function?
>>
>> Unfortunately not, as the domain knowledge required to know what those
>> six lines should look like is significant.
>>
>> Keeping the old unsafe behaviour around with a more obviously dangerous
>> name is much simpler than explaining to people "Here, copy this chunk of
>> code you don't understand".
>>
>> If we were starting with a blank slate there's no way we'd offer such a
>> thing, but as Jim pointed out, we do want to make it relatively easy for
>> Standard Operating Environment maintainers to hack around it if necessary.
>>
>> Cheers,
>> Nick.
>>
>> >
>> > import ssl
>> > import urllib.request
>> > _real_urlopen = urllib.request.urlopen
>> > def _unverified(*args, **kwargs):
>> > if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
>> > ctx = ssl.create_default_context()
>> > ctx.verify_mode = CERT_NONE
>> > ctx.verify_hostname = False
>> > kwargs[“context”] = ctx
>> > return _real_urlopen(*args, **kwargs)
>> >
>> > ---
>> > Donald Stufft
>> > PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>> >
>>
>>
>>   Why isn’t documentation with appropriate red warnings a suitable place
>> if we really must have it? That sounds like a much better solution that
>> some weird function people monkeypatch. It gives them more control over
>> things (maybe they have a valid certificate chain, but an invalid host
>> name!), it’ll work across all Python implementations, and most importantly,
>> it gives us a place where there is some long form location to be like “yea
>> you really probably don’t want to be doing this” in big red letters.
>>
>>  Overall I’m -1 on either offering the function or documenting it at
>> all, but if we must do something then I think documentation is more than
>> enough.
>>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backwards compatibility after certificate autovalidation

[Nick Coghlan]

On 9 Sep 2014 10:48, "Jim J. Jewett"  wrote:
> I assume that adding _unverified_urlopen or urlopen(context=...) do
> provide incremental improvements compatible with the eventual full
> opt-in.  If so, adding them is probably reasonable, but I think the
> PEP should explicitly list all such approved half-measures as a guard
> against API feature creep.

>From Guido's and your feedback, I think we may need two things to approve
this for 3.4.2 (putting 2.7 aside for now):

1. "context" parameter support in urllib.request (to opt out on a per-call
basis)
2. a documented way to restore the old behaviour via sitecustomize (which
may involve monkeypatching)

The former change seems non-controversial.

I think the more fine-grained solution for the latter can wait until 3.5
(and will be an independent PEP), we just need an interim workaround for
3.4 that could conceivably be backported to 2.7.

On that front, ssl currently has two context factories:
get_default_context() and _get_stdlib_context. One possible option would be
to:

1. Rename "_get_stdlib_context" to "_get_unverified_context"
2. Add "_get_https_context" as an alias for "get_default_context"

Opting out on a per-call basis means passing an unverified context.

Opting out globally would mean monkeypatching _get_https_context to refer
to _get_unverified_context instead.

These would both be documented as part of transition, but with clear
security warnings. The use of the leading underscores in the names is
intended to emphasise "you probably don't want to be using this".

Regards,
Nick.


>
> -jJ
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Glenn Linderman]

Well, this thread seems to be top-posted so...

Why not provide _urlopen_with_scary_keyword_parameter as the 
monkey-patch option?


So after the (global to the module) monkeypatch, they would _still_ have 
to add the keyword parameter.



On 9/8/2014 4:31 PM, Guido van Rossum wrote:
I still prefer having a parameter on urlopen (or thereabouts) -- it 
feels wrong to make it easier to change this globally than on a 
per-call basis, and if you don't understand monkey-patching, it's 
impossible to debug if you put the patch in the wrong place.


For the poor soul who has a script with many 
urlopen("https"//") calls, well, they probably don't mind 
the busywork of editing each and every one of them.


I'm fine with giving the actual keyword parameter a scary-sounding 
ugly name.


On Mon, Sep 8, 2014 at 3:48 PM, Donald Stufft > wrote:




On Sep 8, 2014, at 6:43 PM, Nick Coghlan mailto:ncogh...@gmail.com>> wrote:


On 9 Sep 2014 08:30, "Donald Stufft" mailto:don...@stufft.io>> wrote:
>
> If someone wants to do this, can’t they write their own 6 line
function?

Unfortunately not, as the domain knowledge required to know what
those six lines should look like is significant.

Keeping the old unsafe behaviour around with a more obviously
dangerous name is much simpler than explaining to people "Here,
copy this chunk of code you don't understand".

If we were starting with a blank slate there's no way we'd offer
such a thing, but as Jim pointed out, we do want to make it
relatively easy for Standard Operating Environment maintainers to
hack around it if necessary.

Cheers,
Nick.

>
> import ssl
> import urllib.request
> _real_urlopen = urllib.request.urlopen
> def _unverified(*args, **kwargs):
> if not kwargs.keys() & {“context”, “cafile”, “capath”,
“cadefault”}:
> ctx = ssl.create_default_context()
> ctx.verify_mode = CERT_NONE
> ctx.verify_hostname = False
> kwargs[“context”] = ctx
> return _real_urlopen(*args, **kwargs)
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>



Why isn’t documentation with appropriate red warnings a suitable
place if we really must have it? That sounds like a much better
solution that some weird function people monkeypatch. It gives
them more control over things (maybe they have a valid certificate
chain, but an invalid host name!), it’ll work across all Python
implementations, and most importantly, it gives us a place where
there is some long form location to be like “yea you really
probably don’t want to be doing this” in big red letters.

Overall I’m -1 on either offering the function or documenting it
at all, but if we must do something then I think documentation is
more than enough.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backwards compatibility after certificate autovalidation

[Jim J. Jewett]

On Mon, Sep 8, 2014 at 3:44 PM, Cory Benfield  wrote:
> On 8 September 2014 18:23, Jim J. Jewett  wrote:
>> Summary:  There needs to be a simple way to opt out at install time.
>> It would be far better to offer more fine-grained control, but leaving
>> that better solution to downstream is acceptable.

> Does this argument apply to a hypothetical 2.7 backport of this
> change, or does it apply to making the change in 3.5? (Or of course
> both.)

I believe the argument applies even to 3.5, given that there was no
deprecation period.  The concern is obviously stronger for maintenance
releases.

I am not saying that secure-by-default should wait until until 3.6; I
am saying that the "rush" requires even more attention than usual to
backwards compatibility.

This actually argues *for* backporting the fix as at least opt-in, so
that 2.7/3.4 can serve as the "make your changes now, test them
without all the other new features" releases.


Nick's suggestion of a monkey-patching .pth file would be sufficient
backwards compatibility support, if the recipe were referenced from
the release notes (not just the python lib documentation).



Support for partial opt-in -- whether per-process, per call, per
address, etc -- would be nice, but it isn't required for backwards
compatibility.

I think that means an -X option for "noverifyhttps" should NOT be
added.  It doesn't get users closer to the final solution; it just
adds the noise of a different workaround.


I assume that adding _unverified_urlopen or urlopen(context=...) do
provide incremental improvements compatible with the eventual full
opt-in.  If so, adding them is probably reasonable, but I think the
PEP should explicitly list all such approved half-measures as a guard
against API feature creep.

-jJ
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

I still prefer having a parameter on urlopen (or thereabouts) -- it feels
wrong to make it easier to change this globally than on a per-call basis,
and if you don't understand monkey-patching, it's impossible to debug if
you put the patch in the wrong place.

For the poor soul who has a script with many urlopen("https"//")
calls, well, they probably don't mind the busywork of editing each and
every one of them.

I'm fine with giving the actual keyword parameter a scary-sounding ugly
name.

On Mon, Sep 8, 2014 at 3:48 PM, Donald Stufft  wrote:

>
> On Sep 8, 2014, at 6:43 PM, Nick Coghlan  wrote:
>
>
> On 9 Sep 2014 08:30, "Donald Stufft"  wrote:
> >
> > If someone wants to do this, can’t they write their own 6 line function?
>
> Unfortunately not, as the domain knowledge required to know what those six
> lines should look like is significant.
>
> Keeping the old unsafe behaviour around with a more obviously dangerous
> name is much simpler than explaining to people "Here, copy this chunk of
> code you don't understand".
>
> If we were starting with a blank slate there's no way we'd offer such a
> thing, but as Jim pointed out, we do want to make it relatively easy for
> Standard Operating Environment maintainers to hack around it if necessary.
>
> Cheers,
> Nick.
>
> >
> > import ssl
> > import urllib.request
> > _real_urlopen = urllib.request.urlopen
> > def _unverified(*args, **kwargs):
> > if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
> > ctx = ssl.create_default_context()
> > ctx.verify_mode = CERT_NONE
> > ctx.verify_hostname = False
> > kwargs[“context”] = ctx
> > return _real_urlopen(*args, **kwargs)
> >
> > ---
> > Donald Stufft
> > PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> >
>
>
> Why isn’t documentation with appropriate red warnings a suitable place if
> we really must have it? That sounds like a much better solution that some
> weird function people monkeypatch. It gives them more control over things
> (maybe they have a valid certificate chain, but an invalid host name!),
> it’ll work across all Python implementations, and most importantly, it
> gives us a place where there is some long form location to be like “yea you
> really probably don’t want to be doing this” in big red letters.
>
> Overall I’m -1 on either offering the function or documenting it at all,
> but if we must do something then I think documentation is more than enough.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Donald Stufft]

> On Sep 8, 2014, at 6:43 PM, Nick Coghlan  wrote:
> 
> 
> On 9 Sep 2014 08:30, "Donald Stufft"  > wrote:
> >
> > If someone wants to do this, can’t they write their own 6 line function?
> 
> Unfortunately not, as the domain knowledge required to know what those six 
> lines should look like is significant.
> 
> Keeping the old unsafe behaviour around with a more obviously dangerous name 
> is much simpler than explaining to people "Here, copy this chunk of code you 
> don't understand".
> 
> If we were starting with a blank slate there's no way we'd offer such a 
> thing, but as Jim pointed out, we do want to make it relatively easy for 
> Standard Operating Environment maintainers to hack around it if necessary.
> 
> Cheers,
> Nick.
> 
> >
> > import ssl
> > import urllib.request
> > _real_urlopen = urllib.request.urlopen
> > def _unverified(*args, **kwargs):
> > if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
> > ctx = ssl.create_default_context()
> > ctx.verify_mode = CERT_NONE
> > ctx.verify_hostname = False
> > kwargs[“context”] = ctx
> > return _real_urlopen(*args, **kwargs)
> >
> > ---
> > Donald Stufft
> > PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> >


Why isn’t documentation with appropriate red warnings a suitable place if we 
really must have it? That sounds like a much better solution that some weird 
function people monkeypatch. It gives them more control over things (maybe they 
have a valid certificate chain, but an invalid host name!), it’ll work across 
all Python implementations, and most importantly, it gives us a place where 
there is some long form location to be like “yea you really probably don’t want 
to be doing this” in big red letters.

Overall I’m -1 on either offering the function or documenting it at all, but if 
we must do something then I think documentation is more than enough.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 9 Sep 2014 08:30, "Donald Stufft"  wrote:
>
> If someone wants to do this, can’t they write their own 6 line function?

Unfortunately not, as the domain knowledge required to know what those six
lines should look like is significant.

Keeping the old unsafe behaviour around with a more obviously dangerous
name is much simpler than explaining to people "Here, copy this chunk of
code you don't understand".

If we were starting with a blank slate there's no way we'd offer such a
thing, but as Jim pointed out, we do want to make it relatively easy for
Standard Operating Environment maintainers to hack around it if necessary.

Cheers,
Nick.

>
> import ssl
> import urllib.request
> _real_urlopen = urllib.request.urlopen
> def _unverified(*args, **kwargs):
> if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
> ctx = ssl.create_default_context()
> ctx.verify_mode = CERT_NONE
> ctx.verify_hostname = False
> kwargs[“context”] = ctx
> return _real_urlopen(*args, **kwargs)
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backwards compatibility after certificate autovalidation

[Nick Coghlan]

On 9 Sep 2014 03:25, "Jim J. Jewett"  wrote:
> Examples of good enough:
> "Add this file to site-packages"
> "Add this environment variable with this setting"
> "Add this command line switch to your launch script"

The monkeypatching hack I proposed would work correctly in sitecustomize
(or a *.pth file). Cost would be loading httplib in all cases.

A -X option for "noverifyhttps" might also be feasible.

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 9 Sep 2014 08:20, "Nick Coghlan"  wrote:
>
>
> On 9 Sep 2014 04:00, "Barry Warsaw"  wrote:
> > >
> > >This would need to be updated first, once it *did* take such an
argument,
> > >this would be accomplished by:
> > >
> > >context = ssl.create_default_context()
> > >context.verify_mode = CERT_OPTIONACERT_NONE
> > >context.verify_hostname = False
> > >urllib.request.urlopen("
https://something-i-apparently-dont-care-much-about";,
> > >context=context)
> >
> > There's probably an ugly hack possibility that uses
unittest.mock.patch. ;)
>
> We could actually make it an "official" hack:
>
> import urllib.request
> urllib.request.urlopen = urllib.request._unverified_urlopen

Thinking about it a bit more, I suspect httplib would be the right level
for such a hack.

Either way, I actually think a monkeypatching based solution is a
reasonable choice here. You can downgrade back to the old behaviour
selectively (calling the unverified version or monkeypatching the calling
module) or globally (monkeypatching the httplib module)

If folks go "Ewww, I'm going to fix my code or certs instead", that's a
good outcome :)

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Donald Stufft]

> On Sep 8, 2014, at 6:20 PM, Nick Coghlan  wrote:
> 
> 
> On 9 Sep 2014 04:00, "Barry Warsaw"  > wrote:
> > >
> > >This would need to be updated first, once it *did* take such an argument,
> > >this would be accomplished by:
> > >
> > >context = ssl.create_default_context()
> > >context.verify_mode = CERT_OPTIONACERT_NONE
> > >context.verify_hostname = False
> > >urllib.request.urlopen("https://something-i-apparently-dont-care-much-about
> > > ",
> > >context=context)
> >
> > There's probably an ugly hack possibility that uses unittest.mock.patch. ;)
> 
> We could actually make it an "official" hack:
> 
> import urllib.request
> urllib.request.urlopen = urllib.request._unverified_urlopen
> 
> Or else the user can just change the code to call the unverified one directly.
> 
> All we'd have to do is keep the existing version that doesn't validate certs 
> properly around under the name "_unverified_urlopen".
> 
> I like this for a few reasons:
> 
> 1. It doesn't get much easier than calling function A instead of function B
> 2. Monkeypatching lets you do a process global hack 
> 3. The name tells you exactly why this is a bad idea
> 4. It's easy to grep for later after you fix your certs
> 5. The leading underscore acts as a strong "keep away" signal
> 6. The leading underscore makes it clear this function may not always be 
> available (e.g. Jython, older versions of Python)
> 
> 

If someone wants to do this, can’t they write their own 6 line function? 

import ssl
import urllib.request
_real_urlopen = urllib.request.urlopen
def _unverified(*args, **kwargs):
if not kwargs.keys() & {“context”, “cafile”, “capath”, “cadefault”}:
ctx = ssl.create_default_context()
ctx.verify_mode = CERT_NONE
ctx.verify_hostname = False
kwargs[“context”] = ctx
return _real_urlopen(*args, **kwargs)

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Barry Warsaw]

On Sep 09, 2014, at 08:20 AM, Nick Coghlan wrote:

>We could actually make it an "official" hack:
>
>import urllib.request
>urllib.request.urlopen = urllib.request._unverified_urlopen
>
>Or else the user can just change the code to call the unverified one
>directly.
>
>All we'd have to do is keep the existing version that doesn't validate
>certs properly around under the name "_unverified_urlopen".
>
>I like this for a few reasons:
>
>1. It doesn't get much easier than calling function A instead of function B
>2. Monkeypatching lets you do a process global hack
>3. The name tells you exactly why this is a bad idea
>4. It's easy to grep for later after you fix your certs
>5. The leading underscore acts as a strong "keep away" signal
>6. The leading underscore makes it clear this function may not always be
>available (e.g. Jython, older versions of Python)

+1.  This would also make it easy to mock in a context manager if you just
wanted to ignore certs for a small section of code.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 9 Sep 2014 04:00, "Barry Warsaw"  wrote:
> >
> >This would need to be updated first, once it *did* take such an argument,
> >this would be accomplished by:
> >
> >context = ssl.create_default_context()
> >context.verify_mode = CERT_OPTIONACERT_NONE
> >context.verify_hostname = False
> >urllib.request.urlopen("
https://something-i-apparently-dont-care-much-about";,
> >context=context)
>
> There's probably an ugly hack possibility that uses unittest.mock.patch.
;)

We could actually make it an "official" hack:

import urllib.request
urllib.request.urlopen = urllib.request._unverified_urlopen

Or else the user can just change the code to call the unverified one
directly.

All we'd have to do is keep the existing version that doesn't validate
certs properly around under the name "_unverified_urlopen".

I like this for a few reasons:

1. It doesn't get much easier than calling function A instead of function B
2. Monkeypatching lets you do a process global hack
3. The name tells you exactly why this is a bad idea
4. It's easy to grep for later after you fix your certs
5. The leading underscore acts as a strong "keep away" signal
6. The leading underscore makes it clear this function may not always be
available (e.g. Jython, older versions of Python)

Cheers,
Nick.

>
> -Barry
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Gregory P. Smith]

On Wed, Sep 3, 2014 at 3:48 PM, Stephen J. Turnbull 
wrote:

> Guido van Rossum writes:
>
>  > lot: five years ago (when I worked at Google!) it was common to find
>  > internal services that required SSL but had a misconfigured certificate,
>  > and the only way to access those services was to override the browser
>  > complaints. Today (working at Dropbox, a much smaller company!) I don't
>  > even remember the last time I had to deal with such a browser complaint
> --
>
> I would tend to discount your recent experience, then.  Smaller (and
> possibly even more important in this fast-developing area, younger)
> organizations are a lot more nimble about things like this.
>

As a defensive data point: I don't remember a single instance of this
happening for Google internal services, at least since I arrived in 2007.
I'm not doubting that Guido remembers some thing(s) but in general people
here at Google would not stand for that, then or now. I would not call it
common, especially five years ago.

Common things I _have_ encountered over the years everywhere I've been both
internal and external: services that listen on the https port 443 but don't
have a valid cert as they are intended only for http port 80 access. Those
are becoming somewhat less common, the only thing I regularly see that on
anymore is random home router web config UIs as issuing a signed server
certificate for security hole ridden commodity embedded box is... a
challenge.

(I'm not commenting on the PEP plans as it seems the right things are
happening for now)

-gps @ Google
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Sad status of Python 3.x buildbots

[Gregory P. Smith]

On Wed, Sep 3, 2014 at 2:52 PM, Victor Stinner 
wrote:

>
> > ARMv7 3.x: "Read-only file system", Mercurial fails...
>
> I sent an email to Gregory P. Smith (owner).
>

For mine, its a case of me not having any monitoring for it. Sending me an
email worked.

In this case the SSD (a very fast usb3 memory stick) died after 9 months of
use. Not a surprise. I brought it back up on the internal flash for the
time being. I am in the process of setting up a new system (nvidia jetson
tk1) to replace this one that will be 1.5-3x faster depending on how you
measure it.

Monitoring idea for buildbots: If builders for all non-custom branches on a
buildbot are failing for more than some period of time or number of builds
each in a row, that is likely a sign of broken infrastructure or a common
platform specific code change that needs to be made.

anyone want to implement that?

-gps
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backwards compatibility after certificate autovalidation

[Cory Benfield]

On 8 September 2014 18:23, Jim J. Jewett  wrote:
> Summary:  There needs to be a simple way to opt out at install time.
> It would be far better to offer more fine-grained control, but leaving
> that better solution to downstream is acceptable.

Does this argument apply to a hypothetical 2.7 backport of this
change, or does it apply to making the change in 3.5? (Or of course
both.)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Jim Baker]

+1 for the suggested change to 2.7.

Something I have put off in the work on SSL support in Jython 2.7 is what
to do about the possibility of adding a large security hole to support
standard Python behavior here with CERT_NONE. By default, we use the
standard trust database and corresponding manager from Java, as augmented
by any provided ca_certs. In practice, I don't think people are really
noticing that it's currently locked down in the latest beta, which added
this SSL support.

Although it's very easy to open such a hole on Jython/Java, it's much nicer
if we require the user has to do some work to do so, and on a per
connection basis.

- Jim



On Mon, Sep 8, 2014 at 10:58 AM, Guido van Rossum  wrote:

> On Mon, Sep 8, 2014 at 5:08 AM, Nick Coghlan  wrote:
>
>>
>> It would also be good to get Guido's official verdict on PEP 476 (the
>> switch to validating HTTPS by default) in time for 3.4.2. Based on the
>> previous discussion, Alex updated the PEP to suggest "just fix it" for
>> all of 3.5, 3.4.x and 2.7.x (including the httplib SSLContext support
>> backport in the latter case).
>>
>
> My opinion hasn't changed since the last time I opened my mouth
> prematurely. :-) I would very much like these to go in, but for 2.7 I am
> now worried about what we should tell people who have a script that uses an
> https URL to access a service that can only be accessed via SSL/TLS to a
> self-signed or otherwise mis-configured cert. I am not insisting on an
> environment variable to disable this (too easy) but I do think it must be
> possible to make a simple change to the code, on the order of tracking down
> the urlopen() call and adding a new keyword parameter. Such a band-aid
> needn't be backward compatible (we can introduce a new keyword parameter
> for this purpose) and it needn't be totally straightforward (we can assume
> some modicum of understanding of finding and editing .py files) but it
> should definitely not require a refactor of the script (e.g. swapping out
> urlopen and replacing it with httplib or requests would be too much of a
> burden). And we should have prominent documentation (perhaps in FAQ form?)
> with an example of how to do it.
>
>
>> I think that would be feasible with an rc on the 20th, but challenging
>> if the rc is this coming weekend.
>>
>> Note, as I stated in the previous thread, I'm now +1 on that PEP,
>> because I don't see any way to write an automated scan for a large
>> code base that ensures we're not relying on the default handling at
>> all. If the default behaviour is to validate HTTPS, the lack of such a
>> scanner isn't a problem - any failures to cope with self-signed or
>> invalid certs will be noisy, and we can either fix the certs, patch
>> the code to cope with them appropriately, or (for the self-signed cert
>> case) configure OpenSSL via environment variables. If dealing with
>> invalid certs is truly necessary, and the code can't be updated
>> either, then I'm OK with the answer being "keep using an older version
>> of Python, as that's going to be the least of your security concerns".
>>
>
> Yeah, I am not interested in helping out the case where the user is
> incapable (for whatever reason) of tracking down and changing a couple of
> lines of code. Such users are dependent on someone else with wizard powers
> anyway (who gave them the script?).
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Barry Warsaw]

On Sep 08, 2014, at 10:44 AM, Alex Gaynor wrote:

>*Shifts uncomfortably* it looks like presently there's not a good way to
>change anything about the SSL configuration for urllib.request.urlopen. It
>does not take a `context` argument, as the http.client API does:
>https://docs.python.org/3/library/urllib.request.html#module-urllib.request
>and instead takes the cafile, capath, cadefault args.
>
>This would need to be updated first, once it *did* take such an argument,
>this would be accomplished by:
>
>context = ssl.create_default_context()
>context.verify_mode = CERT_OPTIONACERT_NONE
>context.verify_hostname = False
>urllib.request.urlopen("https://something-i-apparently-dont-care-much-about";,
>context=context)

There's probably an ugly hack possibility that uses unittest.mock.patch. ;)

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

Well, get cracking then! :-)

On Mon, Sep 8, 2014 at 10:44 AM, Alex Gaynor  wrote:

> *Shifts uncomfortably* it looks like presently there's not a good way to
> change anything about the SSL configuration for urllib.request.urlopen. It
> does not take a `context` argument, as the http.client API does:
> https://docs.python.org/3/library/urllib.request.html#module-urllib.request
> and instead takes the cafile, capath, cadefault args.
>
> This would need to be updated first, once it *did* take such an argument,
> this would be accomplished by:
>
> context = ssl.create_default_context()
> context.verify_mode = CERT_OPTIONACERT_NONE
> context.verify_hostname = False
> urllib.request.urlopen("
> https://something-i-apparently-dont-care-much-about";, context=context)
>
> Alex
>
>
> On Mon, Sep 8, 2014 at 10:35 AM, Guido van Rossum 
> wrote:
>
>> I will pronounce for 3.4 once you point me to the documentation that
>> explains how to disable cert validation for an example program that
>> currently pulls down an https URL using urlopen. Without adding package
>> dependencies.
>>
>> On Mon, Sep 8, 2014 at 10:25 AM, Alex Gaynor 
>> wrote:
>>
>>> Guido van Rossum  python.org> writes:
>>>
>>> >
>>> >
>>>
>>> Would you be willing to officially pronounce on PEP-476 in the context
>>> of 3.4.x,
>>> so we can get it into the release, and then we can defer on officially
>>> approving
>>> it for 2.7.X until we figure out all the moving pieces?
>>>
>>> Cheers,
>>> Alex
>>>
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Alex Gaynor]

*Shifts uncomfortably* it looks like presently there's not a good way to
change anything about the SSL configuration for urllib.request.urlopen. It
does not take a `context` argument, as the http.client API does:
https://docs.python.org/3/library/urllib.request.html#module-urllib.request
and instead takes the cafile, capath, cadefault args.

This would need to be updated first, once it *did* take such an argument,
this would be accomplished by:

context = ssl.create_default_context()
context.verify_mode = CERT_OPTIONACERT_NONE
context.verify_hostname = False
urllib.request.urlopen("https://something-i-apparently-dont-care-much-about";,
context=context)

Alex


On Mon, Sep 8, 2014 at 10:35 AM, Guido van Rossum  wrote:

> I will pronounce for 3.4 once you point me to the documentation that
> explains how to disable cert validation for an example program that
> currently pulls down an https URL using urlopen. Without adding package
> dependencies.
>
> On Mon, Sep 8, 2014 at 10:25 AM, Alex Gaynor 
> wrote:
>
>> Guido van Rossum  python.org> writes:
>>
>> >
>> >
>>
>> Would you be willing to officially pronounce on PEP-476 in the context of
>> 3.4.x,
>> so we can get it into the release, and then we can defer on officially
>> approving
>> it for 2.7.X until we figure out all the moving pieces?
>>
>> Cheers,
>> Alex
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

I will pronounce for 3.4 once you point me to the documentation that
explains how to disable cert validation for an example program that
currently pulls down an https URL using urlopen. Without adding package
dependencies.

On Mon, Sep 8, 2014 at 10:25 AM, Alex Gaynor  wrote:

> Guido van Rossum  python.org> writes:
>
> >
> >
>
> Would you be willing to officially pronounce on PEP-476 in the context of
> 3.4.x,
> so we can get it into the release, and then we can defer on officially
> approving
> it for 2.7.X until we figure out all the moving pieces?
>
> Cheers,
> Alex
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Alex Gaynor]

Guido van Rossum  python.org> writes:

> 
> 

Would you be willing to officially pronounce on PEP-476 in the context of 3.4.x,
so we can get it into the release, and then we can defer on officially approving
it for 2.7.X until we figure out all the moving pieces?

Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Backwards compatibility after certificate autovalidation

[Jim J. Jewett]

Summary:  There needs to be a simple way to opt out at install time.
It would be far better to offer more fine-grained control, but leaving
that better solution to downstream is acceptable.


On 3 September 2014 01:19, Antoine Pitrou  wrote:

> RFC 2818 (HTTP over TLS) has the following language in section 3.1:

>> If the hostname is available, the client MUST check it against the
>> server's identity as presented in the server's Certificate message,
>> in order to prevent man-in-the-middle attacks.

>> If the client has external information as to the expected identity of
>> the server, the hostname check MAY be omitted.

This second case is pretty common, in my experience.  I still see it on
the public internet, but mismatches are almost the expected case on the
intranet, and many installation guides begin by saying to ignore the
security warnings.

I think it best not to name my employer in this context, but I work for
an IT firm large enough that you've heard of it.  As bad as our internal
situation is, it is still better than a typical client's infrastructure,
except that clients often have fewer surfaces to expose in the first place.

Internal websites and applications tend to have information that "needs
protection" only because saying otherwise requires a long bureaucratic
process with little payoff.  (Also true at many clients.)  Nick has
already posted a subset of the reasons why a site may be "signed" with
a certificate that is self-signed, expired, and/or limited to the wrong
hostname/subdomain.  

In the long run, I agree that it is better to default to secure.  But
in the short and medium term, there has to be a workaround, and I would
prefer that the simplest workaround not be "retire the application, and
don't use python again."

I believe that the minimal acceptable workaround is that the Release
Notes have an URL pointing to an install-time recipe telling the admin
how to change the default back globally.

Examples of good enough:
"Add this file to site-packages" 
"Add this environment variable with this setting" 
"Add this command line switch to your launch script" 

Examples of not good enough:
"Edit your application to change ..."
"Edit your system store ..." (affecting more than python)

Obviously, it would be great to offer finer control, so that the
stricter default can be used when it is OK.  (Per installation?  Per
application?  Per run?  Per domain?  Per protocol?  Per certificate?
Per rejection reason?  Treat anything in subdomain1.example.com as
valid for hostname.example.com?  Self-signing is OK for this IP range?)
I would be pleasantly surprised if this level of API can even be
standardized in time, and I agree that it is reasonable to leave it
to 3rd party modules and downstream distributions.

But I think Python itself should provide at least the single big
hammer -- and that hammer should be something that can be used once
at installation time (perhaps by changing the launch script), instead
of requiring user interaction.


-jJ

--

If there are still threading problems with my replies, please
email me with details, so that I can try to resolve them.  -jJ

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Guido van Rossum]

On Mon, Sep 8, 2014 at 5:08 AM, Nick Coghlan  wrote:

>
> It would also be good to get Guido's official verdict on PEP 476 (the
> switch to validating HTTPS by default) in time for 3.4.2. Based on the
> previous discussion, Alex updated the PEP to suggest "just fix it" for
> all of 3.5, 3.4.x and 2.7.x (including the httplib SSLContext support
> backport in the latter case).
>

My opinion hasn't changed since the last time I opened my mouth
prematurely. :-) I would very much like these to go in, but for 2.7 I am
now worried about what we should tell people who have a script that uses an
https URL to access a service that can only be accessed via SSL/TLS to a
self-signed or otherwise mis-configured cert. I am not insisting on an
environment variable to disable this (too easy) but I do think it must be
possible to make a simple change to the code, on the order of tracking down
the urlopen() call and adding a new keyword parameter. Such a band-aid
needn't be backward compatible (we can introduce a new keyword parameter
for this purpose) and it needn't be totally straightforward (we can assume
some modicum of understanding of finding and editing .py files) but it
should definitely not require a refactor of the script (e.g. swapping out
urlopen and replacing it with httplib or requests would be too much of a
burden). And we should have prominent documentation (perhaps in FAQ form?)
with an example of how to do it.


> I think that would be feasible with an rc on the 20th, but challenging
> if the rc is this coming weekend.
>
> Note, as I stated in the previous thread, I'm now +1 on that PEP,
> because I don't see any way to write an automated scan for a large
> code base that ensures we're not relying on the default handling at
> all. If the default behaviour is to validate HTTPS, the lack of such a
> scanner isn't a problem - any failures to cope with self-signed or
> invalid certs will be noisy, and we can either fix the certs, patch
> the code to cope with them appropriately, or (for the self-signed cert
> case) configure OpenSSL via environment variables. If dealing with
> invalid certs is truly necessary, and the code can't be updated
> either, then I'm OK with the answer being "keep using an older version
> of Python, as that's going to be the least of your security concerns".
>

Yeah, I am not interested in helping out the case where the user is
incapable (for whatever reason) of tracking down and changing a couple of
lines of code. Such users are dependent on someone else with wizard powers
anyway (who gave them the script?).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython and parallel make

[Jonas Wagner]

Hi,

Open an issue on bugs.python.org and attach the patch there (it should also
> ask you so sign the contributor agreement, but if not then please also sign
> that).
>

Submitted as  http://bugs.python.org/issue22359 , and signed the agreement.

Cheers,
Jonas
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Nick Coghlan]

On 8 September 2014 14:28, Ned Deily  wrote:
> In article <540c521c.7070...@hastings.org>,
>  Larry Hastings  wrote:
>> Matthias asked me when I was going to release 3.4.2.  I propose the
>> following schedule:
>>
>> Tag 3.4.2rc1 Friday Sep 12 2014
>> Release 3.4.2rc1 Saturday Sep 13 2014
>> Tag 3.4.2 final Saturday Sep 27 2014
>> Release 3.4.2 final Sunday Sep 28 2014
>>
>> Normally I want to tag on Saturdays and release on Sundays, however I
>> propose moving rc1 a day earlier because I'll be unavailable that Sunday.
>>
>> Any objections?  I will interpret silence as tacit approval.
>
> As I've already discussed with Larry, I think adding a week to the
> scheduled dates would be preferable.  The original dates give pretty
> short notice and there are a number of open issues that would be good to
> resolve now in 3.4.2.

It would also be good to get Guido's official verdict on PEP 476 (the
switch to validating HTTPS by default) in time for 3.4.2. Based on the
previous discussion, Alex updated the PEP to suggest "just fix it" for
all of 3.5, 3.4.x and 2.7.x (including the httplib SSLContext support
backport in the latter case).

I think that would be feasible with an rc on the 20th, but challenging
if the rc is this coming weekend.

Note, as I stated in the previous thread, I'm now +1 on that PEP,
because I don't see any way to write an automated scan for a large
code base that ensures we're not relying on the default handling at
all. If the default behaviour is to validate HTTPS, the lack of such a
scanner isn't a problem - any failures to cope with self-signed or
invalid certs will be noisy, and we can either fix the certs, patch
the code to cope with them appropriately, or (for the self-signed cert
case) configure OpenSSL via environment variables. If dealing with
invalid certs is truly necessary, and the code can't be updated
either, then I'm OK with the answer being "keep using an older version
of Python, as that's going to be the least of your security concerns".

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 01.09.2014 10:09, Nick Coghlan wrote:
> On 1 September 2014 17:13, Christian Heimes  wrote:
>> On 01.09.2014 08:44, Nick Coghlan wrote:
>>> Yes, it would have exactly the same security failure modes as
>>> sitecustomize, except it would only fire if the application
>>> imported the ssl module.
>>>
>>> The "-S" and "-I" switches would need to disable the implied
>>> "sslcustomize", just as they disable "import site".
>>
>> A malicious package can already play havoc with your installation with
>> a custom ssl module. If somebody is able to sneak in a ssl.py then you
>> are screwed anyway. sslcustomize is not going to make the situation worse.
> 
> That's not quite true - we're fairly careful about putting the
> standard library before userspace directories, so aside from the
> "current directory" problem, shadowing "ssl" itself can be tricky to
> arrange.

It's really easy to modify sys.modules to override any module
that has already been loaded or add new ones bypassing sys.path
entirely, so the sys.path layout doesn't provide any protection
against such hacks.

If you gain access to one of the dirs on sys.path, you can play
such tricks in sitecustomize.py. Any 3rd party package can do
the same.

We'd have to add digital API signatures to the ssl module to
prevent such stunts :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 08 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...11 days to go
2014-09-27: PyDDF Sprint 2014 ...  19 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com