Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

(Oops, I had a version of this reply sitting in my Drafts folder for a
week, and only noticed after pushing the most recent PEP update that
it had never been sent)

On 1 December 2015 at 06:32, Barry Warsaw  wrote:
> On Nov 27, 2015, at 04:04 PM, Nick Coghlan wrote:
>
>>New draft pushed: https://hg.python.org/peps/rev/f602a47ea795
>>
>>This is a significant rewrite that switches the PEP to a Standards Track PEP
>>proposing two new features for 2.7.12+: an "ssl._verify_https_certificates()"
>>configuration function, and a "PYTHONHTTPSVERIFY" environment variable
>>(although writing them together like that makes me wonder if the latter
>>should now be "PYTHONVERIFYHTTPS" instead).
>
> Thanks for this, and +1 on Stephen's suggested name change (which you've
> already pushed).
>
> Two comments: the PEP still describes the configuration file implementation.
> Is this slated for 2.7.12 also?  If not, should it just be dropped from the
> PEP?

That recommendation is still needed to backport PEP 476 itself to
versions prior to 2.7.9 - otherwise there's no way to flip the default
from "don't verify" to "verify" for the entire Python installation. It
may be that the system Python in RHEL/CentOS 7 and derivatives ends up
being the only case where that happens (since that's the only instance
I'm aware of with a 2024 support deadline, rather than 2019 or
earlier), but if anyone else does do it, it would be preferable if
they adopted the same approach to configuring it.

However, I just pushed an update that reverses the presentation order
of the two main backporting sections:
https://hg.python.org/peps/rev/17e0e36cbc19

The original order came from the point where this was just an
Informational PEP suggesting some backporting techniques, but now that
it suggests some actual upstream changes for 2.7.12+, it makes more
sense to cover those first, and then be more explicit that it's
acceptable to skip implementing the rest of the PEP entirely.

Accordingly, that change also includes the following new paragraph in
the section on the PEP 476 backport:


This PEP doesn't take a position on whether or not this particular change is a
good idea - rather, it suggests that *if* a redistributor chooses to go down
the path of making the default behaviour configurable in a version of Python
older than Python 2.7.9, then maintaining a consistent approach across
redistributors would be beneficial for users.


> I'd mildly prefer no default value for `enable` in
> _https_verify_certificates().  I'd have preferred a keyword-only argument, but
> of course this is Python 2.  Instead, I'd like to force passing True or False
> (and document using `enable=True` or `enable=False`) and not rely on a default
> argument.  But I'm only +0 on that detail.

My rationale for giving it a default is to make it marginally more
straightforward to turn verification on than it is to turn it off.
That's going to be most relevant in the pre-2.7.9 backport case, since
in 2.7.9+ the HTTPS certificate verification will already be on by
default.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 27, 2015, at 04:04 PM, Nick Coghlan wrote:

>New draft pushed: https://hg.python.org/peps/rev/f602a47ea795
>
>This is a significant rewrite that switches the PEP to a Standards Track PEP
>proposing two new features for 2.7.12+: an "ssl._verify_https_certificates()"
>configuration function, and a "PYTHONHTTPSVERIFY" environment variable
>(although writing them together like that makes me wonder if the latter
>should now be "PYTHONVERIFYHTTPS" instead).

Thanks for this, and +1 on Stephen's suggested name change (which you've
already pushed).

Two comments: the PEP still describes the configuration file implementation.
Is this slated for 2.7.12 also?  If not, should it just be dropped from the
PEP?

I'd mildly prefer no default value for `enable` in
_https_verify_certificates().  I'd have preferred a keyword-only argument, but
of course this is Python 2.  Instead, I'd like to force passing True or False
(and document using `enable=True` or `enable=False`) and not rely on a default
argument.  But I'm only +0 on that detail.

Cheers,
-Barry


pgpkeUFzn5kLf.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Stephen J. Turnbull]

Nick Coghlan writes:

 > This is a significant rewrite that switches the PEP to a Standards
 > Track PEP proposing two new features for 2.7.12+: an
 > "ssl._verify_https_certificates()" configuration function, and a
 > "PYTHONHTTPSVERIFY" environment variable (although writing them
 > together like that makes me wonder if the latter should now be
 > "PYTHONVERIFYHTTPS" instead).

+1 on the same order of "words" in the function name and the
evnironment variable.  I tend to prefer
"ssl._https_verify_certificates" since these aren't "HTTPS
certificates".

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 27 November 2015 at 18:47, Cory Benfield  wrote:
> Perhaps I missed this, Nick, but what happens if multiple third party 
> libraries apply updates to call this function in incompatible ways? For 
> example, if you depend on libfoo which calls 
> ssl._verify_https_certificates(False) and libbar which calls 
> ssl._verify_https_certificates(True)? Is it…last import wins?

Last import wins, but libaries shouldn't be mutating process global
state as a side effect of import - like the sys module, the ssl module
configuration should only be modified (directly or indirectly) from
__main__.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Stephen J. Turnbull]

Nick Coghlan writes:

 > This is a significant rewrite that switches the PEP to a Standards
 > Track PEP proposing two new features for 2.7.12+: an
 > "ssl._verify_https_certificates()" configuration function, and a
 > "PYTHONHTTPSVERIFY" environment variable (although writing them
 > together like that makes me wonder if the latter should now be
 > "PYTHONVERIFYHTTPS" instead).

+1 on the same order of "words" in the function name and the
evnironment variable.  I tend to prefer
"ssl._https_verify_certificates" since these aren't "HTTPS
certificates".

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Cory Benfield]

> On 27 Nov 2015, at 06:04, Nick Coghlan  wrote:
> 
> Feature: Configuration API
> ==
> 
> This change is proposed for inclusion in CPython 2.7.12 and later CPython 
> 2.7.x
> releases. It consists of a new ``ssl._verify_https_certificates()`` to specify
> the default handling of HTTPS certificates in standard library client 
> libraries.
> 
> It is not proposed to forward port this change to Python 3, so Python 3
> applications that need to support skipping certificate verification will still
> need to define their own suitable security context.
> 
> Feature detection
> -
> 
> The marker attribute on the ``ssl`` module related to this feature is the
> ``ssl._verify_https_certificates`` function itself.
> 
> Specification
> -
> 
> The ``ssl._verify_https_certificates`` function will work as follows::
> 
>def _verify_https_certificates(enable=True):
>"""Verify server HTTPS certificates by default?"""
>global _create_default_https_context
>if enable:
>_create_default_https_context = create_default_context
>else:
>_create_default_https_context = _create_unverified_context
> 
> If called without arguments, or with ``enable`` set to a true value, then
> standard library client modules will subsequently verify HTTPS
> certificates by default, otherwise they will skip verification.

Perhaps I missed this, Nick, but what happens if multiple third party libraries 
apply updates to call this function in incompatible ways? For example, if you 
depend on libfoo which calls ssl._verify_https_certificates(False) and libbar 
which calls ssl._verify_https_certificates(True)? Is it…last import wins?


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 26, 2015, at 02:13 PM, Nick Coghlan wrote:

>PEP 476 rejected providing a public indefinitely maintained API for this, so
>PEP 493 is specifically about helping commercial redistributors offer a
>smoother transition plan to customers without affecting the public Python
>level API, and without encouraging a plethora of mutually incompatible
>transition schemes.

Of course, the API would only have to be support for the life of 2.7; it would
never go in 3.x so the burden is minimal.

>PEP 493 isn't about attempting to rehash the PEP 476 discussions in
>search of a different conclusion, so this would need to be a different
>PEP, preferably one that targets Python 3.6 first and covers more than
>just HTTPS.

That seems like overkill.  PEP 493 is specifically about Python 2.7 and
providing ways for downstreams to facilitate more choice for end-users and
end-administrators.  Although I think it could safely sneak in after rc1, that
would be for the RM to decide.  Even if it were deferred to 2.7.12, it would
still provide a better, more consistent experience if implemented upstream.

Cheers,
-Barry


pgp1_rS2ossYp.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 26, 2015, at 03:06 PM, Nick Coghlan wrote:

>I'm not a big fan of it either, but it's the way sustainable
>commercial open source distribution works in practice:

While it's inevitable that redistributors have to deviate from upstream, in
Debian and Ubuntu, we really try to keep that at a minimum.  Debian, of course
is an all volunteer organization so our "customers" are just normal users who
I think want an experience as close to installing from source as possible,
consistent with the principles and policies of the Debian project.  Debian at
least tries to document the handful of deviations from upstream.

Likewise in Ubuntu, we try to keep deviations from Debian at a minimum, and
document them when we must deviate.  Ubuntu is a community driven distro so
while Canonical itself has customers, it's much more likely that feedback
about the Python stack comes from ordinary users.  Again, my personal goal is
to make Python on Ubuntu a pleasant and comfortable environment, as close to
installing from source as possible, consistent with the principles and
policies of the project.

All this to say that IMHO, solutions to problems should go as far upstream as
possible.  Guidelines certainly are useful (e.g. PEP 394) to maintain
consistency across downstream providers where the decisions are primarily in
the sphere of influence of the downstreams, but IMHO there's still more that
upstream can do to promote PEP 493.

>Different redistributors have different customer bases, which also
>differ from the audience for upstream python.org releases, so saying
>"don't patch Python" is denying the reality of the obligations
>commercial vendors have to their customers, while "when you patch
>Python, please ensure you abide by these guidelines" is something
>redistributors can realistically do.

ISTM that the same forces are in play regardless of whether the change is in
code or in an informational PEP.  Best to get consensus where possible, and
manifest those decisions in code, but if competing goals are the outcome of a
code change or informational PEP, downstream consumers will still make what
they judge to be the best decision in the interest of their users, balanced
against their own competing constraints.

>In this particular case, the migration problems were already raised in
>the PEP 476 discussions, and the decision was made to *not* provide a
>public API specifically for globally turning off HTTPS certificate
>verification, since that would either:
>
>1. Need to be added to Python 3 and maintained indefinitely; or
>2. Be implemented in Python 2.7 only, and thus create another barrier
>to 2->3 migration

A minor one, for sure.  It wouldn't significantly bother me if the API were
underscore-prefixed as a big honkin' clue about the guarantees being made.

>* Robert Kuska came up with a way to do a backwards compatible
>backport of PEP 476 to RHEL 7.2 that doesn't raise any new barriers to
>migrating from Python 2 to Python 3

I'm not sure I agree with that.  If you have to set an environment variable to
get your application to work in Python 2.7, it's *still* a migration issue
when you move to Python 3.  You're just moving the change out of the code and
into the environment, where it's arguably more difficult to re-discover down
the road.

Cheers,
-Barry


pgptqBXhMF998.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 27 November 2015 at 03:15, Barry Warsaw  wrote:
> On Nov 26, 2015, at 03:06 PM, Nick Coghlan wrote:
>
>>I'm not a big fan of it either, but it's the way sustainable
>>commercial open source distribution works in practice:
>
> While it's inevitable that redistributors have to deviate from upstream, in
> Debian and Ubuntu, we really try to keep that at a minimum.  Debian, of course
> is an all volunteer organization so our "customers" are just normal users who
> I think want an experience as close to installing from source as possible,
> consistent with the principles and policies of the Debian project.  Debian at
> least tries to document the handful of deviations from upstream.

Yeah, we need to be better about that - while
https://access.redhat.com/articles/2039753 covers the details for the
certificate verification changes, there's currently no easy way to get
a description of the changes backported to the RHEL/CentOS system
Python without trawling through old Errata announcements :(

> Likewise in Ubuntu, we try to keep deviations from Debian at a minimum, and
> document them when we must deviate.  Ubuntu is a community driven distro so
> while Canonical itself has customers, it's much more likely that feedback
> about the Python stack comes from ordinary users.  Again, my personal goal is
> to make Python on Ubuntu a pleasant and comfortable environment, as close to
> installing from source as possible, consistent with the principles and
> policies of the project.

I'd strongly agree with that description for Fedora and
softwarecollections.org, but for the RHEL/CentOS system Python I think
the situation is slightly different: there, the goal is to meet the
long term support commitments involved in being a base RHEL package.
As the nominal base version of the package (2.7.5 in the case of RHEL
7) doesn't change, there is naturally going to be increasing
divergence from the nominal version.

> All this to say that IMHO, solutions to problems should go as far upstream as
> possible.  Guidelines certainly are useful (e.g. PEP 394) to maintain
> consistency across downstream providers where the decisions are primarily in
> the sphere of influence of the downstreams, but IMHO there's still more that
> upstream can do to promote PEP 493.

I tried to go down the "upstream first" path with a properly supported
"off switch" in PEP 476, and didn't succeed (hence the monkeypatch
compromise). It sounds like several folks would like to see us revisit
that decision, though.

>>Different redistributors have different customer bases, which also
>>differ from the audience for upstream python.org releases, so saying
>>"don't patch Python" is denying the reality of the obligations
>>commercial vendors have to their customers, while "when you patch
>>Python, please ensure you abide by these guidelines" is something
>>redistributors can realistically do.
>
> ISTM that the same forces are in play regardless of whether the change is in
> code or in an informational PEP.  Best to get consensus where possible, and
> manifest those decisions in code, but if competing goals are the outcome of a
> code change or informational PEP, downstream consumers will still make what
> they judge to be the best decision in the interest of their users, balanced
> against their own competing constraints.
>
>>In this particular case, the migration problems were already raised in
>>the PEP 476 discussions, and the decision was made to *not* provide a
>>public API specifically for globally turning off HTTPS certificate
>>verification, since that would either:
>>
>>1. Need to be added to Python 3 and maintained indefinitely; or
>>2. Be implemented in Python 2.7 only, and thus create another barrier
>>to 2->3 migration
>
> A minor one, for sure.  It wouldn't significantly bother me if the API were
> underscore-prefixed as a big honkin' clue about the guarantees being made.

OK, you've persuaded me to recast the PEP from an Informational one to
a Standards track one. The section about backporting to versions prior
to 2.7.9 won't change much (except in tone), but the ability to opt
out on a process-wide basis will be pitched as a language level
feature.

>>* Robert Kuska came up with a way to do a backwards compatible
>>backport of PEP 476 to RHEL 7.2 that doesn't raise any new barriers to
>>migrating from Python 2 to Python 3
>
> I'm not sure I agree with that.  If you have to set an environment variable to
> get your application to work in Python 2.7, it's *still* a migration issue
> when you move to Python 3.  You're just moving the change out of the code and
> into the environment, where it's arguably more difficult to re-discover down
> the road.

By that I meant that setting the environment variable won't *break*
anything in Python 3, whereas if we add a new Python level API and
people call it without a hasattr() guard, then even if their
infrastructure has been upgraded to use verifiable certificates, the
client 

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Toshio Kuratomi]

On Nov 26, 2015 4:53 PM, "Nick Coghlan"  wrote:
>
> On 27 November 2015 at 03:15, Barry Warsaw  wrote:

>
> > Likewise in Ubuntu, we try to keep deviations from Debian at a minimum,
and
> > document them when we must deviate.  Ubuntu is a community driven
distro so
> > while Canonical itself has customers, it's much more likely that
feedback
> > about the Python stack comes from ordinary users.  Again, my personal
goal is
> > to make Python on Ubuntu a pleasant and comfortable environment, as
close to
> > installing from source as possible, consistent with the principles and
> > policies of the project.
>
> I'd strongly agree with that description for Fedora and
> softwarecollections.org, but for the RHEL/CentOS system Python I think
> the situation is slightly different: there, the goal is to meet the
> long term support commitments involved in being a base RHEL package.
> As the nominal base version of the package (2.7.5 in the case of RHEL
> 7) doesn't change, there is naturally going to be increasing
> divergence from the nominal version.

I think the goal in rhel/centos is similar, actually.  The maintenance
burden for non upstream changes has been acknowledged as a problem to be
avoided by rhel maintainers before.  The caveat for those distributions is
that they accumulate more *backports*.

However, backports are easier to maintain than non upstream changes.  The
test of the upstream community helps to find and fix bugs in the code; the
downstream maintainer just needs to stay aware of whether fixes are going
into the code they've backported.

> I tried to go down the "upstream first" path with a properly supported
> "off switch" in PEP 476, and didn't succeed (hence the monkeypatch
> compromise). It sounds like several folks would like to see us revisit
> that decision, though.
>
That's the rub.  If there's now enough support to push this upstream I
think everyone downstream will be happier.  If it turns out there's still
enough resistance to keep it from upstream then I suppose you cross that
bridge if it becomes necessary.

-Toshio
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 27 November 2015 at 10:52, Nick Coghlan  wrote:
> On 27 November 2015 at 03:15, Barry Warsaw  wrote:
>> On Nov 26, 2015, at 03:06 PM, Nick Coghlan wrote:
>>>In this particular case, the migration problems were already raised in
>>>the PEP 476 discussions, and the decision was made to *not* provide a
>>>public API specifically for globally turning off HTTPS certificate
>>>verification, since that would either:
>>>
>>>1. Need to be added to Python 3 and maintained indefinitely; or
>>>2. Be implemented in Python 2.7 only, and thus create another barrier
>>>to 2->3 migration
>>
>> A minor one, for sure.  It wouldn't significantly bother me if the API were
>> underscore-prefixed as a big honkin' clue about the guarantees being made.
>
> OK, you've persuaded me to recast the PEP from an Informational one to
> a Standards track one. The section about backporting to versions prior
> to 2.7.9 won't change much (except in tone), but the ability to opt
> out on a process-wide basis will be pitched as a language level
> feature.

New draft pushed: https://hg.python.org/peps/rev/f602a47ea795

This is a significant rewrite that switches the PEP to a Standards
Track PEP proposing two new features for 2.7.12+: an
"ssl._verify_https_certificates()" configuration function, and a
"PYTHONHTTPSVERIFY" environment variable (although writing them
together like that makes me wonder if the latter should now be
"PYTHONVERIFYHTTPS" instead).

There are then 3 backport recommendation sections:

* one for backporting the ability to enable HTTPS verification by
default for the entire Python installation
* one for backporting the new features described in this PEP
* one for combining both backports in a single implementation

I tried to trim some of the resulting duplication between the
sections, but I suspect the overall text is still longer than it needs
to be. I've also added links describing the origin of the
configuration file based backport in RHEL development, and where to
find the source code for the patches in Fedora.

Cheers,
Nick.

===
PEP: 493
Title: HTTPS verification migration tools for Python 2.7
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan ,
Robert Kuska ,
Marc-André Lemburg 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 10-May-2015
Python-Version: 2.7.12
Post-History: 06-Jul-2015, 11-Nov-2015, 24-Nov-2015


Abstract


PEP 476 updated Python's default handling of HTTPS certificates in client
modules to align with certificate handling in web browsers, by validating
that the certificates received belonged to the server the client was attempting
to contact. The Python 2.7 long term maintenance series was judged to be in
scope for this change, with the new behaviour introduced in the Python 2.7.9
maintenance release.

This has created a non-trivial barrier to adoption for affected Python 2.7
maintenance releases, so this PEP proposes additional Python 2.7 specific
features that allow system administrators and other users to more easily
decouple the decision to verify server certificates in HTTPS client modules
from the decision to update to newer Python 2.7 maintenance releases.


Rationale
=

PEP 476 changed Python's default behaviour to align with expectations
established by web browsers in regards to the semantics of HTTPS URLs:
starting with Python 2.7.9 and 3.4.3, HTTPS clients in the standard library
validate server certificates by default.

However, it is also the case that this change *does* cause problems for
infrastructure administrators operating private intranets that rely on
self-signed certificates, or otherwise encounter problems with the new default
certificate verification settings.

To manage these kinds of situations, web browsers provide users with "click
through" warnings that allow the user to add the server's certificate to the
browser's certificate store. Network client tools like ``curl`` and ``wget``
offer options to switch off certificate checking entirely (by way of
``curl --insecure`` and ``wget --no-check-certificate``, respectively).

At a different layer of the technology stack, Linux security modules like
`SELinux` and `AppArmor`, while enabled by default by distribution vendors,
offer relatively straightforward mechanisms for turning them off.

At the moment, no such convenient mechanisms exist to disable Python's
default certificate checking for a whole process.

PEP 476 did attempt to address this question, by covering how to revert to the
old settings process wide by monkeypatching the ``ssl`` module to restore the
old behaviour. Unfortunately, the ``sitecustomize.py`` based technique proposed
to allow system administrators to disable the feature by default in their
Standard Operating Environment definition has been determined to be
insufficient in at least some cases. The specific case 

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 27 November 2015 at 17:42, Stephen J. Turnbull  wrote:
> Nick Coghlan writes:
>
>  > This is a significant rewrite that switches the PEP to a Standards
>  > Track PEP proposing two new features for 2.7.12+: an
>  > "ssl._verify_https_certificates()" configuration function, and a
>  > "PYTHONHTTPSVERIFY" environment variable (although writing them
>  > together like that makes me wonder if the latter should now be
>  > "PYTHONVERIFYHTTPS" instead).
>
> +1 on the same order of "words" in the function name and the
> evnironment variable.  I tend to prefer
> "ssl._https_verify_certificates" since these aren't "HTTPS
> certificates".

Ah, yes, that would make sense, so I've just pushed an update that
switches the function name order to match the environment variable.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 26 November 2015 at 05:57, Barry Warsaw  wrote:
> There's a lot to process in this thread, but as I see it, the issue breaks
> down to these questions:
>
> * How should PEP 493 be implemented?
>
> * What should the default be?
>
> * How should PEP 493 be worded to express the right tone to redistributors?
>
> Let me take on the implementation details here.
>
> On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote:
>
>>I would still find having built-in support for the recommendations
>>in the Python stdlib a better approach
>
> As would I.
>
> Currently the PEP contains some sample code for reading a configuration file
> and an environment variable, and assigning the default behavior based on these
> values.
>
> The first improvement to the stdlib I would have made would have been to add
> a convenience function to ssl to enable and disable the verification.
> Something like this:
>
> def ssl.verify_certificates(enable)
>
> Then at the simplest level, an application that wants to enable higher
> security could do this early in its main entry point:
>
> import ssl
> if hasattr(ssl, 'verify_certificates'):
> ssl.verify_certificates(True)
>
> or
>
> ...
> ssl.verify_certificates(False)
>
> and that would work regardless of the platform default, and it would work
> properly on older versions of Python 2.7 that didn't have the backport.  This
> seems so straightforward, I'm wondering why PEP 493 doesn't consider it.

PEP 476 rejected providing a public indefinitely maintained API for
this, so PEP 493 is specifically about helping commercial
redistributors offer a smoother transition plan to customers without
affecting the public Python level API, and without encouraging a
plethora of mutually incompatible transition schemes.

> I would like to see the sample code in PEP 493 implemented in the stdlib.  It
> would then be a matter of cherry picking that should a distro decide to
> backport it.  It means one less non-trivial delta from upstream which has to
> be managed separately.  The only difference to the combined approach I'd make
> is to handle the case where the _cert_verification_config file doesn't exist.
>
> (If you included the convenience method above, then the sample code could be
> simplified down to a default for the `enable` parameter and a call to
> ssl.verify_certificates().)
>
> Is it too late to include both of those suggestions in PEP 493 and Python
> 2.7.11?

PEP 493 isn't about attempting to rehash the PEP 476 discussions in
search of a different conclusion, so this would need to be a different
PEP, preferably one that targets Python 3.6 first and covers more than
just HTTPS.  Such an API could then be considered as a backport
candidate for 2.7.12. (2.7.11rc1 has already been published, so there
isn't any scope for updates there). I actually think that's a nicer
approach, but also think the recommendations in PEP 493 are much
simpler to implement and test, while still meeting the bar of "good
enough to solve the problem at hand".

If anyone does decide to write that PEP, then the fact that SELinux
offers "setenforce 0" and AppArmor can be turned off with a kernel
boot option can be used as examples of platform developers letting end
users make their own security decisions, even if the platform provider
recommends leaving the more opinionated default behaviour enabled.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

There's a lot to process in this thread, but as I see it, the issue breaks
down to these questions:

* How should PEP 493 be implemented?

* What should the default be?

* How should PEP 493 be worded to express the right tone to redistributors?

Let me take on the implementation details here.

On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote:

>I would still find having built-in support for the recommendations
>in the Python stdlib a better approach

As would I.

Currently the PEP contains some sample code for reading a configuration file
and an environment variable, and assigning the default behavior based on these
values.

The first improvement to the stdlib I would have made would have been to add
a convenience function to ssl to enable and disable the verification.
Something like this:

def ssl.verify_certificates(enable)

Then at the simplest level, an application that wants to enable higher
security could do this early in its main entry point:

import ssl
if hasattr(ssl, 'verify_certificates'):
ssl.verify_certificates(True)

or

...
ssl.verify_certificates(False)

and that would work regardless of the platform default, and it would work
properly on older versions of Python 2.7 that didn't have the backport.  This
seems so straightforward, I'm wondering why PEP 493 doesn't consider it.

I would like to see the sample code in PEP 493 implemented in the stdlib.  It
would then be a matter of cherry picking that should a distro decide to
backport it.  It means one less non-trivial delta from upstream which has to
be managed separately.  The only difference to the combined approach I'd make
is to handle the case where the _cert_verification_config file doesn't exist.

(If you included the convenience method above, then the sample code could be
simplified down to a default for the `enable` parameter and a call to
ssl.verify_certificates().)

Is it too late to include both of those suggestions in PEP 493 and Python
2.7.11?

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 24, 2015, at 09:16 AM, Toshio Kuratomi wrote:

>It sounds like the implementation sections of the PEP are acceptable

Well, see my follow up to MAL's email.  From Rob's and RDM's responses, it
seems we're not alone. :)

I like all your other proposed changes, although I'll note the preference to
minimize the unique-to-downstream changes.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Laura Creighton]

In a message of Wed, 25 Nov 2015 15:39:54 -0500, "R. David Murray" writes:
>I think we should include the environment variable support in CPython
>and be done with it (nuke the PEP otherwise).  Which is what I've
>thought from the beginning :)
>
>--David

I like this idea too.

Laura
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 24, 2015, at 02:12 PM, Nick Coghlan wrote:

>On 24 November 2015 at 11:59, Barry Warsaw  wrote:

>> We'd have to also handle
>> migration paths to newer Ubuntu releases, which probably means removing the
>> config file on future upgrades.
>
>You'll need to figure out those migration paths anyway, as the certificate
>verification compatibility break will exist between 14.04 and 16.04
>regardless of what happens with PEP 493.

Sure, but that's a very different situation.  If someone upgraded from 14.04.3
to 14.04.4, they definitely wouldn't expect it to break working code.  If it
did, it would be a pretty serious regression.  Upgrading to LTS maintenance
releases should be a no-brainer.

Upgrading from one LTS to a newer one, you *know* things are going to change,
so you'll review the release notes and do some very careful analysis of your
mission critical code.

Cheers,
-Barry


pgpMgKEH4ygoN.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[R. David Murray]

On Thu, 26 Nov 2015 09:17:02 +1300, Robert Collins  
wrote:
> On 26 November 2015 at 08:57, Barry Warsaw  wrote:
> > There's a lot to process in this thread, but as I see it, the issue breaks
> > down to these questions:
> >
> > * How should PEP 493 be implemented?
> >
> > * What should the default be?
> >
> > * How should PEP 493 be worded to express the right tone to redistributors?
> >
> > Let me take on the implementation details here.
> >
> > On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote:
> >
> >>I would still find having built-in support for the recommendations
> >>in the Python stdlib a better approach
> >
> > As would I.
> 
> For what its worth: a PEP telling distributors to patch the standard
> library is really distasteful to me.
> 
> We've spent a long time trying to build close relations such that when
> something doesn't work distributors can share their needs with us and
> we can make Python out of the box be a good fit. This seems to fly in
> the exact opposite direction: we're explicitly making it so that
> Python builds on these vendor's platforms will not be the same as you
> get by checking out the Python source code.

I think we should include the environment variable support in CPython
and be done with it (nuke the PEP otherwise).  Which is what I've
thought from the beginning :)

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 25, 2015, at 03:39 PM, R. David Murray wrote:

>I think we should include the environment variable support in CPython
>and be done with it (nuke the PEP otherwise).  Which is what I've
>thought from the beginning :)

+1.  I'd like to see my proposed convenience function added too.  I'm -0 on
the config file though since while I think it might be useful, I'm not sure
we'd actually install that on Ubuntu.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Robert Collins]

On 26 November 2015 at 08:57, Barry Warsaw  wrote:
> There's a lot to process in this thread, but as I see it, the issue breaks
> down to these questions:
>
> * How should PEP 493 be implemented?
>
> * What should the default be?
>
> * How should PEP 493 be worded to express the right tone to redistributors?
>
> Let me take on the implementation details here.
>
> On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote:
>
>>I would still find having built-in support for the recommendations
>>in the Python stdlib a better approach
>
> As would I.

For what its worth: a PEP telling distributors to patch the standard
library is really distasteful to me.

We've spent a long time trying to build close relations such that when
something doesn't work distributors can share their needs with us and
we can make Python out of the box be a good fit. This seems to fly in
the exact opposite direction: we're explicitly making it so that
Python builds on these vendor's platforms will not be the same as you
get by checking out the Python source code.

Ugh.

-Rob



-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 26 November 2015 at 06:17, Robert Collins  wrote:
> On 26 November 2015 at 08:57, Barry Warsaw  wrote:
>> There's a lot to process in this thread, but as I see it, the issue breaks
>> down to these questions:
>>
>> * How should PEP 493 be implemented?
>>
>> * What should the default be?
>>
>> * How should PEP 493 be worded to express the right tone to redistributors?
>>
>> Let me take on the implementation details here.
>>
>> On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote:
>>
>>>I would still find having built-in support for the recommendations
>>>in the Python stdlib a better approach
>>
>> As would I.
>
> For what its worth: a PEP telling distributors to patch the standard
> library is really distasteful to me.

I'm not a big fan of it either, but it's the way sustainable
commercial open source distribution works in practice: customers want
selective upgrades, so customers get selective upgrades; customers
want greater platform integration, so customers get greater platform
integration; etc, etc. Large Python shops like Google will typically
maintain their own custom Python variants with their own patches
applied that are suitable and sufficient for their use cases, but may
not be suitable for pushing upstream (e.g. they may be Linux specific
patches, without support for other platforms).

That's why I've focused on providing guidance to redistributors to
keep their patches within reasonably consistent bounds for PEPs like
bundling pip, backporting the network security enhancements to Python
2.7, and changing the default behaviour for HTTPS verification.
Different redistributors have different customer bases, which also
differ from the audience for upstream python.org releases, so saying
"don't patch Python" is denying the reality of the obligations
commercial vendors have to their customers, while "when you patch
Python, please ensure you abide by these guidelines" is something
redistributors can realistically do.

> We've spent a long time trying to build close relations such that when
> something doesn't work distributors can share their needs with us and
> we can make Python out of the box be a good fit. This seems to fly in
> the exact opposite direction: we're explicitly making it so that
> Python builds on these vendor's platforms will not be the same as you
> get by checking out the Python source code.

This has always been true for the RHEL system Python - it gets
backported changes, just like the kernel does (albeit nowhere near as
many). That's one of the main things long term support customers are
paying for - developers continuing to care about a code base after the
volunteers working on the project upstream have (entirely reasonably)
shifted their attentions to newer versions. We try to keep the patches
to a minimum, and avoid having to carry large patches indefinitely by
only backporting changes that have already been accepted in later
versions, but "never apply patches to upstream projects as a
downstream redistributor" isn't a realistic goal.

In this particular case, the migration problems were already raised in
the PEP 476 discussions, and the decision was made to *not* provide a
public API specifically for globally turning off HTTPS certificate
verification, since that would either:

1. Need to be added to Python 3 and maintained indefinitely; or
2. Be implemented in Python 2.7 only, and thus create another barrier
to 2->3 migration

A HTTPS specific flag wasn't considered a suitable long term API
design for Python 3, while a multi-protocol configuration design that
also handled setting the defaults for other protocols like secure
SMTP, IMAP, POP3, SFTP, etc seemed generally desirable (but also
doesn't exist anywhere, and poses signficant challenges in designing
and implementing a good test matrix).

The only concrete things that have changed since PEP 476 was approved is that:

* we figured out the monkeypatching based migration recommendations in
that PEP didn't actually work for the redistributor case
* Robert Kuska came up with a way to do a backwards compatible
backport of PEP 476 to RHEL 7.2 that doesn't raise any new barriers to
migrating from Python 2 to Python 3

PEP 493 originally only described the latter idea - I added MAL's
design for the environment variable based security downgrade based on
the original PEP 493 discussions in July. Splitting the latter out to
its own PEP that proposes it as a forwards compatible configuration
setting akin to "curl --insecure" and "wget --no-check-certificate"
for Python 2.7.12, 3.4.4, 3.5.2 and 3.6 upstream would be fine by me,
since that would also significantly clarify the fact that the two
ideas are entirely orthogonal to each other (except for the need to
define a relative priority when you combine the two).

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Cory Benfield]

> On 24 Nov 2015, at 12:53, Christian Heimes  wrote:
> Right, with Antoine and Alex out of scope and you, Victor and me working
> for Red Hat, the air is getting thin. Benjamin is familiar with the ssl
> module. Or we can follow Alex's advice and ask somebody from the PyCA
> group (Donald, Paul, lvh) or requests (Cory) to get some outside
> perspective.

I’m not a CPython core developer, so I think that excludes me.

For what it’s worth, I think that this is a good PEP, and I’m strongly in 
favour of it being approved in some form.


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 25 Nov 2015 05:42, "Toshio Kuratomi"  wrote:

> >
>
> Yeah, I think you are correct in your understanding of what actual
> changes to the python distrribution are being proposed for
> redistributors in the PEP.  Reading through the PEP again, I'm not
> sure if I'm correct in thinking that this only applies to
> backporting... it may be that the environment section of the PEP
> applies to any python-2 while the config file section only applies to
> backporting.  Nick, could you clarify?

Yep, the environment variable section is the part MAL persuaded me was a
good idea during the original discussion of this PEP, based on the
interests of eGenix's customers. It's a cross-platform suggestion for
distributors wanting to provide a smoother upgrade path, applicable to both
backports and rebasing on 2.7.9+. (I originally didn't like it, but became
amenable once it was pointed out that anyone with the ability to set
process environment variables can already alter OpenSSL's settings to use a
different CA cert file or directory)

The config file recommendation is specific to Linux distro backports, with
the aim of saying "IF you backport PEP 476 to an older Python 2.7 point
release, THEN you should do it like this for cross-distro consistency". As
the current discussion shows, I'm *personally* an advocate for backporting,
but I don't think the PEP itself should reflect that - the suggestions to
adjust the tone are good ones, so I'll update the text accordingly.

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Toshio Kuratomi]

On Tue, Nov 24, 2015 at 10:08 AM, Paul Moore  wrote:

> I'm not actually sure that it's the place of this PEP to even comment
> on what the long term answer for such environments should be (or
> indeed, any answer, long term or not). I've argued the position that
> in some organisations it feels like security don't actually understand
> the issues of carefully balancing secure operation against flexible
> development practices,

I agree with this.

> but conversely it's certainly true that in many
> organisations, they *have* weighed the various arguments and made an
> informed decision on how to set up their internal network. It's
> entirely possible that self-signed certificates are entirely the right
> decision for their circumstances. Why would a Python PEP be qualified
> to comment on that decision?

I don't quite agree with this but it's probably moot in the face of
the previous and certain cornercases.  Self-signed certs work just
fine with the backports to python-2.7.9+ but you have to add the ca to
the clients.  An organization that has weighed the arguments and made
an informed decision to use self-signed certs should either do this
(to prevent MITM) or they should switch to using http instead of https
(because MITM isn't a feasible attack here).  The cornercases come
into play because you don't always control all of the devices and
services on your network.  The site could evaluate and decide that
MITM isn't a threat to their switch's configuration interface but that
interface might be served over https using a certificate signed by
their network vendor who doesn't give out their ca certificate (simply
stated: your security team knows what they're doing but your vendor's
does not).

> In my opinion, we should take all of the value judgements out of this
> paragraph, and just state the facts. How about:
>
> """
> In order to provide additional flexibility to allow infrastructure
> administrators to provide the appropriate solution for their
> environment, this PEP offers a way for administrators to upgrade to
> later versions of the Python 2.7 series without being forced to update
> their existing security certificate management infrastructure as a
> prerequisite.
> """

Two notes:

* python-2.7.9+ doesn't give you flexibility in this regard so
organizations do have to update their certificate management
infrastructure.  The cornercase described above becomes something that
has to be addressed at the code level.  Environments that are simply
misconfigured have to be fixed.  So in that regard, a value judgement
does seem appropriate here.  the judgement is "Listen guys, this PEP
advises redistributors on how they might provide a migration path for
you but it *does not bandaid the problem indefinitely*.  So long term,
you have to change your practices or you'll be out in the cold when
your redistributor upgrades to python-2.7.9+"

* Your proposed text actually removes the fix that I was adding --
this version of the paragraph implies that if your environment is
compatible with the redistributors' python-2.7.8 (or less) then it
will also be compatible with the redistributors' python-2.7.9+.  That
is not true.  Whether or not we take out any value judgement as to the
user's present environment this paragraph needs to be fixed to make it
clear that this only affects redistributor's packages which have
backported pep 476 to python-2.7.8 or older.  Once the redistributor
updates to a newer python sites which relied on this crutch will
break.

-Toshio
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Paul Moore]

On 24 November 2015 at 17:16, Toshio Kuratomi  wrote:
> The long term answer for such environments is to update their internal
>   certificate management to at least match the standards set by the public
>   internet, but in the meantime, it is desirable to offer these administrators
>   a way to continue receiving maintenance updates to the Python 2.7 series,
>   without having to gate that on upgrades to their certificate management
>   infrastructure.
>
> + The wording here seems to reflect a different scope than merely
> backporting by distros.  Perhaps we should change it to: "[...]set by
> the public internet.  Distributions may wish to help these sites
> transition by backporting the PEP 476 changes to earlier versions of
> python in a way that does not require the administrators to upgrade
> their certificate management infrastructure immediately.  This would
> allow sites to choose to use the distribution suppiied python in a
> backwards compatible fashion until their certificate management
> infrastructure was updated and then toggle their site to utilize the
> more secure features provided by PEP 476."

I'm not actually sure that it's the place of this PEP to even comment
on what the long term answer for such environments should be (or
indeed, any answer, long term or not). I've argued the position that
in some organisations it feels like security don't actually understand
the issues of carefully balancing secure operation against flexible
development practices, but conversely it's certainly true that in many
organisations, they *have* weighed the various arguments and made an
informed decision on how to set up their internal network. It's
entirely possible that self-signed certificates are entirely the right
decision for their circumstances. Why would a Python PEP be qualified
to comment on that decision?

In my opinion, we should take all of the value judgements out of this
paragraph, and just state the facts. How about:

"""
In order to provide additional flexibility to allow infrastructure
administrators to provide the appropriate solution for their
environment, this PEP offers a way for administrators to upgrade to
later versions of the Python 2.7 series without being forced to update
their existing security certificate management infrastructure as a
prerequisite.
"""

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Toshio Kuratomi]

On Mon, Nov 23, 2015 at 5:59 PM, Barry Warsaw  wrote:

> I'm concerned about accepting PEP 493 making a strong recommendation to
> downstreams.  Yes, in an ideal world we all want security by default, but I
> think the backward compatibility concerns of the PEP are understated,
> especially as they relate to a maintenance release of a stable long term
> support version of the OS.  I don't want PEP 493 to be a cudgel that people
> beat us up with instead of having an honest discussion of the difficult
> trade-offs involved.
>
It sounds like the implementation sections of the PEP are acceptable
but that the PEP's general tone seems to assume that distributors are
champing at the bit to backport and that the recommendations here make
it so that backporting is a no-brainer -- which does not seem to
reflect the real-world?

I think the tone could be changed to address that as it doesn't seem
like forcing distros to backport is a real goal of the PEP.  The main
purposes of the PEP seem to be:

* Enumerate several ways that distributors can backport these 2.7.9
features to older releases
* Allow programmers to detect the presence of the features from their code
* Give end-users the ability to choose between backwards compatibility
and enhanced security

Here's some ideas for changing the tone:

  Abstract
  

  PEP 476 updated Python's default handling of HTTPS certificates to be
  appropriate for communication over the public internet. The Python 2.7 long
  term maintenance series was judged to be in scope for this change, with the
  new behaviour introduced in the Python 2.7.9 maintenance release.

+ Change to "PEP 476 updated Python's default handling of HTTPS
certificates to validate that the certs belonged to the server".  This
way we're saying what the change is rather than making a value
judgement of whether people who don't choose to backport are
"appropriate" or not.  Appropriate-ness is probably best left as an
argument in the text of PEP 476.

  This PEP provides recommendations to downstream redistributors wishing to
  provide a smoother migration experience when helping their users to manage
  this change in Python's default behaviour.

+ Change to "downstream redistributors wishing to backport the
enhancements in a way that allows users to choose between backwards
compatible behaviour or more secure certificate handling."  As barry
noted, this PEP doesn't change the amount of work needed to migrate.
It does, however, give users some choice in when they are going to
perform that work.  Additionally, this isn't simply about distributors
who want to make the transition smoother... (there's no downstreams
that want to make it "more painful" are there? ;-)  It's really about
making backporting of the enhancements less painful for users.

  Rationale
  =

  PEP 476 changed Python's default behaviour to better match the needs and
  expectations of developers operating over the public internet, a category
  which appears to include most new Python developers. It is the position of
  the authors of this PEP that this was a correct decision.

  However, it is also the case that this change *does* cause problems for
  infrastructure administrators operating private intranets that rely on
  self-signed certificates, or otherwise encounter problems with the new default
  certificate verification settings.

+ per barry's mesage, it would be good to either devote a paragraph to
the backwards compatibility implications here or link to
https://www.python.org/dev/peps/pep-0476/#backwards-compatibility

  The long term answer for such environments is to update their internal
  certificate management to at least match the standards set by the public
  internet, but in the meantime, it is desirable to offer these administrators
  a way to continue receiving maintenance updates to the Python 2.7 series,
  without having to gate that on upgrades to their certificate management
  infrastructure.

+ The wording here seems to reflect a different scope than merely
backporting by distros.  Perhaps we should change it to: "[...]set by
the public internet.  Distributions may wish to help these sites
transition by backporting the PEP 476 changes to earlier versions of
python in a way that does not require the administrators to upgrade
their certificate management infrastructure immediately.  This would
allow sites to choose to use the distribution suppiied python in a
backwards compatible fashion until their certificate management
infrastructure was updated and then toggle their site to utilize the
more secure features provided by PEP 476."

[...]

  These designs are being proposed as a recommendation for
redistributors, rather
  than as new upstream features, as they are needed purely to support legacy
  environments migrating from older versions of Python 2.7. Neither approach
  is being proposed as an upstream Python 2.7 feature, nor as a feature in any
  version of Python 3 (whether 

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Paul Moore]

On 24 November 2015 at 18:37, Toshio Kuratomi  wrote:
> I don't quite agree with this but it's probably moot in the face of
> the previous and certain cornercases.  Self-signed certs work just
> fine with the backports to python-2.7.9+ but you have to add the ca to
> the clients.  An organization that has weighed the arguments and made
> an informed decision to use self-signed certs should either do this
> (to prevent MITM) or they should switch to using http instead of https
> (because MITM isn't a feasible attack here).

There's an assumption (which is untrue for the big organisations I'm
familiar with) that security are involved with the full stack -
typically in my experience "corporate security" set rules that are
filtered down through many layers, and development teams have no
influence on those decisions - conversely, the people setting security
policies aren't aware of the details of development day to day needs.

It's easy to say that users just click on the "allow" button in their
browser if they want to use self-signed certificates, but not have any
viable solution for automation code that doesn't use a browser - and
such automation code is "not corporately supported", so policy won't
get updated to address the disconnect.

> The cornercases come
> into play because you don't always control all of the devices and
> services on your network.  The site could evaluate and decide that
> MITM isn't a threat to their switch's configuration interface but that
> interface might be served over https using a certificate signed by
> their network vendor who doesn't give out their ca certificate (simply
> stated: your security team knows what they're doing but your vendor's
> does not).

This sounds like a similar situation to what I described above. I'm
not sure I'd see these as corner cases, though - they are pretty much
day to day business in my experience :-(

>> In my opinion, we should take all of the value judgements out of this
>> paragraph, and just state the facts. How about:
>>
>> """
>> In order to provide additional flexibility to allow infrastructure
>> administrators to provide the appropriate solution for their
>> environment, this PEP offers a way for administrators to upgrade to
>> later versions of the Python 2.7 series without being forced to update
>> their existing security certificate management infrastructure as a
>> prerequisite.
>> """
>
> Two notes:
>
> * python-2.7.9+ doesn't give you flexibility in this regard so
> organizations do have to update their certificate management
> infrastructure.  The cornercase described above becomes something that
> has to be addressed at the code level.  Environments that are simply
> misconfigured have to be fixed.  So in that regard, a value judgement
> does seem appropriate here.  the judgement is "Listen guys, this PEP
> advises redistributors on how they might provide a migration path for
> you but it *does not bandaid the problem indefinitely*.  So long term,
> you have to change your practices or you'll be out in the cold when
> your redistributor upgrades to python-2.7.9+"

Hmm, maybe I misread the PEP (I only skimmed it - as I say, Linux is
of limited interest to me). I thought that the environment variable
gave developers a "get out" clause. Maybe it's not what we want them
to do (for some value of "we") but isn't that the point of the PEP?

Admittedly if distributions don't *implement* that part of the PEP
(and I understand Red Hat haven't) then people are still stuck. But
"this PEP offers a way" is not incompatible with "your vendor didn't
implement the PEP so you're still stuck, sorry"...

> * Your proposed text actually removes the fix that I was adding --
> this version of the paragraph implies that if your environment is
> compatible with the redistributors' python-2.7.8 (or less) then it
> will also be compatible with the redistributors' python-2.7.9+.  That
> is not true.  Whether or not we take out any value judgement as to the
> user's present environment this paragraph needs to be fixed to make it
> clear that this only affects redistributor's packages which have
> backported pep 476 to python-2.7.8 or older.  Once the redistributor
> updates to a newer python sites which relied on this crutch will
> break.

Sorry for that. Certainly getting the facts right is crucial, and it
looks like my suggestion didn't do that. But hopefully someone can fix
it up (if people think it's a good way to go).

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Toshio Kuratomi]

On Tue, Nov 24, 2015 at 10:56 AM, Paul Moore  wrote:
> On 24 November 2015 at 18:37, Toshio Kuratomi  wrote:

>> The cornercases come
>> into play because you don't always control all of the devices and
>> services on your network.  The site could evaluate and decide that
>> MITM isn't a threat to their switch's configuration interface but that
>> interface might be served over https using a certificate signed by
>> their network vendor who doesn't give out their ca certificate (simply
>> stated: your security team knows what they're doing but your vendor's
>> does not).
>
> This sounds like a similar situation to what I described above. I'm
> not sure I'd see these as corner cases, though - they are pretty much
> day to day business in my experience :-(
>
It sounds like you're coming from a Windows background and I'm coming
from a Linux background which might be a small disconnect here -- we
do seem to be in agreement that what's "right to do" isn't always easy
or possible for the client to accomplish so I think we should probably
leave it at that.

>>> In my opinion, we should take all of the value judgements out of this
>>> paragraph, and just state the facts. How about:
>>>
>>> """
>>> In order to provide additional flexibility to allow infrastructure
>>> administrators to provide the appropriate solution for their
>>> environment, this PEP offers a way for administrators to upgrade to
>>> later versions of the Python 2.7 series without being forced to update
>>> their existing security certificate management infrastructure as a
>>> prerequisite.
>>> """
>>
>> Two notes:
>>
>> * python-2.7.9+ doesn't give you flexibility in this regard so
>> organizations do have to update their certificate management
>> infrastructure.  The cornercase described above becomes something that
>> has to be addressed at the code level.  Environments that are simply
>> misconfigured have to be fixed.  So in that regard, a value judgement
>> does seem appropriate here.  the judgement is "Listen guys, this PEP
>> advises redistributors on how they might provide a migration path for
>> you but it *does not bandaid the problem indefinitely*.  So long term,
>> you have to change your practices or you'll be out in the cold when
>> your redistributor upgrades to python-2.7.9+"
>
> Hmm, maybe I misread the PEP (I only skimmed it - as I say, Linux is
> of limited interest to me). I thought that the environment variable
> gave developers a "get out" clause. Maybe it's not what we want them
> to do (for some value of "we") but isn't that the point of the PEP?
>
> Admittedly if distributions don't *implement* that part of the PEP
> (and I understand Red Hat haven't) then people are still stuck. But
> "this PEP offers a way" is not incompatible with "your vendor didn't
> implement the PEP so you're still stuck, sorry"...
>

Yeah, I think you are correct in your understanding of what actual
changes to the python distrribution are being proposed for
redistributors in the PEP.  Reading through the PEP again, I'm not
sure if I'm correct in thinking that this only applies to
backporting... it may be that the environment section of the PEP
applies to any python-2 while the config file section only applies to
backporting.  Nick, could you clarify?

The PEP is clear that it doesn't apply to python-3 or cross-distro.
So that means that sites still can't rely on this long-term (but their
long term would extend to the lifetime of their vendor supporting
python2 rather than when their vendor updated to 2.7.9+) and also that
developers can't depend on this if they're developing portable code.

>> * Your proposed text actually removes the fix that I was adding --
>> this version of the paragraph implies that if your environment is
>> compatible with the redistributors' python-2.7.8 (or less) then it
>> will also be compatible with the redistributors' python-2.7.9+.  That
>> is not true.  Whether or not we take out any value judgement as to the
>> user's present environment this paragraph needs to be fixed to make it
>> clear that this only affects redistributor's packages which have
>> backported pep 476 to python-2.7.8 or older.  Once the redistributor
>> updates to a newer python sites which relied on this crutch will
>> break.
>
> Sorry for that. Certainly getting the facts right is crucial, and it
> looks like my suggestion didn't do that. But hopefully someone can fix
> it up (if people think it's a good way to go).

Could be that I'm wrong -- will wait for Nick to clarify before I
think about what could be done to make this wording better.

-Toshio
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Paul Moore]

On 24 November 2015 at 03:46, Nick Coghlan  wrote:
> I think there are three relevant categories here:
>
> 1. Folks who assume that "https" means the same thing in Python that
> it means in web browsers, and are currently experiencing a silent
> security failure
> 2. Folks who already know it doesn't, and are relying on that to keep
> their infrastructure working
> 3. Folks currently in group 2 who would like to improve their
> infrastructure to default to verifying certificates

I'm not directly interested in this PEP (any code I write that is
relevant will run on Windows) but there's another group:

4. People who use https because it's the published API, but are only
looking at internal systems (where the threat window is minimised) and
the internal infrastructure is out of their control but insecure.
Browser users and other apps have to go through pain to work
(accepting self-signed certs, repeatedly responding "Yes" to security
warnings). Python code written for internal use just works at the
moment. Maybe it "shouldn't", in some abstract sense, but it *does*
and that is important to users.

This group may know that they are taking risks, and may want to
improve the infrastructure, but they *can't*. And this change breaks
their apps (possibly in ways they can't fix easily - not all client
environments offer ways to add trust, there's a reason tools like wget
have "ignore certificate checks" flags even though everyone knows they
are a bad idea).

The push for "secure by default" is important and needed, but needs to
be handled in a more pragmatic way, recognising that Python is used in
situations like the above, where people know they aren't doing the
right thing, but have no better options. Shutting down their least bad
option on a point of principle isn't helpful.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Paul Moore]

On 24 November 2015 at 13:20, Nick Coghlan  wrote:
> I believe you're referring mainly to the original PEP 476 change there. In
> the context of PEP 493, this is another group that would potentially benefit
> from the suggested "security downgrade" environment variable (if any
> redistributors decide to implement that - RHEL doesn't as yet), since it
> would provide a way to restore the old behaviour without changing their
> client code or monkeypatching the SSL module as described in PEP 476.

I'm actually referring to the fact that your classification didn't
seem to include people who have no control over their infrastructure
(except in class 1 which implies ignorance rather than
powerlessness...). PEP 493 is of benefit to such people, so there's
now downside in explicitly noting this.

My concern is that *because* people consistently forget about the
class of people who have to put up with bad infrastructure but can't
do anything about it, we risk promoting a sense of "security as the
enemy" - which is the direct opposite of what we're trying to do.

I have no interest or opinion regarding this PEP itself, but I would
like to see "people who have to put up with whatever infrastructure
they are dumped with, and use Python to ease that burden" recognised
as an important class of user. They are very under-represented in
discussions, as it's usually big business closed source and similar
environments that are in that situation.

Simply adding "people who have no control over their broken
infrastructure" with a note that this PEP helps them, would be
sufficient here (and actually helps the case for the PEP, so why not?
;-))

Apologies, this is a bit of a hobby horse of mine, I'll pipe down now.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Laura Creighton]

In a message of Tue, 24 Nov 2015 14:05:53 +, Paul Moore writes:
>Simply adding "people who have no control over their broken
>infrastructure" with a note that this PEP helps them, would be
>sufficient here (and actually helps the case for the PEP, so why not?
>;-))

But does it help them?  Or does it increase the power of those who
hand out certificates and who are intensely security conscious over
those who would like to get some work done this afternoon?

Laura
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 25 November 2015 at 00:27, Laura Creighton  wrote:
> In a message of Tue, 24 Nov 2015 14:05:53 +, Paul Moore writes:
>>Simply adding "people who have no control over their broken
>>infrastructure" with a note that this PEP helps them, would be
>>sufficient here (and actually helps the case for the PEP, so why not?
>>;-))
>
> But does it help them?  Or does it increase the power of those who
> hand out certificates and who are intensely security conscious over
> those who would like to get some work done this afternoon?

In situations where IT are still the "Department of No", rather than
focusing on facilitating folks in getting their work done, I think the
most likely outcome of the configuration file recommendation in PEP
493 is preservation of the status quo: admins simply won't change the
config setting, even if they deploy a version of Linux that adopts the
approach suggested in the PEP. If they do enable full certificate
verification (or upgrade the environments they provide to a version of
Python that has it enabled by default) without doing appropriate
compatibility testing first, then they're going to hit the
compatibility problems Paul is talking about.

The aspect of the PEP that has the potential to help in the case of
poor infrastructure management is providing the ability to globally
turn off certificate verification on a per-process basis. It's the
networking equivalent of monkeypatching - you know there are risks
with doing it, but also judge the near term benefits to outweigh those
longer term risks in your particular situation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[M.-A. Lemburg]

I think the PEP is a good step forward to compromise between
the crypto purists (use whatever technologies makes us more
secure even if it breaks things) and those who cannot upgrade
their Python 2.7 because of the PEP 476 change, since it causes their
applications to fail (e.g. because the embedded devices they want to
interface to only support self signed certs).

I would still find having built-in support for the recommendations
in the Python stdlib a better approach, but PEP 493 is good enough
in at least solving real problems people are having.

PS: Would be great to have a PyPI package which implements these
recommendations so that you can simply add it as dependency - and
then please for Python 3 as well, since people with embedded
devices will want to be able to use Python 3 as well ;-)


On 24.11.2015 15:27, Laura Creighton wrote:
> In a message of Tue, 24 Nov 2015 14:05:53 +, Paul Moore writes:
>> Simply adding "people who have no control over their broken
>> infrastructure" with a note that this PEP helps them, would be
>> sufficient here (and actually helps the case for the PEP, so why not?
>> ;-))
> 
> But does it help them?  Or does it increase the power of those who
> hand out certificates and who are intensely security conscious over
> those who would like to get some work done this afternoon?
> 
> Laura
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 24 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Christian Heimes]

On 2015-11-24 01:18, Nick Coghlan wrote:
> On 24 November 2015 at 05:35, Christian Heimes  wrote:
>> On 2015-11-17 01:00, Guido van Rossum wrote:
>>> Hm, making Christian the BDFL-delegate would mean two out of three
>>> authors *and* the BDFL-delegate all working for Red Hat, which clearly
>>> has a stake (and IIUC has already committed to this approach ahead of
>>> PEP approval). SO then it would look like this is just rubber-stamping
>>> Red Hat's internal decision process (if it's a process -- sounds more
>>> like an accident :-).
>>>
>>> So, Alex, do you want to approve this PEP?
>>
>> I haven't read this thread until now. Independently from your objection
>> I have raised the same concern with Nick today. I'd be willing to BDFL
>> the PEP but I'd rather have somebody outside of Red Hat.
> 
> Likewise, but the intersection between "wants to get PEP 476 into the
> hands of as many system operators as possible as soon as possible",
> "is a CPython core developer", and "doesn't work for Red Hat" is
> looking to be a rather select group :)

Right, with Antoine and Alex out of scope and you, Victor and me working
for Red Hat, the air is getting thin. Benjamin is familiar with the ssl
module. Or we can follow Alex's advice and ask somebody from the PyCA
group (Donald, Paul, lvh) or requests (Cory) to get some outside
perspective.


> Since we already know Red Hat are OK with the draft recommendations,
> and I missed the RHEL 7.2 release date anyway, perhaps Barry or
> Matthias might be interested in tilting at the Ubuntu 14.04 LTS stable
> release update windmill? I know there was previously a decision from
> Ubuntu Security not to backport PEPs 466 & 476 to 2.7.5 due to the
> stability risks [1], but the configuration file based approach
> recommended in PEP 493 is backwards compatible by default, with the
> decision to opt in to the improved settings after upgrading current
> systems being made by system administrators rather than the distro
> vendor. With around 3 1/2 years still to run on 14.04's support
> lifecycle, that has the potential to reach quite a few systems that
> otherwise wouldn't benefit from the change until well after Ubuntu
> 16.04 is released next year.

Yes, that makes a lot of sense.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Donald Stufft]

> On Nov 24, 2015, at 7:53 AM, Christian Heimes  wrote:
> 
> On 2015-11-24 01:18, Nick Coghlan wrote:
>> On 24 November 2015 at 05:35, Christian Heimes  wrote:
>>> On 2015-11-17 01:00, Guido van Rossum wrote:
 Hm, making Christian the BDFL-delegate would mean two out of three
 authors *and* the BDFL-delegate all working for Red Hat, which clearly
 has a stake (and IIUC has already committed to this approach ahead of
 PEP approval). SO then it would look like this is just rubber-stamping
 Red Hat's internal decision process (if it's a process -- sounds more
 like an accident :-).
 
 So, Alex, do you want to approve this PEP?
>>> 
>>> I haven't read this thread until now. Independently from your objection
>>> I have raised the same concern with Nick today. I'd be willing to BDFL
>>> the PEP but I'd rather have somebody outside of Red Hat.
>> 
>> Likewise, but the intersection between "wants to get PEP 476 into the
>> hands of as many system operators as possible as soon as possible",
>> "is a CPython core developer", and "doesn't work for Red Hat" is
>> looking to be a rather select group :)
> 
> Right, with Antoine and Alex out of scope and you, Victor and me working
> for Red Hat, the air is getting thin. Benjamin is familiar with the ssl
> module. Or we can follow Alex's advice and ask somebody from the PyCA
> group (Donald, Paul, lvh) or requests (Cory) to get some outside
> perspective.
> 

Under normal circumstances I'd probably be willing to do it even though I have
doubts about the value of it. However, I'm less than two weeks away from
closing on a house and will be moving into it after that, so my time is very
limited right now.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 24 Nov 2015 8:12 pm, "Paul Moore"  wrote:
>
> On 24 November 2015 at 03:46, Nick Coghlan  wrote:
> > I think there are three relevant categories here:
> >
> > 1. Folks who assume that "https" means the same thing in Python that
> > it means in web browsers, and are currently experiencing a silent
> > security failure
> > 2. Folks who already know it doesn't, and are relying on that to keep
> > their infrastructure working
> > 3. Folks currently in group 2 who would like to improve their
> > infrastructure to default to verifying certificates
>
> I'm not directly interested in this PEP (any code I write that is
> relevant will run on Windows) but there's another group:
>
> 4. People who use https because it's the published API, but are only
> looking at internal systems (where the threat window is minimised) and
> the internal infrastructure is out of their control but insecure.
> Browser users and other apps have to go through pain to work
> (accepting self-signed certs, repeatedly responding "Yes" to security
> warnings). Python code written for internal use just works at the
> moment. Maybe it "shouldn't", in some abstract sense, but it *does*
> and that is important to users.
>
> This group may know that they are taking risks, and may want to
> improve the infrastructure, but they *can't*. And this change breaks
> their apps (possibly in ways they can't fix easily - not all client
> environments offer ways to add trust, there's a reason tools like wget
> have "ignore certificate checks" flags even though everyone knows they
> are a bad idea).

I believe you're referring mainly to the original PEP 476 change there. In
the context of PEP 493, this is another group that would potentially
benefit from the suggested "security downgrade" environment variable (if
any redistributors decide to implement that - RHEL doesn't as yet), since
it would provide a way to restore the old behaviour without changing their
client code or monkeypatching the SSL module as described in PEP 476.

Regards,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Christian Heimes]

On 2015-11-24 00:47, Nick Coghlan wrote:
> Updated version of the PEP posted: https://hg.python.org/peps/rev/8decac213ebf
> 
> On 24 November 2015 at 05:35, Christian Heimes  wrote:
>> 1) The example implementation of the function doesn't check the
>> sys.flags.ignore_environment. Internally CPython has specialized getenv
>> function that ignores env vars with PYTHON prefix when the flag is set.
>> PYTHON* env vars aren't removed from os.environ. Modules have to check
>> the flag.
> 
> I guarded the relevant sections in the examples with "if not
> sys.flags.ignore_environment:"

Thanks!


>> 2) The PEP is rather Linux-centric. What's the recommended path to the
>> config file on other platforms like BDS (/usr/local/etc/ is preferred
>> for additional dependencies on FreeBSD), OSX and Windows?
> 
> The environment variable section should be generic, and the
> configuration file section is largely specific to vendors offering
> long term commercial support options for Linux distros.
> 
> There was already a qualification in the "Recommended file location"
> section limiting the scope to *nix systems, so rather than trying to
> cover non-Linux systems, I've instead updated that qualification to
> limit the recommendation even further to specifically Linux systems.
> If *BSD folks pipe up saying they'd like to be included, then adding
> another filename recommendation wouldn't be difficult.
>> 3) What's the interaction between the location of the config file and
>> virtual envs? Would it make sense to search for the file in a venv's
>> etc/ first and then dispatch to global /etc/? That way venvs can
>> influence the setting, too.
> 
> This is a system level configuration setting to preserve backwards
> compatible behaviour in the system Python, so scoping it at the
> interpreter level was a deliberate design decision. However, I added a
> new section to both recommendations describing the lack of interaction
> with virtual environments.

I mentioned 2) and 3) because I suspect that some people will want to
use the new setting to configure applications. You know how people
think. They get a new shiny tool and then they abuse it as a hammer to
drive in all their problems. :)


>> 4) It makes sense to make the cert-verification.cfg file future-proof
>> and reserve it for other cert-related configuration in the future. For
>> example it could be used to define new contexts, set protocols, ciphers
>> or hashes for cert pinning. It should be enough to say that CPython
>> reserves the right to add more sections and keys later.
> 
> For future releases, I think a different filename makes sense, as we
> don't really want a global "turn off HTTPS certificate verification"
> flag. Perhaps something like "/etc/python/network-security.cfg", for
> example.

Are you planning to remove the "disable verification flag" in the future?


>> 5) I'm not particular fond of the section name [https]. For one It is
>> ambiguous because it doesn't distinguish between server certs and client
>> certs.
> 
> I added a note to clarify that the section name comes from the "https"
> URL schema passed to the relevant client APIs.
> 
>> It's also not correct. The default context is used for other
>> protocols like imap, smtp etc. over TLS.
> 
> That changed in PEP 476 - there's a separate private context for HTTPS
> standard library clients now, which allows HTTPS to verify hostnames
> by default, while other clients are still permissive. This is noted in
> PEP 476: https://www.python.org/dev/peps/pep-0476/#other-protocols
> 
> The reason for this is that browsers have provided a pretty good
> forcing function in getting folks to use properly signed certificates,
> at least on the public internet, but self-signed and improperly signed
> certificates are still significantly more common for other protocols.

I'm sorry, I forgot about PEP 476 and the separate context function for
HTTPS. Somehow I expected the setting to influence all TLS/SSL
connection, not just HTTPS. I should have read the topic line.

In the light of the new information the setting name makes sense.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Paul Moore]

On 24 November 2015 at 14:27, Laura Creighton  wrote:
> In a message of Tue, 24 Nov 2015 14:05:53 +, Paul Moore writes:
>>Simply adding "people who have no control over their broken
>>infrastructure" with a note that this PEP helps them, would be
>>sufficient here (and actually helps the case for the PEP, so why not?
>>;-))
>
> But does it help them?  Or does it increase the power of those who
> hand out certificates and who are intensely security conscious over
> those who would like to get some work done this afternoon?

My reading is that if fully implemented (and Nick has already
confirmed that Red Hat didn't do this) it would add an environment
variable that would allow the user to (in effect) say "I can't fix my
security infrastructure, so just leave me alone and let me take the
risk".

So in theory this PEP would give back some of the ability to ignore
the problem that previous PEPs took away. (And by "ignore the
problem", here I mean "just try to get some work done in spite of a
security and infrastructure group that don't understand how to
implement security and infrastructure, dammit!")

Like it or not, in many organisations, security and development are a
huge "us and them" battle. For me, it's important that Python doesn't
take sides in that battle, while still offering education to anyone
willing to listen. (All I've learned about security is as a result of
working with Python - sadly, that knowledge has not made my job one
iota easier, it's just increased my stress levels :-()

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Alex Gaynor]

Hi all,

While I appreciate the vote of confidence from everyone, I'm not interested
in being the BDFL-delegate for this. I don't think it's a good idea, and
I'm not willing to put further time into.

If he's interested, Donald Stufft would make a good choice for delegate.

Really do appreciate everyone's confidence.

Cheers,
Alex

On Mon, Nov 23, 2015 at 2:35 PM, Christian Heimes 
wrote:

> On 2015-11-17 01:00, Guido van Rossum wrote:
> > Hm, making Christian the BDFL-delegate would mean two out of three
> > authors *and* the BDFL-delegate all working for Red Hat, which clearly
> > has a stake (and IIUC has already committed to this approach ahead of
> > PEP approval). SO then it would look like this is just rubber-stamping
> > Red Hat's internal decision process (if it's a process -- sounds more
> > like an accident :-).
> >
> > So, Alex, do you want to approve this PEP?
>
> I haven't read this thread until now. Independently from your objection
> I have raised the same concern with Nick today. I'd be willing to BDFL
> the PEP but I'd rather have somebody outside of Red Hat. Alex is a great
> choice.
>
>
> In the same mail I sent Nick a quick review of the latest PEP version in
> private.
>
>
> 1) The example implementation of the function doesn't check the
> sys.flags.ignore_environment. Internally CPython has specialized getenv
> function that ignores env vars with PYTHON prefix when the flag is set.
> PYTHON* env vars aren't removed from os.environ. Modules have to check
> the flag.
>
>
> 2) The PEP is rather Linux-centric. What's the recommended path to the
> config file on other platforms like BDS (/usr/local/etc/ is preferred
> for additional dependencies on FreeBSD), OSX and Windows?
>
>
> 3) What's the interaction between the location of the config file and
> virtual envs? Would it make sense to search for the file in a venv's
> etc/ first and then dispatch to global /etc/? That way venvs can
> influence the setting, too.
>
>
> 4) It makes sense to make the cert-verification.cfg file future-proof
> and reserve it for other cert-related configuration in the future. For
> example it could be used to define new contexts, set protocols, ciphers
> or hashes for cert pinning. It should be enough to say that CPython
> reserves the right to add more sections and keys later.
>
>
> 5) I'm not particular fond of the section name [https]. For one It is
> ambiguous because it doesn't distinguish between server certs and client
> certs. It's also not correct. The default context is used for other
> protocols like imap, smtp etc. over TLS.
>
> Christian
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Christian Heimes]

On 2015-11-17 01:00, Guido van Rossum wrote:
> Hm, making Christian the BDFL-delegate would mean two out of three
> authors *and* the BDFL-delegate all working for Red Hat, which clearly
> has a stake (and IIUC has already committed to this approach ahead of
> PEP approval). SO then it would look like this is just rubber-stamping
> Red Hat's internal decision process (if it's a process -- sounds more
> like an accident :-).
> 
> So, Alex, do you want to approve this PEP?

I haven't read this thread until now. Independently from your objection
I have raised the same concern with Nick today. I'd be willing to BDFL
the PEP but I'd rather have somebody outside of Red Hat. Alex is a great
choice.


In the same mail I sent Nick a quick review of the latest PEP version in
private.


1) The example implementation of the function doesn't check the
sys.flags.ignore_environment. Internally CPython has specialized getenv
function that ignores env vars with PYTHON prefix when the flag is set.
PYTHON* env vars aren't removed from os.environ. Modules have to check
the flag.


2) The PEP is rather Linux-centric. What's the recommended path to the
config file on other platforms like BDS (/usr/local/etc/ is preferred
for additional dependencies on FreeBSD), OSX and Windows?


3) What's the interaction between the location of the config file and
virtual envs? Would it make sense to search for the file in a venv's
etc/ first and then dispatch to global /etc/? That way venvs can
influence the setting, too.


4) It makes sense to make the cert-verification.cfg file future-proof
and reserve it for other cert-related configuration in the future. For
example it could be used to define new contexts, set protocols, ciphers
or hashes for cert pinning. It should be enough to say that CPython
reserves the right to add more sections and keys later.


5) I'm not particular fond of the section name [https]. For one It is
ambiguous because it doesn't distinguish between server certs and client
certs. It's also not correct. The default context is used for other
protocols like imap, smtp etc. over TLS.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 24 November 2015 at 11:59, Barry Warsaw  wrote:
> On Nov 24, 2015, at 10:18 AM, Nick Coghlan wrote:
>
>>Since we already know Red Hat are OK with the draft recommendations,
>>and I missed the RHEL 7.2 release date anyway, perhaps Barry or
>>Matthias might be interested in tilting at the Ubuntu 14.04 LTS stable
>>release update windmill? I know there was previously a decision from
>>Ubuntu Security not to backport PEPs 466 & 476 to 2.7.5 due to the
>>stability risks [1], but the configuration file based approach
>>recommended in PEP 493 is backwards compatible by default
>
> Right, but this isn't a patch we'd particularly want to carry ourselves.
> Maybe if it were available upstream, tried and tested, it could be considered
> for backporting, but it still wouldn't be zero cost.

If anyone wants to look through the patches implementing the backport,
they're available through the CentOS git repositories:
https://git.centos.org/commit/rpms!python.git/refs!heads!c7

> We'd have to also handle
> migration paths to newer Ubuntu releases, which probably means removing the
> config file on future upgrades.

You'll need to figure out those migration paths anyway, as the
certificate verification compatibility break will exist between 14.04
and 16.04 regardless of what happens with PEP 493.

> There's also the possibility of implementing
> different defaults on new installs of 14.04 versus upgrades to 14.04.

For those aspects, we left the defaults in RHEL 7.2 as implementing
the 2.7.5 behaviour (i.e. no cert verification), but used the
"platform default" mechanism in PEP 493 to ensure we had the ability
to change our mind if circumstances change.

For the immediately foreseeable future though, it will be up to
operations teams to decide how they want the Python 2.7 standard
library to behave with respect to HTTPS client certificate
verification on RHEL & CentOS 7.

> And
> even if a system administrator enabled it for one particular application, it
> could break other applications on the same machine, so it just punts a
> difficult decision down the line.
>
> We're also not seeing much (any?) demand from our users, and the initial
> attempt at turning this on by default *did* get a strong negative reaction
> because of the compatibility break.

My own main concern is providing an additional layer of defence
against certificate verification related security bugs in network
services and clients written in Python like OpenStack, Ansible,
FreeIPA, Pulp, etc, which is why I'm happy with the approach of giving
system operators the tools they need to make their own decision about
whether or not they want that additional layer of protection on a
given host. Folks building Docker images can also easily opt in or out
by including a suitable configuration file.

>From a change management perspective, there's also a very big
difference between "We made this decision for you, and you'll need to
update your systems accordingly in order to upgrade", and "We made it
possible for you to make your own decision about this new
configuration setting, based on what you think is best for your
particular situation".

There's still some risk in the latter case (since you may accidentally
break things), but the upstream releases flushed out a lot of those
problems for this particular update (e.g. the gevent compatibility
issue that affected the upstream PEP 466 backport).

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 24 November 2015 at 06:47, Wes Turner  wrote:
> 1. Does this affect easy_install?

easy_install has validated certificates since distribute was merged
back into the project as part of setuptools 0.7 [1], and aside from
one issue with HTTPS tunnelling [2], the certificate verification code
has been stable since setuptools 1.3 [3].

> 2. If/because this affects easy_install,
>   should the guidance / suggested package installation tool be [pip];
> because pip install_requires backports.ssl_match_hostname

setuptools/easy_install uses backports.ssl_match_hostname if it's
available, and otherwise has its own implementation.

Cheers,
Nick.

[1] https://pythonhosted.org/setuptools/history.html#id159
[2] https://pythonhosted.org/setuptools/history.html#id80
[3] https://pythonhosted.org/setuptools/history.html#id123

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Wes Turner]

On Mon, Nov 23, 2015 at 5:56 PM, Nick Coghlan  wrote:

> On 24 November 2015 at 06:47, Wes Turner  wrote:
> > 1. Does this affect easy_install?
>
> easy_install has validated certificates since distribute was merged
> back into the project as part of setuptools 0.7 [1], and aside from
> one issue with HTTPS tunnelling [2], the certificate verification code
> has been stable since setuptools 1.3 [3].
>

Got it; thanks.

* https://bitbucket.org/pypa/setuptools/src/tip/setuptools/ssl_support.py


>
> > 2. If/because this affects easy_install,
> >   should the guidance / suggested package installation tool be [pip];
> > because pip install_requires backports.ssl_match_hostname
>
> setuptools/easy_install uses backports.ssl_match_hostname if it's
> available, and otherwise has its own implementation.
>

* setuptools 0.7 [src] -- 2013-06-?? --
https://bitbucket.org/pypa/setuptools/src/7f2c08e9/?at=0.7
* setuptools 0.7 [distro] -- 201X-XX-XX [see: whohas, ]

Setuptools latest is now at version 18.5 (2015-11-02).






>
> Cheers,
> Nick.
>
> [1] https://pythonhosted.org/setuptools/history.html#id159
> [2] https://pythonhosted.org/setuptools/history.html#id80
> [3] https://pythonhosted.org/setuptools/history.html#id123
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

Updated version of the PEP posted: https://hg.python.org/peps/rev/8decac213ebf

On 24 November 2015 at 05:35, Christian Heimes  wrote:
> 1) The example implementation of the function doesn't check the
> sys.flags.ignore_environment. Internally CPython has specialized getenv
> function that ignores env vars with PYTHON prefix when the flag is set.
> PYTHON* env vars aren't removed from os.environ. Modules have to check
> the flag.

I guarded the relevant sections in the examples with "if not
sys.flags.ignore_environment:"

> 2) The PEP is rather Linux-centric. What's the recommended path to the
> config file on other platforms like BDS (/usr/local/etc/ is preferred
> for additional dependencies on FreeBSD), OSX and Windows?

The environment variable section should be generic, and the
configuration file section is largely specific to vendors offering
long term commercial support options for Linux distros.

There was already a qualification in the "Recommended file location"
section limiting the scope to *nix systems, so rather than trying to
cover non-Linux systems, I've instead updated that qualification to
limit the recommendation even further to specifically Linux systems.
If *BSD folks pipe up saying they'd like to be included, then adding
another filename recommendation wouldn't be difficult.

> 3) What's the interaction between the location of the config file and
> virtual envs? Would it make sense to search for the file in a venv's
> etc/ first and then dispatch to global /etc/? That way venvs can
> influence the setting, too.

This is a system level configuration setting to preserve backwards
compatible behaviour in the system Python, so scoping it at the
interpreter level was a deliberate design decision. However, I added a
new section to both recommendations describing the lack of interaction
with virtual environments.

> 4) It makes sense to make the cert-verification.cfg file future-proof
> and reserve it for other cert-related configuration in the future. For
> example it could be used to define new contexts, set protocols, ciphers
> or hashes for cert pinning. It should be enough to say that CPython
> reserves the right to add more sections and keys later.

For future releases, I think a different filename makes sense, as we
don't really want a global "turn off HTTPS certificate verification"
flag. Perhaps something like "/etc/python/network-security.cfg", for
example.

> 5) I'm not particular fond of the section name [https]. For one It is
> ambiguous because it doesn't distinguish between server certs and client
> certs.

I added a note to clarify that the section name comes from the "https"
URL schema passed to the relevant client APIs.

> It's also not correct. The default context is used for other
> protocols like imap, smtp etc. over TLS.

That changed in PEP 476 - there's a separate private context for HTTPS
standard library clients now, which allows HTTPS to verify hostnames
by default, while other clients are still permissive. This is noted in
PEP 476: https://www.python.org/dev/peps/pep-0476/#other-protocols

The reason for this is that browsers have provided a pretty good
forcing function in getting folks to use properly signed certificates,
at least on the public internet, but self-signed and improperly signed
certificates are still significantly more common for other protocols.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 24 November 2015 at 05:35, Christian Heimes  wrote:
> On 2015-11-17 01:00, Guido van Rossum wrote:
>> Hm, making Christian the BDFL-delegate would mean two out of three
>> authors *and* the BDFL-delegate all working for Red Hat, which clearly
>> has a stake (and IIUC has already committed to this approach ahead of
>> PEP approval). SO then it would look like this is just rubber-stamping
>> Red Hat's internal decision process (if it's a process -- sounds more
>> like an accident :-).
>>
>> So, Alex, do you want to approve this PEP?
>
> I haven't read this thread until now. Independently from your objection
> I have raised the same concern with Nick today. I'd be willing to BDFL
> the PEP but I'd rather have somebody outside of Red Hat.

Likewise, but the intersection between "wants to get PEP 476 into the
hands of as many system operators as possible as soon as possible",
"is a CPython core developer", and "doesn't work for Red Hat" is
looking to be a rather select group :)

Since we already know Red Hat are OK with the draft recommendations,
and I missed the RHEL 7.2 release date anyway, perhaps Barry or
Matthias might be interested in tilting at the Ubuntu 14.04 LTS stable
release update windmill? I know there was previously a decision from
Ubuntu Security not to backport PEPs 466 & 476 to 2.7.5 due to the
stability risks [1], but the configuration file based approach
recommended in PEP 493 is backwards compatible by default, with the
decision to opt in to the improved settings after upgrading current
systems being made by system administrators rather than the distro
vendor. With around 3 1/2 years still to run on 14.04's support
lifecycle, that has the potential to reach quite a few systems that
otherwise wouldn't benefit from the change until well after Ubuntu
16.04 is released next year.

Regards,
Nick.

[1] http://people.canonical.com/~ubuntu-security/cve/2014/CVE-2014-9365.html

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 24 November 2015 at 12:05, Barry Warsaw  wrote:
> On Nov 17, 2015, at 11:44 PM, Nick Coghlan wrote:
>
>>For Debian, Ubuntu and SUSE, their original determinations for the
>>relevant CVE were "too intrusive to backport", so folks currently need
>>to upgrade to newer versions of those distros to get the improved
>>default behaviour:
>
> This is an example of my problem with the tone of PEP 493 (sorry Nick, nothing
> personal!).  "Improved default behavior"... for whom?  It's not improved for
> the folks whose applications are broken by changing the default.

I think there are three relevant categories here:

1. Folks who assume that "https" means the same thing in Python that
it means in web browsers, and are currently experiencing a silent
security failure
2. Folks who already know it doesn't, and are relying on that to keep
their infrastructure working
3. Folks currently in group 2 who would like to improve their
infrastructure to default to verifying certificates

The upstream change in PEP 476 was driven by the interests of folks in
group 1. This makes a lot of sense, given that a popular way of
introducing folks to programming now is by writing programs that
interact with web APIs over HTTPS. Rebasing is fine for reaching
desktop audiences, so this group should already have been covered by
the updated python.org binaries for Windows and Mac OS X, new binary
releases from the cross-platform redistributors, and updates to
non-LTS Linux distros.

For folks in group 2, the main goal is "Don't break their stuff as a
side effect of updating a supported version", which is why PEP 493
recommends leaving certificate verification disabled by default if
backporting the PEP 476 changes.

The target audience for PEP 493 is then the folks in group 3:
infrastructure operators that *want* to turn failures to validate
certificates in Python applications into a noisy failure, but *don't*
want to have to switch to a new major version of their entire base
operating system to do it. With the PEP, opting in to PEP 476 for
Python 2 applications and services is just a configuration file
setting, readily managed with any configuration management tool. If
just the configuration file approach is supported (as is the case for
RHEL 7.2), then that decision needs to be made on a system-wide basis,
and you need to use a chroot (or a full Linux container) in order to
opt out (or in) for a single application. If the environment variable
downgrade is also supported, then the environments for individual
applications can be tweaked, even while the overall environment is
upgraded to complain about missing security checks by default.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 24, 2015, at 10:18 AM, Nick Coghlan wrote:

>Since we already know Red Hat are OK with the draft recommendations,
>and I missed the RHEL 7.2 release date anyway, perhaps Barry or
>Matthias might be interested in tilting at the Ubuntu 14.04 LTS stable
>release update windmill? I know there was previously a decision from
>Ubuntu Security not to backport PEPs 466 & 476 to 2.7.5 due to the
>stability risks [1], but the configuration file based approach
>recommended in PEP 493 is backwards compatible by default

Right, but this isn't a patch we'd particularly want to carry ourselves.
Maybe if it were available upstream, tried and tested, it could be considered
for backporting, but it still wouldn't be zero cost.  We'd have to also handle
migration paths to newer Ubuntu releases, which probably means removing the
config file on future upgrades.  There's also the possibility of implementing
different defaults on new installs of 14.04 versus upgrades to 14.04.  And
even if a system administrator enabled it for one particular application, it
could break other applications on the same machine, so it just punts a
difficult decision down the line.

We're also not seeing much (any?) demand from our users, and the initial
attempt at turning this on by default *did* get a strong negative reaction
because of the compatibility break.

I'm concerned about accepting PEP 493 making a strong recommendation to
downstreams.  Yes, in an ideal world we all want security by default, but I
think the backward compatibility concerns of the PEP are understated,
especially as they relate to a maintenance release of a stable long term
support version of the OS.  I don't want PEP 493 to be a cudgel that people
beat us up with instead of having an honest discussion of the difficult
trade-offs involved.

Having said all that, I think informing people of the issue, and letting any
future reconsideration be demand driven is the right approach for now.

$0.02-ly y'rs,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Barry Warsaw]

On Nov 17, 2015, at 11:44 PM, Nick Coghlan wrote:

>For Debian, Ubuntu and SUSE, their original determinations for the
>relevant CVE were "too intrusive to backport", so folks currently need
>to upgrade to newer versions of those distros to get the improved
>default behaviour:

This is an example of my problem with the tone of PEP 493 (sorry Nick, nothing
personal!).  "Improved default behavior"... for whom?  It's not improved for
the folks whose applications are broken by changing the default.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Wes Turner]

... Just had this discussion in regards to easy_install, Ubuntu 14.04 LTS,
and the ReadTheDocs Docker images (as well as: ~why should I have to
wget/curl get-pip.py)
https://github.com/rtfd/readthedocs-docker-images/pull/3
On Nov 23, 2015 2:47 PM, "Wes Turner"  wrote:

> 1. Does this affect easy_install?
> 2. If/because this affects easy_install,
>   should the guidance / suggested package installation tool be [pip];
> because pip install_requires backports.ssl_match_hostname
>
> https://pypi.python.org/pypi/backports.ssl_match_hostname
> On Nov 10, 2015 6:48 PM, "Nick Coghlan"  wrote:
>
>> Hi folks,
>>
>> I have a confession to make - I dropped the ball on the HTTPS
>> verification backport proposals in PEP 493, and let the upstream and
>> downstream approval processes get out of sequence.
>>
>> As a result, the RHEL 7.2 beta released back in September incorporates
>> the HTTPS verification feature backport based on the current PEP 493
>> draft, even though that hasn't formally been pronounced as an Active
>> recommendation by python-dev yet.
>>
>> Accordingly, I'm belatedly submitting it for pronouncement now:
>> https://www.python.org/dev/peps/pep-0493/
>>
>> There's currently no BDFL-Delegate assigned, so if Guido doesn't want
>> to handle it, we'll need to address that question first.
>>
>> Our last discussion back in July seemed to show that folks either
>> didn't care about the question (because they're using unmodified
>> upstream versions so the PEP didn't affect them), or else thought the
>> approach described in the PEP was reasonable, so I'm hoping the
>> consequences of my mistake won't be too severe.
>>
>> Regards,
>> Nick.
>>
>> P.S. I'm aware that this looks like presenting a fait accompli at a
>> point where it's too late to realistically say "No", but the truth is
>> that preparation for the Python in Education miniconf at PyCon
>> Australia ramped up immediately after the July discussion, and then I
>> personally got confused as to the scope of what was being included in
>> 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493
>> being deferred to a later release, but it's actually the whole package
>> of 466+476+493). That's my fault for trying to keep track of too many
>> things at once (and thus failing at some of them), not anyone else's.
>>
>> 
>>
>> PEP: 493
>> Title: HTTPS verification recommendations for Python 2.7 redistributors
>> Version: $Revision$
>> Last-Modified: $Date$
>> Author: Nick Coghlan ,
>> Robert Kuska ,
>> Marc-André Lemburg 
>> Status: Draft
>> Type: Informational
>> Content-Type: text/x-rst
>> Created: 10-May-2015
>> Post-History: 06-Jul-2015
>>
>>
>> Abstract
>> 
>>
>> PEP 476 updated Python's default handling of HTTPS certificates to be
>> appropriate for communication over the public internet. The Python 2.7
>> long
>> term maintenance series was judged to be in scope for this change, with
>> the
>> new behaviour introduced in the Python 2.7.9 maintenance release.
>>
>> This PEP provides recommendations to downstream redistributors wishing to
>> provide a smoother migration experience when helping their users to manage
>> this change in Python's default behaviour.
>>
>>
>> Rationale
>> =
>>
>> PEP 476 changed Python's default behaviour to better match the needs and
>> expectations of developers operating over the public internet, a category
>> which appears to include most new Python developers. It is the position of
>> the authors of this PEP that this was a correct decision.
>>
>> However, it is also the case that this change *does* cause problems for
>> infrastructure administrators operating private intranets that rely on
>> self-signed certificates, or otherwise encounter problems with the new
>> default
>> certificate verification settings.
>>
>> The long term answer for such environments is to update their internal
>> certificate management to at least match the standards set by the public
>> internet, but in the meantime, it is desirable to offer these
>> administrators
>> a way to continue receiving maintenance updates to the Python 2.7 series,
>> without having to gate that on upgrades to their certificate management
>> infrastructure.
>>
>> PEP 476 did attempt to address this question, by covering how to revert
>> the
>> new settings process wide by monkeypatching the ``ssl`` module to restore
>> the
>> old behaviour. Unfortunately, the ``sitecustomize.py`` based technique
>> proposed
>> to allow system administrators to disable the feature by default in their
>> Standard Operating Environment definition has been determined to be
>> insufficient in at least some cases. The specific case of interest to the
>> authors of this PEP is the one where a Linux distributor aims to provide
>> their users with a
>> `smoother migration path
>> 

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Wes Turner]

1. Does this affect easy_install?
2. If/because this affects easy_install,
  should the guidance / suggested package installation tool be [pip];
because pip install_requires backports.ssl_match_hostname

https://pypi.python.org/pypi/backports.ssl_match_hostname
On Nov 10, 2015 6:48 PM, "Nick Coghlan"  wrote:

> Hi folks,
>
> I have a confession to make - I dropped the ball on the HTTPS
> verification backport proposals in PEP 493, and let the upstream and
> downstream approval processes get out of sequence.
>
> As a result, the RHEL 7.2 beta released back in September incorporates
> the HTTPS verification feature backport based on the current PEP 493
> draft, even though that hasn't formally been pronounced as an Active
> recommendation by python-dev yet.
>
> Accordingly, I'm belatedly submitting it for pronouncement now:
> https://www.python.org/dev/peps/pep-0493/
>
> There's currently no BDFL-Delegate assigned, so if Guido doesn't want
> to handle it, we'll need to address that question first.
>
> Our last discussion back in July seemed to show that folks either
> didn't care about the question (because they're using unmodified
> upstream versions so the PEP didn't affect them), or else thought the
> approach described in the PEP was reasonable, so I'm hoping the
> consequences of my mistake won't be too severe.
>
> Regards,
> Nick.
>
> P.S. I'm aware that this looks like presenting a fait accompli at a
> point where it's too late to realistically say "No", but the truth is
> that preparation for the Python in Education miniconf at PyCon
> Australia ramped up immediately after the July discussion, and then I
> personally got confused as to the scope of what was being included in
> 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493
> being deferred to a later release, but it's actually the whole package
> of 466+476+493). That's my fault for trying to keep track of too many
> things at once (and thus failing at some of them), not anyone else's.
>
> 
>
> PEP: 493
> Title: HTTPS verification recommendations for Python 2.7 redistributors
> Version: $Revision$
> Last-Modified: $Date$
> Author: Nick Coghlan ,
> Robert Kuska ,
> Marc-André Lemburg 
> Status: Draft
> Type: Informational
> Content-Type: text/x-rst
> Created: 10-May-2015
> Post-History: 06-Jul-2015
>
>
> Abstract
> 
>
> PEP 476 updated Python's default handling of HTTPS certificates to be
> appropriate for communication over the public internet. The Python 2.7 long
> term maintenance series was judged to be in scope for this change, with the
> new behaviour introduced in the Python 2.7.9 maintenance release.
>
> This PEP provides recommendations to downstream redistributors wishing to
> provide a smoother migration experience when helping their users to manage
> this change in Python's default behaviour.
>
>
> Rationale
> =
>
> PEP 476 changed Python's default behaviour to better match the needs and
> expectations of developers operating over the public internet, a category
> which appears to include most new Python developers. It is the position of
> the authors of this PEP that this was a correct decision.
>
> However, it is also the case that this change *does* cause problems for
> infrastructure administrators operating private intranets that rely on
> self-signed certificates, or otherwise encounter problems with the new
> default
> certificate verification settings.
>
> The long term answer for such environments is to update their internal
> certificate management to at least match the standards set by the public
> internet, but in the meantime, it is desirable to offer these
> administrators
> a way to continue receiving maintenance updates to the Python 2.7 series,
> without having to gate that on upgrades to their certificate management
> infrastructure.
>
> PEP 476 did attempt to address this question, by covering how to revert the
> new settings process wide by monkeypatching the ``ssl`` module to restore
> the
> old behaviour. Unfortunately, the ``sitecustomize.py`` based technique
> proposed
> to allow system administrators to disable the feature by default in their
> Standard Operating Environment definition has been determined to be
> insufficient in at least some cases. The specific case of interest to the
> authors of this PEP is the one where a Linux distributor aims to provide
> their users with a
> `smoother migration path
> `__
> than the standard one provided by consuming upstream CPython 2.7 releases
> directly, but other potential challenges have also been pointed out with
> updating embedded Python runtimes and other user level installations of
> Python.
>
> Rather than allowing a plethora of mutually incompatibile migration
> techniques
> to bloom, this PEP proposes two alternative approaches that redistributors
> 

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 11 November 2015 at 10:47, Nick Coghlan  wrote:

> Our last discussion back in July seemed to show that folks either
> didn't care about the question (because they're using unmodified
> upstream versions so the PEP didn't affect them), or else thought the
> approach described in the PEP was reasonable, so I'm hoping the
> consequences of my mistake won't be too severe.
>

RHEL 7.2 is out today, together with details of what we've now committed to
supporting for certification verification configuration in the RHEL 7
system Python: https://access.redhat.com/articles/2039753

That article also describes the PEP 476 status across the different Python
builds Red Hat supports, which I've summarised below.

Versions with PEP 476 implemented:

* Python 3.4 SCL: cert verification always enabled by default
* system Python in RHEL 7.2+: still off by default due to the compatibility
break, but with PEP 493's file based configuration setting to enable it by
default

Versions without PEP 476 implemented:

* Python 2.7 SCL
* Python 3.3 SCL
* system Python in RHEL 7.1 and earlier
* system Python in all versions of RHEL 6

I assume that status of the Python 2.7 SCL will change at some point, but
don't have an ETA or any technical details to share at this point.

The system Python versions in RHEL 5 and earlier didn't include the ssl
module at all (since that was only added to the standard library in Python
2.6), so they're not affected by the concerns raised in PEP 476 in the
first place.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 17 November 2015 at 20:33, Victor Stinner  wrote:
> 2015-11-17 1:00 GMT+01:00 Guido van Rossum :
>> Hm, making Christian the BDFL-delegate would mean two out of three
>> authors *and* the BDFL-delegate all working for Red Hat, which clearly
>> has a stake (and IIUC has already committed to this approach ahead of
>> PEP approval). SO then it would look like this is just rubber-stamping
>> Red Hat's internal decision process (if it's a process -- sounds more
>> like an accident :-).
>
> Can we try to get a vote from maintainers of the Python2/3 packages of
> other Linux distributions? Debian, Ubuntu, OpenSUSE, etc.?

I know Oracle were interested based on a discussion between them and a
member of Red Hat's product security team about it on oss-security,
but their devs never followed up on it upstream (even after an
explicit suggestion that they do so), so I'm interpreting that as
willingness to go along with whatever happens in RHEL.

For Debian, Ubuntu and SUSE, their original determinations for the
relevant CVE were "too intrusive to backport", so folks currently need
to upgrade to newer versions of those distros to get the improved
default behaviour:

* http://people.canonical.com/~ubuntu-security/cve/2014/CVE-2014-9365.html
* https://security-tracker.debian.org/tracker/CVE-2014-9365
* https://www.suse.com/security/cve/CVE-2014-9365.html

If having an opt-in backwards-compatible-by-default approach available
(albeit as a PEP 466+476+493 patch set in the RHEL/CentOS system
Python 2.7.5 package) prompts other distro security teams to
reconsider those initial assessments, that would be a nice outcome,
but it isn't my own main priority (so Guido makes a good point in
favouring finding a non-Red-Hatter willing to act as BDFL-Delegate)

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Victor Stinner]

2015-11-17 1:00 GMT+01:00 Guido van Rossum :
> Hm, making Christian the BDFL-delegate would mean two out of three
> authors *and* the BDFL-delegate all working for Red Hat, which clearly
> has a stake (and IIUC has already committed to this approach ahead of
> PEP approval). SO then it would look like this is just rubber-stamping
> Red Hat's internal decision process (if it's a process -- sounds more
> like an accident :-).

Can we try to get a vote from maintainers of the Python2/3 packages of
other Linux distributions? Debian, Ubuntu, OpenSUSE, etc.?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Guido van Rossum]

Hm, making Christian the BDFL-delegate would mean two out of three
authors *and* the BDFL-delegate all working for Red Hat, which clearly
has a stake (and IIUC has already committed to this approach ahead of
PEP approval). SO then it would look like this is just rubber-stamping
Red Hat's internal decision process (if it's a process -- sounds more
like an accident :-).

So, Alex, do you want to approve this PEP?

On Mon, Nov 16, 2015 at 3:54 PM, Nick Coghlan  wrote:
> On 17 November 2015 at 09:07, Guido van Rossum  wrote:
>> So I dropped the ball on this too -- I was going to either have a look
>> or tell you quickly to find a BDFL-delegate, but ended up doing
>> neither. Skimming it now I really don't think I'm the right person to
>> review this. Maybe you could ask Alex Gaynor?
>
> If Alex is interested, that could definitely work. I've also asked
> Christian Heimes if he might be interested in handling it, as he
> worked on several of the SSL module changes that were backported, and
> he also works on FreeIPA [1] now, which will potentially be affected
> by the changes to RHEL & CentOS 7.
>
> Regards,
> Nick.
>
> [1] FreeIPA is an open source Identity Management & Authentication
> system: http://www.freeipa.org/
>
>>
>> On Tue, Nov 10, 2015 at 4:47 PM, Nick Coghlan  wrote:
>>> Hi folks,
>>>
>>> I have a confession to make - I dropped the ball on the HTTPS
>>> verification backport proposals in PEP 493, and let the upstream and
>>> downstream approval processes get out of sequence.
>>>
>>> As a result, the RHEL 7.2 beta released back in September incorporates
>>> the HTTPS verification feature backport based on the current PEP 493
>>> draft, even though that hasn't formally been pronounced as an Active
>>> recommendation by python-dev yet.
>>>
>>> Accordingly, I'm belatedly submitting it for pronouncement now:
>>> https://www.python.org/dev/peps/pep-0493/
>>>
>>> There's currently no BDFL-Delegate assigned, so if Guido doesn't want
>>> to handle it, we'll need to address that question first.
>>>
>>> Our last discussion back in July seemed to show that folks either
>>> didn't care about the question (because they're using unmodified
>>> upstream versions so the PEP didn't affect them), or else thought the
>>> approach described in the PEP was reasonable, so I'm hoping the
>>> consequences of my mistake won't be too severe.
>>>
>>> Regards,
>>> Nick.
>>>
>>> P.S. I'm aware that this looks like presenting a fait accompli at a
>>> point where it's too late to realistically say "No", but the truth is
>>> that preparation for the Python in Education miniconf at PyCon
>>> Australia ramped up immediately after the July discussion, and then I
>>> personally got confused as to the scope of what was being included in
>>> 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493
>>> being deferred to a later release, but it's actually the whole package
>>> of 466+476+493). That's my fault for trying to keep track of too many
>>> things at once (and thus failing at some of them), not anyone else's.
>>>
>>> 
>>>
>>> PEP: 493
>>> Title: HTTPS verification recommendations for Python 2.7 redistributors
>>> Version: $Revision$
>>> Last-Modified: $Date$
>>> Author: Nick Coghlan ,
>>> Robert Kuska ,
>>> Marc-André Lemburg 
>>> Status: Draft
>>> Type: Informational
>>> Content-Type: text/x-rst
>>> Created: 10-May-2015
>>> Post-History: 06-Jul-2015
>>>
>>>
>>> Abstract
>>> 
>>>
>>> PEP 476 updated Python's default handling of HTTPS certificates to be
>>> appropriate for communication over the public internet. The Python 2.7 long
>>> term maintenance series was judged to be in scope for this change, with the
>>> new behaviour introduced in the Python 2.7.9 maintenance release.
>>>
>>> This PEP provides recommendations to downstream redistributors wishing to
>>> provide a smoother migration experience when helping their users to manage
>>> this change in Python's default behaviour.
>>>
>>>
>>> Rationale
>>> =
>>>
>>> PEP 476 changed Python's default behaviour to better match the needs and
>>> expectations of developers operating over the public internet, a category
>>> which appears to include most new Python developers. It is the position of
>>> the authors of this PEP that this was a correct decision.
>>>
>>> However, it is also the case that this change *does* cause problems for
>>> infrastructure administrators operating private intranets that rely on
>>> self-signed certificates, or otherwise encounter problems with the new 
>>> default
>>> certificate verification settings.
>>>
>>> The long term answer for such environments is to update their internal
>>> certificate management to at least match the standards set by the public
>>> internet, but in the meantime, it is desirable to offer these administrators
>>> a way to continue 

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Guido van Rossum]

So I dropped the ball on this too -- I was going to either have a look
or tell you quickly to find a BDFL-delegate, but ended up doing
neither. Skimming it now I really don't think I'm the right person to
review this. Maybe you could ask Alex Gaynor?

On Tue, Nov 10, 2015 at 4:47 PM, Nick Coghlan  wrote:
> Hi folks,
>
> I have a confession to make - I dropped the ball on the HTTPS
> verification backport proposals in PEP 493, and let the upstream and
> downstream approval processes get out of sequence.
>
> As a result, the RHEL 7.2 beta released back in September incorporates
> the HTTPS verification feature backport based on the current PEP 493
> draft, even though that hasn't formally been pronounced as an Active
> recommendation by python-dev yet.
>
> Accordingly, I'm belatedly submitting it for pronouncement now:
> https://www.python.org/dev/peps/pep-0493/
>
> There's currently no BDFL-Delegate assigned, so if Guido doesn't want
> to handle it, we'll need to address that question first.
>
> Our last discussion back in July seemed to show that folks either
> didn't care about the question (because they're using unmodified
> upstream versions so the PEP didn't affect them), or else thought the
> approach described in the PEP was reasonable, so I'm hoping the
> consequences of my mistake won't be too severe.
>
> Regards,
> Nick.
>
> P.S. I'm aware that this looks like presenting a fait accompli at a
> point where it's too late to realistically say "No", but the truth is
> that preparation for the Python in Education miniconf at PyCon
> Australia ramped up immediately after the July discussion, and then I
> personally got confused as to the scope of what was being included in
> 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493
> being deferred to a later release, but it's actually the whole package
> of 466+476+493). That's my fault for trying to keep track of too many
> things at once (and thus failing at some of them), not anyone else's.
>
> 
>
> PEP: 493
> Title: HTTPS verification recommendations for Python 2.7 redistributors
> Version: $Revision$
> Last-Modified: $Date$
> Author: Nick Coghlan ,
> Robert Kuska ,
> Marc-André Lemburg 
> Status: Draft
> Type: Informational
> Content-Type: text/x-rst
> Created: 10-May-2015
> Post-History: 06-Jul-2015
>
>
> Abstract
> 
>
> PEP 476 updated Python's default handling of HTTPS certificates to be
> appropriate for communication over the public internet. The Python 2.7 long
> term maintenance series was judged to be in scope for this change, with the
> new behaviour introduced in the Python 2.7.9 maintenance release.
>
> This PEP provides recommendations to downstream redistributors wishing to
> provide a smoother migration experience when helping their users to manage
> this change in Python's default behaviour.
>
>
> Rationale
> =
>
> PEP 476 changed Python's default behaviour to better match the needs and
> expectations of developers operating over the public internet, a category
> which appears to include most new Python developers. It is the position of
> the authors of this PEP that this was a correct decision.
>
> However, it is also the case that this change *does* cause problems for
> infrastructure administrators operating private intranets that rely on
> self-signed certificates, or otherwise encounter problems with the new default
> certificate verification settings.
>
> The long term answer for such environments is to update their internal
> certificate management to at least match the standards set by the public
> internet, but in the meantime, it is desirable to offer these administrators
> a way to continue receiving maintenance updates to the Python 2.7 series,
> without having to gate that on upgrades to their certificate management
> infrastructure.
>
> PEP 476 did attempt to address this question, by covering how to revert the
> new settings process wide by monkeypatching the ``ssl`` module to restore the
> old behaviour. Unfortunately, the ``sitecustomize.py`` based technique 
> proposed
> to allow system administrators to disable the feature by default in their
> Standard Operating Environment definition has been determined to be
> insufficient in at least some cases. The specific case of interest to the
> authors of this PEP is the one where a Linux distributor aims to provide
> their users with a
> `smoother migration path
> `__
> than the standard one provided by consuming upstream CPython 2.7 releases
> directly, but other potential challenges have also been pointed out with
> updating embedded Python runtimes and other user level installations of 
> Python.
>
> Rather than allowing a plethora of mutually incompatibile migration techniques
> to bloom, this PEP proposes two alternative approaches that redistributors
> may take 

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 17 November 2015 at 09:07, Guido van Rossum  wrote:
> So I dropped the ball on this too -- I was going to either have a look
> or tell you quickly to find a BDFL-delegate, but ended up doing
> neither. Skimming it now I really don't think I'm the right person to
> review this. Maybe you could ask Alex Gaynor?

If Alex is interested, that could definitely work. I've also asked
Christian Heimes if he might be interested in handling it, as he
worked on several of the SSL module changes that were backported, and
he also works on FreeIPA [1] now, which will potentially be affected
by the changes to RHEL & CentOS 7.

Regards,
Nick.

[1] FreeIPA is an open source Identity Management & Authentication
system: http://www.freeipa.org/

>
> On Tue, Nov 10, 2015 at 4:47 PM, Nick Coghlan  wrote:
>> Hi folks,
>>
>> I have a confession to make - I dropped the ball on the HTTPS
>> verification backport proposals in PEP 493, and let the upstream and
>> downstream approval processes get out of sequence.
>>
>> As a result, the RHEL 7.2 beta released back in September incorporates
>> the HTTPS verification feature backport based on the current PEP 493
>> draft, even though that hasn't formally been pronounced as an Active
>> recommendation by python-dev yet.
>>
>> Accordingly, I'm belatedly submitting it for pronouncement now:
>> https://www.python.org/dev/peps/pep-0493/
>>
>> There's currently no BDFL-Delegate assigned, so if Guido doesn't want
>> to handle it, we'll need to address that question first.
>>
>> Our last discussion back in July seemed to show that folks either
>> didn't care about the question (because they're using unmodified
>> upstream versions so the PEP didn't affect them), or else thought the
>> approach described in the PEP was reasonable, so I'm hoping the
>> consequences of my mistake won't be too severe.
>>
>> Regards,
>> Nick.
>>
>> P.S. I'm aware that this looks like presenting a fait accompli at a
>> point where it's too late to realistically say "No", but the truth is
>> that preparation for the Python in Education miniconf at PyCon
>> Australia ramped up immediately after the July discussion, and then I
>> personally got confused as to the scope of what was being included in
>> 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493
>> being deferred to a later release, but it's actually the whole package
>> of 466+476+493). That's my fault for trying to keep track of too many
>> things at once (and thus failing at some of them), not anyone else's.
>>
>> 
>>
>> PEP: 493
>> Title: HTTPS verification recommendations for Python 2.7 redistributors
>> Version: $Revision$
>> Last-Modified: $Date$
>> Author: Nick Coghlan ,
>> Robert Kuska ,
>> Marc-André Lemburg 
>> Status: Draft
>> Type: Informational
>> Content-Type: text/x-rst
>> Created: 10-May-2015
>> Post-History: 06-Jul-2015
>>
>>
>> Abstract
>> 
>>
>> PEP 476 updated Python's default handling of HTTPS certificates to be
>> appropriate for communication over the public internet. The Python 2.7 long
>> term maintenance series was judged to be in scope for this change, with the
>> new behaviour introduced in the Python 2.7.9 maintenance release.
>>
>> This PEP provides recommendations to downstream redistributors wishing to
>> provide a smoother migration experience when helping their users to manage
>> this change in Python's default behaviour.
>>
>>
>> Rationale
>> =
>>
>> PEP 476 changed Python's default behaviour to better match the needs and
>> expectations of developers operating over the public internet, a category
>> which appears to include most new Python developers. It is the position of
>> the authors of this PEP that this was a correct decision.
>>
>> However, it is also the case that this change *does* cause problems for
>> infrastructure administrators operating private intranets that rely on
>> self-signed certificates, or otherwise encounter problems with the new 
>> default
>> certificate verification settings.
>>
>> The long term answer for such environments is to update their internal
>> certificate management to at least match the standards set by the public
>> internet, but in the meantime, it is desirable to offer these administrators
>> a way to continue receiving maintenance updates to the Python 2.7 series,
>> without having to gate that on upgrades to their certificate management
>> infrastructure.
>>
>> PEP 476 did attempt to address this question, by covering how to revert the
>> new settings process wide by monkeypatching the ``ssl`` module to restore the
>> old behaviour. Unfortunately, the ``sitecustomize.py`` based technique 
>> proposed
>> to allow system administrators to disable the feature by default in their
>> Standard Operating Environment definition has been determined to be
>> insufficient in at least some cases. The specific case of 

[Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

Hi folks,

I have a confession to make - I dropped the ball on the HTTPS
verification backport proposals in PEP 493, and let the upstream and
downstream approval processes get out of sequence.

As a result, the RHEL 7.2 beta released back in September incorporates
the HTTPS verification feature backport based on the current PEP 493
draft, even though that hasn't formally been pronounced as an Active
recommendation by python-dev yet.

Accordingly, I'm belatedly submitting it for pronouncement now:
https://www.python.org/dev/peps/pep-0493/

There's currently no BDFL-Delegate assigned, so if Guido doesn't want
to handle it, we'll need to address that question first.

Our last discussion back in July seemed to show that folks either
didn't care about the question (because they're using unmodified
upstream versions so the PEP didn't affect them), or else thought the
approach described in the PEP was reasonable, so I'm hoping the
consequences of my mistake won't be too severe.

Regards,
Nick.

P.S. I'm aware that this looks like presenting a fait accompli at a
point where it's too late to realistically say "No", but the truth is
that preparation for the Python in Education miniconf at PyCon
Australia ramped up immediately after the July discussion, and then I
personally got confused as to the scope of what was being included in
7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493
being deferred to a later release, but it's actually the whole package
of 466+476+493). That's my fault for trying to keep track of too many
things at once (and thus failing at some of them), not anyone else's.



PEP: 493
Title: HTTPS verification recommendations for Python 2.7 redistributors
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan ,
Robert Kuska ,
Marc-André Lemburg 
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 10-May-2015
Post-History: 06-Jul-2015


Abstract


PEP 476 updated Python's default handling of HTTPS certificates to be
appropriate for communication over the public internet. The Python 2.7 long
term maintenance series was judged to be in scope for this change, with the
new behaviour introduced in the Python 2.7.9 maintenance release.

This PEP provides recommendations to downstream redistributors wishing to
provide a smoother migration experience when helping their users to manage
this change in Python's default behaviour.


Rationale
=

PEP 476 changed Python's default behaviour to better match the needs and
expectations of developers operating over the public internet, a category
which appears to include most new Python developers. It is the position of
the authors of this PEP that this was a correct decision.

However, it is also the case that this change *does* cause problems for
infrastructure administrators operating private intranets that rely on
self-signed certificates, or otherwise encounter problems with the new default
certificate verification settings.

The long term answer for such environments is to update their internal
certificate management to at least match the standards set by the public
internet, but in the meantime, it is desirable to offer these administrators
a way to continue receiving maintenance updates to the Python 2.7 series,
without having to gate that on upgrades to their certificate management
infrastructure.

PEP 476 did attempt to address this question, by covering how to revert the
new settings process wide by monkeypatching the ``ssl`` module to restore the
old behaviour. Unfortunately, the ``sitecustomize.py`` based technique proposed
to allow system administrators to disable the feature by default in their
Standard Operating Environment definition has been determined to be
insufficient in at least some cases. The specific case of interest to the
authors of this PEP is the one where a Linux distributor aims to provide
their users with a
`smoother migration path
`__
than the standard one provided by consuming upstream CPython 2.7 releases
directly, but other potential challenges have also been pointed out with
updating embedded Python runtimes and other user level installations of Python.

Rather than allowing a plethora of mutually incompatibile migration techniques
to bloom, this PEP proposes two alternative approaches that redistributors
may take when addressing these problems. Redistributors may choose to implement
one, both, or neither of these approaches based on their assessment of the
needs of their particular userbase.

These designs are being proposed as a recommendation for redistributors, rather
than as new upstream features, as they are needed purely to support legacy
environments migrating from older versions of Python 2.7. Neither approach
is being proposed as an upstream Python 2.7 feature, nor as a feature in any
version of Python 3