Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Gregory P. Smith]

On Wed, Sep 3, 2014 at 3:48 PM, Stephen J. Turnbull 
wrote:

> Guido van Rossum writes:
>
>  > lot: five years ago (when I worked at Google!) it was common to find
>  > internal services that required SSL but had a misconfigured certificate,
>  > and the only way to access those services was to override the browser
>  > complaints. Today (working at Dropbox, a much smaller company!) I don't
>  > even remember the last time I had to deal with such a browser complaint
> --
>
> I would tend to discount your recent experience, then.  Smaller (and
> possibly even more important in this fast-developing area, younger)
> organizations are a lot more nimble about things like this.
>

As a defensive data point: I don't remember a single instance of this
happening for Google internal services, at least since I arrived in 2007.
I'm not doubting that Guido remembers some thing(s) but in general people
here at Google would not stand for that, then or now. I would not call it
common, especially five years ago.

Common things I _have_ encountered over the years everywhere I've been both
internal and external: services that listen on the https port 443 but don't
have a valid cert as they are intended only for http port 80 access. Those
are becoming somewhat less common, the only thing I regularly see that on
anymore is random home router web config UIs as issuing a signed server
certificate for security hole ridden commodity embedded box is... a
challenge.

(I'm not commenting on the PEP plans as it seems the right things are
happening for now)

-gps @ Google
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 01.09.2014 10:09, Nick Coghlan wrote:
> On 1 September 2014 17:13, Christian Heimes  wrote:
>> On 01.09.2014 08:44, Nick Coghlan wrote:
>>> Yes, it would have exactly the same security failure modes as
>>> sitecustomize, except it would only fire if the application
>>> imported the ssl module.
>>>
>>> The "-S" and "-I" switches would need to disable the implied
>>> "sslcustomize", just as they disable "import site".
>>
>> A malicious package can already play havoc with your installation with
>> a custom ssl module. If somebody is able to sneak in a ssl.py then you
>> are screwed anyway. sslcustomize is not going to make the situation worse.
> 
> That's not quite true - we're fairly careful about putting the
> standard library before userspace directories, so aside from the
> "current directory" problem, shadowing "ssl" itself can be tricky to
> arrange.

It's really easy to modify sys.modules to override any module
that has already been loaded or add new ones bypassing sys.path
entirely, so the sys.path layout doesn't provide any protection
against such hacks.

If you gain access to one of the dirs on sys.path, you can play
such tricks in sitecustomize.py. Any 3rd party package can do
the same.

We'd have to add digital API signatures to the ssl module to
prevent such stunts :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 08 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...11 days to go
2014-09-27: PyDDF Sprint 2014 ...  19 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 4 September 2014 22:39, Antoine Pitrou  wrote:
> On Thu, 4 Sep 2014 13:11:38 +1000
> Nick Coghlan  wrote:
>> That leaves Python 2.7, and I have to say I'm now persuaded that a
>> backport (including any required httplib and urllib features) is the
>> right way to go. One of the tasks I'd been dreading as a follow-on
>> from PEP 466 was organising the code audit to make sure our existing
>> Python 2 applications are properly configuring SSL. If we instead
>> change Python 2.7.9 to validate certificates by default, then the need
>> to do that audit *goes away*, replaced by the far more mundane tasking
>> of doing integration testing on 2.7.9, which we'd have to do *anyway*.
>
> What are "our existing Python 2 applications"? Is it a Red Hat-specific
> statement? What is the "code audit" you are talking about?

Yes, that's a Red Hat specific statement. In addition to OpenStack,
many of our other products include Python components. We mostly rely
on pycurl, pyopenssl, python-nss, etc for SSL/TLS for historical
reasons (since ssl wasn't available in the Python standard library
until RHEL 6), which was why these concerns got picked up via the
OpenStack community rather than the Fedora community. However, being
aware of the problem creates that element of doubt as to whether there
are *other* cases where someone used the stdlib ssl module rather than
one of the alternatives that verifies HTTPS by default.

I'd previously been looking at the question while wearing my upstream
core developer & CPython redistributor hats. Glyph et al finally
prompted me to look at it while wearing my "automated quality
assurance toolchain architect for an application vendor" hat (i.e. my
actual day job), and that's a *radically* different perspective when
it comes to a language runtime that fails to verify HTTPS connections
correctly by default.

That's the core reason I changed my mind: this conversation forced me
to think through how I could ensure correct HTTPS handling in all our
Python code, and the only workable solution I now see is to fix it at
the standard library level. The alternative is to prepare for the
likelihood of future CVEs in Python based tools based on failure to
properly verify SSL certificates, as I don't see a practical way of
automatically detecting "tried to use HTTPS without verifying the SSL
certificate properly".

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Thu, 4 Sep 2014 13:11:38 +1000
Nick Coghlan  wrote:
> That leaves Python 2.7, and I have to say I'm now persuaded that a
> backport (including any required httplib and urllib features) is the
> right way to go. One of the tasks I'd been dreading as a follow-on
> from PEP 466 was organising the code audit to make sure our existing
> Python 2 applications are properly configuring SSL. If we instead
> change Python 2.7.9 to validate certificates by default, then the need
> to do that audit *goes away*, replaced by the far more mundane tasking
> of doing integration testing on 2.7.9, which we'd have to do *anyway*.

What are "our existing Python 2 applications"? Is it a Red Hat-specific
statement? What is the "code audit" you are talking about?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 4 September 2014 10:00, Ethan Furman  wrote:
> On 09/03/2014 04:36 PM, Antoine Pitrou wrote:
>>
>> On Thu, 4 Sep 2014 09:19:56 +1000
>> Nick Coghlan  wrote:


 Python is routinely updated to bugfix releases by Linux distributions
 and other distribution channels, you usually have no say over what's
 shipped in those updates. This is not like changing the major version
 used for executing the script, which is normally a manual change.
>>>
>>>
>>> We can potentially deal with the more conservative part of the user base
>>> on
>>> the redistributor side - so long as the PEP says it's OK for us to not
>>> apply this particular change if we deem it appropriate to do so.
>>
>>
>> So people would believe python.org that they would get HTTPS cert
>> validation by default, but their upstream distributor would have
>> disabled it for them? That's even worse...
>
>
> I agree.  If the vendors don't want to have validation by default, they
> should stick with 2.7.8.

Yes, that's the way it would work in practice - we'd call it 2.7.8,
and backport fixes from upstream 2.7.x as needed (as someone put it to
me recently, a useful way to think of component version numbers in
RHEL is as referring to the *oldest* pieces of code in that
component).

I've spent quite a bit more time considering the proposal, and I'm now
satisfied that making this change upstream isn't likely to cause any
major problems, due to the fact that folks who are likely to suffer
from this change aren't likely to even be on 2.7 yet.

While we managed not to be last this time, the RHEL/CentOS ecosystem
is still a useful benchmark for the roll out of Python versions into
conservative enterprise organisations, and the more conservative users
within *that* ecosystem are likely to wait for the x.1 release (at the
earliest) rather than upgrading as soon as x.0 is out. RHEL 7.0 only
came out in June, so most of those conservative environments are still
going to be on Python 2.6 in RHEL 6. While we shipped 2.7 support well
before the release of RHEL 7 as part of Software Collections and
OpenShift, the kinds of environments where properly validating SSL by
default may cause problems aren't likely to be on the leading edge of
adopting new approaches to software deployment like SCL and PaaS.

Fixing the HTTPS validation defaults would have several significant
positive consequences:

- lowers a major barrier to Python adoption (and continued usage) for
public internet focused services
- fixes a latent security defect for Python applications operating
over the public internet
- fixes a latent security defect for Python applications operating in
a properly configured intranet environment
- reveals a latent security defect for Python applications
communicating with improperly configured public internet services
- reveals a latent security defect for Python applications operating
in improperly configured intranet environments

The debate is thus solely over how to manage the consequences of the
last two, since going from "silent failure" to "noisy failure" looks
like going from "working" to "broken" to someone that isn't already
intimately familiar with the underlying issues.

That question needs to be considered separately for 3 different versions:

- 3.5
- 3.4
- 2.7

3.5 is not controversial, the answer is that the standard library's
handling of HTTPS URLs should change to verify certificates properly.
No ifs, buts, or maybes - Python 3.5 should automatically verify all
HTTPS connections, with explicit developer action required to skip (or
otherwise relax) the validation check.

So far, we have assumed that 3.4 will get at most a warning. However,
I have changed my mind on that, because Python 3 is still largely an
early adopter driven technology (it's making inroads into more
conservative environments, but it's still at least a few years away
from catching up to Python 2 on that front). As a result, the kinds of
environments RDM and I are worried about will generally *not* be using
Python 3, or if they are, it will be for custom scripts that they can
fix. I wouldn't suggest actually making that change without getting an
explicit +1 from the Canonical folks (since 3.4 is in Ubuntu LTS
14.04), but I would now personally be +1 on just *fixing it* in 3.4.2,
rather than doing a bunch of additional development work purely so we
can make folks wait another year for the Python 3 standard library to
correctly handle HTTPS.

That leaves Python 2.7, and I have to say I'm now persuaded that a
backport (including any required httplib and urllib features) is the
right way to go. One of the tasks I'd been dreading as a follow-on
from PEP 466 was organising the code audit to make sure our existing
Python 2 applications are properly configuring SSL. If we instead
change Python 2.7.9 to validate certificates by default, then the need
to do that audit *goes away*, replaced by the far more mundane tasking
of doing integration testing on 2.7.9, which we'd have 

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Ethan Furman]

On 09/03/2014 05:00 PM, Ethan Furman wrote:

On 09/03/2014 04:36 PM, Antoine Pitrou wrote:

On Thu, 4 Sep 2014 09:19:56 +1000
Nick Coghlan  wrote:


Python is routinely updated to bugfix releases by Linux distributions
and other distribution channels, you usually have no say over what's
shipped in those updates. This is not like changing the major version
used for executing the script, which is normally a manual change.


We can potentially deal with the more conservative part of the user base on
the redistributor side - so long as the PEP says it's OK for us to not
apply this particular change if we deem it appropriate to do so.


So people would believe python.org that they would get HTTPS cert
validation by default, but their upstream distributor would have
disabled it for them? That's even worse...


I agree.  If the vendors don't want to have validation by default, they should 
stick with 2.7.8.


If good argument can be made for why we should make validation by default optional, then that point should be well made 
in Python's release notes, and some easy programmatic way to tell if validation is on or off  (which may just be more 
docs saying call SSLContext and examine the results:  xxx means you're validating, yyy means you are not).


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Ethan Furman]

On 09/03/2014 04:36 PM, Antoine Pitrou wrote:

On Thu, 4 Sep 2014 09:19:56 +1000
Nick Coghlan  wrote:


Python is routinely updated to bugfix releases by Linux distributions
and other distribution channels, you usually have no say over what's
shipped in those updates. This is not like changing the major version
used for executing the script, which is normally a manual change.


We can potentially deal with the more conservative part of the user base on
the redistributor side - so long as the PEP says it's OK for us to not
apply this particular change if we deem it appropriate to do so.


So people would believe python.org that they would get HTTPS cert
validation by default, but their upstream distributor would have
disabled it for them? That's even worse...


I agree.  If the vendors don't want to have validation by default, they should 
stick with 2.7.8.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Thu, 4 Sep 2014 09:19:56 +1000
Nick Coghlan  wrote:
> >
> > Python is routinely updated to bugfix releases by Linux distributions
> > and other distribution channels, you usually have no say over what's
> > shipped in those updates. This is not like changing the major version
> > used for executing the script, which is normally a manual change.
> 
> We can potentially deal with the more conservative part of the user base on
> the redistributor side - so long as the PEP says it's OK for us to not
> apply this particular change if we deem it appropriate to do so.

So people would believe python.org that they would get HTTPS cert
validation by default, but their upstream distributor would have
disabled it for them? That's even worse...

Of course, people could read distribution-specific package changelogs,
but nobody does that.

> 2.7.9 is going to be a somewhat "interesting" release that requires careful
> attention anyway (due to the completion of the PEP 466 backports), so if
> Guido's OK with it, sure, let's kill the "HTTPS isn't" problem for Python 2
> as well.

Possible unvoluntary breakage due to a large backport is one thing.
Deliberate breakage is another.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 4 Sep 2014 06:39, "Alex Gaynor"  wrote:
>
> Guido van Rossum  python.org> writes:
>
> > OK, that changes my position for 2.7 (but not for 3.5). I had assumed
there
> > was a way to disable the cert check by changing one parameter to the
> > urlopen() call. (And I had wanted to add that there should be a clear
FAQ
> > about the subject.) If this isn't possible that changes the situation.
(But I
> > still think that once we do have that simple change option we should do
it,
> > in a later 2.7 upgrade.) I apologize for speaking before I had read all
> > facts, and I'll await what you and Nick come up with.
> > --Guido
>
> This probably doesn't surprise anyone, but I'm more than happy to do the
back-
> porting work for httplib, and any other modules which need SSLContext
support;
> does this require an additional PEP, or does it fit under PEP466 or
PEP476?

I suggest writing up a separate PEP for 2.7 that covers exactly what would
need to be backported in order to make the same HTTPS handling change in
Python 2.

For 476, I suggest taking my list of modules that call
"_create_stdlib_cert" and being completely explicit as to which ones are
*not* changing (as that will help clarify the scope of the backport
proposal).

I learned that lesson with PEP 453 - it's well worth making the Python 3
PEP easier to accept by making it independent of the inevitably more
controversial Python 2 backport PEP :)

Cheers,
Nick.

>
> Alex
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 4 Sep 2014 04:39, "Antoine Pitrou"  wrote:
>
> On Wed, 3 Sep 2014 10:54:55 -0700
> Guido van Rossum  wrote:
> >
> > Let's take the plunge on this issue for the next 2.7 release (3.5 being
a
> > done deal).
>
> I'm entirely against this.
>
> > Yes, some people will find that they have an old script
> > accessing an old service which breaks. Surely some of the other changes
in
> > the same 2.7 bugfix release will also break some other scripts. People
deal
> > with it. Probably 90% of the time it's an annoyance (but no worse than
any
> > other minor-release upgrade -- you should test upgrades before
committing
> > to them, and if all else fails, roll it back).
>
> Python is routinely updated to bugfix releases by Linux distributions
> and other distribution channels, you usually have no say over what's
> shipped in those updates. This is not like changing the major version
> used for executing the script, which is normally a manual change.

We can potentially deal with the more conservative part of the user base on
the redistributor side - so long as the PEP says it's OK for us to not
apply this particular change if we deem it appropriate to do so.

That would make this a case of upstream asking us to do the kind of risk
assessment that people pay us for, which is a kind of request I'm generally
happy to get :)

That way, if downstream users get upset, we can point them at their vendor
support department, rather than having them take out their ire on upstream
volunteers.

Also, after thinking through the implications a bit more, my guess is that
Fedora & Software Collections will accept the change without any fuss, but
the CentOS/RHEL side could be a more involved discussion. On the other
hand, orgs with these kinds of problems aren't likely to have rolled out
RHEL 7 or CentOS 7 yet either, so they're probably still back on Python 2.6
(RHEL 6) or even 2.4 (RHEL 5).

2.7.9 is going to be a somewhat "interesting" release that requires careful
attention anyway (due to the completion of the PEP 466 backports), so if
Guido's OK with it, sure, let's kill the "HTTPS isn't" problem for Python 2
as well.

One additional wrinkle in that case: we will likely need to backport the
SSLContext related changes to httplib as well.

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Benjamin Peterson]

On Wed, Sep 3, 2014, at 13:37, Alex Gaynor wrote:
> Guido van Rossum  python.org> writes:
> 
> > OK, that changes my position for 2.7 (but not for 3.5). I had assumed there
> > was a way to disable the cert check by changing one parameter to the
> > urlopen() call. (And I had wanted to add that there should be a clear FAQ
> > about the subject.) If this isn't possible that changes the situation. (But 
> > I
> > still think that once we do have that simple change option we should do it,
> > in a later 2.7 upgrade.) I apologize for speaking before I had read all
> > facts, and I'll await what you and Nick come up with.
> > --Guido
> 
> This probably doesn't surprise anyone, but I'm more than happy to do the
> back-
> porting work for httplib, and any other modules which need SSLContext
> support;
> does this require an additional PEP, or does it fit under PEP466 or
> PEP476?

I imagine you could put it into 476.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Stephen J. Turnbull]

Guido van Rossum writes:

 > lot: five years ago (when I worked at Google!) it was common to find
 > internal services that required SSL but had a misconfigured certificate,
 > and the only way to access those services was to override the browser
 > complaints. Today (working at Dropbox, a much smaller company!) I don't
 > even remember the last time I had to deal with such a browser complaint --

I would tend to discount your recent experience, then.  Smaller (and
possibly even more important in this fast-developing area, younger)
organizations are a lot more nimble about things like this.

That is not intended to express an opinion about a backport, though.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Guido van Rossum  python.org> writes:

> OK, that changes my position for 2.7 (but not for 3.5). I had assumed there
> was a way to disable the cert check by changing one parameter to the
> urlopen() call. (And I had wanted to add that there should be a clear FAQ
> about the subject.) If this isn't possible that changes the situation. (But I
> still think that once we do have that simple change option we should do it,
> in a later 2.7 upgrade.) I apologize for speaking before I had read all
> facts, and I'll await what you and Nick come up with.
> --Guido

This probably doesn't surprise anyone, but I'm more than happy to do the back-
porting work for httplib, and any other modules which need SSLContext support;
does this require an additional PEP, or does it fit under PEP466 or PEP476?

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 03.09.2014 21:37, Guido van Rossum wrote:
> OK, that changes my position for 2.7 (but not for 3.5). I had
> assumed there was a way to disable the cert check by changing one
> parameter to the urlopen() call. (And I had wanted to add that
> there should be a clear FAQ about the subject.) If this isn't
> possible that changes the situation. (But I still think that once
> we do have that simple change option we should do it, in a later
> 2.7 upgrade.)
> 
> I apologize for speaking before I had read all facts, and I'll
> await what you and Nick come up with.

You are welcome! :)

I like to see cert checks in Python 2.7, too. Eventually Python 2.7
should have them enabled by default or at least have one very simple
way to enable them globally. Right now we aren't there yet. Perhaps
Python 2.7.10 or 2.7.11 will have the necessary bits and backports.

Christian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 03.09.2014 21:37, Victor Stinner wrote:
> Thanks, you replied before I asked the question :-) (If
> certificates are validated by default, how do you disable the
> checks?)
> 
> Sorry, I didn't follow the whole discussion and Python 2.7 changes 
> related to security. Does Python 2.7 support loading
> (automatically) system certificate authorities? Like the Windows
> certificate store: 
> https://docs.python.org/dev/whatsnew/3.4.html#whatsnew34-win-cert-store

Yes!

Alex and others have ported all SSL features back to Python 2.7.
The SSL module in Python 2.7.9 will have the same feature set as
Python 3.4.

Christian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Ethan Furman]

On 09/03/2014 12:10 PM, R. David Murray wrote:

On Wed, 03 Sep 2014 10:09:36 -0700, Ethan Furman  wrote:

On 09/03/2014 08:58 AM, R. David Murray wrote:


I'm OK with letting go of this invalid-cert issue myself, given the lack
of negative feedback Twisted got.  I'll just keep my fingers crossed.


I apologize if I missed this point, but if we have the source code then it is 
possible to go in and directly modify the
application/utility to be able to talk over https to a router with an invalid 
certificate?  This is an option when
creating the ssl_context?


The immediately preceding paragraph that you didn't quote said that the
context was 3rd party applications, not source code under your control.
Yes, you can (usually) still hack the source, but there are good reasons to
prefer to not do that, unfamiliarity with the codebase being just one of
them.


I appreciate that there is a distinction, yet in most cases we have the source code available (it is the nature of 
Python) and if push comes to shove (and a bunch of other colloquialisms) then modifying that source code can get you up 
and running again.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Guido van Rossum]

OK, that changes my position for 2.7 (but not for 3.5). I had assumed there
was a way to disable the cert check by changing one parameter to the
urlopen() call. (And I had wanted to add that there should be a clear FAQ
about the subject.) If this isn't possible that changes the situation. (But
I still think that once we do have that simple change option we should do
it, in a later 2.7 upgrade.)

I apologize for speaking before I had read all facts, and I'll await what
you and Nick come up with.

--Guido


On Wed, Sep 3, 2014 at 12:26 PM, Christian Heimes 
wrote:

> On 03.09.2014 19:54, Guido van Rossum wrote:
> > Let's take the plunge on this issue for the next 2.7 release (3.5 being
> > a done deal). Yes, some people will find that they have an old script
> > accessing an old service which breaks. Surely some of the other changes
> > in the same 2.7 bugfix release will also break some other scripts.
> > People deal with it. Probably 90% of the time it's an annoyance (but no
> > worse than any other minor-release upgrade -- you should test upgrades
> > before committing to them, and if all else fails, roll it back). But at
> > least some of the time it will be a wake-up call and an expired
> > certificate will be replaced, resulting in more security for all.
>
> I'm +1 for Python 3.5 but -1 for Python 2.7.
>
> The SSLContext backport will landed in Python 2.7.9 (to be released). No
> Python 2 user is familiar with the feature yet. But more importantly:
> None of the stdlib modules support the new feature, too. httplib,
> imaplib ... they all don't take a SSLContext object as an argument.
> PEP-466 does not include the backport for the network modules. Without
> the context argument there is simply no clean way to configure the SSL
> handshake properly.
>
> The default settings must stay until we decide to backport the context
> argument and have a way to configure the default behavior. Nick and me
> are planing a PEP.
>
> Christian
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Victor Stinner]

2014-09-03 21:26 GMT+02:00 Christian Heimes :
> On 03.09.2014 19:54, Guido van Rossum wrote:
> I'm +1 for Python 3.5 but -1 for Python 2.7.
>
> The SSLContext backport will landed in Python 2.7.9 (to be released). No
> Python 2 user is familiar with the feature yet. But more importantly:
> None of the stdlib modules support the new feature, too. httplib,
> imaplib ... they all don't take a SSLContext object as an argument.
> PEP-466 does not include the backport for the network modules. Without
> the context argument there is simply no clean way to configure the SSL
> handshake properly.

Thanks, you replied before I asked the question :-) (If certificates
are validated by default, how do you disable the checks?)

Sorry, I didn't follow the whole discussion and Python 2.7 changes
related to security. Does Python 2.7 support loading (automatically)
system certificate authorities? Like the Windows certificate store:
https://docs.python.org/dev/whatsnew/3.4.html#whatsnew34-win-cert-store

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 03.09.2014 19:54, Guido van Rossum wrote:
> Let's take the plunge on this issue for the next 2.7 release (3.5 being
> a done deal). Yes, some people will find that they have an old script
> accessing an old service which breaks. Surely some of the other changes
> in the same 2.7 bugfix release will also break some other scripts.
> People deal with it. Probably 90% of the time it's an annoyance (but no
> worse than any other minor-release upgrade -- you should test upgrades
> before committing to them, and if all else fails, roll it back). But at
> least some of the time it will be a wake-up call and an expired
> certificate will be replaced, resulting in more security for all.

I'm +1 for Python 3.5 but -1 for Python 2.7.

The SSLContext backport will landed in Python 2.7.9 (to be released). No
Python 2 user is familiar with the feature yet. But more importantly:
None of the stdlib modules support the new feature, too. httplib,
imaplib ... they all don't take a SSLContext object as an argument.
PEP-466 does not include the backport for the network modules. Without
the context argument there is simply no clean way to configure the SSL
handshake properly.

The default settings must stay until we decide to backport the context
argument and have a way to configure the default behavior. Nick and me
are planing a PEP.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Guido van Rossum]

Antoine, I think we are well past the point where arguments can sway
positions. There clearly is no agreement on this issue. So please treat my
post as a BDFL tie-breaker. I will just give you one thing to ponder --
those small/non-profit websites that can't afford proper certs are exactly
the ones that will be hosting malware soon. Sorry for them, and the
certificate vendors certainly aren't in it for charity, but they must fix
their certificate issues (and probably improve many other sysadmin
practices).

--Guido


On Wed, Sep 3, 2014 at 11:37 AM, Antoine Pitrou  wrote:

> On Wed, 3 Sep 2014 10:54:55 -0700
> Guido van Rossum  wrote:
> >
> > Let's take the plunge on this issue for the next 2.7 release (3.5 being a
> > done deal).
>
> I'm entirely against this.
>
> > Yes, some people will find that they have an old script
> > accessing an old service which breaks. Surely some of the other changes
> in
> > the same 2.7 bugfix release will also break some other scripts. People
> deal
> > with it. Probably 90% of the time it's an annoyance (but no worse than
> any
> > other minor-release upgrade -- you should test upgrades before committing
> > to them, and if all else fails, roll it back).
>
> Python is routinely updated to bugfix releases by Linux distributions
> and other distribution channels, you usually have no say over what's
> shipped in those updates. This is not like changing the major version
> used for executing the script, which is normally a manual change.
>
> > Today (working at Dropbox, a much smaller company!) I don't
> > even remember the last time I had to deal with such a browser
> > complaint -- internal services here all redirect to SSL, and not a
> > browser that can find fault with their certs.
>
> Good for you. I still sometimes get warnings about expired certificates
> - and sometimes ones that don't exactly match the domain being
> fetched (for example, the certificate wouldn't be valid for that
> specific subdomain - note that CAs often charge a premium for multiple
> subdomains, which why small or non-profit Web sites sometimes skimp on
> them).
>
> You shouldn't assume that the experience of well-connected people in
> the Silicon Valley is representative of what people over the world
> encounter. Yes, where there's a lot of money and a lot of accumulated
> domain competence, security procedures are updated and followed more
> scrupulously...
>
> > But at least some of the
> > time it will be a wake-up call and an expired certificate will be
> replaced,
> > resulting in more security for all.
>
> Only if you are actually the one managing that certificate and the
> machine it's installed one...
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Wed, 03 Sep 2014 10:09:36 -0700, Ethan Furman  wrote:
> On 09/03/2014 08:58 AM, R. David Murray wrote:
> >
> > I'm OK with letting go of this invalid-cert issue myself, given the lack
> > of negative feedback Twisted got.  I'll just keep my fingers crossed.
> 
> I apologize if I missed this point, but if we have the source code then it is 
> possible to go in and directly modify the 
> application/utility to be able to talk over https to a router with an invalid 
> certificate?  This is an option when 
> creating the ssl_context?

The immediately preceding paragraph that you didn't quote said that the
context was 3rd party applications, not source code under your control.
Yes, you can (usually) still hack the source, but there are good reasons to
prefer to not do that, unfamiliarity with the codebase being just one of
them.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 03.09.2014 19:29, Ethan Furman wrote:
> Excellent.  Last question (I hope): it is possible to (easily) create an
> SSLContext that will verify against a self-signed certificate?

Yes:

   context = ssl.create_default_context(cafile="/path/to/selfsigned.pem")

That works iff the certificate is valid, not expired and its CN or SAN
matches the hostname of the service. When the hostname doesn't match
then you have to set

   context.check_hostname = False

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Wed, 03 Sep 2014 20:37:38 +0200, Antoine Pitrou  wrote:
> On Wed, 3 Sep 2014 10:54:55 -0700
> Guido van Rossum  wrote:
> > Today (working at Dropbox, a much smaller company!) I don't
> > even remember the last time I had to deal with such a browser
> > complaint -- internal services here all redirect to SSL, and not a
> > browser that can find fault with their certs.
> 
> Good for you. I still sometimes get warnings about expired certificates
> - and sometimes ones that don't exactly match the domain being
> fetched (for example, the certificate wouldn't be valid for that
> specific subdomain - note that CAs often charge a premium for multiple
> subdomains, which why small or non-profit Web sites sometimes skimp on
> them).
> 
> You shouldn't assume that the experience of well-connected people in
> the Silicon Valley is representative of what people over the world
> encounter. Yes, where there's a lot of money and a lot of accumulated
> domain competence, security procedures are updated and followed more
> scrupulously...

Heck, yesterday I got invalid certs from...I think it was roku.com, but
in any case not some obscure little company...the actual cert was an
akamai cert, which means something is configured wrong somewhere.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Donald Stufft]

> On Sep 3, 2014, at 1:54 PM, Guido van Rossum  wrote:
> 
> On Wed, Sep 3, 2014 at 8:58 AM, R. David Murray  > wrote:
> I'm OK with letting go of this invalid-cert issue myself, given the lack
> of negative feedback Twisted got.  I'll just keep my fingers crossed.
> 
> I'm with this sentiment (cautiously +1) -- and not just because of Twisted's 
> experience or Glyph's passion.
> 
> Network security is much more important now than it was five years ago -- and 
> yet Python 2.7 is at least that old. My own experience has changed a lot: 
> five years ago (when I worked at Google!) it was common to find internal 
> services that required SSL but had a misconfigured certificate, and the only 
> way to access those services was to override the browser complaints. Today 
> (working at Dropbox, a much smaller company!) I don't even remember the last 
> time I had to deal with such a browser complaint -- internal services here 
> all redirect to SSL, and not a browser that can find fault with their certs. 
> If I did get a complaint about a certificate I would fire off an email to a 
> sysadmin alerting them to the issue.
> 
> Let's take the plunge on this issue for the next 2.7 release (3.5 being a 
> done deal). Yes, some people will find that they have an old script accessing 
> an old service which breaks. Surely some of the other changes in the same 2.7 
> bugfix release will also break some other scripts. People deal with it. 
> Probably 90% of the time it's an annoyance (but no worse than any other 
> minor-release upgrade -- you should test upgrades before committing to them, 
> and if all else fails, roll it back). But at least some of the time it will 
> be a wake-up call and an expired certificate will be replaced, resulting in 
> more security for all.

+1, this makes me unreasonably happy.

> 
> I don't want to start preaching security doom and gloom (the experts are 
> doing enough of that :-), but the scale and sophistication of attacks 
> (whether publicized or not) is constantly increasing, and routine maintenance 
> checks on old software are just one of the small ways that we can help the 
> internet become more secure. (And please let the PSF sysadmin team beef up 
> *.python.org  -- sooner or later some forgotten part of 
> our infrastructure *will* come under attack.)

This is an ongoing effort amongst the Infra team, part of the process is moving 
infrastructure away from hand crafted servers towards servers managed by config 
management as well as making sure all our services are behind TLS as well.

> 
> -- 
> --Guido van Rossum (python.org/~guido )
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Wed, 3 Sep 2014 10:54:55 -0700
Guido van Rossum  wrote:
> 
> Let's take the plunge on this issue for the next 2.7 release (3.5 being a
> done deal).

I'm entirely against this.

> Yes, some people will find that they have an old script
> accessing an old service which breaks. Surely some of the other changes in
> the same 2.7 bugfix release will also break some other scripts. People deal
> with it. Probably 90% of the time it's an annoyance (but no worse than any
> other minor-release upgrade -- you should test upgrades before committing
> to them, and if all else fails, roll it back).

Python is routinely updated to bugfix releases by Linux distributions
and other distribution channels, you usually have no say over what's
shipped in those updates. This is not like changing the major version
used for executing the script, which is normally a manual change.

> Today (working at Dropbox, a much smaller company!) I don't
> even remember the last time I had to deal with such a browser
> complaint -- internal services here all redirect to SSL, and not a
> browser that can find fault with their certs.

Good for you. I still sometimes get warnings about expired certificates
- and sometimes ones that don't exactly match the domain being
fetched (for example, the certificate wouldn't be valid for that
specific subdomain - note that CAs often charge a premium for multiple
subdomains, which why small or non-profit Web sites sometimes skimp on
them).

You shouldn't assume that the experience of well-connected people in
the Silicon Valley is representative of what people over the world
encounter. Yes, where there's a lot of money and a lot of accumulated
domain competence, security procedures are updated and followed more
scrupulously...

> But at least some of the
> time it will be a wake-up call and an expired certificate will be replaced,
> resulting in more security for all.

Only if you are actually the one managing that certificate and the
machine it's installed one...

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Guido van Rossum]

On Wed, Sep 3, 2014 at 8:58 AM, R. David Murray 
wrote:

> I'm OK with letting go of this invalid-cert issue myself, given the lack
> of negative feedback Twisted got.  I'll just keep my fingers crossed.
>

I'm with this sentiment (cautiously +1) -- and not just because of
Twisted's experience or Glyph's passion.

Network security is much more important now than it was five years ago --
and yet Python 2.7 is at least that old. My own experience has changed a
lot: five years ago (when I worked at Google!) it was common to find
internal services that required SSL but had a misconfigured certificate,
and the only way to access those services was to override the browser
complaints. Today (working at Dropbox, a much smaller company!) I don't
even remember the last time I had to deal with such a browser complaint --
internal services here all redirect to SSL, and not a browser that can find
fault with their certs. If I did get a complaint about a certificate I
would fire off an email to a sysadmin alerting them to the issue.

Let's take the plunge on this issue for the next 2.7 release (3.5 being a
done deal). Yes, some people will find that they have an old script
accessing an old service which breaks. Surely some of the other changes in
the same 2.7 bugfix release will also break some other scripts. People deal
with it. Probably 90% of the time it's an annoyance (but no worse than any
other minor-release upgrade -- you should test upgrades before committing
to them, and if all else fails, roll it back). But at least some of the
time it will be a wake-up call and an expired certificate will be replaced,
resulting in more security for all.

I don't want to start preaching security doom and gloom (the experts are
doing enough of that :-), but the scale and sophistication of attacks
(whether publicized or not) is constantly increasing, and routine
maintenance checks on old software are just one of the small ways that we
can help the internet become more secure. (And please let the PSF sysadmin
team beef up *.python.org -- sooner or later some forgotten part of our
infrastructure *will* come under attack.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Ethan Furman]

On 09/03/2014 10:15 AM, Alex Gaynor wrote:

Ethan Furman writes:


I apologize if I missed this point, but if we have the source code then it is
possible to go in and directly modify the application/utility to be able to
talk over https to a router with an invalid certificate?  This is an option
when creating the ssl_context?


Yes, it's totally possible to create (and pass to ``http.client``) an
``SSLContext`` which doesn't verify various things. My proposal is only about
changing what happens when you don't explicitly pass a context.


Excellent.  Last question (I hope): it is possible to (easily) create an SSLContext that will verify against a 
self-signed certificate?


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Ethan Furman  stoneleaf.us> writes:

> 
> I apologize if I missed this point, but if we have the source code then it is
> possible to go in and directly modify the application/utility to be able to
> talk over https to a router with an invalid certificate?  This is an option
> when creating the ssl_context?
> 
> --
> ~Ethan~
> 


Yes, it's totally possible to create (and pass to ``http.client``) an
``SSLContext`` which doesn't verify various things. My proposal is only about
changing what happens when you don't explicitly pass a context.

Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Ethan Furman]

On 09/03/2014 08:58 AM, R. David Murray wrote:


I'm OK with letting go of this invalid-cert issue myself, given the lack
of negative feedback Twisted got.  I'll just keep my fingers crossed.


I apologize if I missed this point, but if we have the source code then it is possible to go in and directly modify the 
application/utility to be able to talk over https to a router with an invalid certificate?  This is an option when 
creating the ssl_context?


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Wed, 03 Sep 2014 16:31:13 +0200, Antoine Pitrou  wrote:
> On Tue, 02 Sep 2014 21:29:16 -0400
> "R. David Murray"  wrote:
> > 
> > The top proposal so far is an sslcustomize.py file that could be used to
> > either decrease or increase the default security.  This is a much less
> > handy solution than application options (eg, curl, wget) that allow
> > disabling security for "this cert" or "this CLI session".  It also is
> > more prone to unthinking abuse since it is persistent.  So perhaps
> > it is indeed not worth it.  (That's why I suggested an environment
> > variable...something you could specify on the command line for a one-off.)
> 
> I'll be fine with not adding any hooks at all, and letting people
> configure their application code correctly :-)

Again, the problem arises when it is not *their* application code, but
a third party tool that hasn't been ported to 3.5.

I'm OK with letting go of this invalid-cert issue myself, given the lack
of negative feedback Twisted got.  I'll just keep my fingers crossed.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Tue, 02 Sep 2014 21:29:16 -0400
"R. David Murray"  wrote:
> 
> The top proposal so far is an sslcustomize.py file that could be used to
> either decrease or increase the default security.  This is a much less
> handy solution than application options (eg, curl, wget) that allow
> disabling security for "this cert" or "this CLI session".  It also is
> more prone to unthinking abuse since it is persistent.  So perhaps
> it is indeed not worth it.  (That's why I suggested an environment
> variable...something you could specify on the command line for a one-off.)

I'll be fine with not adding any hooks at all, and letting people
configure their application code correctly :-)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Wed, 3 Sep 2014 20:34:32 +1000
Nick Coghlan  wrote:
> 
> The backwards compatibility argument only applies to Python 2 maintenance
> releases (where dreid indicated an intention to request backporting the
> change), and there I'm quite happy to take the position of "use requests,
> Twisted or Python 3.5+ to get HTTPS done right".

or asyncio.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 3 Sep 2014 18:28, "Cory Benfield"  wrote:

> This is definitely true, and this change is both. The only question
> that matters is whether we believe we're doing users a service by
> breaking their code. I'd argue, along with Glyph, Alex and Donald,
> that we are. I've been on the losing side of this debate a number of
> times though, and I expect I will be again.

The default stdlib behaviour will change in 3.5, I don't think anyone is
disputing that. While I earlier said that should depend on the sslcustomize
PEP, I now think they should be made orthogonal so the SSL customisation
PEP can focus on its potential for *increasing* security in properly
configured environments rather than deliberately decreasing it after
upgrading to Python 3.5 in improperly configured ones.

The backwards compatibility argument only applies to Python 2 maintenance
releases (where dreid indicated an intention to request backporting the
change), and there I'm quite happy to take the position of "use requests,
Twisted or Python 3.5+ to get HTTPS done right".

There are a variety of reasons not to use the Python 2 stdlib for modern
networking, and making better tools more readily accessible to Python 2
users by backporting ensurepip is my preferred answer.

Regards,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Cory Benfield]

On 3 September 2014 01:19, Antoine Pitrou  wrote:
> Keeping saying it doesn't make it magically true.

Sure, but it *is* true, at the very least for HTTP.

RFC 2818 (HTTP over TLS) has the following language in section 3.1:

> If the hostname is available, the client MUST check it against the
> server's identity as presented in the server's Certificate message,
> in order to prevent man-in-the-middle attacks.
>
> If the client has external information as to the expected identity of
> the server, the hostname check MAY be omitted.

The default behaviour of httplib is in contravention of the normative
language of this specification and is therefore bugged. (For those
unclear about the relevance of RFC 2818, it's normatively referenced
by RFC 7230, which is the HTTP/1.1 spec.)

This should silence the debate about whether or not httplib's
behaviour is a bug or not.

> Besides, it can perfectly well be a bug fix *as well as* a break in
> backwards compatibility.

This is definitely true, and this change is both. The only question
that matters is whether we believe we're doing users a service by
breaking their code. I'd argue, along with Glyph, Alex and Donald,
that we are. I've been on the losing side of this debate a number of
times though, and I expect I will be again.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Stephen J. Turnbull]

Antoine Pitrou writes:
 > On Tue, 2 Sep 2014 16:47:35 -0700
 > Glyph Lefkowitz  wrote:

 > > As we keep saying, this is not a break in backwards
 > > compatibility, it's a bug fix.
 > 
 > Keeping saying it doesn't make it magically true.

It's not "magically" true, it is "just" true.  What the hardliners
fail to acknowledge is that this is *not a bug in Python, it's a bug
in the whole system*, and *mostly* in the environment.  Changing
Python will not change the environment, and applications will fail,
with unknown consequences.  Saying they "should" fail *right* now is
bogus when you don't even know what those applications are, or what
other security measures may be in place:

Now is better than never.
Although never is often better than *right* now.

On the other hand, I commend the Twisted developers for putting their
values into their code with their reputation on the line.  I hope they
win big with this move!  Shouldn't we all hope for that?

Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Tue, 02 Sep 2014 20:59:54 -0400, Terry Reedy  wrote:
> On 9/2/2014 7:47 PM, Glyph Lefkowitz wrote:
> >
> > On Sep 2, 2014, at 4:28 PM, Nick Coghlan  > > wrote:
> >
> >> On 3 Sep 2014 09:08, "David Reid"  >> > wrote:
> 
> >> > Clearly this change should be backported to Python2.
> >>
> >> Proposing to break backwards compatibility in a maintenance release (...)
> 
> For code that depends on a bug, fixing the bug breaks compatibility 
> between releases without and with the fix.  Nothing new.  We 
> occasionally don't backpart disruptive bugfixes because of that. We also 
> backport some security fixes to otherwise unmaintained versions.
> 
> > As we keep saying, this is not a break in backwards compatibility, it's
> > a bug fix.
> 
> Disagreement on whether changing the default for certificate validation 
> is a 'bugfix' or 'enhancement' (and in the colloquial or python tracker 
> sense) the seems like the crux of the disagreement here.  The case for 
> 'bug', at least in the colloquial meaning, is at least plausible. On the 
> other hand, the tracker meaning of 'behavior' issue (contrary to 
> intention and doc) is more strict in including implementation bugs but 
> not necessarily design bugs.

Up to this point the suggestion has been to change the default only in
3.5, and what we are arguing about is what kind of and how much aid to
give to those for whom 3.5 breaks backward compatibility.  So arguing
about feature versus bug is irrelevant.

The top proposal so far is an sslcustomize.py file that could be used to
either decrease or increase the default security.  This is a much less
handy solution than application options (eg, curl, wget) that allow
disabling security for "this cert" or "this CLI session".  It also is
more prone to unthinking abuse since it is persistent.  So perhaps
it is indeed not worth it.  (That's why I suggested an environment
variable...something you could specify on the command line for a one-off.)

The more I think about it the less I like sslcustomize.  I think I'd
rather do without the ability to turn off checking without modifying
code rather than introduce an intentional vector for a system-wide
decrease in security.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Terry Reedy]

On 9/2/2014 7:47 PM, Glyph Lefkowitz wrote:


On Sep 2, 2014, at 4:28 PM, Nick Coghlan mailto:ncogh...@gmail.com>> wrote:


On 3 Sep 2014 09:08, "David Reid" mailto:dr...@dreid.org>> wrote:



> Clearly this change should be backported to Python2.

Proposing to break backwards compatibility in a maintenance release (...)


For code that depends on a bug, fixing the bug breaks compatibility 
between releases without and with the fix.  Nothing new.  We 
occasionally don't backpart disruptive bugfixes because of that. We also 
backport some security fixes to otherwise unmaintained versions.



As we keep saying, this is not a break in backwards compatibility, it's
a bug fix.


Disagreement on whether changing the default for certificate validation 
is a 'bugfix' or 'enhancement' (and in the colloquial or python tracker 
sense) the seems like the crux of the disagreement here.  The case for 
'bug', at least in the colloquial meaning, is at least plausible. On the 
other hand, the tracker meaning of 'behavior' issue (contrary to 
intention and doc) is more strict in including implementation bugs but 
not necessarily design bugs.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Tue, 2 Sep 2014 16:47:35 -0700
Glyph Lefkowitz  wrote:
> 
> On Sep 2, 2014, at 4:28 PM, Nick Coghlan  wrote:
> 
> > On 3 Sep 2014 09:08, "David Reid"  wrote:
> > >
> > > Nick Coghlan  gmail.com> writes:
> > >
> > > > Creating *new* incompatibilities between Python 2 & Python 3 is a major 
> > > > point
> > > > of concern.
> > >
> > > Clearly this change should be backported to Python2.
> > 
> > Proposing to break backwards compatibility in a maintenance release (...)
> > 
> 
> As we keep saying, this is not a break in backwards compatibility, it's a bug 
> fix.

Keeping saying it doesn't make it magically true.
Besides, it can perfectly well be a bug fix *as well as* a break in
backwards compatibility. Which is why we sometimes choose to fix bugs
only in the feature development branch.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Donald Stufft]

> On Sep 2, 2014, at 7:47 PM, Glyph Lefkowitz  wrote:
> 
> 
> On Sep 2, 2014, at 4:28 PM, Nick Coghlan  > wrote:
> 
>> On 3 Sep 2014 09:08, "David Reid" mailto:dr...@dreid.org>> 
>> wrote:
>> >
>> > Nick Coghlan  gmail.com > writes:
>> >
>> > > Creating *new* incompatibilities between Python 2 & Python 3 is a major 
>> > > point
>> > > of concern.
>> >
>> > Clearly this change should be backported to Python2.
>> 
>> Proposing to break backwards compatibility in a maintenance release (...)
>> 
> 
> As we keep saying, this is not a break in backwards compatibility, it's a bug 
> fix.  Yes, systems might break, but that breakage represents an increase in 
> security which may well be operationally important.  Not everyone with a 
> "working" application has the relevant understanding an expertise to know 
> that Python's HTTP client is exposing them to surveillance.  These 
> applications should break. That is the very nature of the fix.  It is not a 
> "compatibility break" that the system starts correctly rejecting invalid 
> connections.
> 
> By way of analogy, here's another kind of breach in security: an arbitrary 
> remote code execution vulnerability in XML-RPC.  I think we all agree that 
> any 0day RCE vulnerabilities in Python really ought to be fixed and could be 
> legitimately included without worrying about backwards compatibility breaks.  
> (At least... gosh, I hope so.)
> 
> Perhaps this arbitrary remote execution looks harmless; the use of an eval() 
> instead of an int() someplace.  Perhaps someone discovered that they can do 
> "3 + 4" in their XML-RPC and the server does the computation for them.  
> Great!  They start relying on this in their applications to use symbolic 
> values in their requests instead of having explicit enumerations.  This can 
> save you quite a bit of code!
> 
> When the RCE is fixed, this application will break, and that's fine.  In fact 
> that's the whole point of issuing the fix, that people will no longer be able 
> to make arbitrary computation requests of your server any more.  If that 
> server's maintainer has the relevant context and actually wants the XML-RPC 
> endpoint to enable arbitrary RCE, they can easily modify their application to 
> start doing eval() on the data that they received, just as someone can easily 
> modify their application to intentionally disable all connection security.  
> (Let's stop calling it "certificate verification" because that sounds like 
> some kind of clerical detail: if you disable certificate verification, TLS 
> connections are unauthenticated and unidentified and therefore insecure.)
> 
> For what it's worth, on the equivalent Twisted change, I originally had just 
> these concerns, but my mind was changed when I considered what exactly the 
> user-interface ramifications were for people typing that 's' for 'secure' in 
> URLs.  I was convinced, and we made the change, and there have been no ill 
> effects that I'm aware of as a result.  In fact, there has been a renewed 
> interest in Twisted for HTTP client work, because we finally made security 
> work more or less like it's supposed to, and the standard library is so 
> broken.
> 
> I care about the health of the broader Python community, so I will 
> passionately argue that this change should be made, but for me personally 
> it's a lot easier to justify that everyone should use Twisted (at least since 
> 14+) because transport security in the stdlib is such a wreck and even if it 
> gets fixed it's going to have easy options to turn it off unilaterally so 
> your application can never really be sure if it's getting transport security 
> when it's requesting transport security.
> 

I completely agree with everything Glyph has said in this post. (To the shock 
of everyone involved I’m sure!).

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Glyph Lefkowitz]

On Sep 2, 2014, at 4:28 PM, Nick Coghlan  wrote:

> On 3 Sep 2014 09:08, "David Reid"  wrote:
> >
> > Nick Coghlan  gmail.com> writes:
> >
> > > Creating *new* incompatibilities between Python 2 & Python 3 is a major 
> > > point
> > > of concern.
> >
> > Clearly this change should be backported to Python2.
> 
> Proposing to break backwards compatibility in a maintenance release (...)
> 

As we keep saying, this is not a break in backwards compatibility, it's a bug 
fix.  Yes, systems might break, but that breakage represents an increase in 
security which may well be operationally important.  Not everyone with a 
"working" application has the relevant understanding an expertise to know that 
Python's HTTP client is exposing them to surveillance.  These applications 
should break. That is the very nature of the fix.  It is not a "compatibility 
break" that the system starts correctly rejecting invalid connections.

By way of analogy, here's another kind of breach in security: an arbitrary 
remote code execution vulnerability in XML-RPC.  I think we all agree that any 
0day RCE vulnerabilities in Python really ought to be fixed and could be 
legitimately included without worrying about backwards compatibility breaks.  
(At least... gosh, I hope so.)

Perhaps this arbitrary remote execution looks harmless; the use of an eval() 
instead of an int() someplace.  Perhaps someone discovered that they can do "3 
+ 4" in their XML-RPC and the server does the computation for them.  Great!  
They start relying on this in their applications to use symbolic values in 
their requests instead of having explicit enumerations.  This can save you 
quite a bit of code!

When the RCE is fixed, this application will break, and that's fine.  In fact 
that's the whole point of issuing the fix, that people will no longer be able 
to make arbitrary computation requests of your server any more.  If that 
server's maintainer has the relevant context and actually wants the XML-RPC 
endpoint to enable arbitrary RCE, they can easily modify their application to 
start doing eval() on the data that they received, just as someone can easily 
modify their application to intentionally disable all connection security.  
(Let's stop calling it "certificate verification" because that sounds like some 
kind of clerical detail: if you disable certificate verification, TLS 
connections are unauthenticated and unidentified and therefore insecure.)

For what it's worth, on the equivalent Twisted change, I originally had just 
these concerns, but my mind was changed when I considered what exactly the 
user-interface ramifications were for people typing that 's' for 'secure' in 
URLs.  I was convinced, and we made the change, and there have been no ill 
effects that I'm aware of as a result.  In fact, there has been a renewed 
interest in Twisted for HTTP client work, because we finally made security work 
more or less like it's supposed to, and the standard library is so broken.

I care about the health of the broader Python community, so I will passionately 
argue that this change should be made, but for me personally it's a lot easier 
to justify that everyone should use Twisted (at least since 14+) because 
transport security in the stdlib is such a wreck and even if it gets fixed it's 
going to have easy options to turn it off unilaterally so your application can 
never really be sure if it's getting transport security when it's requesting 
transport security.

-glyph

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 3 Sep 2014 09:08, "David Reid"  wrote:
>
> Nick Coghlan  gmail.com> writes:
>
> > Creating *new* incompatibilities between Python 2 & Python 3 is a major
point
> > of concern.
>
> Clearly this change should be backported to Python2.

Proposing to break backwards compatibility in a maintenance release would
require an *even better* migration story for handling legacy
infrastructure. There's a reason the Py3 compatibility break was
accompanied by a long term support commitment for Python 2 (and why that
commitment was extended another 5 years when it became clear the
implications of the binary and text handling were more significant than we
originally realised).

It's important to remember that the tech industry in general, and open
source focused companies in particular, tend to be amongst the best of the
best when it comes to infrastructure management (and we're still pretty
terrible at it).

The vast majority of infrastructure in the world *isn't* in the hands of
those companies though - it's in the hands of companies where having any
kind of computing infrastructure at all is merely a cost of doing business.

Alex's "ridiculous" scenario doesn't sound ridiculous to me - it sounds
like a typical corporate computing environment.

Regards,
Nick.

>
> -David
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Glyph Lefkowitz]

On Sep 2, 2014, at 4:01 PM, Nick Coghlan  wrote:

> 
> On 3 Sep 2014 08:18, "Alex Gaynor"  wrote:
> >
> > Antoine Pitrou  pitrou.net> writes:
> >
> > >
> > > And how many people are using Twisted as an HTTPS client?
> > > (compared to e.g. Python's httplib, and all the third-party libraries
> > > building on it?)
> > >
> >
> > I don't think anyone could give an honest estimate of these counts, however
> > there's two factors to bare in mind: a) It's extremely strongly recommended 
> > to
> > use requests to make any HTTP requests precisely because httplib is 
> > negligent
> > in certificate and hostname checking by default, b) We're talking about
> > Python3, which has fewer users than Python2.
> 
> Creating *new* incompatibilities between Python 2 & Python 3 is a major point 
> of concern. One key focus of 3.5 is *reducing* barriers to migration, and 
> this PEP would be raising a new one.
> 
No.  Providing the security that the user originally asked for is not a 
"backwards incompatible change".  It is a bug fix.  And believe me: I care a 
_LOT_ about reducing barriers to migration.  This would not be on my list of 
the top 1000 things that make migration difficult.
> It's a change worth making, but we have time to ensure there are easy ways to 
> do things like skipping cert validation, or tolerate expired certificates.
> 

The API already supports both of these things.  What I believe you're 
implicitly saying is that there needs to be a way to do this without editing 
code, and... no, there really doesn't.  Not to mention the fact that you could 
already craft a horrific monkeypatch to allow operators to cause the ssl module 
to malfunction by 'pip install'ing a separate package, which is about as 
supported as this should be.

-glyph

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Tue, 02 Sep 2014 22:16:18 -, Alex Gaynor  wrote:
> This whole scenario seems to be predicated on a siutation where: You have a
> peer whose certificate you can't change, and you have a piece of code you 
> can't
> change, and you're going to upgrade your Python installation, and you want to
> talk to this peer, and you need to use an encrypted channel, but don't really
> care if it's being MITM'd. It doesn't seem to me that this is reasonably
> Python's responsibility to deal with the fact that you have no ability to
> upgrade any of your infrastructure, except your Python version.

I would say that is an excellent summary, except that you are sometimes
*forced* to use an encrypted channel (the device only opens the https port).
That is, in the real messy world of network devices, accepting an
invalid cert is not a nonsensical operation.  (Of course, what I really
want is to be able to say "I know this cert has invalid fields, accept
it anyway, but warn me if it changes when I connect again".)

It's a good point that the actual number of people who will be hit by
this and can't hack the code in question will be (possibly vanishingly)
small, especially if you only consider python3.  But we've already had
one call for a backport :)

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[David Reid]

Nick Coghlan  gmail.com> writes:

> Creating *new* incompatibilities between Python 2 & Python 3 is a major point
> of concern. 

Clearly this change should be backported to Python2.

-David


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 3 Sep 2014 08:18, "Alex Gaynor"  wrote:
>
> Antoine Pitrou  pitrou.net> writes:
>
> >
> > And how many people are using Twisted as an HTTPS client?
> > (compared to e.g. Python's httplib, and all the third-party libraries
> > building on it?)
> >
>
> I don't think anyone could give an honest estimate of these counts,
however
> there's two factors to bare in mind: a) It's extremely strongly
recommended to
> use requests to make any HTTP requests precisely because httplib is
negligent
> in certificate and hostname checking by default, b) We're talking about
> Python3, which has fewer users than Python2.

Creating *new* incompatibilities between Python 2 & Python 3 is a major
point of concern. One key focus of 3.5 is *reducing* barriers to migration,
and this PEP would be raising a new one.

It's a change worth making, but we have time to ensure there are easy ways
to do things like skipping cert validation, or tolerate expired
certificates.

Regards,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 02.09.2014 23:32, Antoine Pitrou wrote:
>> Furthermore, "disable verification" is a nonsensical thing to do with TLS.
> 
> It's not. For example, if you have an expired cert, all you can do
> AFAIK is to disable verification.

It's possible to ignore or just warn about expired certs with simple
verify callback. The callback looks like this:

int verify_callback(int ok, X509_STORE_CTX *ctx)
{
if (X509_STORE_CTX_get_error(ctx) == X509_V_ERR_CERT_HAS_EXPIRED)
return 1;
return ok;
}

It's installed like this:

PySSLContext *self;
X509_STORE *store = SSL_CTX_get_cert_store(self->ctx);
X509_STORE_set_verify_cb(store, verify_callback);

The X509_STORE_CTX struct is created when a certificate chain is
verified. It holds all sorts of states like chain, leaf cert, current
cert that is tested, validation depth, error flags and more. In order to
write useful verify callbacks me or somebody else has to write a
X509_STORE_CTX type and X509 cert type. It's something I want to do for
more than a year but I don't find any spare time. :(

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Tue, 2 Sep 2014 22:16:18 + (UTC)
Alex Gaynor  wrote:
> > > Furthermore, "disable verification" is a nonsensical thing to do with TLS.
> > 
> > It's not. For example, if you have an expired cert, all you can do
> > AFAIK is to disable verification.
> 
> It really is a nonsensical operation, accepting any random TLS certificate
> without pinning or use of a certificate authorities makes a valid connection
> completely indistinguishable from a MITM attack.

It's still distinguishable from a passively-eavesdroppable clear-text
connection, though.

> This whole scenario seems to be predicated on a siutation where: You have a
> peer whose certificate you can't change, and you have a piece of code you 
> can't
> change, and you're going to upgrade your Python installation, and you want to
> talk to this peer, and you need to use an encrypted channel, but don't really
> care if it's being MITM'd. It doesn't seem to me that this is reasonably
> Python's responsibility to deal with the fact that you have no ability to
> upgrade any of your infrastructure, except your Python version.

Oh, I agree that whoever upgrades their Python version should be able
to fix any of their applications should they start failing. That's why
I don't want to let new command-line options and environment variables
proliferate in the name of damage control.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Antoine Pitrou  pitrou.net> writes:

> 
> And how many people are using Twisted as an HTTPS client?
> (compared to e.g. Python's httplib, and all the third-party libraries
> building on it?)
> 

I don't think anyone could give an honest estimate of these counts, however
there's two factors to bare in mind: a) It's extremely strongly recommended to
use requests to make any HTTP requests precisely because httplib is negligent
in certificate and hostname checking by default, b) We're talking about
Python3, which has fewer users than Python2.

> > Furthermore, "disable verification" is a nonsensical thing to do with TLS.
> 
> It's not. For example, if you have an expired cert, all you can do
> AFAIK is to disable verification.
> 

It really is a nonsensical operation, accepting any random TLS certificate
without pinning or use of a certificate authorities makes a valid connection
completely indistinguishable from a MITM attack.

If I were the emperor of the universe (or even just Python ;-)) I wouldn't
allow this operation at all, however, I'm not and you can still disable any and
all verification. It just requires you to pass a different argument, which
doesn't seem overly burdensome.

This whole scenario seems to be predicated on a siutation where: You have a
peer whose certificate you can't change, and you have a piece of code you can't
change, and you're going to upgrade your Python installation, and you want to
talk to this peer, and you need to use an encrypted channel, but don't really
care if it's being MITM'd. It doesn't seem to me that this is reasonably
Python's responsibility to deal with the fact that you have no ability to
upgrade any of your infrastructure, except your Python version.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 1 Sep 2014 16:05, "Nick Coghlan"  wrote:
>
> The final change would be to seed the context factory map
> appropriately for the standard library modules where we wanted to keep
> the *old* default:
>
> for modname in ("nntplib", "poplib", "imaplib", "ftplib",
> "smtplib", "asyncio.selector_events", "urllib.request",
> "http.client"):
> named_contexts[modname] = create_legacy_context
>
> The list I have above is for *all* current uses of
> "sss._create_stdlib_context". The backwards incompatible part of PEP
> 476 would then just be about removing names from that list (currently
> just "http.client", but I'd suggest "asyncio.selector_events" as
> another candidate, taking advantage of asyncio's provisional API
> status).

Update on this: Christian & I both like the SSL named context and
customisation idea independently of the proposal to change the default
behaviour, so we're going to write it up as a separate PEP.

Changing the default behaviour would then be reduced to a proposal to take
a couple of module names off the "uses legacy SSL settings" list.

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Tue, 2 Sep 2014 14:00:02 -0700
Glyph Lefkowitz  wrote:
> 
> I would strongly recommend against such a mechanism.
> 
> For what it's worth, Twisted simply unconditionally started verifying 
> certificates in 14.0 with no "disable" switch, and (to my knowledge) 
> literally no users have complained.

And how many people are using Twisted as an HTTPS client?
(compared to e.g. Python's httplib, and all the third-party libraries
building on it?)

> Furthermore, "disable verification" is a nonsensical thing to do with TLS.

It's not. For example, if you have an expired cert, all you can do
AFAIK is to disable verification.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Glyph Lefkowitz]

On Aug 29, 2014, at 7:44 PM, Alex Gaynor  wrote:
>  Disabling verification entirely externally to the program, through a CLI flag
>  or environment variable. I'm pretty down on this idea, the problem you hit is
>  that it's a pretty blunt instrument to swing, and it's almost impossible to
>  imagine it not hitting things it shouldn't; it's far too likely to be used in
>  applications that make two sets of outbound connections: 1) to some internal
>  service which you want to disable verification on, and 2) some external
>  service which needs strong validation. A global flag causes the latter to
>  fail silently when subjected to a MITM attack, and that's exactly what we're
>  trying to avoid. It also makes things much harder for library authors: I
>  write an API client for some API, and make TLS connections to it. I want
>  those to be verified by default. I can't even rely on the httplib defaults,
>  because someone might disable them from the outside.


I would strongly recommend against such a mechanism.

For what it's worth, Twisted simply unconditionally started verifying 
certificates in 14.0 with no "disable" switch, and (to my knowledge) literally 
no users have complained.

Twisted has a very, very strict backwards compatibility policy.  For example, I 
once refused to accept the deletion of a class that raised an exception upon 
construction, on the grounds that someone might have been inadvertently 
importing that class, and they shouldn't see an exception until they've seen a 
deprecation for one release.

Despite that, we classified failing to verify certificates as a security bug, 
and fixed it with no deprecation period.  When users type the 's' after the 'p' 
and before the ':' in a URL, they implicitly expect browser-like certificate 
verification.

The lack of complaints is despite the fact that 14.0 has been out for several 
months now, and, thanks to the aforementioned strict policy, users tend to 
upgrade fairly often (since they know they can almost always do so without fear 
of application-breaking consequences).  According to PyPI metadata, 14.0.0 has 
had 273283 downloads so far.

Furthermore, "disable verification" is a nonsensical thing to do with TLS.  
"select a trust root" is a valid configuration option, and OpenSSL already 
provides it via the SSL_CERT_DIR environment variable, so there's no need for 
Python to provide anything beyond that.

-glyph

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 2 Sep 2014 03:08, "Donald Stufft"  wrote:
>
>
>> On Sep 1, 2014, at 1:01 PM, Christian Heimes 
wrote:
>>
>> On 01.09.2014 17:35, Nick Coghlan wrote:
>>>
>>> Oh, now I get what you mean - yes, sitecustomize already poses the same
>>> kind of problem as the proposed sslcustomize (hence the existence of the
>>> related command line options).
>>
>>
>> If an attacker is able to place a module like sitecustomize.py in an
>> import directory or any .pth file in a site-packages directory than this
>> Python installation is compromised. .pth files are insidious because
>> they are always loaded and their code is always executed. I don't see
>> how sslcustomize is going to make a difference here.
>>
>
> Right, this is the point I was trying to make. If you’ve installed a
malicious
> package it’s game over. There’s nothing Python can do to help you.

Yes, that's what I said originally when pointing out that isolated mode and
the switch to disable site module processing would need to disable
sslcustomize processing as well.

Antoine was replying to a side comment about it being tricky to shadow
stdlib modules. I left out the qualifier "directly" in my original comment,
and he left out "indirectly through sitecustomize" in his initial reply, so
we were talking past each for a while.

Cheers,
Nick.

>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Donald Stufft]

> On Sep 1, 2014, at 1:01 PM, Christian Heimes  wrote:
> 
> On 01.09.2014 17:35, Nick Coghlan wrote:
>> Oh, now I get what you mean - yes, sitecustomize already poses the same
>> kind of problem as the proposed sslcustomize (hence the existence of the
>> related command line options).
> 
> If an attacker is able to place a module like sitecustomize.py in an
> import directory or any .pth file in a site-packages directory than this
> Python installation is compromised. .pth files are insidious because
> they are always loaded and their code is always executed. I don't see
> how sslcustomize is going to make a difference here.
> 

Right, this is the point I was trying to make. If you’ve installed a malicious
package it’s game over. There’s nothing Python can do to help you.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 01.09.2014 17:35, Nick Coghlan wrote:
> Oh, now I get what you mean - yes, sitecustomize already poses the same
> kind of problem as the proposed sslcustomize (hence the existence of the
> related command line options).

If an attacker is able to place a module like sitecustomize.py in an
import directory or any .pth file in a site-packages directory than this
Python installation is compromised. .pth files are insidious because
they are always loaded and their code is always executed. I don't see
how sslcustomize is going to make a difference here.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Donald Stufft]

> On Sep 1, 2014, at 11:35 AM, Nick Coghlan  wrote:
> 
> 
> On 2 Sep 2014 00:59, "Antoine Pitrou"  > wrote:
> >
> > On Tue, 2 Sep 2014 00:53:11 +1000
> > Nick Coghlan mailto:ncogh...@gmail.com>> wrote:
> > > >
> > > > To be frank I don't understand what you're arguing about.
> > >
> > > When I said "shadowing ssl can be tricky to arrange", Chris correctly
> > > interpreted it as referring to the filesystem based privilege escalation
> > > scenario that isolated mode handles, not to normal in-process
> > > monkeypatching or module injection.
> >
> > There's no actual difference. You can have a sitecustomize.py that does
> > the monkeypatching or the shadowing. There doesn't seem to be anything
> > "tricky" about that.
> 
> Oh, now I get what you mean - yes, sitecustomize already poses the same kind 
> of problem as the proposed sslcustomize (hence the existence of the related 
> command line options).
> 
> I missed that you had switched to talking about using that attack vector, 
> rather than trying to shadow stdlib modules directly through the filesystem 
> (which is the only tricky thing I was referring to).
> 
> Cheers,
> Nick.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


Or you can just install something with easy_install, or you can drop a .pth 
file and monkey patch there. You can’t stop people from overriding modules, 
it’s trivial to do. The sys.path ordering just makes it slightly less trivial.

—
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 2 Sep 2014 00:59, "Antoine Pitrou"  wrote:
>
> On Tue, 2 Sep 2014 00:53:11 +1000
> Nick Coghlan  wrote:
> > >
> > > To be frank I don't understand what you're arguing about.
> >
> > When I said "shadowing ssl can be tricky to arrange", Chris correctly
> > interpreted it as referring to the filesystem based privilege escalation
> > scenario that isolated mode handles, not to normal in-process
> > monkeypatching or module injection.
>
> There's no actual difference. You can have a sitecustomize.py that does
> the monkeypatching or the shadowing. There doesn't seem to be anything
> "tricky" about that.

Oh, now I get what you mean - yes, sitecustomize already poses the same
kind of problem as the proposed sslcustomize (hence the existence of the
related command line options).

I missed that you had switched to talking about using that attack vector,
rather than trying to shadow stdlib modules directly through the filesystem
(which is the only tricky thing I was referring to).

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Tue, 2 Sep 2014 00:53:11 +1000
Nick Coghlan  wrote:
> On 2 Sep 2014 00:08, "Antoine Pitrou"  wrote:
> >
> > On Mon, 1 Sep 2014 23:42:10 +1000
> > Chris Angelico  wrote:
> > > >>
> > > >> That has to be done inside the same process. But imagine this
> > > >> scenario: You have a program that gets invoked as root (or some other
> > > >> user than yourself), and you're trying to fiddle with what it sees.
> > > >> You don't have root access, but you can manipulate the file system,
> to
> > > >> the extent that your userid has access. What can you do to affect
> this
> > > >> other program?
> > > >
> > > > If you're root you shouldn't run untrusted code. See
> > > > https://docs.python.org/3/using/cmdline.html#cmdoption-I
> > >
> > > Right, which is why sslcustomize has to be controlled by that, but the
> > > possibility of patching (or monkeypatching) ssl.py isn't as big a
> > > deal.
> >
> > To be frank I don't understand what you're arguing about.
> 
> When I said "shadowing ssl can be tricky to arrange", Chris correctly
> interpreted it as referring to the filesystem based privilege escalation
> scenario that isolated mode handles, not to normal in-process
> monkeypatching or module injection.

There's no actual difference. You can have a sitecustomize.py that does
the monkeypatching or the shadowing. There doesn't seem to be anything
"tricky" about that.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 2 Sep 2014 00:08, "Antoine Pitrou"  wrote:
>
> On Mon, 1 Sep 2014 23:42:10 +1000
> Chris Angelico  wrote:
> > >>
> > >> That has to be done inside the same process. But imagine this
> > >> scenario: You have a program that gets invoked as root (or some other
> > >> user than yourself), and you're trying to fiddle with what it sees.
> > >> You don't have root access, but you can manipulate the file system,
to
> > >> the extent that your userid has access. What can you do to affect
this
> > >> other program?
> > >
> > > If you're root you shouldn't run untrusted code. See
> > > https://docs.python.org/3/using/cmdline.html#cmdoption-I
> >
> > Right, which is why sslcustomize has to be controlled by that, but the
> > possibility of patching (or monkeypatching) ssl.py isn't as big a
> > deal.
>
> To be frank I don't understand what you're arguing about.

When I said "shadowing ssl can be tricky to arrange", Chris correctly
interpreted it as referring to the filesystem based privilege escalation
scenario that isolated mode handles, not to normal in-process
monkeypatching or module injection. I don't consider the latter cases to be
interesting attack scenarios, as they imply the attacker is *already*
running arbitrary Python code inside your CPython process, so you've
already lost.

Cheers,
Nick.

>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Mon, 1 Sep 2014 23:42:10 +1000
Chris Angelico  wrote:
> On Mon, Sep 1, 2014 at 11:34 PM, Antoine Pitrou  wrote:
> > On Mon, 1 Sep 2014 23:24:39 +1000
> > Chris Angelico  wrote:
> >> On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou  wrote:
> >> > Not sure why. Just put another module named "ssl" in sys.modules 
> >> > directly.
> >> > You can also monkeypatch the genuine ssl module.
> >>
> >> That has to be done inside the same process. But imagine this
> >> scenario: You have a program that gets invoked as root (or some other
> >> user than yourself), and you're trying to fiddle with what it sees.
> >> You don't have root access, but you can manipulate the file system, to
> >> the extent that your userid has access. What can you do to affect this
> >> other program?
> >
> > If you're root you shouldn't run untrusted code. See
> > https://docs.python.org/3/using/cmdline.html#cmdoption-I
> 
> Right, which is why sslcustomize has to be controlled by that, but the
> possibility of patching (or monkeypatching) ssl.py isn't as big a
> deal.

To be frank I don't understand what you're arguing about.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Chris Angelico]

On Mon, Sep 1, 2014 at 11:34 PM, Antoine Pitrou  wrote:
> On Mon, 1 Sep 2014 23:24:39 +1000
> Chris Angelico  wrote:
>> On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou  wrote:
>> > Not sure why. Just put another module named "ssl" in sys.modules directly.
>> > You can also monkeypatch the genuine ssl module.
>>
>> That has to be done inside the same process. But imagine this
>> scenario: You have a program that gets invoked as root (or some other
>> user than yourself), and you're trying to fiddle with what it sees.
>> You don't have root access, but you can manipulate the file system, to
>> the extent that your userid has access. What can you do to affect this
>> other program?
>
> If you're root you shouldn't run untrusted code. See
> https://docs.python.org/3/using/cmdline.html#cmdoption-I

Right, which is why sslcustomize has to be controlled by that, but the
possibility of patching (or monkeypatching) ssl.py isn't as big a
deal.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Mon, 1 Sep 2014 23:24:39 +1000
Chris Angelico  wrote:
> On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou  wrote:
> > Not sure why. Just put another module named "ssl" in sys.modules directly.
> > You can also monkeypatch the genuine ssl module.
> 
> That has to be done inside the same process. But imagine this
> scenario: You have a program that gets invoked as root (or some other
> user than yourself), and you're trying to fiddle with what it sees.
> You don't have root access, but you can manipulate the file system, to
> the extent that your userid has access. What can you do to affect this
> other program?

If you're root you shouldn't run untrusted code. See
https://docs.python.org/3/using/cmdline.html#cmdoption-I

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Chris Angelico]

On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou  wrote:
> Not sure why. Just put another module named "ssl" in sys.modules directly.
> You can also monkeypatch the genuine ssl module.

That has to be done inside the same process. But imagine this
scenario: You have a program that gets invoked as root (or some other
user than yourself), and you're trying to fiddle with what it sees.
You don't have root access, but you can manipulate the file system, to
the extent that your userid has access. What can you do to affect this
other program?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

Le 01/09/2014 10:09, Nick Coghlan a écrit :
> On 1 September 2014 17:13, Christian Heimes  wrote:
>> On 01.09.2014 08:44, Nick Coghlan wrote:
>>> Yes, it would have exactly the same security failure modes as
>>> sitecustomize, except it would only fire if the application
>>> imported the ssl module.
>>>
>>> The "-S" and "-I" switches would need to disable the implied
>>> "sslcustomize", just as they disable "import site".
>>
>> A malicious package can already play havoc with your installation with
>> a custom ssl module. If somebody is able to sneak in a ssl.py then you
>> are screwed anyway. sslcustomize is not going to make the situation worse.
> 
> That's not quite true - we're fairly careful about putting the
> standard library before userspace directories, so aside from the
> "current directory" problem, shadowing "ssl" itself can be tricky to
> arrange.

Not sure why. Just put another module named "ssl" in sys.modules directly.
You can also monkeypatch the genuine ssl module.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 1 September 2014 17:13, Christian Heimes  wrote:
> On 01.09.2014 08:44, Nick Coghlan wrote:
>> Yes, it would have exactly the same security failure modes as
>> sitecustomize, except it would only fire if the application
>> imported the ssl module.
>>
>> The "-S" and "-I" switches would need to disable the implied
>> "sslcustomize", just as they disable "import site".
>
> A malicious package can already play havoc with your installation with
> a custom ssl module. If somebody is able to sneak in a ssl.py then you
> are screwed anyway. sslcustomize is not going to make the situation worse.

That's not quite true - we're fairly careful about putting the
standard library before userspace directories, so aside from the
"current directory" problem, shadowing "ssl" itself can be tricky to
arrange. "sslcustomize" would be more like "sitecustomize" - since it
wouldn't normally be in the standard library, it can appear anywhere
on sys.path, rather than having to be injected ahead of the standard
library.

I think that's OK though - compared to the security nightmare that is
downloading modules from PyPI and running "./setup.py install" (or,
even worse, "sudo ./setup.py install"), this would be a rather
esoteric attack vector, and the existing -S and -I mechanisms could be
used to defend against it :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 01.09.2014 08:44, Nick Coghlan wrote:
> Yes, it would have exactly the same security failure modes as 
> sitecustomize, except it would only fire if the application
> imported the ssl module.
> 
> The "-S" and "-I" switches would need to disable the implied 
> "sslcustomize", just as they disable "import site".

A malicious package can already play havoc with your installation with
a custom ssl module. If somebody is able to sneak in a ssl.py then you
are screwed anyway. sslcustomize is not going to make the situation worse.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 1 September 2014 16:07, Paul Moore  wrote:
> On 31 August 2014 23:10, Nick Coghlan  wrote:
>> Assuming sslcustomize was in site-packages rather than the standard library
>> directories, you would also be able to use virtual environments with an
>> appropriate sslcustomize module to disable cert checking even if the
>> application you were running didn't support direct configuration.
>
> Would this mean that a malicious package could install a custom
> sslcustomize.py and so add unwanted certs to the system? I guess we
> have to assume that installed packages are trusted, but I just wanted
> to be explicit.

Yes, it would have exactly the same security failure modes as
sitecustomize, except it would only fire if the application imported
the ssl module.

The "-S" and "-I" switches would need to disable the implied
"sslcustomize", just as they disable "import site".

Cheers,
Nick.



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Paul Moore]

On 31 August 2014 23:10, Nick Coghlan  wrote:
> Assuming sslcustomize was in site-packages rather than the standard library
> directories, you would also be able to use virtual environments with an
> appropriate sslcustomize module to disable cert checking even if the
> application you were running didn't support direct configuration.

Would this mean that a malicious package could install a custom
sslcustomize.py and so add unwanted certs to the system? I guess we
have to assume that installed packages are trusted, but I just wanted
to be explicit.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 1 September 2014 11:10, R. David Murray  wrote:
>
> It sounds like this would address my concerns as well (I don't really
> care *how* it is implemented as long as I don't have to touch the
> code of a third party application when I upgrade my python version to
> 3.5...remember, the context here is backward compatibility concerns).
>
> Does it address the issue of accepting an invalid cert, though?

That's actually an interesting question, as the PEP doesn't currently
propose adding any new global configuration knobs to the ssl or
httplib modules - it just proposes switching httplib from the legacy
(but fully backwards compatible) ssl._create_stdlib_context() API to
the newer (but potentially backwards incompatible in some
environments) ssl.create_default_context() API.

Having the ssl module import an sslcustomize module at the end
wouldn't be enough unless the appropriate APIs were put in place to
allow things to be configured at a process global level.

One possible way to do that would be to provide a central context
factory mapping that provide a module specific SSL context creator.
We'd seed it appropriately for the stdlib modules where we wanted to
use the legacy context definition, but it would default to using
ssl.create_default_context.

Under that kind of model, the first change we would actually make is
to make ssl._create_stdlib_context() public under a suitable name,
let's say ssl.create_legacy_context()

Independenting of any other changes, exposing
ssl.create_legacy_context() like that would also make it
straightforward for folks to opt in to the old behaviour as an interim
hack in a way that is easy to grep for and fix later (it's also
something a linter can easily disallow).

The second change would be to provide a mapping from arbitrary names
to context factories in the ssl module that defaults to
ssl.create_default_context:

named_contexts = defaultdict((lambda name: create_default_context))

(A more accurate name would be "named_context_factory", but I think
"named_contexts" reads better. Folks will learn quickly enough that it
actually stores context factories rather than prebuilt context
objects)

The third change would be to replace all calls to
"ssl._create_stdlib_context()" with calls to
"ssl.named_contexts[__name__]()" instead.

The final change would be to seed the context factory map
appropriately for the standard library modules where we wanted to keep
the *old* default:

for modname in ("nntplib", "poplib", "imaplib", "ftplib",
"smtplib", "asyncio.selector_events", "urllib.request",
"http.client"):
named_contexts[modname] = create_legacy_context

The list I have above is for *all* current uses of
"sss._create_stdlib_context". The backwards incompatible part of PEP
476 would then just be about removing names from that list (currently
just "http.client", but I'd suggest "asyncio.selector_events" as
another candidate, taking advantage of asyncio's provisional API
status).

The "revert to 3.4 behaviour" content for sslcustomize.py would then just be:

import ssl
ssl.named_contexts["http.client"] = ssl.create_legacy_context

However, someone that wanted to also enforce SSL properly for other
standard library modules could go the other way:

import ssl
for modname in ("nntplib", "poplib", "imaplib", "ftplib",
"smtplib", "urllib.request"):
   ssl.named_contexts[modname] = ssl.create_default_context

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Mon, 01 Sep 2014 08:10:58 +1000, Nick Coghlan  wrote:
> On 1 Sep 2014 07:43, "Christian Heimes"  wrote:
> >
> > On 31.08.2014 08:09, Nick Coghlan wrote:
> > > As Antoine says here, I'm also opposed to adding more Python specific
> > > configuration options. However, I think there may be something
> > > worthwhile we can do that's closer to the way browsers work, and has
> > > the significant benefit of being implementable as a PyPI module first
> > > (more on that in a separate reply).
> >
> > I'm on your and Antoine's side and strictly against any additional
> > environment variables or command line arguments. That would make the
> > whole validation process even more complex and harder to understand.
> >
> > There might be a better option to give people and companies the option
> > to tune the SSL module to their needs. Python already have a
> > customization hook for the site module called sitecustomize. How about
> > another module named sslcustomize? Such a module could be used to tune
> > the ssl module to the needs of users, e.g. configure a different default
> > context, add certificates to a default context etc.
> >
> > Companies could install them in a system global directory on their
> > servers. Users could put them in their own user site directory and even
> > each virtual env can have one sslcustomize of its own. It's fully
> > backward compatible, doesn't add any flags and developers have the full
> > power of Python for configuration and customization.
> 
> And means a user specific store (if one became available) could be
> configured there.
> 
> Yes, I think this would address my concerns, especially if combined with a
> clear recipe in the documentation on how to optionally disable cert
> validation at the application layer.
> 
> Assuming sslcustomize was in site-packages rather than the standard library
> directories, you would also be able to use virtual environments with an
> appropriate sslcustomize module to disable cert checking even if the
> application you were running didn't support direct configuration.

It sounds like this would address my concerns as well (I don't really
care *how* it is implemented as long as I don't have to touch the
code of a third party application when I upgrade my python version to
3.5...remember, the context here is backward compatibility concerns).

Does it address the issue of accepting an invalid cert, though?

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 1 Sep 2014 08:15, "Donald Stufft"  wrote:
>
>
>> On Aug 31, 2014, at 5:43 PM, Christian Heimes 
wrote:
>>
>> Companies could install them in a system global directory on their
>> servers. Users could put them in their own user site directory and even
>> each virtual env can have one sslcustomize of its own. It's fully
>> backward compatible, doesn't add any flags and developers have the full
>> power of Python for configuration and customization.
>
> This may be a dumb question, but why can’t sitecustomize do this already?

It can. The advantage of a separate file is that it won't conflict with
existing sitecustomize modules, so (for example) redistributors can add a
default sslcustomize, and you can add one to your virtual environments that
are integrated with the system Python environment without needing to worry
about whether or not there's a global sitecustomize (you'd only have
trouble if there was a global sslcustomize).

Cheers,
Nick.

>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Donald Stufft]

> On Aug 31, 2014, at 5:43 PM, Christian Heimes  wrote:
> 
> On 31.08.2014 08:09, Nick Coghlan wrote:
>> As Antoine says here, I'm also opposed to adding more Python specific
>> configuration options. However, I think there may be something
>> worthwhile we can do that's closer to the way browsers work, and has
>> the significant benefit of being implementable as a PyPI module first
>> (more on that in a separate reply).
> 
> I'm on your and Antoine's side and strictly against any additional
> environment variables or command line arguments. That would make the
> whole validation process even more complex and harder to understand.
> 
> There might be a better option to give people and companies the option
> to tune the SSL module to their needs. Python already have a
> customization hook for the site module called sitecustomize. How about
> another module named sslcustomize? Such a module could be used to tune
> the ssl module to the needs of users, e.g. configure a different default
> context, add certificates to a default context etc.
> 
> Companies could install them in a system global directory on their
> servers. Users could put them in their own user site directory and even
> each virtual env can have one sslcustomize of its own. It's fully
> backward compatible, doesn't add any flags and developers have the full
> power of Python for configuration and customization.
> 
> 

This may be a dumb question, but why can’t sitecustomize do this already?

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 1 Sep 2014 07:43, "Christian Heimes"  wrote:
>
> On 31.08.2014 08:09, Nick Coghlan wrote:
> > As Antoine says here, I'm also opposed to adding more Python specific
> > configuration options. However, I think there may be something
> > worthwhile we can do that's closer to the way browsers work, and has
> > the significant benefit of being implementable as a PyPI module first
> > (more on that in a separate reply).
>
> I'm on your and Antoine's side and strictly against any additional
> environment variables or command line arguments. That would make the
> whole validation process even more complex and harder to understand.
>
> There might be a better option to give people and companies the option
> to tune the SSL module to their needs. Python already have a
> customization hook for the site module called sitecustomize. How about
> another module named sslcustomize? Such a module could be used to tune
> the ssl module to the needs of users, e.g. configure a different default
> context, add certificates to a default context etc.
>
> Companies could install them in a system global directory on their
> servers. Users could put them in their own user site directory and even
> each virtual env can have one sslcustomize of its own. It's fully
> backward compatible, doesn't add any flags and developers have the full
> power of Python for configuration and customization.

And means a user specific store (if one became available) could be
configured there.

Yes, I think this would address my concerns, especially if combined with a
clear recipe in the documentation on how to optionally disable cert
validation at the application layer.

Assuming sslcustomize was in site-packages rather than the standard library
directories, you would also be able to use virtual environments with an
appropriate sslcustomize module to disable cert checking even if the
application you were running didn't support direct configuration.

Cheers,
Nick.

>
> Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 31.08.2014 22:30, Paul Moore wrote:
> On 31 August 2014 21:15, Antoine Pitrou  wrote:
>> What do you call your local cert store?
> 
> I was referring to Christian's comment
>> It's very simple to trust a self-signed certificate: just download it and 
>> stuff it into the trust store.

I was referring to the the trust store of the SSLContext object and not
to any kind of cert store of the operating system. Sorry for the confusion.


> a) Is there really no OS-level personal trust store? I'm thinking of
> Windows here for my own personal use, but the same question applies
> elsewhere.

Windows and OSX have superior cert stores compared to Linux and BSD.
They have means for user and system wide cert stores and trust settings
Linux just have one central directory or file with all trusted certs. My
KDE has some options to disable certs but I don't know how to make use
of the configuration.

Even worse: Linux distros doesn't make a different between purposes. On
Windows a user can trust a certificate for S/MIME but not for server
auth or client auth. Ubuntu just puts all certification in one directory
but it's wrong. :(

https://bugs.launchpad.net/ubuntu/+source/ca-certificates/+bug/1207004

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

Le 31/08/2014 23:41, Nick Coghlan a écrit :

Right, this is why I came to the conclusion we need to follow the browser
vendors lead here and support a per-user Python specific supplementary
certificate cache before we can start validating certs by default at the
*Python* level. There are still too many failure modes for cert management
on private networks for us to safely ignore the use case of needing to
force connections to services with invalid certs.


We are not ignoring that use case. The proper solution is simply to 
disable cert validation in the application code (or, for more 
sophisticated needs, provide an application configuration setting for 
cert validation).


> In the meantime, we can update the security considerations for the ssl
> module to make it clearer that the defaults are set up for trusted 
networks
> and that using it safely on the public internet may mean you're 
better off

> with a third party library like requests or Twisted.

No, you simply have to select the proper validation settings.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 31.08.2014 08:09, Nick Coghlan wrote:
> As Antoine says here, I'm also opposed to adding more Python specific
> configuration options. However, I think there may be something
> worthwhile we can do that's closer to the way browsers work, and has
> the significant benefit of being implementable as a PyPI module first
> (more on that in a separate reply).

I'm on your and Antoine's side and strictly against any additional
environment variables or command line arguments. That would make the
whole validation process even more complex and harder to understand.

There might be a better option to give people and companies the option
to tune the SSL module to their needs. Python already have a
customization hook for the site module called sitecustomize. How about
another module named sslcustomize? Such a module could be used to tune
the ssl module to the needs of users, e.g. configure a different default
context, add certificates to a default context etc.

Companies could install them in a system global directory on their
servers. Users could put them in their own user site directory and even
each virtual env can have one sslcustomize of its own. It's fully
backward compatible, doesn't add any flags and developers have the full
power of Python for configuration and customization.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 1 Sep 2014 06:32, "Paul Moore"  wrote:
>
> On 31 August 2014 21:15, Antoine Pitrou  wrote:
> > What do you call your local cert store?
>
> I was referring to Christian's comment
> > It's very simple to trust a self-signed certificate: just download it
and stuff it into the trust store.
>
> From his recent response, I guess he meant the system store, and he
> agrees that this is a bad option.
>
> OK, that's fair, but:
>
> a) Is there really no OS-level personal trust store? I'm thinking of
> Windows here for my own personal use, but the same question applies
> elsewhere.
> b) I doubt my confusion over Christian's response is atypical. Based
> on what he said, if we hadn't had the subsequent discussion, I would
> probably have found a way to add a cert to "the store" without
> understanding the implications. While it's not Python's job to educate
> users, it would be a shame if its default behaviour led people to make
> ill-informed decisions.

Right, this is why I came to the conclusion we need to follow the browser
vendors lead here and support a per-user Python specific supplementary
certificate cache before we can start validating certs by default at the
*Python* level. There are still too many failure modes for cert management
on private networks for us to safely ignore the use case of needing to
force connections to services with invalid certs.

We don't need to *solve* that problem here today - we can push it back to
Alex (and anyone else interested) as a building block to investigate
providing as part of cryptography.io or certi.fi, with a view to making a
standard library version of that (along with any SSL module updates) part
of PEP 476.

In the meantime, we can update the security considerations for the ssl
module to make it clearer that the defaults are set up for trusted networks
and that using it safely on the public internet may mean you're better off
with a third party library like requests or Twisted. (I'll start another
thread shortly that is highly relevant to that topic)

Regards,
Nick.

>
> Maybe an SSL HOWTO would be a useful addition to the docs, if anyone
> feels motivated to write one.
>
> Regardless, thanks for the education!
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Paul Moore]

On 31 August 2014 21:15, Antoine Pitrou  wrote:
> What do you call your local cert store?

I was referring to Christian's comment
> It's very simple to trust a self-signed certificate: just download it and 
> stuff it into the trust store.

>From his recent response, I guess he meant the system store, and he
agrees that this is a bad option.

OK, that's fair, but:

a) Is there really no OS-level personal trust store? I'm thinking of
Windows here for my own personal use, but the same question applies
elsewhere.
b) I doubt my confusion over Christian's response is atypical. Based
on what he said, if we hadn't had the subsequent discussion, I would
probably have found a way to add a cert to "the store" without
understanding the implications. While it's not Python's job to educate
users, it would be a shame if its default behaviour led people to make
ill-informed decisions.

Maybe an SSL HOWTO would be a useful addition to the docs, if anyone
feels motivated to write one.

Regardless, thanks for the education!

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 31.08.2014 19:29, Antoine Pitrou wrote:
> You certainly shouldn't do so. If an application has special needs that
> require trusting a self-signed certificate, then it should expose a
> configuration setting to let users specify the cert's location. Stuffing
> self-signed certs into the system trust store is really a measure of
> last resort.

Correct!

I merely wanted to state that OpenSSL can verify a self-signed
certificate easily. The certificate 'just' have to be added to the
SSLContext's store of trusted root certs. Somebody has to figure out how
Python can accomplish the task.

> There's another case which isn't solved by this, though, which is when a
> cert is invalid. The common situation being that it has expired
> (renewing certs is a PITA and therefore expired certs are more common
> than it sounds they should be). In this case, there is no way to
> whitelist it: you have to disable certificate checking altogether. This
> can be exposed by the application as configuration option if necessary,
> as well.

It's possible to ignore errors with a verify callback. OpenSSL's wiki
has an example for the expired certs
http://wiki.openssl.org/index.php/Manual:X509_STORE_CTX_set_verify_cb%283%29#EXAMPLES

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

Le 31/08/2014 21:12, Paul Moore a écrit :

On 31 August 2014 19:37, Antoine Pitrou  wrote:

Well, it's certainly pip's responsibility more than Python's. What would
Python do? Provide a setting that would blindly add a cert for all uses of
httplib?


That's more or less my point, pip doesn't have that much better idea
than Python. I was talking about putting the cert in my local cert
store, so that *I* can decide, and applications don't need to take
special care to allow me to handle this case.


What do you call your local cert store?

If you mean the system cert store, then that will affect all users.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Paul Moore]

On 31 August 2014 19:37, Antoine Pitrou  wrote:
> Well, it's certainly pip's responsibility more than Python's. What would
> Python do? Provide a setting that would blindly add a cert for all uses of
> httplib?

That's more or less my point, pip doesn't have that much better idea
than Python. I was talking about putting the cert in my local cert
store, so that *I* can decide, and applications don't need to take
special care to allow me to handle this case. You said that doing so
was bad, but I don't see why. It seems to me that you're saying that I
should raise a feature request for pip instead, which seems
unreasonable. Am I missing something?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

Le 31/08/2014 20:28, Paul Moore a écrit :


I can't see how that would be something the application would know.
For example, pip allows me to specify an "alternate cert bundle" but
not a single additional cert. So IIUC, I can't use my local index that
serves https using a self-signed cert. I'd find it hard to argue that
it's pip's responsibility to think of that use case - pretty much any
program that interacts with a web service *might* need to interact
with a self-signed dummy version, if only under test conditions.


Well, it's certainly pip's responsibility more than Python's. What would 
Python do? Provide a setting that would blindly add a cert for all uses 
of httplib?


pip knows about the use cases here, Python doesn't.

(perhaps you want to serve your local index using http, though)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Paul Moore]

On 31 August 2014 18:29, Antoine Pitrou  wrote:
> If an application has special needs that require trusting a self-signed
> certificate, then it should expose a configuration setting to let users
> specify the cert's location.

I can't see how that would be something the application would know.
For example, pip allows me to specify an "alternate cert bundle" but
not a single additional cert. So IIUC, I can't use my local index that
serves https using a self-signed cert. I'd find it hard to argue that
it's pip's responsibility to think of that use case - pretty much any
program that interacts with a web service *might* need to interact
with a self-signed dummy version, if only under test conditions.

Or did you mean that Python should provide such a setting that would
cover all applications written in Python?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

Le 31/08/2014 19:03, Paul Moore a écrit :

On 31 August 2014 17:27, Christian Heimes  wrote:

It's very simple to trust a self-signed certificate: just download it
and stuff it into the trust store.


"Stuff it into the trust store" is the hard bit, though. I have
honestly no idea how to do that.


You certainly shouldn't do so. If an application has special needs that 
require trusting a self-signed certificate, then it should expose a 
configuration setting to let users specify the cert's location. Stuffing 
self-signed certs into the system trust store is really a measure of 
last resort.


There's another case which isn't solved by this, though, which is when a 
cert is invalid. The common situation being that it has expired 
(renewing certs is a PITA and therefore expired certs are more common 
than it sounds they should be). In this case, there is no way to 
whitelist it: you have to disable certificate checking altogether. This 
can be exposed by the application as configuration option if necessary, 
as well.


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 31.08.2014 08:24, Nick Coghlan wrote:
> To answer David's specific question, the existing knobs at the OpenSSL
> level (SSL_CERT_DIR and SSL_CERT_FILE ) let people add an internal CA,
> opt out of the default CA system, and trust *specific* self-signed
> certs.

This works only on Unix platforms iff SSL_CERT_DIR and SSL_CERT_FILE are
both set to a non-empty string that points to non-existing files or
something like /dev/null.

On Windows my enhancement will always cause the system trust store to
kick in. There is currently no way to disable the Windows system store
for ssl.create_default_context() and ssl._create_stdlib_context() with
the functions' default arguments.

On Mac OS X the situation is even more obscure. Apple's OpenSSL binaries
are using Apple's Trust Evaluation Agent. You have to set
OPENSSL_X509_TEA_DISABLE=1 in order to prevent the agent from adding
trusted certs from OSX key chain. Hynek Schlawack did a deep dive into
it. https://hynek.me/articles/apple-openssl-verification-surprises/


> A Python-specific user level cert store is something that could be
> developed as a PyPI library for Python 2.7.9+ and 3.4+ (Is cert
> management considered in scope for cryptography.io? If so, that could
> be a good home).

Python's SSL module is lacking some functionalities in order to
implement a fully functional cert store.

* no verify hook to verify each certificate in the chain like

https://www.openssl.org/docs/ssl/SSL_CTX_set_cert_verify_callback.html
http://linux.die.net/man/3/x509_store_ctx_set_verify_cb
/api/ssl.html#OpenSSL.SSL.Context.set_verify

* no way to get the full cert chain including the root certificate.

* no API to get the subject public key information (SPKI). The SPKI hash
can be used to identify a certificate. For example it's used in Google's
CRLSet. http://dev.chromium.org/Home/chromium-security/crlsets

* the cert validation exception could use some additional information.

There are probably some more things mising. An X509 object would help, too.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Paul Moore]

On 31 August 2014 17:27, Christian Heimes  wrote:
> It's very simple to trust a self-signed certificate: just download it
> and stuff it into the trust store.

"Stuff it into the trust store" is the hard bit, though. I have
honestly no idea how to do that. Or if it's temporary (which it likely
is) how to manage it - delete it when I no longer need it, list what
junk I've added over time, etc. And equally relevantly, no idea how to
do that in a way that won't clash with my company's policies...

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 31.08.2014 16:16, R. David Murray wrote:
> Self -signed certificates are not crazy in an internal corporate
> environment even when properly playing the defense in depth game.  Once
> you've acked the cert the first time, you will be warned if it changes
> (like an ssh host key).  Sure, as Nick says the corp could set up an
> internal signing authority and make sure everyone has their CA...and
> they *should*...but realistically, that is probably relatively rare at
> the moment, because it is not particularly easy to accomplish
> (distributing the CA everywhere it needs to go is still a Hard Problem,
> though it has gotten a lot better).

It's very simple to trust a self-signed certificate: just download it
and stuff it into the trust store. That's all. A self-signed certificate
acts as its own root CA (so to speak). But there is a downside, too. The
certificate is trusted for any and all connections. Python's SSL module
has no way to trust a specific certificate for a host.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Sun, 31 Aug 2014 16:45:42 +1000, Nick Coghlan  wrote:
> On 31 August 2014 16:16, Donald Stufft  wrote:
> >
> > On Aug 31, 2014, at 2:09 AM, Nick Coghlan  wrote:
> >
> > At the same time, we need to account for the fact that most existing
> > organisations still trust in perimeter defence for their internal
> > network security, and hence tolerate (or even actively encourage) the
> > use of unsecured connections, or skipping certificate validation,
> > internally. This is actually a really terrible idea, but it's still
> > incredibly common due to the general failure of the technology
> > industry to take usability issues seriously when we design security
> > systems (at least until recently) - doing the wrong "unsafe" thing is
> > genuinely easier than doing things right.
> >
> >
> > Just a quick clarification in order to be a little clearer, this change will
> > (obviously) only effect those who trust perimeter security *and* decided to
> > install an invalid certificate instead of just using HTTP. I'm not saying
> > that
> > this doesn't happen, just being specific (I'm not actually sure why they
> > would
> > install a TLS certificate at all if they are trusting perimeter security,
> > but
> > I'm sure folks do).
> 
> It's the end result when a company wide edict to use HTTPS isn't
> backed up by the necessary documentation and training on how to get a
> properly signed cert from your internal CA (or, even better, when such
> an edict comes down without setting up an internal CA first). Folks
> hit the internet instead, find instructions on creating a self-signed
> cert, install that, and tell their users to ignore the security
> warning and accept the cert. Historically, Python clients have "just
> worked" in environments that required a click-through on the browser
> side, since you had to opt in to checking the certificates properly.
> 
> Self-signed certificates can also be really handy for doing local
> testing - you're not really aiming to authenticate the connection in
> that case, you're just aiming to test that the secure connection
> machinery is all working properly.

Self-signed certificates are not crazy in an internal corporate
environment even when properly playing the defense in depth game.  Once
you've acked the cert the first time, you will be warned if it changes
(like an ssh host key).  Sure, as Nick says the corp could set up an
internal signing authority and make sure everyone has their CA...and
they *should*...but realistically, that is probably relatively rare at
the moment, because it is not particularly easy to accomplish
(distributing the CA everywhere it needs to go is still a Hard Problem,
though it has gotten a lot better).

Given the reality of human nature, even when the documentation
accompanying the HTTPS initiative is good, there will *still* be someone
who hasn't followed the internal rules, yet you really need to talk to
the piece of infrastructure they are maintaining.  At least that one is
short term problem (for some definition of "short" that may be several
months long), but it does exist.

In addition, as has been mentioned before, self-signed certs are often
embedded in *devices* from vendors (I'm looking at you, Cisco).  This is
another area where security conciousness has gotten better (the cert
exists) but isn't good yet (the cert is self-signed and replacing it
isn't trivial when it is even possible; and, because the self-signed cert
happens by defaultit gets left in place).  And in the case of those
embedded certs, the cert can wind up *invalid* (expired) as well as
self-signed.  (This last item is where my concern about being able
to talk to invalid certs comes from.)

And yes, I have encountered all of this in the wild.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 30.08.2014 00:22, Antoine Pitrou wrote:
> SSL_CERT_DIR and SSL_CERT_FILE are used, if set, when
> SSLContext.load_verify_locations() is called.
> 
> Actually, come to think of it, this allows us to write a better
> test for that method. Patch welcome!

The environment vars are used only when
SSLContext.set_default_verify_paths() is called. load_verify_locations()
loads certificates from a given file, directory or memory but it doesn't
look at the env vars.

create_default_context() calls SSLContext.load_default_certs() when
neither cafile, capath nor cadata is given as an argument.
SSLContext.load_default_certs() then calls
SSLContext.set_default_verify_paths(). However there is a catch:
SSLContext.set_default_verify_paths() is not called on Windows. In
retrospective it was a bad decision by me to omit the call.

http://hg.python.org/cpython/file/164a17eca081/Lib/ssl.py#l376

Christian

PS: SSL_CERT_DIR and SSL_CERT_FILE are the default names. It's possible
to change the names in OpenSSL. ssl.get_default_verify_paths() returns
the names and paths to the default verify locations.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Cory Benfield]

On 31 August 2014 07:45, Nick Coghlan  wrote:
> There's also the fact that most corporate Python users are
> unlikely to know that PyPI exists, let alone that it contains a module
> called "requests" that does SSL certificate validation by default.
> Those of us in the corporate world that interact directly with
> upstream are still the exception rather than the rule)

I think this point deserves just a little bit more emphasis. This is
why any solution that begins with 'use PyPI' is insufficient. I've
worked on requests for 3 years now and most of my colleagues have
never heard of it, and it's not because I don't talk about it (I talk
about it all the time!).

When building internal tools, corporate environments frequently
restrict themselves to the standard library. This is because it's hard
enough to get adoption of a tool when it requires a new language
runtime, let alone if you have to get people ramped up on package
distribution as well! I have enough trouble getting people to upgrade
Python versions at work: trying to get them up to speed on pip and
PyPI is worse.

It is no longer tenable in the long term for Python to trust the
network: you're right in this regard Nick. In the past, on this very
list, I've been bullish about fixing up Python's network security
position. I was an aggressive supporter of PEP 466 (and there are some
corners of PEP 466 that I think didn't go far enough). However, I'm
with you here: we should do this once and do it right. Corporate users
*will* bump into it, and they will look to the docs to fix it. That
fix needs to be easy and painless.

A user-level cert store is a good start, and if cryptography.io aren't
interested in it I might take a look at implementing it under the
certifi.io umbrella instead.

Cory
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 31 August 2014 16:16, Donald Stufft  wrote:
>
> On Aug 31, 2014, at 2:09 AM, Nick Coghlan  wrote:
>
> At the same time, we need to account for the fact that most existing
> organisations still trust in perimeter defence for their internal
> network security, and hence tolerate (or even actively encourage) the
> use of unsecured connections, or skipping certificate validation,
> internally. This is actually a really terrible idea, but it's still
> incredibly common due to the general failure of the technology
> industry to take usability issues seriously when we design security
> systems (at least until recently) - doing the wrong "unsafe" thing is
> genuinely easier than doing things right.
>
>
> Just a quick clarification in order to be a little clearer, this change will
> (obviously) only effect those who trust perimeter security *and* decided to
> install an invalid certificate instead of just using HTTP. I'm not saying
> that
> this doesn't happen, just being specific (I'm not actually sure why they
> would
> install a TLS certificate at all if they are trusting perimeter security,
> but
> I'm sure folks do).

It's the end result when a company wide edict to use HTTPS isn't
backed up by the necessary documentation and training on how to get a
properly signed cert from your internal CA (or, even better, when such
an edict comes down without setting up an internal CA first). Folks
hit the internet instead, find instructions on creating a self-signed
cert, install that, and tell their users to ignore the security
warning and accept the cert. Historically, Python clients have "just
worked" in environments that required a click-through on the browser
side, since you had to opt in to checking the certificates properly.

Self-signed certificates can also be really handy for doing local
testing - you're not really aiming to authenticate the connection in
that case, you're just aiming to test that the secure connection
machinery is all working properly.

(As far as the "what about requests?" question goes - that's in a
similar situation to Go, where being new allows it to choose different
defaults, and folks for whom those defaults don't work just won't use
it. There's also the fact that most corporate Python users are
unlikely to know that PyPI exists, let alone that it contains a module
called "requests" that does SSL certificate validation by default.
Those of us in the corporate world that interact directly with
upstream are still the exception rather than the rule)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 31 August 2014 12:21, R. David Murray  wrote:
> Do those knobs allow one to instruct urllib to accept an invalid
> certificate without changing the program code?

My first reply ended up being a context dump of the challenges created
by legacy corporate intranets that may not be immediately obvious to
folks that spend most of their time working on or with the public
internet. I decided to split these more technical details out to a new
reply for the benefit of folks that already know all that history :)

To answer David's specific question, the existing knobs at the OpenSSL
level (SSL_CERT_DIR and SSL_CERT_FILE ) let people add an internal CA,
opt out of the default CA system, and trust *specific* self-signed
certs.

What they don't allow is a global "trust any cert" setting -
exceptions need to be added at the individual cert level or at the CA
level, or the application needs to offer an option to not do cert
validation at all. That "trust anything" option at the platform level
is the setting that is a really bad idea - if an organisation thinks
it needs that (because they have a lot of self-signed certs, but
aren't verifying their HTTPS connections to those servers), then what
they really need is an internal CA, where their systems just need to
be set up to trust the internal CA in addition to the platform CA
certs.

With Alex's proposal, organisations that are already running an
internal CA should be just fine - Python 3.5 will see the CA cert in
the platform cert store and accept certs signed by it as valid. (Note:
the Python 3.4 warning should take this into account, which could be a
problem since we don't currently do validity checks against the
platform store by default. The PEP needs to cover the mechanics of
that in more detail, as I think it means we'll need to make *some*
changes to the default configuration even in Python 3.4 to get
accurate validity data back from OpenSSL)

However, we also need to accept that there's a reason browser vendors
still offer "click through insecurity" for sites with self-signed
certificates, and tools like wget/curl offer the option to say "don't
check the certificate": these are necessary compromises to make SSL
based network connections actually work on many current corporate
intranets.

It is corporate environments that also make it desirable to be able to
address this potential problem at a *user* level, since many Python
users in a large organisations are actually running Python entirely
out of their home directories, rather than as a system installation
(they may not even have admin access to their own systems).

My suggestion at this point is that we take a leaf from both browser
vendors and the design of SSH: make it easy to *add* a specific
self-signed cert to the set a *particular user* trusts by default
(preferably *only* for a particular host, to limit the power of such
certs). "python -m ssl" doesn't currently do anything interesting, so
it could be used to provide an API for managing that user level
certificate store.

A Python-specific user level cert store is something that could be
developed as a PyPI library for Python 2.7.9+ and 3.4+ (Is cert
management considered in scope for cryptography.io? If so, that could
be a good home).

So while I agree with the intent of PEP 476, and like the suggested
end state, I'm back to thinking that the transition plan for existing
corporate users needs more work before it can be accepted. This is
especially true since it becomes another barrier to migrating from
Python 2.7 to Python 3.5+ (a warning in Python 3.4 doesn't help with
that aspect, although a new -3 warning might).

A third party module that offers a user level certificate store, and a
gevent.monkey style way of opting in to this behaviour for existing
Python versions would be one way to provide a more compelling
transition plan.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Donald Stufft]

> On Aug 31, 2014, at 2:09 AM, Nick Coghlan  wrote:
> 
> At the same time, we need to account for the fact that most existing
> organisations still trust in perimeter defence for their internal
> network security, and hence tolerate (or even actively encourage) the
> use of unsecured connections, or skipping certificate validation,
> internally. This is actually a really terrible idea, but it's still
> incredibly common due to the general failure of the technology
> industry to take usability issues seriously when we design security
> systems (at least until recently) - doing the wrong "unsafe" thing is
> genuinely easier than doing things right.
> 


Just a quick clarification in order to be a little clearer, this change will
(obviously) only effect those who trust perimeter security *and* decided to
install an invalid certificate instead of just using HTTP. I'm not saying that
this doesn't happen, just being specific (I'm not actually sure why they would
install a TLS certificate at all if they are trusting perimeter security, but
I'm sure folks do).

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 31 August 2014 12:21, R. David Murray  wrote:
> On Sun, 31 Aug 2014 03:25:25 +0200, Antoine Pitrou  
> wrote:
>> On Sun, 31 Aug 2014 09:26:30 +1000
>> Nick Coghlan  wrote:
>> > In relation to changing the Python CLI API to offer some of the wget/curl
>> > style command line options, I like the idea of providing recipes in the
>> > docs for implementing them at the application layer, but postponing making
>> > the *default* behaviour configurable that way.
>>
>> I'm against any additional environment variables and command-line
>> options. It will only complicate and obscure the security parameters of
>> certificate validation.

As Antoine says here, I'm also opposed to adding more Python specific
configuration options. However, I think there may be something
worthwhile we can do that's closer to the way browsers work, and has
the significant benefit of being implementable as a PyPI module first
(more on that in a separate reply).

>> The existing knobs have already been mentioned in this thread, I won't
>> mention them here again.
>
> Do those knobs allow one to instruct urllib to accept an invalid
> certificate without changing the program code?

Only if you add the specific certificate concerned to the certificate
store that Python is using (which PEP 476 currently suggests will be
the platform wide certificate store). Whether or not that is an
adequate solution is the point currently in dispute.

My view is that the core problem/concern we need to address here is
how we manage the migration away from a network communication model
that trusts the network by default. That transition will happen
regardless of whether or not we adapt Python as a platform - the
challenge for us is how we can address it in a way that minimises the
impact on existing users, while still ensuring future users are
protected by default.

This would be relatively easy if we only had to worry about the public
internet (since we're followers rather than leaders in that
environment), but we don't. Python made the leap into enterprise
environments long ago, so we not only need to cope with corporate
intranets, we need to cope with corporate intranets that aren't
necessarily being well managed. That's what makes this a harder
problem for us than it is for a new language like Go that was created
by a public internet utility, specifically for use over the public
internet - they didn't *have* an installed base to manage, they could
just build a language specifically tailored for the task of running
network services on Linux, without needing to account for any other
use cases.

The reason our existing installed base creates a problem is because
corporate network security has historically focused on "perimeter
defence": carving out a trusted island behind the corporate firewall
where users and other computer systems could be "safely" assumed not
to be malicious.

As an industry, we have learned though harsh experience that *this
model doesn't work*. You can't trust the network, period. A corporate
intranet is *less* dangerous than the public internet, but you still
can't trust it. This "don't trust the network" ethos is also
reinforced by the broad shift to "utility computing" where more and
more companies are running distributed networks, where some of their
systems are actually running on vendor provided servers. The "network
perimeter" is evaporating, as corporate "intranets" start to look a
lot more like recreations of the internet in miniature, with the only
difference being the existence of more formal contractual
relationships than typically exist between internet peers.

Unfortunately, far too many organisations (especially those outside
the tech industry) still trust in perimeter defence for their internal
network security, and hence tolerate the use of unsecured connections,
or skipping certificate validation internally. This is actually a
really terrible idea, but it's still incredibly common due to the
general failure of the technology industry to take usability issues
seriously when we design security systems - doing the wrong "unsafe"
thing is genuinely easier than doing things right.

We have enough evidence now to be able to say (as Alex does in PEP
476) that it has been comprehensively demonstrated that "opt-in
security" really just means "security failures are common and silent
by default". We've seen it with C buffer overflow vulnerabilities,
we've seen it with plain text communication links, we've seen it with
SSL certificate validation - the vast majority of users and developers
will just run with the default behaviour of the platform or
application they're using, even if those defaults have serious
problems. As the saying goes, "you can't document your way out of a
usability problem" - uncovered connections, or that are vulnerable to
a man-in-the-middle attack appear to work for all functional purposes,
they're just vulnerable to monitoring and subversion.

It turns out "opt-out security with a global off switch" isn'

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Stephen J. Turnbull]

mar...@v.loewis.de writes:

 > BTW, it's patented:
 > 
 > http://www.google.de/patents/US6816900

Damn them.  I hope they never get a look at my crontab.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[R. David Murray]

On Sun, 31 Aug 2014 03:25:25 +0200, Antoine Pitrou  wrote:
> On Sun, 31 Aug 2014 09:26:30 +1000
> Nick Coghlan  wrote:
> > >>
> > >>   * configuration:
> > >>
> > >> It would be good to be able to switch this on or off
> > >> without having to change the code, e.g. via a command
> > >> line switch and environment variable; perhaps even
> > >> controlling whether or not to raise an exception or
> > >> warning.
> > >>
> > >>   * choice of trusted certificate:
> > >>
> > >> Instead of hard wiring using the system CA roots into
> > >> Python it would be good to just make this default and
> > >> permit the user to point Python to a different set of
> > >> CA roots.
> > >>
> > >> This would enable using self signed certs more easily.
> > >> Since these are often used for tests, demos and education,
> > >> I think it's important to allow having more control of
> > >> the trusted certs.
> > >
> > >
> > > +1 for PEP with above changes.
> > 
> > Ditto from me.
> > 
> > In relation to changing the Python CLI API to offer some of the wget/curl
> > style command line options, I like the idea of providing recipes in the
> > docs for implementing them at the application layer, but postponing making
> > the *default* behaviour configurable that way.
> 
> I'm against any additional environment variables and command-line
> options. It will only complicate and obscure the security parameters of
> certificate validation.
> 
> The existing knobs have already been mentioned in this thread, I won't
> mention them here again.

Do those knobs allow one to instruct urllib to accept an invalid
certificate without changing the program code?

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Antoine Pitrou]

On Sun, 31 Aug 2014 09:26:30 +1000
Nick Coghlan  wrote:
> >>
> >>   * configuration:
> >>
> >> It would be good to be able to switch this on or off
> >> without having to change the code, e.g. via a command
> >> line switch and environment variable; perhaps even
> >> controlling whether or not to raise an exception or
> >> warning.
> >>
> >>   * choice of trusted certificate:
> >>
> >> Instead of hard wiring using the system CA roots into
> >> Python it would be good to just make this default and
> >> permit the user to point Python to a different set of
> >> CA roots.
> >>
> >> This would enable using self signed certs more easily.
> >> Since these are often used for tests, demos and education,
> >> I think it's important to allow having more control of
> >> the trusted certs.
> >
> >
> > +1 for PEP with above changes.
> 
> Ditto from me.
> 
> In relation to changing the Python CLI API to offer some of the wget/curl
> style command line options, I like the idea of providing recipes in the
> docs for implementing them at the application layer, but postponing making
> the *default* behaviour configurable that way.

I'm against any additional environment variables and command-line
options. It will only complicate and obscure the security parameters of
certificate validation.

The existing knobs have already been mentioned in this thread, I won't
mention them here again.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 30 Aug 2014 06:08, "Ethan Furman"  wrote:
>
> On 08/29/2014 01:00 PM, M.-A. Lemburg wrote:
>>
>> On 29.08.2014 21:47, Alex Gaynor wrote:
>>>
>>>
>>> I've just submitted PEP 476, on enabling certificate validation by
default for
>>> HTTPS clients in Python. Please have a look and let me know what you
think.
>>
>>
>> Thanks for the PEP. I think this is generally a good idea,
>> but some important parts are missing from the PEP:
>>
>>   * transition plan:
>>
>> I think starting with warnings in Python 3.5 and going
>> for exceptions in 3.6 would make a good transition
>>
>> Going straight for exceptions in 3.5 is not in line with
>> our normal procedures for backwards incompatible changes.
>>
>>   * configuration:
>>
>> It would be good to be able to switch this on or off
>> without having to change the code, e.g. via a command
>> line switch and environment variable; perhaps even
>> controlling whether or not to raise an exception or
>> warning.
>>
>>   * choice of trusted certificate:
>>
>> Instead of hard wiring using the system CA roots into
>> Python it would be good to just make this default and
>> permit the user to point Python to a different set of
>> CA roots.
>>
>> This would enable using self signed certs more easily.
>> Since these are often used for tests, demos and education,
>> I think it's important to allow having more control of
>> the trusted certs.
>
>
> +1 for PEP with above changes.

Ditto from me.

In relation to changing the Python CLI API to offer some of the wget/curl
style command line options, I like the idea of providing recipes in the
docs for implementing them at the application layer, but postponing making
the *default* behaviour configurable that way.

Longer term, I'd like to actually have a per-runtime configuration file for
some of these things that also integrated with the pyvenv support, but that
requires untangling the current startup code first (and there are only so
many hours in the day).

Regards,
Nick.


>
> --
> ~Ethan~
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[martin]

Zitat von Christian Heimes :


On 30.08.2014 17:22, Alex Gaynor wrote:

The Windows certificate store is used by ``load_default_certs``:

* https://github.com/python/cpython/blob/master/Lib/ssl.py#L379-L381
* https://docs.python.org/3.4/library/ssl.html#ssl.enum_certificates


The Windows part of load_default_certs() has one major flaw: it can only
load certificates that are already in Windows's cert store. However
Windows comes only with a small set of default certs and downloads more
certs on demand. In order to trigger a download Python or OpenSSL would
have to use the Windows API to verify root certificates.


It's better than you think. Vista+ has a weekly prefetching procedure that
should assure that virtually all root certificates are available:

http://support.microsoft.com/kb/931125/en-us

BTW, it's patented:

http://www.google.de/patents/US6816900

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Christian Heimes]

On 30.08.2014 17:22, Alex Gaynor wrote:
> The Windows certificate store is used by ``load_default_certs``:
> 
> * https://github.com/python/cpython/blob/master/Lib/ssl.py#L379-L381
> * https://docs.python.org/3.4/library/ssl.html#ssl.enum_certificates

The Windows part of load_default_certs() has one major flaw: it can only
load certificates that are already in Windows's cert store. However
Windows comes only with a small set of default certs and downloads more
certs on demand. In order to trigger a download Python or OpenSSL would
have to use the Windows API to verify root certificates.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Barry Warsaw]

On Aug 30, 2014, at 12:19 PM, M.-A. Lemburg wrote:

>The reasoning here is the same as for hash randomization. There
>are cases where you want to test your application using self-signed
>certificates which don't validate against the system CA root list.
>
>In those cases, you do know what you're doing. The test would fail
>otherwise and the reason is not a bug in your code, it's just
>the fact that the environment you're running it in is a test
>environment.

Exactly.  I have test cases where I have to load up a self-signed cert via
.load_cert_chain() and in the good-path tests, I expect to make successful
https connections.  I also have test cases that expect to fail when:

 * I load bogus self-signed certs
 * I have an http server masquerading as an https server
 * I load an expired self-signed cert

It certainly makes sense for the default to be the most secure, but other use
cases must be preserved.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com