Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 1:58 AM, Benjamin Peterson wrote: > 2013/6/2 Donald Stufft : >> As of right now, as far as I can tell, Python does not validate HTTPS >> certificates by default. As far as I can tell this is because there is no >> guaranteed certificates available. >> >> So I would like to propose that CPython adopt the Mozilla SSL certificate >> list and include it in core, and switch over the API's so that they verify >> HTTPS by default. > > +1 > >> >> Ideally this would take the shape of attempting to locate the system >> certificate store if possible, and if that doesn't work falling back to the >> bundled certificates. That way the various Linux distros can easily have >> their copies of Python depend soley on their built in certs, but Windows, >> OSX, Source compiles etc will all still have a fallback value. > > My preference would be actually be for the included certificates file > to be used by default. This would provide a consistent experience > across platforms. We could provide options to look for system cert > repositories if desired. That's fine with me too. My only reason for wanting to use the system certs first is so if someone has modified their system certs (say to include a corporate cert) that it would ideally take affect for Python as well. But honestly the Linux distros will probably modify things to use system certs anyways and non Linux (esp Windows) probably doesn't have a way to get those system certs into OpenSSL. > > > > -- > Regards, > Benjamin - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
2013/6/3 Donald Stufft : > > On Jun 3, 2013, at 1:58 AM, Benjamin Peterson wrote: > > 2013/6/2 Donald Stufft : > > As of right now, as far as I can tell, Python does not validate HTTPS > certificates by default. As far as I can tell this is because there is no > guaranteed certificates available. > > So I would like to propose that CPython adopt the Mozilla SSL certificate > list and include it in core, and switch over the API's so that they verify > HTTPS by default. > > > +1 > > > Ideally this would take the shape of attempting to locate the system > certificate store if possible, and if that doesn't work falling back to the > bundled certificates. That way the various Linux distros can easily have > their copies of Python depend soley on their built in certs, but Windows, > OSX, Source compiles etc will all still have a fallback value. > > > My preference would be actually be for the included certificates file > to be used by default. This would provide a consistent experience > across platforms. We could provide options to look for system cert > repositories if desired. > > > That's fine with me too. My only reason for wanting to use the system certs > first is so > if someone has modified their system certs (say to include a corporate cert) > that it > would ideally take affect for Python as well. I don't think users should be able to modify stdlib behaviors (in this case could be unintentionally) without application consent. > > But honestly the Linux distros will probably modify things to use system > certs anyways > and non Linux (esp Windows) probably doesn't have a way to get those system > certs > into OpenSSL. Yes, I'm happy to let them figure it out. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
Love this idea. Some third-party HTTP libraries turn this on by default in any case (eg: requests, and I think others), so this would mean Python would get their "safe-by-default" behaviour in its stdlib. > > Ideally this would take the shape of attempting to locate the system > > certificate store if possible, and if that doesn't work falling back to the > > bundled certificates. That way the various Linux distros can easily have > > their copies of Python depend soley on their built in certs, but Windows, > > OSX, Source compiles etc will all still have a fallback value. > > My preference would be actually be for the included certificates file > to be used by default. This would provide a consistent experience > across platforms. We could provide options to look for system cert > repositories if desired. Very much agreed. When the Windows version of the mimetypes module tried to use Windows' system mimetype mappings by default, chaos and bugs ensued (for example, http://bugs.python.org/issue15207 and http://bugs.python.org/issue10551). -Ben ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 3 Jun 2013 01:20:42 -0400 Donald Stufft wrote: > As of right now, as far as I can tell, Python does not validate HTTPS > certificates by default. As far as I can tell this is because there is > no guaranteed certificates available. Also for backwards compatibility. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Sun, 2 Jun 2013 22:57:15 -0700 Chris Rebert wrote: > On Jun 2, 2013 10:22 PM, "Donald Stufft" wrote: > > > > As of right now, as far as I can tell, Python does not validate HTTPS > > certificates by default. As far as I can tell this is because there is no > > guaranteed certificates available. > > Relevant: http://bugs.python.org/issue13647 > > > So I would like to propose that CPython adopt the Mozilla SSL certificate > > list and include it in core, and switch over the API's so that they verify > > HTTPS by default. This is what most people are going to expect when using a > > https url (Especially after learning that Python 2.x doesn't verify TLS, > > but Python 3.x "does"). > > > > Ideally this would take the shape of attempting to locate the system > > certificate store if possible, and if that doesn't work falling back to the > > bundled certificates. That way the various Linux distros can easily have > > their copies of Python depend solely on their built in certs, but Windows, > > OSX, Source compiles etc will all still have a fallback value. > > There's an existing request for this: > http://bugs.python.org/issue13655 See also http://bugs.python.org/issue17134 Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 03 Jun 2013 19:38:45 +1200, Ben Hoyt wrote: > Love this idea. Some third-party HTTP libraries turn this on by > default in any case (eg: requests, and I think others), so this would > mean Python would get their "safe-by-default" behaviour in its stdlib. > > > > Ideally this would take the shape of attempting to locate the system > > > certificate store if possible, and if that doesn't work falling back to > > > the > > > bundled certificates. That way the various Linux distros can easily have > > > their copies of Python depend soley on their built in certs, but Windows, > > > OSX, Source compiles etc will all still have a fallback value. > > > > My preference would be actually be for the included certificates file > > to be used by default. This would provide a consistent experience > > across platforms. We could provide options to look for system cert > > repositories if desired. > > Very much agreed. When the Windows version of the mimetypes module > tried to use Windows' system mimetype mappings by default, chaos and > bugs ensued (for example, http://bugs.python.org/issue15207 and > http://bugs.python.org/issue10551). On the other hand, the fact that it uses /etc/mimetypes on unix does *not* cause chaos, quite the opposite: it means Python recognizes new mime types provided by the distro automatically, which provides for a more consistent experience for the Python application user. The situation for certs is probably fairly parallel: on unix, it would probably be an advantage as Python would automatically follow distro decisions about cert chains, while on windows trying to use a system cert store would probably be a disaster. An application can choose to explicitly ignore the system mimetypes file, by the way. That said, I don't feel strongly either way about the cert store case. --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
> The situation for certs is probably fairly parallel: on unix, it would > probably be an advantage as Python would automatically follow distro > decisions about cert chains, while on windows trying to use a system > cert store would probably be a disaster. Yeah, fair enough. If it's stable and just works on Linux, this would be an advantage. > An application can choose to explicitly ignore the system mimetypes > file, by the way. My main concern is that it could be broken out of the box on Windows (mimetypes currently is), and callers have to go out of their way to find this workaround. I'm not familiar with Unix/Linux, but on Windows, if it's anything like mimetypes it'll be really hard to get consistent behaviour across different boxes/versions from the registry, or wherever certs might be stored on Windows. I'd much rather have a slightly outdated but consistent experience by default. -Ben ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
Am 03.06.2013 07:20, schrieb Donald Stufft: > Ideally this would take the shape of attempting to locate the system > certificate store if possible, and if that doesn't work falling back to > the bundled certificates. That way the various Linux distros can easily > have their copies of Python depend soley on their built in certs, but > Windows, OSX, Source compiles etc will all still have a fallback value. On Windows it's rather easy to access the cert storage with a couple of calls to crypt32.dll. I have created a smallish ctypes interface: https://bitbucket.org/tiran/wincertstore/overview . setuptools 0.7 is using it to for its ssl support. I'm going to write a patch for Python 3.4, too. See http://bugs.python.org/issue17134 I welcome a cert bundle in CPython. But how are we suppose to handle updates and invalidation of CAs? A couple of months ago there was an intense discussion about a copy of the Olson time zone database in Python's stdlib. People argued against it as they feared outdated tz information. IMHO outdated CA certs are much more severe than tz ... I'd like to see OCSP support, too. Unfortunately OpenSSL's OCSP API is far from trivial. I had a look at it once. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 3 Jun 2013 21:37:10 +1200 Ben Hoyt wrote: > > I'm not familiar with Unix/Linux, but on Windows, if it's anything > like mimetypes it'll be really hard to get consistent behaviour across > different boxes/versions from the registry, or wherever certs might be > stored on Windows. I'd much rather have a slightly outdated but > consistent experience by default. The problem with a "slightly outdated" CA store is that it can be a security risk. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 442: Safe object finalization
On Sun, 2 Jun 2013 19:27:49 -0700 Benjamin Peterson wrote: > 2013/5/18 Antoine Pitrou : > > > > Hello, > > > > I would like to submit the following PEP for discussion and evaluation. > > Will the API of the gc module be at all affected? I assume nothing > will just be printed for DEBUG_UNCOLLECTABLE. Objects with tp_del may still exist (third-party extensions perhaps). > Maybe there should be a > way to discover when a cycle is resurrected? Is it more important than discovering when a non-cycle is resurrected? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 442: Safe object finalization
On Sun, 2 Jun 2013 19:16:17 -0700 Benjamin Peterson wrote: > 2013/5/18 Antoine Pitrou : > > Calling finalizers only once is fine with me, but it would be a change > > in behaviour; I don't know if it may break existing code. > > I agree with Armin that this is better behavior. (Mostly significantly > consistent with weakrefs.) Keep in mind that it is a limitation of weakrefs, not a feature: you can't "unclear" a weakref. > > (for example, say someone is using __del__ to manage a freelist) > > Do you know if it breaks any of the projects you tested it with? I haven't tested it (yet). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
>> I'm not familiar with Unix/Linux, but on Windows, if it's anything >> like mimetypes it'll be really hard to get consistent behaviour across >> different boxes/versions from the registry, or wherever certs might be >> stored on Windows. I'd much rather have a slightly outdated but >> consistent experience by default. > > The problem with a "slightly outdated" CA store is that it can be a > security risk. True. This is different from mimetypes in that respect. Also, with Christian's post about Windows cert store, it does look like Windows has a stable/documented way to get this from the system. I wonder, are the crypt32.dll Cert* functions what IE uses? -Ben ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, Jun 3, 2013 at 1:20 AM, Donald Stufft wrote: > As of right now, as far as I can tell, Python does not validate HTTPS > certificates by default. As far as I can tell this is because there is no > guaranteed certificates available. > > So I would like to propose that CPython adopt the Mozilla SSL certificate > list and include it in core, and switch over the API's so that they verify > HTTPS by default. This is what most people are going to expect when using a > https url (Especially after learning that Python 2.x doesn't verify TLS, > but Python 3.x "does"). > > Ideally this would take the shape of attempting to locate the system > certificate store if possible, and if that doesn't work falling back to the > bundled certificates. That way the various Linux distros can easily have > their copies of Python depend soley on their built in certs, but Windows, > OSX, Source compiles etc will all still have a fallback value. > +1. I bundle a copy of the Mozilla CA list with Tornado, but would love to access the system's CA roots and/or use a Python-provided copy. I'd prefer to use the certificates from the operating system when possible, as that list is most likely to receive timely security updates (or be updated with a local corporate CA, for example). It's better to aim for consistency with the user's browser than consistency of Python applications across different installations. The data is analogous to the time zone database (PEP 431) in that it may need to be updated independently of Python's own release schedule, so we may want to use similar techniques to manage both. Also see certifi ( https://pypi.python.org/pypi/certifi), which is a copy of the Mozilla list in a pip-installable form. -Ben > > - > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ben%40bendarnell.com > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On 06/02/2013 10:20 PM, Donald Stufft wrote: So I would like to propose that CPython adopt the Mozilla SSL certificate list and include it in core, and switch over the API's so that they verify HTTPS by default. +1 Also, +1 on using system certs when available. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 5:51 AM, Antoine Pitrou wrote: > On Mon, 3 Jun 2013 21:37:10 +1200 > Ben Hoyt wrote: >> >> I'm not familiar with Unix/Linux, but on Windows, if it's anything >> like mimetypes it'll be really hard to get consistent behaviour across >> different boxes/versions from the registry, or wherever certs might be >> stored on Windows. I'd much rather have a slightly outdated but >> consistent experience by default. > > The problem with a "slightly outdated" CA store is that it can be a > security risk. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io Tracking the Mozilla store isn't difficult. New additions can be ignored for currently released Pythons so we'd just need to watch them for blacklisting certs and roll that into a security update. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 3 Jun 2013 12:43:32 -0400 Donald Stufft wrote: > > On Jun 3, 2013, at 5:51 AM, Antoine Pitrou wrote: > > > On Mon, 3 Jun 2013 21:37:10 +1200 > > Ben Hoyt wrote: > >> > >> I'm not familiar with Unix/Linux, but on Windows, if it's anything > >> like mimetypes it'll be really hard to get consistent behaviour across > >> different boxes/versions from the registry, or wherever certs might be > >> stored on Windows. I'd much rather have a slightly outdated but > >> consistent experience by default. > > > > The problem with a "slightly outdated" CA store is that it can be a > > security risk. > > > > Regards > > > > Antoine. > > > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io > > Tracking the Mozilla store isn't difficult. New additions can be ignored for > currently released Pythons so we'd just need to watch them for blacklisting > certs and roll that into a security update. Let's see if our security release managers want to do that job. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On 06/03/2013 09:43 AM, Donald Stufft wrote: On Jun 3, 2013, at 5:51 AM, Antoine Pitrou wrote: The problem with a "slightly outdated" CA store is that it can be a security risk. Tracking the Mozilla store isn't difficult. New additions can be ignored for currently released Pythons so we'd just need to watch them for blacklisting certs and roll that into a security update. Personally, I'm not interested in waiting six months for an update. And why can't I have the new additions? Seems to me a better solution is to have routines that can query and update at will (meaning the app has to call them), as well as having the bundle (black lists as well as new additions) in the regular updates. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 12:52 PM, Ethan Furman wrote: > On 06/03/2013 09:43 AM, Donald Stufft wrote: >> On Jun 3, 2013, at 5:51 AM, Antoine Pitrou wrote: >>> >>> The problem with a "slightly outdated" CA store is that it can be a >>> security risk. >> >> Tracking the Mozilla store isn't difficult. New additions can be ignored for >> currently released Pythons so we'd just >> need to watch them for blacklisting certs and roll that into a security >> update. > > Personally, I'm not interested in waiting six months for an update. And why > can't I have the new additions? > > Seems to me a better solution is to have routines that can query and update > at will (meaning the app has to call them), as well as having the bundle > (black lists as well as new additions) in the regular updates. > > -- > ~Ethan~ > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io Personally I view that as a nice to have but if an app is willing to call hem they can easily depend on certifi and just replace the default certificates. The goal here is to get secure by default without any work on the part of the developers. If a developer knows to update the certs they can know to use certifi. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 03, 2013, at 01:20 AM, Donald Stufft wrote: >So I would like to propose that CPython adopt the Mozilla SSL certificate >list and include it in core, and switch over the API's so that they verify >HTTPS by default. This is what most people are going to expect when using a >https url (Especially after learning that Python 2.x doesn't verify TLS, but >Python 3.x "does"). For the "verify HTTPS by default", do you mean specifically changing the cadefault argument to urllib.request.urlopen() to True? Note that I recently closed a bug related to this: http://bugs.python.org/issue17977 +1 for changing the default to True. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 03, 2013, at 09:05 AM, Ben Darnell wrote: >The data is analogous to the time zone database (PEP 431) in that it may >need to be updated independently of Python's own release schedule, so we >may want to use similar techniques to manage both. Also see certifi ( >https://pypi.python.org/pypi/certifi), which is a copy of the Mozilla list >in a pip-installable form. Right, this is very much analogous, except with the additional twist that out-of-date certificates can pose a significant security risk. I'm fairly certain that Debian and Ubuntu would explicitly not use any certificates shipped with Python, for two main reasons: 1) our security teams already manage the certificate store distro-wide and we want to make sure that one update fixes everything; 2) we don't want to duplicate code in multiple packages[1]. So *if* Python decides to do this (and I'm -0, but from a decidedly Linux-distro bias), it must be easily disabled. I generally like the way PEP 431 handles the tzdata, so I think we should do the same here. -Barry [1] This gives us headaches in upstreams like coverage caused by bundling externally available JavaScript libraries, or like urllib3 bundling chardet and urllib3, not to mention their own certificates yet again. :( ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 12:36 PM, Barry Warsaw wrote: > On Jun 03, 2013, at 01:20 AM, Donald Stufft wrote: > >> So I would like to propose that CPython adopt the Mozilla SSL certificate >> list and include it in core, and switch over the API's so that they verify >> HTTPS by default. This is what most people are going to expect when using a >> https url (Especially after learning that Python 2.x doesn't verify TLS, but >> Python 3.x "does"). > > For the "verify HTTPS by default", do you mean specifically changing the > cadefault argument to urllib.request.urlopen() to True? Note that I recently > closed a bug related to this: > > http://bugs.python.org/issue17977 > > +1 for changing the default to True. > > -Barry > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io If that's all it takes to make sure that people have to opt out of an invalid certificate throwing an error than yes :) My goal here is to make it that when someone accesses a TLS secured network resource (I said HTTP, but if there are other things, xmlrpclib, or what not where people can reasonably expect valid TLS certificates those too) they are protected by a MITM attack by default. I worry with the current situation people will just use TLS connections without realizing it's not being verified and thinking they are "safe". As with most things security the outcome of "failing" to do it right when the default is insecure is well nothing until someone attacks you. So Hopefully we make things secure by default for folks :) - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 03, 2013, at 03:12 AM, Donald Stufft wrote: >That's fine with me too. My only reason for wanting to use the system certs >first is so if someone has modified their system certs (say to include a >corporate cert) that it would ideally take affect for Python as well. This reminds me of one other thing. We have to make sure that the APIs (e.g urlopen()) continue to allow us to use self-signed certificates, if for no other reason than for testing purposes. OTOH, taking this away would be a backward incompatible change in API so probably wouldn't happen anyway. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 12:52 PM, Barry Warsaw wrote: > On Jun 03, 2013, at 03:12 AM, Donald Stufft wrote: > >> That's fine with me too. My only reason for wanting to use the system certs >> first is so if someone has modified their system certs (say to include a >> corporate cert) that it would ideally take affect for Python as well. > > This reminds me of one other thing. We have to make sure that the APIs > (e.g urlopen()) continue to allow us to use self-signed certificates, if for > no other reason than for testing purposes. OTOH, taking this away would be a > backward incompatible change in API so probably wouldn't happen anyway. > > -Barry > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io The other additional comment I'd like to throw in here is that if we don't bundle SSL certs I think we should still verify by default (which means HTTPS urls will throw an error by default if we can't locate a certificate store) because I think the risk to people unknowingly thinking that their HTTPS urls are protected are significant enough that this "error" shouldn't be silent by default. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 03, 2013, at 02:21 PM, Donald Stufft wrote: >The other additional comment I'd like to throw in here is that if we don't >bundle SSL certs I think we should still verify by default (which means HTTPS >urls will throw an error by default if we can't locate a certificate store) >because I think the risk to people unknowingly thinking that their HTTPS urls >are protected are significant enough that this "error" shouldn't be silent by >default. +1, especially if we ensure that the APIs are available to not verify, as is currently the case with urlopen(). I don't think people will want to do that in production, but it will be useful for testing (e.g. guess how I found issues 17977 :). -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 12:48 PM, Barry Warsaw wrote: > On Jun 03, 2013, at 09:05 AM, Ben Darnell wrote: > >> The data is analogous to the time zone database (PEP 431) in that it may >> need to be updated independently of Python's own release schedule, so we >> may want to use similar techniques to manage both. Also see certifi ( >> https://pypi.python.org/pypi/certifi), which is a copy of the Mozilla list >> in a pip-installable form. > > Right, this is very much analogous, except with the additional twist that > out-of-date certificates can pose a significant security risk. > > I'm fairly certain that Debian and Ubuntu would explicitly not use any > certificates shipped with Python, for two main reasons: 1) our security teams > already manage the certificate store distro-wide and we want to make sure that > one update fixes everything; 2) we don't want to duplicate code in multiple > packages[1]. > > So *if* Python decides to do this (and I'm -0, but from a decidedly > Linux-distro bias), it must be easily disabled. I generally like the way PEP > 431 handles the tzdata, so I think we should do the same here. I'd actually prefer for Linux to not use the bundled certs when installed from a package manager because it should use the system certs, but people can't depend on certs being there if they are only there on linux. Adding them into Python means people _can_ depend on them being there, and Windows and other systems without system integrators to modify it to use the system store will still get certs and Ubuntu can make it just work(™). > > -Barry > > [1] This gives us headaches in upstreams like coverage caused by bundling > externally available JavaScript libraries, or like urllib3 bundling chardet > and urllib3, not to mention their own certificates yet again. :( This would probably (eventually) make the bundling of certificates better too. Meaning that once it's been in long enough people are willing to depend on it, they won't need to bundle their own certs and ubuntu/debian can just modify the one location instead of needing to modify it for every package that does it. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 03, 2013, at 02:17 PM, Donald Stufft wrote: >I'd actually prefer for Linux to not use the bundled certs when installed >from a package manager because it should use the system certs, but people >can't depend on certs being there if they are only there on linux. I think we agree on that. >Adding them into Python means people _can_ depend on them being there, and >Windows and other systems without system integrators to modify it to use the >system store will still get certs and Ubuntu can make it just work(™). Again, I think PEP 431 provides a pretty good model for how this should be done. Maybe it's worth factoring out this specific part of PEP 431 into an informational PEP? >This would probably (eventually) make the bundling of certificates better >too. > >Meaning that once it's been in long enough people are willing to depend on >it, they won't need to bundle their own certs and ubuntu/debian can just >modify the one location instead of needing to modify it for every package >that does it. Can we do the same for the JavaScript libraries? :) -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On 06/03/2013 10:55 AM, Donald Stufft wrote: On Jun 3, 2013, at 12:52 PM, Ethan Furman wrote: On 06/03/2013 09:43 AM, Donald Stufft wrote: On Jun 3, 2013, at 5:51 AM, Antoine Pitrou wrote: The problem with a "slightly outdated" CA store is that it can be a security risk. Tracking the Mozilla store isn't difficult. New additions can be ignored for currently released Pythons so we'd just need to watch them for blacklisting certs and roll that into a security update. Personally, I'm not interested in waiting six months for an update. And why can't I have the new additions? Seems to me a better solution is to have routines that can query and update at will (meaning the app has to call them), as well as having the bundle (black lists as well as new additions) in the regular updates. Personally I view that as a nice to have but if an app is willing to call hem they can easily depend on certifi and just replace the default certificates. The goal here is to get secure by default without any work on the part of the developers. If a developer knows to update the certs they can know to use certifi. Let's pretend I want to do something with https. I'm pretty much a complete novice in that area, but I do know about these certificate things and that I should use them. (Which will probably happen sooner rather than later. ;) From my POV, having the bundled certs in the stdlib is great, but also having something in the stdlib to query for and download updates would also be great. I just looked at certifi and the only thing it has on there is `where` which tells me where the certs are on the system. This on only half the problem. If we're going to solve it, why not solve the whole problem? -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 3 Jun 2013 14:12:34 -0400 Donald Stufft wrote: > > I worry with the current situation people will just use TLS connections > without realizing it's not being verified and thinking they are "safe". Yet there's quite a visible warning in the docs: http://docs.python.org/dev/library/urllib.request.html Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 1:07 PM, Barry Warsaw wrote: > On Jun 03, 2013, at 02:17 PM, Donald Stufft wrote: > >> I'd actually prefer for Linux to not use the bundled certs when installed >> from a package manager because it should use the system certs, but people >> can't depend on certs being there if they are only there on linux. > > I think we agree on that. > >> Adding them into Python means people _can_ depend on them being there, and >> Windows and other systems without system integrators to modify it to use the >> system store will still get certs and Ubuntu can make it just work(™). > > Again, I think PEP 431 provides a pretty good model for how this should be > done. Maybe it's worth factoring out this specific part of PEP 431 into an > informational PEP? Looks fine to me minus the not updating in security releases (but that's just a difference in the type of data). > >> This would probably (eventually) make the bundling of certificates better >> too. >> >> Meaning that once it's been in long enough people are willing to depend on >> it, they won't need to bundle their own certs and ubuntu/debian can just >> modify the one location instead of needing to modify it for every package >> that does it. > > Can we do the same for the JavaScript libraries? :) > > -Barry - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, Jun 3, 2013 at 1:04 PM, Barry Warsaw wrote: > On Jun 03, 2013, at 02:21 PM, Donald Stufft wrote: > > >The other additional comment I'd like to throw in here is that if we don't > >bundle SSL certs I think we should still verify by default (which means > HTTPS > >urls will throw an error by default if we can't locate a certificate > store) > >because I think the risk to people unknowingly thinking that their HTTPS > urls > >are protected are significant enough that this "error" shouldn't be > silent by > >default. > > +1, especially if we ensure that the APIs are available to not verify, as > is > currently the case with urlopen(). I don't think people will want to do > that > in production, but it will be useful for testing (e.g. guess how I found > issues 17977 :). > +1 from me as well. Whether we bundle or simply provide a command to download the certs I think making this default is the bare-minimum, especially if setting nothing more than cadefault=True is all that is needed to get this behaviour since that's backwards-compatible to Python 3.3. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On 06/03/2013 11:34 AM, Antoine Pitrou wrote: On Mon, 3 Jun 2013 14:12:34 -0400 Donald Stufft wrote: I worry with the current situation people will just use TLS connections without realizing it's not being verified and thinking they are "safe". Yet there's quite a visible warning in the docs: http://docs.python.org/dev/library/urllib.request.html As has been been mentioned elsewhere, at the very least we should change cadefault to True for secure-type connections. The dangerous/unexpected behavior should not be the default. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 03 Jun 2013 11:56:45 -0700 Ethan Furman wrote: > On 06/03/2013 11:34 AM, Antoine Pitrou wrote: > > On Mon, 3 Jun 2013 14:12:34 -0400 > > Donald Stufft wrote: > >> > >> I worry with the current situation people will just use TLS connections > >> without realizing it's not being verified and thinking they are "safe". > > > > Yet there's quite a visible warning in the docs: > > http://docs.python.org/dev/library/urllib.request.html > > As has been been mentioned elsewhere, at the very least we should change > cadefault to True for secure-type connections. > The dangerous/unexpected behavior should not be the default. cadefault=True will probably be fail if the system certs are not properly configured in OpenSSL, e.g. under Windows or with a hand-made OpenSSL build. And, because of the way the OpenSSL API works, there's no way of knowing if it is the case or not: http://docs.python.org/3.4/library/ssl.html#ssl.SSLContext.set_default_verify_paths While I'm not saying we shouldn't try to verify certs by default, doing so is a little more delicate than "setting cadefault to True", unfortunately. Bundling our own CA cert store is an option, assuming Donald or someone else wants to take responsibility for it. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Tue, Jun 4, 2013 at 4:56 AM, Ethan Furman wrote:
> On 06/03/2013 11:34 AM, Antoine Pitrou wrote:
>>
>> On Mon, 3 Jun 2013 14:12:34 -0400
>> Donald Stufft wrote:
>>>
>>>
>>> I worry with the current situation people will just use TLS connections
>>> without realizing it's not being verified and thinking they are "safe".
>>
>>
>> Yet there's quite a visible warning in the docs:
>> http://docs.python.org/dev/library/urllib.request.html
>
>
> As has been been mentioned elsewhere, at the very least we should change
> cadefault to True for secure-type connections. The dangerous/unexpected
> behavior should not be the default.
+1 for having the default be safe, but this will have to be very
loudly announced ("when migrating from 3.3 to 3.4, TLS connections
will cease to work if blah blah"). Some legit sites with proper
certificates still manage to muck something up administratively
(developer.quicksales.com.au has a cert from RapidSSL but doesn't
bundle the intermediates, and I've told their devs about it, but all I
can do is disable cert checking). This will break code in ways that
will surprise people greatly. But I'd still rather the default be
True.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 2013-06-03 at 12:48 -0400, Barry Warsaw wrote: > On Jun 03, 2013, at 09:05 AM, Ben Darnell wrote: > > >The data is analogous to the time zone database (PEP 431) in that it may > >need to be updated independently of Python's own release schedule, so we > >may want to use similar techniques to manage both. Also see certifi ( > >https://pypi.python.org/pypi/certifi), which is a copy of the Mozilla list > >in a pip-installable form. > > Right, this is very much analogous, except with the additional twist that > out-of-date certificates can pose a significant security risk. > > I'm fairly certain that Debian and Ubuntu would explicitly not use any > certificates shipped with Python, for two main reasons: 1) our security teams > already manage the certificate store distro-wide and we want to make sure that > one update fixes everything; 2) we don't want to duplicate code in multiple > packages[1]. Fedora/RHEL are in a similar position; I expect we'd rip out the bundled certs in our builds shortly after unzipping the tarball, and use a system-wide cert store (I "rm -rf" bundled libraries in our builds, to make sure we're not using them). [...snip...] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
Am 03.06.2013 21:52, schrieb Antoine Pitrou: > cadefault=True will probably be fail if the system certs are not > properly configured in OpenSSL, e.g. under Windows or with a hand-made > OpenSSL build. > And, because of the way the OpenSSL API works, there's no way of > knowing if it is the case or not: > http://docs.python.org/3.4/library/ssl.html#ssl.SSLContext.set_default_verify_paths I only see an issue for uncommon Linux distributions and exotic Unices. For Windows an interface to crypt32 API solves the CA issue as shown in my wincertstore module. It gives the user the same SSL experience as Internet Explorer. Most Linux and BSD-ish operating systems have SSL certs at some standard location. https://bitbucket.org/pypa/setuptools/src/6de3186fdfd9f5b543380e9aca2d48976cfc38cd/setuptools/ssl_support.py?at=default#cl-15 lists a couple of standard locations. Under which conditions do we need to ship a CA cert file? Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 4:19 PM, Christian Heimes wrote: > Am 03.06.2013 21:52, schrieb Antoine Pitrou: >> cadefault=True will probably be fail if the system certs are not >> properly configured in OpenSSL, e.g. under Windows or with a hand-made >> OpenSSL build. >> And, because of the way the OpenSSL API works, there's no way of >> knowing if it is the case or not: >> http://docs.python.org/3.4/library/ssl.html#ssl.SSLContext.set_default_verify_paths > > I only see an issue for uncommon Linux distributions and exotic Unices. > > For Windows an interface to crypt32 API solves the CA issue as shown in > my wincertstore module. It gives the user the same SSL experience as > Internet Explorer. > > Most Linux and BSD-ish operating systems have SSL certs at some standard > location. > https://bitbucket.org/pypa/setuptools/src/6de3186fdfd9f5b543380e9aca2d48976cfc38cd/setuptools/ssl_support.py?at=default#cl-15 > lists a couple of standard locations. > > Under which conditions do we need to ship a CA cert file? > > Christian > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io What about OSX? - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
In article <699d9ab5-26e0-4cce-89c5-8e92c3305...@stufft.io>, Donald Stufft wrote: > On Jun 3, 2013, at 4:19 PM, Christian Heimes wrote: > > Most Linux and BSD-ish operating systems have SSL certs at some standard > > location. > > https://bitbucket.org/pypa/setuptools/src/6de3186fdfd9f5b543380e9aca2d48976c > > fc38cd/setuptools/ssl_support.py?at=default#cl-15 > > lists a couple of standard locations. > > Under which conditions do we need to ship a CA cert file? > What about OSX? OS X has an actively managed set of CA certs, by Apple and with provision for users to tailor their security policies (add/modify/delete certs and their policies via either a gui or cli). For python.org OS X installers, we currently depend on the system-provided versions of libssl and libcrypto which use the system/user cert infrastructure. But Apple has deprecated the OpenSSL-compatible APIs and we have an open issue (Issue17128) to move to supplying our own copy of OpenSSL libs. One hangup has been what to do about the certs. One solution would be to use a Python distribution of them as discussed here. Another, probably better for users, solution is to continue to use the existing system/user cert infrastructure; recently, we've seen a new approach to that which we need to look into: http://bugs.python.org/issue17128#msg189244. Third-party distributors of Python on OS X may use other solutions; for example, MacPorts already ships and links with its own version of OpenSSL. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On 3 June 2013 21:05, Chris Angelico wrote:
> +1 for having the default be safe, but this will have to be very
> loudly announced ("when migrating from 3.3 to 3.4, TLS connections
> will cease to work if blah blah").
>
+1 on the default being safe, certainly. But with the proviso that the same
code should work in 3.3 and 3.4, with no user impact (other than that the
connection is safer, but that's not user-visible unless there's an attack
:-))
In other words, that "will cease to work" clause should not exist - but see
below...
> Some legit sites with proper
> certificates still manage to muck something up administratively
> (developer.quicksales.com.au has a cert from RapidSSL but doesn't
> bundle the intermediates, and I've told their devs about it, but all I
> can do is disable cert checking). This will break code in ways that
> will surprise people greatly. But I'd still rather the default be
> True.
>
I'm happy if the "will cease to work" clause only says "some sites with
broken security configurations may stop working" with a clear explanation
that it is *their* fault, not Python's. I'd also expect that the same sites
would fail in browsers - if not, we should also be able to make them work
(or face cries of "well, Internet Explorer/Firefox doesn't have a problem
with my site, why does Python?").
Also, we should consider the issue for application users. Suppose I'm using
a Python application that downloads something from the web. I upgrade to
3.4, and the app stops working because of a "will cease to work" case. As
an end user, how can I get the app working again? Having to patch the
sources isn't an option, and reverting to 3.3 provokes the reaction "Python
broke my app".
Summary - I'm +1 as long as either the "will cease to work" list is empty,
or we have a *very* good story for (legitimate) sites which do cease to
work.
Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 3 Jun 2013 22:31:40 +0100 Paul Moore wrote: > > > Some legit sites with proper > > certificates still manage to muck something up administratively > > (developer.quicksales.com.au has a cert from RapidSSL but doesn't > > bundle the intermediates, and I've told their devs about it, but all I > > can do is disable cert checking). This will break code in ways that > > will surprise people greatly. But I'd still rather the default be > > True. > > > > I'm happy if the "will cease to work" clause only says "some sites with > broken security configurations may stop working" with a clear explanation > that it is *their* fault, not Python's. I'd also expect that the same sites > would fail in browsers - if not, we should also be able to make them work > (or face cries of "well, Internet Explorer/Firefox doesn't have a problem > with my site, why does Python?"). Keep in mind that not every HTTPS service is a Web site that is meant to be readable with a browser. Some are Web services, possibly internal, possibly without a domain name (and, therefore, probably a non-matching certificate subject name). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Tue, Jun 4, 2013 at 7:31 AM, Paul Moore wrote: > > On 3 June 2013 21:05, Chris Angelico wrote: >> Some legit sites with proper >> certificates still manage to muck something up administratively >> (developer.quicksales.com.au has a cert from RapidSSL but doesn't >> bundle the intermediates, and I've told their devs about it, but all I >> can do is disable cert checking). This will break code in ways that >> will surprise people greatly. But I'd still rather the default be >> True. > > I'm happy if the "will cease to work" clause only says "some sites with > broken security configurations may stop working" with a clear explanation > that it is *their* fault, not Python's. I'd also expect that the same sites > would fail in browsers - if not, we should also be able to make them work > (or face cries of "well, Internet Explorer/Firefox doesn't have a problem > with my site, why does Python?"). In this specific case, I get no warning from Google Chrome on either Windows or Linux, but I do get a warning from Iceweasel on the same Linux (it's Debian Wheezy, fwiw). So there probably will be cries of "well, Chrome doesn't etc etc". > Also, we should consider the issue for application users. Suppose I'm using > a Python application that downloads something from the web. I upgrade to > 3.4, and the app stops working because of a "will cease to work" case. As an > end user, how can I get the app working again? Having to patch the sources > isn't an option, and reverting to 3.3 provokes the reaction "Python broke my > app". Right. The instance I'm citing comes from what I'm doing at work; I had to disable certificate checking in order to get a non-interactive application to work. If I'd never thought to make sure cert checking was happening, the app could easily have been unexpectedly broken by this change. But you're absolutely right. It's a failing at the remote end, or... On Tue, Jun 4, 2013 at 7:41 AM, Antoine Pitrou wrote: > Keep in mind that not every HTTPS service is a Web site that is meant > to be readable with a browser. Some are Web services, possibly internal, > possibly without a domain name (and, therefore, probably a non-matching > certificate subject name). ... this. This isn't a problem, as checking can easily be disabled, but I'd be inclined to recommend that this one be solved without changing code by explicitly importing that certificate. That would resolve everything, but would require administrative action. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 5:41 PM, Antoine Pitrou wrote: > On Mon, 3 Jun 2013 22:31:40 +0100 > Paul Moore wrote: >> >>> Some legit sites with proper >>> certificates still manage to muck something up administratively >>> (developer.quicksales.com.au has a cert from RapidSSL but doesn't >>> bundle the intermediates, and I've told their devs about it, but all I >>> can do is disable cert checking). This will break code in ways that >>> will surprise people greatly. But I'd still rather the default be >>> True. >>> >> >> I'm happy if the "will cease to work" clause only says "some sites with >> broken security configurations may stop working" with a clear explanation >> that it is *their* fault, not Python's. I'd also expect that the same sites >> would fail in browsers - if not, we should also be able to make them work >> (or face cries of "well, Internet Explorer/Firefox doesn't have a problem >> with my site, why does Python?"). > > Keep in mind that not every HTTPS service is a Web site that is meant > to be readable with a browser. Some are Web services, possibly internal, > possibly without a domain name (and, therefore, probably a non-matching > certificate subject name). They should need to explicitly opt in to disabling the checks that allow that to work. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 5:31 PM, Paul Moore wrote:
>
> On 3 June 2013 21:05, Chris Angelico wrote:
> +1 for having the default be safe, but this will have to be very
> loudly announced ("when migrating from 3.3 to 3.4, TLS connections
> will cease to work if blah blah").
>
> +1 on the default being safe, certainly. But with the proviso that the same
> code should work in 3.3 and 3.4, with no user impact (other than that the
> connection is safer, but that's not user-visible unless there's an attack :-))
If we bundle certs that will be the case sans connections where it doesn't
validate.
>
> In other words, that "will cease to work" clause should not exist - but see
> below...
>
> Some legit sites with proper
> certificates still manage to muck something up administratively
> (developer.quicksales.com.au has a cert from RapidSSL but doesn't
> bundle the intermediates, and I've told their devs about it, but all I
> can do is disable cert checking). This will break code in ways that
> will surprise people greatly. But I'd still rather the default be
> True.
>
> I'm happy if the "will cease to work" clause only says "some sites with
> broken security configurations may stop working" with a clear explanation
> that it is *their* fault, not Python's. I'd also expect that the same sites
> would fail in browsers - if not, we should also be able to make them work (or
> face cries of "well, Internet Explorer/Firefox doesn't have a problem with my
> site, why does Python?").
Browsers tend to download intermediate certs while I don't think Python does.
>
> Also, we should consider the issue for application users. Suppose I'm using a
> Python application that downloads something from the web. I upgrade to 3.4,
> and the app stops working because of a "will cease to work" case. As an end
> user, how can I get the app working again? Having to patch the sources isn't
> an option, and reverting to 3.3 provokes the reaction "Python broke my app".
Supply a SSL vert using the environment variable?
>
> Summary - I'm +1 as long as either the "will cease to work" list is empty, or
> we have a *very* good story for (legitimate) sites which do cease to work.
>
> Paul.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/donald%40stufft.io
-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Mon, 3 Jun 2013 17:47:31 -0400 Donald Stufft wrote: > > On Jun 3, 2013, at 5:41 PM, Antoine Pitrou wrote: > > > On Mon, 3 Jun 2013 22:31:40 +0100 > > Paul Moore wrote: > >> > >>> Some legit sites with proper > >>> certificates still manage to muck something up administratively > >>> (developer.quicksales.com.au has a cert from RapidSSL but doesn't > >>> bundle the intermediates, and I've told their devs about it, but all I > >>> can do is disable cert checking). This will break code in ways that > >>> will surprise people greatly. But I'd still rather the default be > >>> True. > >>> > >> > >> I'm happy if the "will cease to work" clause only says "some sites with > >> broken security configurations may stop working" with a clear explanation > >> that it is *their* fault, not Python's. I'd also expect that the same sites > >> would fail in browsers - if not, we should also be able to make them work > >> (or face cries of "well, Internet Explorer/Firefox doesn't have a problem > >> with my site, why does Python?"). > > > > Keep in mind that not every HTTPS service is a Web site that is meant > > to be readable with a browser. Some are Web services, possibly internal, > > possibly without a domain name (and, therefore, probably a non-matching > > certificate subject name). > > They should need to explicitly opt in to disabling the checks that allow that > to work. Obviously, which means compatibility is broken with existing code. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 5:51 PM, Antoine Pitrou wrote: > On Mon, 3 Jun 2013 17:47:31 -0400 > Donald Stufft wrote: >> >> On Jun 3, 2013, at 5:41 PM, Antoine Pitrou wrote: >> >>> On Mon, 3 Jun 2013 22:31:40 +0100 >>> Paul Moore wrote: > Some legit sites with proper > certificates still manage to muck something up administratively > (developer.quicksales.com.au has a cert from RapidSSL but doesn't > bundle the intermediates, and I've told their devs about it, but all I > can do is disable cert checking). This will break code in ways that > will surprise people greatly. But I'd still rather the default be > True. > I'm happy if the "will cease to work" clause only says "some sites with broken security configurations may stop working" with a clear explanation that it is *their* fault, not Python's. I'd also expect that the same sites would fail in browsers - if not, we should also be able to make them work (or face cries of "well, Internet Explorer/Firefox doesn't have a problem with my site, why does Python?"). >>> >>> Keep in mind that not every HTTPS service is a Web site that is meant >>> to be readable with a browser. Some are Web services, possibly internal, >>> possibly without a domain name (and, therefore, probably a non-matching >>> certificate subject name). >> >> They should need to explicitly opt in to disabling the checks that allow >> that to work. > > Obviously, which means compatibility is broken with existing code. > > Regards > > Antoine. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/donald%40stufft.io Yes in that case compat will be broken and they'll need to either specify a cert that can be used to validate the connection or disable the protection. I think it's very surprising for people that they need to *enable* secure mode when most tools have that on by default. It's handing users a security foot gun, and like most things security related "broken" is silent until it's too late. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On 3 June 2013 22:46, Donald Stufft wrote: > Also, we should consider the issue for application users. Suppose I'm > using a Python application that downloads something from the web. I upgrade > to 3.4, and the app stops working because of a "will cease to work" case. > As an end user, how can I get the app working again? Having to patch the > sources isn't an option, and reverting to 3.3 provokes the reaction "Python > broke my app". > > > Supply a SSL vert using the environment variable? > Hmm, that would be acceptable, I guess, for many users (although Windows users are somewhat more environment-variable-averse than Unix users). But you say that as if it's obvious how to do that (or where to get a cert). It's certainly not obvious to me, and if "it works in Internet Explorer", I'd have no idea where to get a cert from that I could use in an environment variable. Just to repeat - I agree with the principle, but in many environments, users are pretty much clueless about security and actively object to being educated "for their own safety". These users will disable all security quite happily if it stops the internal app failing, and will blame Python for "making things harder" and breaking backward compatibility. On the other hand, I suspect we're talking about an extremely low percentage of cases, so let's not blow the issue out of proportion :-) Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Validating SSL By Default (aka Including a Cert Bundle in CPython)
On Jun 3, 2013, at 6:01 PM, Paul Moore wrote: > > On 3 June 2013 22:46, Donald Stufft wrote: >> Also, we should consider the issue for application users. Suppose I'm using >> a Python application that downloads something from the web. I upgrade to >> 3.4, and the app stops working because of a "will cease to work" case. As an >> end user, how can I get the app working again? Having to patch the sources >> isn't an option, and reverting to 3.3 provokes the reaction "Python broke my >> app". > > Supply a SSL vert using the environment variable? > > Hmm, that would be acceptable, I guess, for many users (although Windows > users are somewhat more environment-variable-averse than Unix users). But you > say that as if it's obvious how to do that (or where to get a cert). It's > certainly not obvious to me, and if "it works in Internet Explorer", I'd have > no idea where to get a cert from that I could use in an environment variable. > > Just to repeat - I agree with the principle, but in many environments, users > are pretty much clueless about security and actively object to being educated > "for their own safety". These users will disable all security quite happily > if it stops the internal app failing, and will blame Python for "making > things harder" and breaking backward compatibility. On the other hand, I > suspect we're talking about an extremely low percentage of cases, so let's > not blow the issue out of proportion :-) > > Paul Let me make myself a bit more clear because maybe I haven't been. If someone is relying on the insecure by default behavior and Python 3.4 gets this change, their code will break and their options will be: Client side: - Disable Verification, essentially reverting back to the old behavior. - Setup their Python install to be able to validate that certificate - This may be importing it into their OS certificate store - This may be using an env var to specify - This may be passing it directly somewhere Server side: - Make the certificate valid - Purchase a Valid by default certificate - Serve immedaries etc. Generally any of these will be completely valid options, even disabling the checks. The idea behind my proposal is that people generally only use TLS for a reason and that reason is they want to protect against the kinds of attacks that TLS protects against. You really only get the bulk of those protections if you validate the certificates. So for the vast bulk of people validation is the option they want. It also happens that validation on by default is "secure" by default which aims a fairly large foot gun away from peoples feet. To expand on something else, I'm completely willing to do whatever I can to hep make this happen. I don't really know what the process is. If I need to write a PEP I'll do that. If I need to volunteer to manage the certificates, I'll do that. If we need docs or tooling to help the people who this change might break I'll do that too. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 442: Safe object finalization
On Sat, May 18, 2013 at 10:33 PM, Antoine Pitrou wrote: > On Sat, 18 May 2013 16:22:55 +0200 > Armin Rigo wrote: >> Hi Antoine, >> >> On Sat, May 18, 2013 at 3:45 PM, Antoine Pitrou wrote: >> >> How is this done? I don't see a clear way to determine it by looking >> >> only at the objects in the CI, given that arbitrary modifications of >> >> the object graph may have occurred. >> > >> > The same way a generation is traversed, but restricted to the CI. >> > >> > First the gc_refs field of each CI object is initialized to its >> > ob_refcnt (again). >> > >> > Then, tp_traverse is called on each CI object, and each visited >> > CI object has its gc_refs decremented. This substracts CI-internal >> > references from the gc_refs fields. >> > >> > At the end of the traversal, if all CI objects have their gc_refs equal >> > to 0, then the CI has no external reference to it and can be cleared. >> > If at least one CI object has non-zero gc_refs, the CI cannot be >> > cleared. >> >> Ok, indeed. Then you really should call finalizers only once: in case >> one of the finalizers in a cycle did a trivial change like I >> described, the algorithm above will conservatively assume the cycle >> should be kept alive. At the next GC collection we must not call the >> finalizer again, because it's likely to just do a similar trivial >> change. > > Well, the finalizer will only be called if the resurrected object is > dereferenced again; otherwise the object won't be considered by the GC. > So, this will only happen if someone keeps trying to destroy a > resurrected object. > > Calling finalizers only once is fine with me, but it would be a change > in behaviour; I don't know if it may break existing code. > > (for example, say someone is using __del__ to manage a freelist) > > Regards > > Antoine. PyPy already ever calls finalizers once. If you resurrect an object, it'll be alive, but it's finalizer will not be called again. We discussed a few changes a while ago and we decided (I think even debated on python-dev) than even such behavior is correct: * you have a reference cycle A <-> B, C references A. C references itself. * you collect the stuff. We do topological order, so C finalizer is called first (they're only undefined inside a cycle) * then A and B finalizers are called in undefined order, even if C finalizer would resurrect C. * no more finalizers for those objects are called I'm not sure if it's cool for CPython or not to do such changes Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Let's get rid of unbound methods
___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Putting the Mac Build in the Apple App Store
Does anyone know what we would need to do to get Python in the Apple application store as a free App? The default security settings on OS X 10.8 block the installation of the DMG (or any software downloaded outside the app store). A number of my students are having difficulty getting around it will help. If we were in the app store, installation and upgrade would be a piece of cake. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Putting the Mac Build in the Apple App Store
On Tue, Jun 4, 2013 at 12:44 PM, Raymond Hettinger wrote: > Does anyone know what we would need to do to get Python in the Apple > application store as a free App? > > The default security settings on OS X 10.8 block the installation of the DMG > (or any software downloaded outside the app store). A number of my students > are having difficulty getting around it will help. > > If we were in the app store, installation and upgrade would be a piece of > cake. > > > Raymond > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com doesn't it break apple's rules? generally you should not be able to download content from the internet and run it (like say javascript in the browser). I suppose python interpreter counts as that but hey, who knows. cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Putting the Mac Build in the Apple App Store
On Jun 3, 2013 9:45 PM, "Raymond Hettinger" wrote: > > Does anyone know what we would need to do to get Python in the Apple application store as a free App? > > The default security settings on OS X 10.8 block the installation of the DMG (or any software downloaded outside the app store). A number of my students are having difficulty getting around it will help. > > If we were in the app store, installation and upgrade would be a piece of cake. If you need another option, you could try installing Python via Homebrew ( http://brew.sh ). Homebrew itself is also easy to install. (Although admittedly I haven't tried Homebrew's Python, but my experience with their other packages has been good.) Cheers, Chris ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
