Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
BTW, my intention was also to set a precedent for future additions to the list. "We did this before" is a great argument to help smooth the path, without bypassing deliberations altogether. On Mar 27, 2014 2:33 AM, "Nick Coghlan" wrote: > On 27 March 2014 18:02, Stephen J. Turnbull wrote: > > Alex Gaynor writes: > > > > > Here's my proposed list of such featuers: > > > > And suppose that list grows over time? After all, it once was []. > > > > If we go for a feature-by-feature list, that has two more-or-less > > hidden costs. (1) Python-Dev has to specify which ones, and either > > risks a new specification debate in the future, or needs to spend > > time now describing criteria and processes for extending the list. > > It's not a hidden cost - it's a deliberately chosen one. Guido was > wary of an open-ended agreement, so by enumerating the precise set of > missing features in Python 2.7 that are causing concern for the > network security folks, we get to address the immediate problem, > without granting permission to backport further arbitrary features > without additional discussion. > > > (2) Users may need to worry about the list. (OTOH, as long as the > > list is restricted to features in certain modules, users can choose to > > assume anything in those modules may have changed behavior and that's > > no different from Nick's proposal for them.) > > The PEP already specifically advises that cross-version compatible > code use feature detection rather than version checks. For network > security, it's recommended to avoid using the low level modules > directly, anyway - it's much better to use a higher level library like > requests, to reduce the number of places where we need to get the > security design right. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 27 March 2014 18:02, Stephen J. Turnbull wrote: > Alex Gaynor writes: > > > Here's my proposed list of such featuers: > > And suppose that list grows over time? After all, it once was []. > > If we go for a feature-by-feature list, that has two more-or-less > hidden costs. (1) Python-Dev has to specify which ones, and either > risks a new specification debate in the future, or needs to spend > time now describing criteria and processes for extending the list. It's not a hidden cost - it's a deliberately chosen one. Guido was wary of an open-ended agreement, so by enumerating the precise set of missing features in Python 2.7 that are causing concern for the network security folks, we get to address the immediate problem, without granting permission to backport further arbitrary features without additional discussion. > (2) Users may need to worry about the list. (OTOH, as long as the > list is restricted to features in certain modules, users can choose to > assume anything in those modules may have changed behavior and that's > no different from Nick's proposal for them.) The PEP already specifically advises that cross-version compatible code use feature detection rather than version checks. For network security, it's recommended to avoid using the low level modules directly, anyway - it's much better to use a higher level library like requests, to reduce the number of places where we need to get the security design right. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Alex Gaynor writes: > Here's my proposed list of such featuers: And suppose that list grows over time? After all, it once was []. If we go for a feature-by-feature list, that has two more-or-less hidden costs. (1) Python-Dev has to specify which ones, and either risks a new specification debate in the future, or needs to spend time now describing criteria and processes for extending the list. (2) Users may need to worry about the list. (OTOH, as long as the list is restricted to features in certain modules, users can choose to assume anything in those modules may have changed behavior and that's no different from Nick's proposal for them.) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
At this I think this PEP has become a little too vague and abstract, and I think we'd probably be better served by getting more concrete: Problem: Some of Python 2's modules which are fundamentally necessary for interop with the broader internet, and the security thereof, are missing really important features. Right now Python 2 has a policy of getting absolutely new features. Solution: We're going to ignore that policy for a couple of pretty important features to that end. Here's my proposed list of such featuers: * hmac * constant_time_compare * os * Persisant FD for os.urandom() * ssl * SNI * SSLContext * A giant suite of constants from OpenSSL * The functions for checking a hostname against a certificate * The functions for finding the platform's certificate store Alex ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Mar 25, 2014, at 7:03 PM, Nick Coghlan wrote: > > On 26 Mar 2014 08:35, "Antoine Pitrou" wrote: > > > > On Tue, 25 Mar 2014 23:09:45 +1000 > > Nick Coghlan wrote: > > > > > > Alternative: selectively backport particular APIs > > > - > > > > > > An instinctively minimalist reaction to this proposal is to only backport > > > particular APIs in the affected modules that are judged to be "security > > > critical". However, this ends up providing a worse end user experience, > > > as well as a worse developer experience. > > > > > > For end users, the selective backporting approach means learning not only > > > the legacy Python 2.7 API and the current Python 3 APIs, but also the > > > hybrid API created by the selective backporting process. > > > > I think this is a strawman, since you are also advocating for a > > "feature detection" approach to writing cross-version code. It is > > already required, actually, if wanting to write code compatible from > > 3.2 to 3.4 (for example, SSLContext exists in 3.2 but > > create_default_context appears in 3.4 while OP_NO_COMPRESSION appears > > in 3.3). > > > > I would much rather selectively backport a minimal set of APIs than the > > whole range of ssl APIs. There are things there (RAND_bytes, > > RAND_pseudo_bytes) that are not even useful for network security, or > > only in a rather uncommon manner (such as channel bindings). > > Yeah, I think this is a valid point, and, as Guido noted, we also want the > option to skip backporting things if they depend on other aspects of Python 3 > that we decide can't be backported. > > So a feature-by-feature decision making process actually does make more sense > than a blanket exemption. > > Looking at the ssl module, just about the only thing that wouldn’t be helpful to have backported is the rand functions that AP mentioned (and those mostly because everyone should just use os.urandom for everything ever). The NPN stuff isn’t important for security related stuff either though it’d probably be more work to rip it out and more disruptive as well. Looking at the hmac module, the only thing that really matters for a backport would be the constant time compare. Lokoing at hashlib, the guaranteed and supported algorithms would be nice to have but not really security sensitive. Adding pbkdf2_hmac would be really nice for the security of web services though. Looking at os.urandom, back porting the lazily opened and held file descriptor would be a nice to have, but not strictly required. (Not sure if that would be considered a backwards incompat change). Nothing in random.py really stands out to me looking at it’s docs. > Cheers, > Nick. > > > > > Regards > > > > Antoine. > > > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 26 Mar 2014 08:35, "Antoine Pitrou" wrote: > > On Tue, 25 Mar 2014 23:09:45 +1000 > Nick Coghlan wrote: > > > > Alternative: selectively backport particular APIs > > - > > > > An instinctively minimalist reaction to this proposal is to only backport > > particular APIs in the affected modules that are judged to be "security > > critical". However, this ends up providing a worse end user experience, > > as well as a worse developer experience. > > > > For end users, the selective backporting approach means learning not only > > the legacy Python 2.7 API and the current Python 3 APIs, but also the > > hybrid API created by the selective backporting process. > > I think this is a strawman, since you are also advocating for a > "feature detection" approach to writing cross-version code. It is > already required, actually, if wanting to write code compatible from > 3.2 to 3.4 (for example, SSLContext exists in 3.2 but > create_default_context appears in 3.4 while OP_NO_COMPRESSION appears > in 3.3). > > I would much rather selectively backport a minimal set of APIs than the > whole range of ssl APIs. There are things there (RAND_bytes, > RAND_pseudo_bytes) that are not even useful for network security, or > only in a rather uncommon manner (such as channel bindings). Yeah, I think this is a valid point, and, as Guido noted, we also want the option to skip backporting things if they depend on other aspects of Python 3 that we decide can't be backported. So a feature-by-feature decision making process actually does make more sense than a blanket exemption. Cheers, Nick. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 26 Mar 2014 00:25, "Paul Moore" wrote: > > On 25 March 2014 13:47, Nick Coghlan wrote: > > It's not like we're going to just be giving the PEP to vendors as a spec and > > leaving them to it - it's largely an invitation to participate more directly > > upstream to help resolve a particularly thorny problem, not a Statement of > > Work :) > > :-) > > I don't really know the APIs involved, but AIUI one of the 3.4 > enhancements is exposing the SSLContext. Is the code to do this > compatible with the version of OpenSSL bundled with Python 2.7 on > Windows? If not, suppose that Red Hat provide resources that work on > backporting the code, but they don't have Windows experts so no-one > deals with integrating the new OpenSSL into the Windows binaries. > Would the backport be blocked until someone is found to do the Windows > work? We'll get it done. For example, while *I* definitely approach the problem from a Linux vendor perspective (and that's reflected in the PEP), I also know several folks at Rackspace have expressed concern about the status quo, and the client side of OpenStack is cross platform. > This "I've written a patch but it hasn't been applied" is the type of > scenario that puts people off contributing. If it's likely to happen, > I think Red Hat have a right to know that in advance. And I don't know > that it's something they would appreciate without python-dev pointing > it out. If we're reasonably sure (not necessarily certain, there's > always grey areas) that this isn't going to be an issue, then that's > also fine. We can simply say that. Yeah, I think we can make sure the right folks are involved to make it happen. The PEP is about me getting agreement in advance that we actually want to see the problem fixed, and the constraints we want to impose on the solution. Once we have that agreement, I won't be sitting around idly *waiting* for assistance to magically appear - I'll go looking for it, and I know there are others that will do the same :) > That's all I'm saying. Not trying to require anything of contributors, > just trying to be open and explicit about the criteria that will apply > to accepting contributions. I won't expect Linux folks to fix Windows problems (as that rarely works well). If the PEP is accepted, I *will* ensure we get the policy implemented on all supported platforms for 2.7.7+ by getting appropriate people involved (and will also work on securing the appropriate longer term support commitments). Cheers, Nick. > > Paul. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, 25 Mar 2014 23:09:45 +1000 Nick Coghlan wrote: > > Alternative: selectively backport particular APIs > - > > An instinctively minimalist reaction to this proposal is to only backport > particular APIs in the affected modules that are judged to be "security > critical". However, this ends up providing a worse end user experience, > as well as a worse developer experience. > > For end users, the selective backporting approach means learning not only > the legacy Python 2.7 API and the current Python 3 APIs, but also the > hybrid API created by the selective backporting process. I think this is a strawman, since you are also advocating for a "feature detection" approach to writing cross-version code. It is already required, actually, if wanting to write code compatible from 3.2 to 3.4 (for example, SSLContext exists in 3.2 but create_default_context appears in 3.4 while OP_NO_COMPRESSION appears in 3.3). I would much rather selectively backport a minimal set of APIs than the whole range of ssl APIs. There are things there (RAND_bytes, RAND_pseudo_bytes) that are not even useful for network security, or only in a rather uncommon manner (such as channel bindings). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Urk. Premature send. As it turns out, I had no specific comments to make after the one comment on > Alternative: create and release Python 2.8 > -- My apologies to anybody who read to the bottom for wasting their time. I do have one generic comment on style: I think at this point the statements tagged with "I (Nick Coghlan) think" should either be edited to remove the reference to the author, or moved to the "Caveat: I work for RH" section or the "Open Issues" section. If they're not pretty much consensus by now, they probably don't belong in the main body of the PEP. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Nick Coghlan writes: > Changes to these modules will still need to undergo normal backwards > compatibility assessments to ensure their default behaviour remains > consistent with earlier Python 2.7 releases, I find this wording confusing. Does this mean that (possibly imaginary example) where Python 3.n enables a new "host match" check by default that Python 2.7 doesn't have, Python 2.7 gets the ability to check "host match", but it defaults to off? > While this PEP does not make any changes to the core development team's > handling of security-fix-only branches that are no longer in active > maintenance, I wonder if it would be possible to generalize the approach to 2.7 in such a way that our repos can collect backports of security *enhancements* which we have reviewed, even though they wouldn't be distributed in a release? That way the distros can share their patches to 2.x, x < 7. > infrastructure on their platform. Accordingly, and independently of this > PEP, the Mac OS X binary installers were already going to be switched to > statically linker newer versions of OpenSSL [4]_ Typo: statically linker -> statically-linked > For better or for worse (mostly worse), there are some environments where Is that "For better or for worse (mostly worse)" really necessary? Consenting adults and all that. > the risk of latent security defects is more tolerated than even a slightly > increased risk of regressions in maintenance releases. This policy largely > excludes these environments from consideration where the modules covered by > the exemption are concerned Again, I think you can put a period here. The people making such decisions know what they're doing, at least they think they do. Just tell them they're on they're own now, and their complaints, if any, will fall on deaf ears. > Downstream redistributors may still choose to cater to such environments, > but they will need to handle the process of downgrading the security > related modules and doing the associated regression testing themselves. > The main CPython continuous integration infrastructure will not cover this > scenario. This is how to write it, I think. > Motivation and Rationale > > > This PEP can be seen as a more targeted version of the "faster standard > library release cycle" proposals discussed in PEP 407 and PEP 413, > focusing specifically on those areas which have implications beyond the > Python community. FWIW, I don't see it that way at all. > It is worth comparing the approach described in this PEP with Red Hat's > handling of its long term support commitments: it isn't the RHEL 6.0 release > itself that receives 10 years worth of support, but the overall RHEL 6 > *series*. The individual RHEL 6.x point releases within the series then > receive a wide variety of new features, including security enhancements, > all while meeting strict backwards compatibility guarantees for existing > software. I don't understand this comparison. I think it requires too much familiarity with Red Hat policy to be useful to the great majority of readers, ie, it tells them a lot more about Red Hat then it does about Python. > However, now that we're fully aware of the impact the limitations of the > Python 2 standard library may be having on the evolution of internet > security standards, I suggest s/evolution/implementation/ here. > As Terry Reedy noted, URL? > Alternative: create and release Python 2.8 > -- I don't see how this can possibly be viable. It would have to involve essentially the same restrictions as this PEP, or it completely fails to satisfy the requirements of downstreams. Even then, it would be a huge, uphill PR battle for downstreams trying to enhance their current 2.7 with the content of this PEP. > > With sufficient corporate support, it likely *would* be possible to create > and release Python 2.8 (it's highly unlikely such a project would garner > enough interest to be achievable with only volunteers). However, this > wouldn't actually solve the problem, as the aim is to provide a *relatively > low impact* way to incorporate enhanced security features into integrated > products and deployments that make use of Python 2. > > Upgrading to a new Python feature release would mean both more work for the > core development team, as well as a more disruptive update that most > potential end users would likely just skip entirely. > > Attempting to create a Python 2.8 release would also bring in suggestions > to backport many additional features from Python 3 (such as ``tracemalloc`` > and the improved coroutine support), making the migration from Python 2.7 > to this hypothetical 2.8 release even riskier and more disruptive. > > This is not a recommended approach, as it would involve substantial > additional work for a result that is actually less effective in achieving > the origina
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Am 25.03.14 14:47, schrieb Nick Coghlan: > The PEP says to sync with Python 3, and that has full cross platform > support. The Linux focus just comes from the fact that Linux is where > the problem is most evident. However, it fails to address a critical detail: the upcoming maintenance end for 2.7. This makes it less platform-agnostic than you say: For Linux, many users rely on the system vendor to provide Python binary, and they may chose to continue improving the backported APIs even after python-dev's maintenance ends. OTOH, for Windows, most users rely on the binaries from python.org, which will stop evolving in May 2015. Of course, there is still ActiveState, but there isn't the same kind of vendor choice that you see for Linux. So I think the PEP should make it clear whether: a) you expect that the maintenance period for Python 2.7 is to be extended beyond 2015, b) you expect that the backports will evolve in the years to come, or whether this will be a one-shot activity c) and if the answer to b was "will evolve", whether you expect this will create forks of 2.7 after maintenance has ended (as the then-security-only 2.7 branch won't see any further improvement to the backport, according to the PEP) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Mar 25, 2014, at 1:16 PM, Guido van Rossum wrote: > On Tue, Mar 25, 2014 at 9:46 AM, Donald Stufft wrote: > > On Mar 25, 2014, at 12:35 PM, Guido van Rossum wrote: > [...] >> >> I do note that the PEP seems to have some weasel-words about breaking >> backward compatibility in the name of security. The phrase "This PEP does >> not grant Python 2.7 any general exemptions to the usual backwards >> compatibility policy for maintenance releases" *could* be interpreted to >> imply that the PEP grants some specific exemptions (regardless of whether >> that was Nick's intention when he wrote that sentence). I'd like clarity on >> this; IIRC we've had to make some compatibility-breaking changes in the past >> for security reasons, but I don't recall the details or how that worked out >> (whether much code broke and whether that was considered a good or a bad >> thing). > > I’m pretty sure Nick was just trying to say that the changes made under this > PEP still have to be backwards compatible in the sense that APIs can’t change > their default behavior and such. In other words we can’t suddenly flip on > hostname checking or anything like that. > > Then the words should be clarified (maybe by removing 'general'?). This PEP > invites interpretation by future generations so it should be as clear as > possible on the intent, to avoid scholarly arguments. > > -- > --Guido van Rossum (python.org/~guido) Yea I agree, was just stating what I understand the PEP to be proposing :) - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, Mar 25, 2014 at 9:46 AM, Donald Stufft wrote: > > On Mar 25, 2014, at 12:35 PM, Guido van Rossum wrote: > [...] > > I do note that the PEP seems to have some weasel-words about breaking > backward compatibility in the name of security. The phrase "This PEP does > *not* grant Python 2.7 any general exemptions to the usual backwards > compatibility policy for maintenance releases" *could* be interpreted to > imply that the PEP grants some *specific* exemptions (regardless of > whether that was Nick's intention when he wrote that sentence). I'd like > clarity on this; IIRC we've had to make some compatibility-breaking changes > in the past for security reasons, but I don't recall the details or how > that worked out (whether much code broke and whether that was considered a > good or a bad thing). > > > I'm pretty sure Nick was just trying to say that the changes made under > this PEP still have to be backwards compatible in the sense that APIs can't > change their default behavior and such. In other words we can't suddenly > flip on hostname checking or anything like that. > Then the words should be clarified (maybe by removing 'general'?). This PEP invites interpretation by future generations so it should be as clear as possible on the intent, to avoid scholarly arguments. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Mar 25, 2014, at 12:35 PM, Guido van Rossum wrote: > On Tue, Mar 25, 2014 at 8:31 AM, Alex Gaynor wrote: > A casual glance at > https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/ > util.py#L610 > which is probably the most widely used consumer of these APIs, outside the > stdlib itself, looks to me like if these names were to suddenly show up, > everything would continue to work just fine, with the advance of being able to > explicitly specify some options. > > All of which is to say: I don't think this is a real concern. > > That would be great, because I have no other major beef with the PEP (but I > still need to read in in full -- it's long and half of it still feels like > weasel words to me, so I can't apply my usual skimming tactics). > > I would like the PEP (or perhaps a companion PEP?) spell out the set of > enhancements that we would *currently* like to see backported from Python 3.4 > to 2.7, without the implication that these would be the *only* enhancements > -- such a list would serve as an example and to focus the understanding. The > PEP currently doesn't even name SSLContext! > > I wouldn't be totally surprised to find that there are some details of some > API added to Python 3.4 that simply cannot be backported due to some > important difference between Python 2 and 3 (e.g. because of differences in > Unicode handling, or a missing socket method). I don't think such things > would be showstoppers, they would just have to be worked around carefully; > but it would be better to know about them now rather than having to figure > out how to comply with the PEP's insistence of a full backport. > > I do note that the PEP seems to have some weasel-words about breaking > backward compatibility in the name of security. The phrase "This PEP does not > grant Python 2.7 any general exemptions to the usual backwards compatibility > policy for maintenance releases" *could* be interpreted to imply that the PEP > grants some specific exemptions (regardless of whether that was Nick's > intention when he wrote that sentence). I'd like clarity on this; IIRC we've > had to make some compatibility-breaking changes in the past for security > reasons, but I don't recall the details or how that worked out (whether much > code broke and whether that was considered a good or a bad thing). I’m pretty sure Nick was just trying to say that the changes made under this PEP still have to be backwards compatible in the sense that APIs can’t change their default behavior and such. In other words we can’t suddenly flip on hostname checking or anything like that. > > -- > --Guido van Rossum (python.org/~guido) > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, Mar 25, 2014 at 8:31 AM, Alex Gaynor wrote: > A casual glance at > > https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/ > util.py#L610 > which is probably the most widely used consumer of these APIs, outside the > stdlib itself, looks to me like if these names were to suddenly show up, > everything would continue to work just fine, with the advance of being > able to > explicitly specify some options. > > All of which is to say: I don't think this is a real concern. That would be great, because I have no other major beef with the PEP (but I still need to read in in full -- it's long and half of it still feels like weasel words to me, so I can't apply my usual skimming tactics). I would like the PEP (or perhaps a companion PEP?) spell out the set of enhancements that we would *currently* like to see backported from Python 3.4 to 2.7, without the implication that these would be the *only* enhancements -- such a list would serve as an example and to focus the understanding. The PEP currently doesn't even name SSLContext! I wouldn't be totally surprised to find that there are some details of some API added to Python 3.4 that simply cannot be backported due to some important difference between Python 2 and 3 (e.g. because of differences in Unicode handling, or a missing socket method). I don't think such things would be showstoppers, they would just have to be worked around carefully; but it would be better to know about them now rather than having to figure out how to comply with the PEP's insistence of a full backport. I do note that the PEP seems to have some weasel-words about breaking backward compatibility in the name of security. The phrase "This PEP does *not* grant Python 2.7 any general exemptions to the usual backwards compatibility policy for maintenance releases" *could* be interpreted to imply that the PEP grants some *specific* exemptions (regardless of whether that was Nick's intention when he wrote that sentence). I'd like clarity on this; IIRC we've had to make some compatibility-breaking changes in the past for security reasons, but I don't recall the details or how that worked out (whether much code broke and whether that was considered a good or a bad thing). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
A casual glance at https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/ util.py#L610 which is probably the most widely used consumer of these APIs, outside the stdlib itself, looks to me like if these names were to suddenly show up, everything would continue to work just fine, with the advance of being able to explicitly specify some options. All of which is to say: I don't think this is a real concern. Alex ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, Mar 25, 2014 at 8:10 AM, Antoine Pitrou wrote: > The problem with backporting SSLContext is really that 1) you are adding > a non-minimal new API set to the 2.7 feature set; 2) you must make it so > that other stdlib modules take advantage of the new APIs (otherwise > why bother?). > I actually worry about another scenario. Suppose we backport SSLContext to 2.7.7. There is plenty of software out there nowadays that uses a single source for Python 2 and 3 compatibility. Given that SSLContext until now has only been available in Python 3, and given the recommendation to use dynamic feature detection, not version checks, it's at least *conceivable* that some library detects SSLContext and switches to some code that uses Python-3-only features (of the variety that don't cause syntax errors in Python 2). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On Tue, 25 Mar 2014 14:25:29 + Paul Moore wrote: > On 25 March 2014 13:47, Nick Coghlan wrote: > > It's not like we're going to just be giving the PEP to vendors as a spec and > > leaving them to it - it's largely an invitation to participate more directly > > upstream to help resolve a particularly thorny problem, not a Statement of > > Work :) > > :-) > > I don't really know the APIs involved, but AIUI one of the 3.4 > enhancements is exposing the SSLContext. No, it was done as soon as 3.2. Note that exposing SSLContext doesn't achieve anything *in itself*, until you also add relevant APIs to query and influence the SSLContext's internal state (which was done gradually in 3.2, 3.3 and 3.4). Separate enhancements have to do with strengthening the default SSL options and accepted SSL ciphers; when they didn't add new APIs or threatened to break legitimate use case, they have even been applied to bugfix branches. > Is the code to do this > compatible with the version of OpenSSL bundled with Python 2.7 on > Windows? It is. Actually, we have (or had) buildbots with older OpenSSL versions! The problem with backporting SSLContext is really that 1) you are adding a non-minimal new API set to the 2.7 feature set; 2) you must make it so that other stdlib modules take advantage of the new APIs (otherwise why bother?). > This "I've written a patch but it hasn't been applied" is the type of > scenario that puts people off contributing. In the case of ssl, I think I've tried to apply most interesting and finished patches. This is how you get features such as NPN protocols or server-side SNI. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 25 March 2014 13:47, Nick Coghlan wrote: > It's not like we're going to just be giving the PEP to vendors as a spec and > leaving them to it - it's largely an invitation to participate more directly > upstream to help resolve a particularly thorny problem, not a Statement of > Work :) :-) I don't really know the APIs involved, but AIUI one of the 3.4 enhancements is exposing the SSLContext. Is the code to do this compatible with the version of OpenSSL bundled with Python 2.7 on Windows? If not, suppose that Red Hat provide resources that work on backporting the code, but they don't have Windows experts so no-one deals with integrating the new OpenSSL into the Windows binaries. Would the backport be blocked until someone is found to do the Windows work? This "I've written a patch but it hasn't been applied" is the type of scenario that puts people off contributing. If it's likely to happen, I think Red Hat have a right to know that in advance. And I don't know that it's something they would appreciate without python-dev pointing it out. If we're reasonably sure (not necessarily certain, there's always grey areas) that this isn't going to be an issue, then that's also fine. We can simply say that. That's all I'm saying. Not trying to require anything of contributors, just trying to be open and explicit about the criteria that will apply to accepting contributions. Paul. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 25 Mar 2014 23:29, "Paul Moore" wrote: > > On 25 March 2014 13:09, Nick Coghlan wrote: > > * MvL has indicated he is not prepared to tackle the task of trying to > > integrate a newer OpenSSL into the also aging Python 2.7 build > > infrastructure on Windows (unfortunately, we've looked into upgrading > > that build infrastructure, and the backwards compatibility issues > > appear to be effectively insurmountable). We would require a commitment > > from another trusted contributor to handle at least this task, and > > potentially also taking over the task of creating the official > > Python 2.7 Windows installers for the remaining Python 2.7 maintenance > > releases. > > One issue that strikes me is that much of the focus of this PEP is on > supporting Linux distributions. This is entirely reasonable, as they > are the ones with the sort of long-term support commitments that > result in this issue (in the Windows world, possibly ActiveState offer > formal support for Python 2.7, but otherwise I'm not aware of actual > paid support options that might be relevant on Windows). With that in > mind, is it reasonable to expect Linux vendors to support delivery of > updated Windows builds of Python 2.7? If not, is it acceptable to > python-dev to release a Python 2.7 maintenance release with backported > security enhancements only available for Linux? (The same questions > can be asked of OSX or Solaris support - this isn't solely a Windows > issue). > > I think the PEP needs to be explicit here about what python-dev expect > in terms of cross-platform support. I would assume that the > expectation is that we deliver exactly the same level of > cross-platform support as for 3.x, but commercial vendors could quite > easily miss that implication if it is not spelled out. The PEP says to sync with Python 3, and that has full cross platform support. The Linux focus just comes from the fact that Linux is where the problem is most evident. It's not like we're going to just be giving the PEP to vendors as a spec and leaving them to it - it's largely an invitation to participate more directly upstream to help resolve a particularly thorny problem, not a Statement of Work :) Cheers, Nick. > > Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
On 25 March 2014 13:09, Nick Coghlan wrote: > * MvL has indicated he is not prepared to tackle the task of trying to > integrate a newer OpenSSL into the also aging Python 2.7 build > infrastructure on Windows (unfortunately, we've looked into upgrading > that build infrastructure, and the backwards compatibility issues > appear to be effectively insurmountable). We would require a commitment > from another trusted contributor to handle at least this task, and > potentially also taking over the task of creating the official > Python 2.7 Windows installers for the remaining Python 2.7 maintenance > releases. One issue that strikes me is that much of the focus of this PEP is on supporting Linux distributions. This is entirely reasonable, as they are the ones with the sort of long-term support commitments that result in this issue (in the Windows world, possibly ActiveState offer formal support for Python 2.7, but otherwise I'm not aware of actual paid support options that might be relevant on Windows). With that in mind, is it reasonable to expect Linux vendors to support delivery of updated Windows builds of Python 2.7? If not, is it acceptable to python-dev to release a Python 2.7 maintenance release with backported security enhancements only available for Linux? (The same questions can be asked of OSX or Solaris support - this isn't solely a Windows issue). I think the PEP needs to be explicit here about what python-dev expect in terms of cross-platform support. I would assume that the expectation is that we deliver exactly the same level of cross-platform support as for 3.x, but commercial vendors could quite easily miss that implication if it is not spelled out. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
Hmm, should probably have more entries in the post history... Anyway: * As Ben Darnell noted, the legacy SSL branch thing was a fundamentally bad idea. We can't stop downstream redistributors doing that if they really want to, but we don't have to encourage them * I also explained why I think selective backports are a bad idea * I've given a lot more context on how Red Hat actually manages to handle such long term support cycles, and that what I'm proposing here really isn't that radical from that perspective (and is, in fact, about the only available solution that can be effectively slotted into that model) * I've been a lot more explicit about the fact that long term maintenance releases are genuinely *different* from CPython's normal release lifetimes * more details on the specific support I think this proposal would need to become a practical reality * removed some unhelpful side comments and obliquely acknowledged the fact that it mat conceivably have been difficult for outsiders to see that it might actually be possible to make this case effectively :) Cheers, Nick. Diff: http://hg.python.org/peps/rev/4eb8de143801 Web: http://www.python.org/dev/peps/pep-0466/ === PEP: 466 Title: Network Security Enhancement Exception for Python 2.7 Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan , Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Mar-2014 Post-History: 23-Mar-2014 Abstract Most CPython tracker issues are classified as errors in behaviour or proposed enhancements. Most patches to fix behavioural errors are applied to all active maintenance branches. Enhancement patches are restricted to the default branch that becomes the next Python version. This cadence works reasonably well during Python's normal 18-24 month feature release cycle, which is still applicable to the Python 3 series. However, the age of the standard library in Python 2 has now reached a point where it is sufficiently far behind the state of the art in network security protocols for it to be causing real problems in commercial use cases where upgrading to Python 3 in the near term may not be practical. In recognition of the additional practical considerations that have arisen during the 4+ year maintenance cycle for Python 2.7, this PEP allows Python 2.7 standard library components that have implications for the overall security of the internet to be updated in line with the corresponding Python 3 feature releases. Specifically, the exception will apply to: * the ``ssl`` module * the ``hashlib`` module * the ``hmac`` module * the components of the ``random`` and ``os`` modules that the above modules rely on for cryptographic randomness * the version of OpenSSL bundled with the binary installers for Windows and Mac OS X Changes to these modules will still need to undergo normal backwards compatibility assessments to ensure their default behaviour remains consistent with earlier Python 2.7 releases, but otherwise they will be kept entirely aligned with the latest feature release of their Python 3 counterparts. This is designed to make it easier to implement secure networked software in Python, even for software that currently needs to remain compatible with the Python 2 series (which includes a lot of network infrastructure software). While this PEP does not make any changes to the core development team's handling of security-fix-only branches that are no longer in active maintenance, it *does* recommend that commercial redistributors providing extended support periods for the Python standard library either adopt a similar approach to ensuring that the secure networking infrastructure keeps pace with the evolution of the internet, or else explicitly disclaim support for the use of older versions in roles that involve connecting directly to the public internet. Exemption Policy Under this policy, the following network security related modules are granted a blanket exemption to the normal restriction against adding new features in Python 2.7 maintenance releases, for the purpose of keeping their APIs aligned with their counterparts in the latest feature release of Python 3: * the ``ssl`` module * the ``hashlib`` module * the ``hmac`` module Under this exemption, these modules are updated to provide identical functionality to their Python 3 counterparts after every new Python 3 feature release. The default behaviour of the backported modules will be adjusted as appropriate to meet the backwards compatibility requirements of a Python 2.7 maintenance release. As part of this policy, permission is also granted to upgrade to newer feature releases of OpenSSL when preparing the binary installers for new maintenance releases of Python 2.7. Note that the ``sha`` and ``md5`` modules are not covered by this policy, but have been deprecated in favour of ``hashlib`` since Python 2.5 and have been removed entirely in the Python 3 series
