Re: [Python-Dev] Mac popups running make test
On 5/10/15 10:29 AM, Tal Einat wrote: On Sun, May 10, 2015 at 5:07 PM, Brett Cannon br...@python.org mailto:br...@python.org wrote: On Sun, May 10, 2015 at 10:04 AM Skip Montanaro skip.montan...@gmail.com mailto:skip.montan...@gmail.com wrote: I haven't run the test suite in awhile. I am in the midst of running it on my Mac running Yosemite 10.10.3. Twice now, I've gotten this popup: I assume this is testing some server listening on localhost. Is this a new thing, either with the Python test suite or with Mac OS X? (I'd normally be hidden behind a NAT firewall, but at the moment I am on a miserable public connection in a Peet's Coffee, so it takes on slightly more importance...) It's not new. Indeed, I've run into this as well. I've also seen the Crash Reporter pop up many times, but as far as I could tell, in all cases the test suite output told me it was expected. Perhaps tests which listen for network connections should also mention that, at least on Macs? Wouldn't hurt. Just requires tracking down which test(s) triggers it (might be more than one and I don't know if answering that popup applies for the rest of the test execution or once per test if you use -j). If anyone starts working on this, let me know if I can help, e.g. trying things on my own Mac. I believe that the message has to do with OS X's sandboxing implementation and the setting of the sandbox's entitlement keys. Here's an Apple doc: https://developer.apple.com/library/ios/documentation/Miscellaneous/Reference/EntitlementKeyReference/Chapters/EnablingAppSandbox.html#//apple_ref/doc/uid/TP40011195-CH4-SW9 I'm unaware of a way to work around this other than using Apple's code signing or adjusting target build settings in XCode :( If anyone knows a good way to workaround or manually set permission (other than clicking the Allow button), I would be interested. Warmly, Carol -- *Carol Willing* Developer | Willing Consulting https://willingconsulting.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anomaly
In case the example given at the start of the thread wasn't
interesting enough, it also works in the other direction:
class str(int): pass
str('2')
2 #- an integer!!!
Mark
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] anomaly
Here's something that might be wrong in Python (tried on v2.7): class int(str): pass int(3) '3' Mark ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode literals in Python 2.7
On 10 May 2015 at 23:28, Adam Bartoš dre...@gmail.com wrote: Glenn Linderman wrote: Is this going to get released in 3.5, I hope? Python 3 is pretty limited without some solution for Unicode on the console... probably the biggest deficiency I have found in Python 3, since its introduction. It has great Unicode support for files and processing, which convinced me to switch from Perl, and I like so much else about it, that I can hardly code in Perl any more (I still support a few Perl programs, but have ported most of them to Python). I'd love to see it included in 3.5, but I doubt that will happen. For one thing, it's only two weeks till beta 1, which is feature freeze. And mainly, my package is mostly hacking into existing Python environment. A proper implementation would need some changes in Python someone would have to do. See for example my proposal http://bugs.python.org/issue17620#msg234439. I'm not competent to write a patch myself and I have also no feedback to the proposed idea. On the other hand, using the package is good enough for me so I didn't further bring attention to the proposal. Right, and while I'm interested in seeing this improved, I'm not especially familiar with the internal details of our terminal interaction implementation, and even less so when it comes to the Windows terminal. Steve Dower's also had his hands full working on the Windows installer changes, and several of our other Windows folks aren't C programmers. PEP 432 (the interpreter startup sequence improvements) will be back on the agenda for Python 3.6, so the 3.6 time frame seems more plausible at this point. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 11 May 2015 at 18:04, M.-A. Lemburg m...@egenix.com wrote: On 10.05.2015 05:04, Robert Collins wrote: On 10 May 2015 at 11:44, Chris Angelico ros...@gmail.com wrote: On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg m...@egenix.com wrote: By providing a way to intentionally switch off the new default, we do make people aware of the risks and that's good enough, while still maintaining the contract people rightly expect of patch level releases of Python. Just as long as it's the sysadmin, and NOT some random attacker over the internet, who has the power to downgrade security. Environment variables can be attacked in various ways. They can, and the bash fun was very good evidence of that. OTOH if someones environment is at risk, PATH and PYTHONPATH are already very effective attack vectors. If an attacker has access to the process environment, you're doomed anyway, so that's not really an argument for or against using environment variables :-) The core issue lies in managing the user vs administrator permissions split. Even for self-administered systems, it's recommended practice to run *without* administrative permissions normally, and only elevate when you need them. One of the things you're watching out for in such cases is that it shouldn't be possible for an attacker to make a change to the user environment, and have that propagate to have an effect on a process running with administrative access. One of the recommended hardening measures against that kind of attack vector is to *turn off* Python's environment variable processing when launching Python processes with administrative access. We didn't care about that in the hash randomisation case, as the compatibility concern there applied on a per application basis, and caring about hash order was technically a bug in its own right. By contrast, in the situation we're worried about for certificate verification compatibility, the issue is environmental: certificate management in many private intranets isn't yet to the same standard as that on the public internet, so administrators may have a valid reason for defaulting Python back to the old behaviour, and redistributors may feel obliged to provide an opt-in period prior to switching the default behaviour to opt-out. Having the new setting be ignored in Python processes run under a hardened configuration means that an environment variable based solution won't have the desired effect in providing that smoother migration path to the more hardened configuration. I've now written a draft recommendations to redistributors PEP for Robert's configuration file based design: https://www.python.org/dev/peps/pep-0493/ (exact file names config setting names TBD) I wouldn't be opposed to seeing that as an upstream Python 2.7.x feature, but agreement amongst redistributors on using a file-based approach is the main outcome I'm looking for. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On May 11, 2015, at 6:15 AM, Nick Coghlan ncogh...@gmail.com wrote: On 11 May 2015 at 19:22, M.-A. Lemburg m...@egenix.com wrote: On 11.05.2015 11:13, Nick Coghlan wrote: I wouldn't be opposed to seeing that as an upstream Python 2.7.x feature, but agreement amongst redistributors on using a file-based approach is the main outcome I'm looking for. Can't we have both ? I'd strongly advise against it, as we're deliberately increasing the attack surface area here by providing a potential path to carry out a downgrade attack on the HTTPS certificate verification by forcing it back to the old behaviour. The main existing environment variable based attack vector would be to manage to run a process with administrative privileges and SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to a user or world-writable directory by the attacker. Providing a don't verify HTTPS flag at the interpreter level would let an attacker skip the first step of writing the certificate file to disk somewhere, making a system compromise harder to detect. (An especially paranoid SSL implementation would disallow reading certs from locations that allow write access to non-administrative users, but I don't believe OpenSSL is that paranoid) By contrast, the configuration file shouldn't provide a new attack vector (or simplify any existing attack vector), as if you have the permissions needed to modify the config file, you likely also have the permissions needed to modify the system certificate store, and the latter is a *far* more interesting attack vector than a downgrade attack solely against Python. Thus the environment variable based off switch is neither necessary (as an administrator controlled configuration file can do the job), nor sufficient (it can't handle the -E switch), *and* it represents an increase in the attack surface area relative to a Python implementation without the capability. I don't think that we can wait for a whole PEP process to run through Matrix multiplication made it through the PEP process in 8 days. If we do this as a redistributor recommendation rather than attempting to get it into upstream Python 2.7, we could potentially propose you take on the role of BDFL-Delegate and mark it as Accepted as soon as the two of us agree on a common approach. The reason I think that's a reasonable way forward is because we already know there are folks opposed to making the change upstream. If the PEP just provides recommendations for redistributors that *do* decide to provide a global off switch to revert to the old behaviour, then the perspective of the folks opposed to the feature is respected by the fact that this is a feature some redistributors *may* choose to add to provide a smoother migration path to more secure default HTTPS handling, rather than something upstream provides by default. I assume the Debian, Ubuntu and Suse folks won't care, as they have all already decided against backporting the change to their long term support releases where the compatibility break would pose a problem (and I can certainly sympathise with that perspective given the dependency on backporting the PEP 466 SSL changes first, and the work involved in seeking consensus on a smoother migration path from the old default to the new one). It would be nice to hear from ActiveState, Enthought Continuum Analytics as well, but if they don't chime in here, I don't see any particular need to go chasing them explicitly. to fix this regression in 2.7.9. We made the decision when PEP 476 was accepted that this change turned a silent security failure into a noisy one, rather than being a regression in its own right. PEP 493 isn't about disagreeing with that decision, it's about providing a smoother upgrade path in contexts where letting the security failure remain silent is deemed to be preferred in the near term. I don't really agree that the decision to disable TLS is an environment one, it's really a per application decision. This is why I was against having some sort of global off switch for all of Python because just because one application needs it turned off doesn't mean you want it turned off for another Python application. You might have some script that is interacting with a custom internal server which doesn’t have a valid TLS certificate but then you also have pip* installed which is reaching out to PyPI and downloading code from the internet. You might want to disable TLS verification for the first but you almost certainly don't want it to disable TLS verification for the second one. In this regard I think that environment variables are somewhat better because they are far easier to set per application instead of in a way that affects every python program. Per application is the *right* scope for this setting, especially in a system where people may or may not realize what is written in Python and what isn't. I think
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 11 May 2015 at 20:23, Donald Stufft don...@stufft.io wrote: On May 11, 2015, at 6:15 AM, Nick Coghlan ncogh...@gmail.com wrote: We made the decision when PEP 476 was accepted that this change turned a silent security failure into a noisy one, rather than being a regression in its own right. PEP 493 isn't about disagreeing with that decision, it's about providing a smoother upgrade path in contexts where letting the security failure remain silent is deemed to be preferred in the near term. I don't really agree that the decision to disable TLS is an environment one, it's really a per application decision. This is why I was against having some sort of global off switch for all of Python because just because one application needs it turned off doesn't mean you want it turned off for another Python application. The scenario I'm interested in is the one where it *was* off globally (i.e. you were already running Python 2.7.8 or earlier) and you want to manage a global rollout of a new Python version that supports being configured to verify HTTPS certificates by default, while making the decision on whether or not to enable HTTPS certificate verification on a server-by-server basis, rather than having that decision be coupled directly to the rollout of the updated version of Python. I agree that the desired end state is where Python 3 is, and where upstream Python 2.7.9+ is, this is solely about how to facilitate folks getting from point A to point B without an intervening window of I broke the world and now my boss is yelling at me about it :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
I'm in entire agreement with Donald below. Regards Antoine. On Mon, 11 May 2015 06:23:11 -0400 Donald Stufft don...@stufft.io wrote: I don't really agree that the decision to disable TLS is an environment one, it's really a per application decision. This is why I was against having some sort of global off switch for all of Python because just because one application needs it turned off doesn't mean you want it turned off for another Python application. You might have some script that is interacting with a custom internal server which doesn’t have a valid TLS certificate but then you also have pip* installed which is reaching out to PyPI and downloading code from the internet. You might want to disable TLS verification for the first but you almost certainly don't want it to disable TLS verification for the second one. In this regard I think that environment variables are somewhat better because they are far easier to set per application instead of in a way that affects every python program. Per application is the *right* scope for this setting, especially in a system where people may or may not realize what is written in Python and what isn't. I think it's absolutely wrong to give people a footgun in the terms of a switch that turns off all of Python's TLS verification when for many applications the fact they use Python is simply an implementation detail. That being said, since it's not being included in Python core and it's only some patch that some downstream's are going to apply I also don't really care that much because it's not going to effect me and if it turns out to be a bad idea and a footgun like I think it is, then the blame can rest on those downstreams and not us :) I'm also not a fan of the environment variable either really for a lot of the reasons you've outlined here. * Ignoring the fact that pip has (via requests/urllib3) worked around this deficiency in Python and isn't going to be affected by this configuration switch at all. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 11.05.2015 12:47, Nick Coghlan wrote: On 11 May 2015 at 20:23, Donald Stufft don...@stufft.io wrote: On May 11, 2015, at 6:15 AM, Nick Coghlan ncogh...@gmail.com wrote: We made the decision when PEP 476 was accepted that this change turned a silent security failure into a noisy one, rather than being a regression in its own right. PEP 493 isn't about disagreeing with that decision, it's about providing a smoother upgrade path in contexts where letting the security failure remain silent is deemed to be preferred in the near term. I don't really agree that the decision to disable TLS is an environment one, it's really a per application decision. This is why I was against having some sort of global off switch for all of Python because just because one application needs it turned off doesn't mean you want it turned off for another Python application. The scenario I'm interested in is the one where it *was* off globally (i.e. you were already running Python 2.7.8 or earlier) and you want to manage a global rollout of a new Python version that supports being configured to verify HTTPS certificates by default, while making the decision on whether or not to enable HTTPS certificate verification on a server-by-server basis, rather than having that decision be coupled directly to the rollout of the updated version of Python. I guess we're approaching this from different perspectives :-) I'm mostly interested in having a switch that can be set on a per application basis, not globally. I think the global default is fine and I'm just looking for a way to have admins disable it on a case-by-case basis for those applications which have problems with the new default. Hence the env var approach - the admin would simply edit the application's startup shell script, add the env var and that's it. For pip et al. which don't use the ssl module, the admins can simply continue using older versions for those applications - ones which don't implement the extra verification. In many cases, this is not necessary, since production environments typically don't use PyPI at all: they use a local directory with the needed distribution files, which is both more secure and reliable. I agree that the desired end state is where Python 3 is, and where upstream Python 2.7.9+ is, this is solely about how to facilitate folks getting from point A to point B without an intervening window of I broke the world and now my boss is yelling at me about it :) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 11 2015) Python Projects, Coaching and Consulting ... http://www.egenix.com/ mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 10.05.2015 05:04, Robert Collins wrote: On 10 May 2015 at 11:44, Chris Angelico ros...@gmail.com wrote: On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg m...@egenix.com wrote: By providing a way to intentionally switch off the new default, we do make people aware of the risks and that's good enough, while still maintaining the contract people rightly expect of patch level releases of Python. Just as long as it's the sysadmin, and NOT some random attacker over the internet, who has the power to downgrade security. Environment variables can be attacked in various ways. They can, and the bash fun was very good evidence of that. OTOH if someones environment is at risk, PATH and PYTHONPATH are already very effective attack vectors. If an attacker has access to the process environment, you're doomed anyway, so that's not really an argument for or against using environment variables :-) You'd just need to create a file os.py and point PYTHONPATH at it. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 11 2015) Python Projects, Coaching and Consulting ... http://www.egenix.com/ mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 11.05.2015 11:13, Nick Coghlan wrote: On 11 May 2015 at 18:04, M.-A. Lemburg m...@egenix.com wrote: On 10.05.2015 05:04, Robert Collins wrote: On 10 May 2015 at 11:44, Chris Angelico ros...@gmail.com wrote: On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg m...@egenix.com wrote: By providing a way to intentionally switch off the new default, we do make people aware of the risks and that's good enough, while still maintaining the contract people rightly expect of patch level releases of Python. Just as long as it's the sysadmin, and NOT some random attacker over the internet, who has the power to downgrade security. Environment variables can be attacked in various ways. They can, and the bash fun was very good evidence of that. OTOH if someones environment is at risk, PATH and PYTHONPATH are already very effective attack vectors. If an attacker has access to the process environment, you're doomed anyway, so that's not really an argument for or against using environment variables :-) The core issue lies in managing the user vs administrator permissions split. Even for self-administered systems, it's recommended practice to run *without* administrative permissions normally, and only elevate when you need them. One of the things you're watching out for in such cases is that it shouldn't be possible for an attacker to make a change to the user environment, and have that propagate to have an effect on a process running with administrative access. One of the recommended hardening measures against that kind of attack vector is to *turn off* Python's environment variable processing when launching Python processes with administrative access. The env var would not be read at Python startup time, only when loading the ssl module, so the -E switch would not have the effect of disabling it - unlike the hash seed logic, which is run (and has to be run) at Python startup time. We didn't care about that in the hash randomisation case, as the compatibility concern there applied on a per application basis, and caring about hash order was technically a bug in its own right. By contrast, in the situation we're worried about for certificate verification compatibility, the issue is environmental: certificate management in many private intranets isn't yet to the same standard as that on the public internet, so administrators may have a valid reason for defaulting Python back to the old behaviour, and redistributors may feel obliged to provide an opt-in period prior to switching the default behaviour to opt-out. Having the new setting be ignored in Python processes run under a hardened configuration means that an environment variable based solution won't have the desired effect in providing that smoother migration path to the more hardened configuration. I've now written a draft recommendations to redistributors PEP for Robert's configuration file based design: https://www.python.org/dev/peps/pep-0493/ (exact file names config setting names TBD) The Fastly cache seems to be having problems again. I only get: 503 Backend is unhealthy - Details: cache-fra1225-FRA 1431335851 2631441948 I wouldn't be opposed to seeing that as an upstream Python 2.7.x feature, but agreement amongst redistributors on using a file-based approach is the main outcome I'm looking for. Can't we have both ? I don't think that we can wait for a whole PEP process to run through to fix this regression in 2.7.9. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 11 2015) Python Projects, Coaching and Consulting ... http://www.egenix.com/ mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 11 May 2015 at 19:22, M.-A. Lemburg m...@egenix.com wrote: On 11.05.2015 11:13, Nick Coghlan wrote: I wouldn't be opposed to seeing that as an upstream Python 2.7.x feature, but agreement amongst redistributors on using a file-based approach is the main outcome I'm looking for. Can't we have both ? I'd strongly advise against it, as we're deliberately increasing the attack surface area here by providing a potential path to carry out a downgrade attack on the HTTPS certificate verification by forcing it back to the old behaviour. The main existing environment variable based attack vector would be to manage to run a process with administrative privileges and SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to a user or world-writable directory by the attacker. Providing a don't verify HTTPS flag at the interpreter level would let an attacker skip the first step of writing the certificate file to disk somewhere, making a system compromise harder to detect. (An especially paranoid SSL implementation would disallow reading certs from locations that allow write access to non-administrative users, but I don't believe OpenSSL is that paranoid) By contrast, the configuration file shouldn't provide a new attack vector (or simplify any existing attack vector), as if you have the permissions needed to modify the config file, you likely also have the permissions needed to modify the system certificate store, and the latter is a *far* more interesting attack vector than a downgrade attack solely against Python. Thus the environment variable based off switch is neither necessary (as an administrator controlled configuration file can do the job), nor sufficient (it can't handle the -E switch), *and* it represents an increase in the attack surface area relative to a Python implementation without the capability. I don't think that we can wait for a whole PEP process to run through Matrix multiplication made it through the PEP process in 8 days. If we do this as a redistributor recommendation rather than attempting to get it into upstream Python 2.7, we could potentially propose you take on the role of BDFL-Delegate and mark it as Accepted as soon as the two of us agree on a common approach. The reason I think that's a reasonable way forward is because we already know there are folks opposed to making the change upstream. If the PEP just provides recommendations for redistributors that *do* decide to provide a global off switch to revert to the old behaviour, then the perspective of the folks opposed to the feature is respected by the fact that this is a feature some redistributors *may* choose to add to provide a smoother migration path to more secure default HTTPS handling, rather than something upstream provides by default. I assume the Debian, Ubuntu and Suse folks won't care, as they have all already decided against backporting the change to their long term support releases where the compatibility break would pose a problem (and I can certainly sympathise with that perspective given the dependency on backporting the PEP 466 SSL changes first, and the work involved in seeking consensus on a smoother migration path from the old default to the new one). It would be nice to hear from ActiveState, Enthought Continuum Analytics as well, but if they don't chime in here, I don't see any particular need to go chasing them explicitly. to fix this regression in 2.7.9. We made the decision when PEP 476 was accepted that this change turned a silent security failure into a noisy one, rather than being a regression in its own right. PEP 493 isn't about disagreeing with that decision, it's about providing a smoother upgrade path in contexts where letting the security failure remain silent is deemed to be preferred in the near term. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anomaly
On 10 May 2015 at 17:34, Mark Rosenblitt-Janssen dreamingforw...@gmail.com wrote: Here's something that might be wrong in Python (tried on v2.7): class int(str): pass int(3) '3' It's not wrong as such. It is allowed to define your own class that subclasses a builtin class, and it's allowed to shadow builtin names. So while this is (obviously) bad practice, it's not wrong. For a simpler example: Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AMD64)] on win32 Type help, copyright, credits or license for more information. str class 'str' str = Hello str 'Hello' Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On May 11, 2015, at 6:47 AM, Nick Coghlan ncogh...@gmail.com wrote: On 11 May 2015 at 20:23, Donald Stufft don...@stufft.io wrote: On May 11, 2015, at 6:15 AM, Nick Coghlan ncogh...@gmail.com wrote: We made the decision when PEP 476 was accepted that this change turned a silent security failure into a noisy one, rather than being a regression in its own right. PEP 493 isn't about disagreeing with that decision, it's about providing a smoother upgrade path in contexts where letting the security failure remain silent is deemed to be preferred in the near term. I don't really agree that the decision to disable TLS is an environment one, it's really a per application decision. This is why I was against having some sort of global off switch for all of Python because just because one application needs it turned off doesn't mean you want it turned off for another Python application. The scenario I'm interested in is the one where it *was* off globally (i.e. you were already running Python 2.7.8 or earlier) and you want to manage a global rollout of a new Python version that supports being configured to verify HTTPS certificates by default, while making the decision on whether or not to enable HTTPS certificate verification on a server-by-server basis, rather than having that decision be coupled directly to the rollout of the updated version of Python. I agree that the desired end state is where Python 3 is, and where upstream Python 2.7.9+ is, this is solely about how to facilitate folks getting from point A to point B without an intervening window of I broke the world and now my boss is yelling at me about it :) Oh, another issue that I forgot to mention-- A fair number of people had no idea that Python wasn't validating TLS before 2.7.9/3.4.3 however as part of the processing of changing that in 2.7.9 a lot of people became aware that Python's before 2.7.9 didn't validate but that Python 2.7.9+ does. I worry that if Redhat (or anyone) ships a Python 2.7.9 that doesn't verify by default then they are going to be shipping something which defies the expectations of those users who were relying on the fact that Python 2.7.9+ was supposed to be secure by default now. You're (understandibly) focusing on I already have my thing running on Python 2.7.8 and I want to yum update and get 2.7.9 and have things not visibly break, however there is the other use case of I'm setting up a new environment, and I installed RHEL and got 2.7.9, I remembered reading in LWN that 2.7.9 verifies now so I must be safe. If you *do* provide such a switch, defaulting it to verify and having people where that breaks go in and turn it off is probably a safer mechanism since the cases where 2.7.9 verification breaks things for people is a visible change where the case that someone expects 2.7.9 to verify and it doesn't isn't a visible change and is easily missed unless they go out of their way to try and test it against a server with an invalid certificate. Either way, if there is some sort of global off switch, having that off switch set to off should raise some kind of warning (like urllib3 does if you use the unverified HTTPS methods). To be clear, I don't mean that using the built in ssl module APIs to disable verification should raise a warning, I mean the hypothetical make my Python insecurely access HTTPS configuration file (or environment variable) that is being proposed. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 11 May 2015 10:16 pm, Robert Kuska rku...@redhat.com wrote: Oh, another issue that I forgot to mention-- A fair number of people had no idea that Python wasn't validating TLS before 2.7.9/3.4.3 however as part of the processing of changing that in 2.7.9 a lot of people became aware that Python's before 2.7.9 didn't validate but that Python 2.7.9+ does. I worry that if Redhat (or anyone) ships a Python 2.7.9 that doesn't verify by default then they are going to be shipping something which defies the expectations of those users who were relying on the fact that Python 2.7.9+ was supposed to be secure by default now. You're (understandibly) focusing on I already have my thing running on Python 2.7.8 and I want to yum update and get 2.7.9 and have things not visibly break, As Robert noted, it would be a matter of updating to a 2.7.5 with more patches backported, rather than rebasing to a newer upstream version. I can make the do not change the default behaviour relative to the corresponding upstream version guidance explicit in the PEP, though. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASE] Python 2.7.10 release candidate 1
It is my privilege to announce the first release candidate of 2.7.10, the next bugfix release in the 2.7 series. Downloads are at https://www.python.org/downloads/release/python-2710rc1/ The full changelog is at https://hg.python.org/cpython/raw-file/80ccce248ba2/Misc/NEWS Please consider testing 2.7.10rc1 with your application and reporting bugs to https://bugs.python.org Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Minimal async event loop and async utilities (Was: PEP 492: async/await in Python; version 4)
On Mon, May 11, 2015 at 1:37 PM, Paul Moore p.f.mo...@gmail.com wrote: On 6 May 2015 at 16:46, Guido van Rossum gu...@python.org wrote: This is actually a great idea, and I encourage you to go forward with it. The biggest piece missing from your inventory is probably Task, which is needed to wrap a Future around a coroutine. OK, I've been doing some work on this. You're right, the asyncio framework makes Future a key component. But I'm not 100% sure why Future (and Task) have to be so fundamental. Ignoring cancellation (see below!) I can build pretty much all of a basic event loop, plus equivalents of the asyncio locks and queues modules, without needing the concept of a Future at all. The create_task function becomes simply a function to add a coroutine to the ready queue, in this context. I can't return a Task (because I haven't implemented the Task or Future classes) but I don't actually know what significant functionality is lost as a result - is there a reasonably accessible example of where using the return value from create_task is important anywhere? In asyncio the Task object is used to wait for the result. Of course if all you need is to wait for the result you don't need to call create_task() -- so in your situation it's uninteresting. But Task is needed for cancellation and Future is needed so I/O completion can be implemented using callback functions. A slightly more complicated issue is with the run_until_complete function, which takes a Future, and hence is fundamentally tied to the Future API. However, it seems to me that a minimal implementation could work by having a run_until_complete() that just took an awaitable (i.e., anything that you can yield from). Again, is there a specific reason that you ended up going with run_until_complete taking a Future rather than just a coroutine? Actually it takes a Future *or* a coroutine. (The docs or the arg name may be confusing.) In asyncio, pretty much everything that takes one takes the other. I think (but haven't confirmed yet by implementing it) that it should be possible to create a coroutine that acts like a Future, in the sense that you can tell it from outside (via send()) that it's completed and set its return value. But this is all theory, and if you have any practical experience that shows I'm going down a dead end, I'd be glad to know. I don't know -- I never explored that. I'm not sure how useful this line of attack will be - if the API isn't compatible with asyncio.BaseEventLoop, it's not very useful in practice. On the other hand, if I can build a loop without Future or Task classes, it may indicate that those classes aren't quite as fundamental as asyncio makes them (which may allow some simplifications or generalisations). Have you tried to implement waiting for I/O yet? OTOH you may look at micropython's uasyncio -- IIRC it doesn't have Futures and it definitely has I/O waiting. I expect you'll also want to build cancellation into your base async framework; and the primitives to wait for multiple awaitables. The next step would be some mechanism to implement call_later()/call_at() (but this needs to be pluggable since for a real event loop it needs to be implemented by the basic I/O selector). These are where I suspect I'll have the most trouble if I haven't got a solid understanding of the role of the Future and Task classes (or alternatively, how to avoid them :-)) So I'm holding off on worrying about them for now. But certainly they need to be covered. In particular, call_later/call_at are the only generic example of any form of wait that actually *waits*, rather than returning immediately. So as you say, implementing them will show how the basic mechanism can be extended with a real selector (whether for I/O, or GUI events, or whatever). Right. If you can get this working it would be great to include this in the stdlib as a separate asynclib library. The original asyncio library would then be a specific implementation (using a subclass of asynclib.EventLoop) that adds I/O, subprocesses, and integrates with the selectors module (or with IOCP, on Windows). One thing I've not really considered in the above, is how a refactoring like this would work. Ignoring the let's try to remove the Future class approach above, my basic event loop is mostly just an alternative implementation of an event loop (or maybe an alternative policy - I'm not sure I understand the need for policies yet). A policy is mostly a wrapper around an event loop factory plus state that records the current event loop. So it may simply be a case of ripping coroutines.py, futures.py, locks.py, log.py, queues.py, and tasks.py out of asyncio and adding a new equivalent of events.py with my minimal loop in it. (So far, when I've tried to do that I get hit with some form of circular import problem - I've not worked out why yet, or how asyncio avoids the
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 11.05.2015 12:15, Nick Coghlan wrote: On 11 May 2015 at 19:22, M.-A. Lemburg m...@egenix.com wrote: On 11.05.2015 11:13, Nick Coghlan wrote: I wouldn't be opposed to seeing that as an upstream Python 2.7.x feature, but agreement amongst redistributors on using a file-based approach is the main outcome I'm looking for. Can't we have both ? I'd strongly advise against it, as we're deliberately increasing the attack surface area here by providing a potential path to carry out a downgrade attack on the HTTPS certificate verification by forcing it back to the old behaviour. The main existing environment variable based attack vector would be to manage to run a process with administrative privileges and SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to a user or world-writable directory by the attacker. Providing a don't verify HTTPS flag at the interpreter level would let an attacker skip the first step of writing the certificate file to disk somewhere, making a system compromise harder to detect. (An especially paranoid SSL implementation would disallow reading certs from locations that allow write access to non-administrative users, but I don't believe OpenSSL is that paranoid) Correct. OpenSSL will happily read the cert files from anywhere you point it to. By contrast, the configuration file shouldn't provide a new attack vector (or simplify any existing attack vector), as if you have the permissions needed to modify the config file, you likely also have the permissions needed to modify the system certificate store, and the latter is a *far* more interesting attack vector than a downgrade attack solely against Python. Thus the environment variable based off switch is neither necessary (as an administrator controlled configuration file can do the job), nor sufficient (it can't handle the -E switch), *and* it represents an increase in the attack surface area relative to a Python implementation without the capability. Whether or not -E will have an effect on the env var depends on the implementation. At the moment, -E only has an effect on the C runtime, while the stdlib happily reads from os.environ without taking the flag into account. As proposed, the PYTHONHTTPSVERIFY would only affect the ssl module and only be checked when loading this module, i.e. not at Python startup time. I don't think that we can wait for a whole PEP process to run through Matrix multiplication made it through the PEP process in 8 days. If we do this as a redistributor recommendation rather than attempting to get it into upstream Python 2.7, we could potentially propose you take on the role of BDFL-Delegate and mark it as Accepted as soon as the two of us agree on a common approach. The reason I think that's a reasonable way forward is because we already know there are folks opposed to making the change upstream. If the PEP just provides recommendations for redistributors that *do* decide to provide a global off switch to revert to the old behaviour, then the perspective of the folks opposed to the feature is respected by the fact that this is a feature some redistributors *may* choose to add to provide a smoother migration path to more secure default HTTPS handling, rather than something upstream provides by default. I assume the Debian, Ubuntu and Suse folks won't care, as they have all already decided against backporting the change to their long term support releases where the compatibility break would pose a problem (and I can certainly sympathise with that perspective given the dependency on backporting the PEP 466 SSL changes first, and the work involved in seeking consensus on a smoother migration path from the old default to the new one). It would be nice to hear from ActiveState, Enthought Continuum Analytics as well, but if they don't chime in here, I don't see any particular need to go chasing them explicitly. I think the approach to only consider a subset of redistributors as viable targets for such a switch is a bit too narrow. You are leaving out all the parties which use custom Python installations to run their applications, e.g. the Plone and Zope community, the ZenOSS community, the many Windows applications built on Python, etc. to fix this regression in 2.7.9. We made the decision when PEP 476 was accepted that this change turned a silent security failure into a noisy one, rather than being a regression in its own right. PEP 493 isn't about disagreeing with that decision, it's about providing a smoother upgrade path in contexts where letting the security failure remain silent is deemed to be preferred in the near term. The change wasn't regression. The missing downgrade path is a regression. Some other comments on PEP 493: * I don't think we really want to add the overhead of having to parse an INI file every time Python starts up. Please remember that we just parsing of the sysconfig data not
Re: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: async() function is deprecated in favour of ensure_future().
On 11 May 2015 at 20:47, Yury Selivanov yselivanov...@gmail.com wrote: Yes, I'm in the process of writing it ;) (as well as for new set_task_factory()) Cool - sorry for being a nag :-) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Minimal async event loop and async utilities (Was: PEP 492: async/await in Python; version 4)
On 6 May 2015 at 16:46, Guido van Rossum gu...@python.org wrote: This is actually a great idea, and I encourage you to go forward with it. The biggest piece missing from your inventory is probably Task, which is needed to wrap a Future around a coroutine. OK, I've been doing some work on this. You're right, the asyncio framework makes Future a key component. But I'm not 100% sure why Future (and Task) have to be so fundamental. Ignoring cancellation (see below!) I can build pretty much all of a basic event loop, plus equivalents of the asyncio locks and queues modules, without needing the concept of a Future at all. The create_task function becomes simply a function to add a coroutine to the ready queue, in this context. I can't return a Task (because I haven't implemented the Task or Future classes) but I don't actually know what significant functionality is lost as a result - is there a reasonably accessible example of where using the return value from create_task is important anywhere? A slightly more complicated issue is with the run_until_complete function, which takes a Future, and hence is fundamentally tied to the Future API. However, it seems to me that a minimal implementation could work by having a run_until_complete() that just took an awaitable (i.e., anything that you can yield from). Again, is there a specific reason that you ended up going with run_until_complete taking a Future rather than just a coroutine? I think (but haven't confirmed yet by implementing it) that it should be possible to create a coroutine that acts like a Future, in the sense that you can tell it from outside (via send()) that it's completed and set its return value. But this is all theory, and if you have any practical experience that shows I'm going down a dead end, I'd be glad to know. I'm not sure how useful this line of attack will be - if the API isn't compatible with asyncio.BaseEventLoop, it's not very useful in practice. On the other hand, if I can build a loop without Future or Task classes, it may indicate that those classes aren't quite as fundamental as asyncio makes them (which may allow some simplifications or generalisations). I expect you'll also want to build cancellation into your base async framework; and the primitives to wait for multiple awaitables. The next step would be some mechanism to implement call_later()/call_at() (but this needs to be pluggable since for a real event loop it needs to be implemented by the basic I/O selector). These are where I suspect I'll have the most trouble if I haven't got a solid understanding of the role of the Future and Task classes (or alternatively, how to avoid them :-)) So I'm holding off on worrying about them for now. But certainly they need to be covered. In particular, call_later/call_at are the only generic example of any form of wait that actually *waits*, rather than returning immediately. So as you say, implementing them will show how the basic mechanism can be extended with a real selector (whether for I/O, or GUI events, or whatever). If you can get this working it would be great to include this in the stdlib as a separate asynclib library. The original asyncio library would then be a specific implementation (using a subclass of asynclib.EventLoop) that adds I/O, subprocesses, and integrates with the selectors module (or with IOCP, on Windows). One thing I've not really considered in the above, is how a refactoring like this would work. Ignoring the let's try to remove the Future class approach above, my basic event loop is mostly just an alternative implementation of an event loop (or maybe an alternative policy - I'm not sure I understand the need for policies yet). So it may simply be a case of ripping coroutines.py, futures.py, locks.py, log.py, queues.py, and tasks.py out of asyncio and adding a new equivalent of events.py with my minimal loop in it. (So far, when I've tried to do that I get hit with some form of circular import problem - I've not worked out why yet, or how asyncio avoids the same problem). That in itself would probably be a useful refactoring, splitting out the IO aspects of asyncio from the event loop / async aspects. I don't see any particular hurry to get this in before 3.5; the refactoring of asyncio can be done later, in a backward compatible way. It would be a good way to test the architecture of asyncio! Agreed. It's also not at all clear to me how the new async/await syntax would fit in with this, so that probably needs some time to settle down. For example, in Python 3.5 would run_until_complete take an awaitable rather than a Future? Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: async() function is deprecated in favour of ensure_future().
Yes, I'm in the process of writing it ;) (as well as for new set_task_factory()) Thanks, Yury On 2015-05-11 3:42 PM, Paul Moore wrote: On 11 May 2015 at 19:50, yury.selivanov python-check...@python.org wrote: https://hg.python.org/cpython/rev/b78127eafad7 changeset: 95956:b78127eafad7 branch: 3.4 parent: 95953:a983d63e3321 user:Yury Selivanov yseliva...@sprymix.com date:Mon May 11 14:48:38 2015 -0400 summary: asyncio: async() function is deprecated in favour of ensure_future(). files: Lib/asyncio/base_events.py | 2 +- Lib/asyncio/tasks.py | 27 - Lib/asyncio/windows_events.py| 2 +- Lib/test/test_asyncio/test_base_events.py| 6 +- Lib/test/test_asyncio/test_tasks.py | 48 + Lib/test/test_asyncio/test_windows_events.py | 2 +- Misc/NEWS| 4 +- 7 files changed, 57 insertions(+), 34 deletions(-) Surely this should include a doc change? Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: async() function is deprecated in favour of ensure_future().
On 11 May 2015 at 19:50, yury.selivanov python-check...@python.org wrote: https://hg.python.org/cpython/rev/b78127eafad7 changeset: 95956:b78127eafad7 branch: 3.4 parent: 95953:a983d63e3321 user:Yury Selivanov yseliva...@sprymix.com date:Mon May 11 14:48:38 2015 -0400 summary: asyncio: async() function is deprecated in favour of ensure_future(). files: Lib/asyncio/base_events.py | 2 +- Lib/asyncio/tasks.py | 27 - Lib/asyncio/windows_events.py| 2 +- Lib/test/test_asyncio/test_base_events.py| 6 +- Lib/test/test_asyncio/test_tasks.py | 48 + Lib/test/test_asyncio/test_windows_events.py | 2 +- Misc/NEWS| 4 +- 7 files changed, 57 insertions(+), 34 deletions(-) Surely this should include a doc change? Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Free lists
Serhiy Storchaka schrieb am 09.05.2015 um 21:01: Here is a statistic for most called PyObject_INIT or PyObject_INIT_VAR for types (collected during running Python tests on 32-bit Linux). I'm aware that this includes lots of tests for the Python code in the stdlib, so these numbers are most likely not too far from what real-world code would give, but wouldn't it be even better to collect statistics from a (quick) benchmark suite run? Test suites tend to be very regular, flat and broad, unlike most production code. Stefan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode literals in Python 2.7
On 5/11/2015 1:09 AM, Nick Coghlan wrote: On 10 May 2015 at 23:28, Adam Bartoš dre...@gmail.com wrote: Glenn Linderman wrote: Is this going to get released in 3.5, I hope? Python 3 is pretty limited without some solution for Unicode on the console... probably the biggest deficiency I have found in Python 3, since its introduction. It has great Unicode support for files and processing, which convinced me to switch from Perl, and I like so much else about it, that I can hardly code in Perl any more (I still support a few Perl programs, but have ported most of them to Python). I'd love to see it included in 3.5, but I doubt that will happen. For one thing, it's only two weeks till beta 1, which is feature freeze. And mainly, my package is mostly hacking into existing Python environment. A proper implementation would need some changes in Python someone would have to do. See for example my proposal http://bugs.python.org/issue17620#msg234439. I'm not competent to write a patch myself and I have also no feedback to the proposed idea. On the other hand, using the package is good enough for me so I didn't further bring attention to the proposal. Right, and while I'm interested in seeing this improved, I'm not especially familiar with the internal details of our terminal interaction implementation, and even less so when it comes to the Windows terminal. Steve Dower's also had his hands full working on the Windows installer changes, and several of our other Windows folks aren't C programmers. PEP 432 (the interpreter startup sequence improvements) will be back on the agenda for Python 3.6, so the 3.6 time frame seems more plausible at this point. Cheers, Nick. Wow! Another bug that'll reach a decade in age before being fixed... ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode literals in Python 2.7
On 12 May 2015 at 06:38, Glenn Linderman v+pyt...@g.nevcal.com wrote: On 5/11/2015 1:09 AM, Nick Coghlan wrote: On 10 May 2015 at 23:28, Adam Bartoš dre...@gmail.com wrote: I'd love to see it included in 3.5, but I doubt that will happen. For one thing, it's only two weeks till beta 1, which is feature freeze. And mainly, my package is mostly hacking into existing Python environment. A proper implementation would need some changes in Python someone would have to do. See for example my proposal http://bugs.python.org/issue17620#msg234439. I'm not competent to write a patch myself and I have also no feedback to the proposed idea. On the other hand, using the package is good enough for me so I didn't further bring attention to the proposal. Right, and while I'm interested in seeing this improved, I'm not especially familiar with the internal details of our terminal interaction implementation, and even less so when it comes to the Windows terminal. Steve Dower's also had his hands full working on the Windows installer changes, and several of our other Windows folks aren't C programmers. PEP 432 (the interpreter startup sequence improvements) will be back on the agenda for Python 3.6, so the 3.6 time frame seems more plausible at this point. Cheers, Nick. Wow! Another bug that'll reach a decade in age before being fixed... Yep, that tends to happen with complex cross-platform bugs RFEs that require domain expertise in multiple areas to resolve. It's one of the areas that operating system vendors are typically best equipped to handle, but we haven't historically had that kind of major institutional backing for CPython core development (that *is* changing, but it's a relatively recent phenomenon). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mac popups running make test
In article CALWZvp4D4C3+z=xw2usvqtdxszhqmw9o6zoyzvm2ueecaw2...@mail.gmail.com, Tal Einat talei...@gmail.com wrote: On Sun, May 10, 2015 at 9:07 PM, Carol Willing willi...@willingconsulting.com wrote: On 5/10/15 10:29 AM, Tal Einat wrote: On Sun, May 10, 2015 at 5:07 PM, Brett Cannon br...@python.org wrote: On Sun, May 10, 2015 at 10:04 AM Skip Montanaro skip.montan...@gmail.com wrote: I haven't run the test suite in awhile. I am in the midst of running it on my Mac running Yosemite 10.10.3. Twice now, I've gotten this popup: I assume this is testing some server listening on localhost. Is this a new thing, either with the Python test suite or with Mac OS X? (I'd normally be hidden behind a NAT firewall, but at the moment I am on a miserable public connection in a Peet's Coffee, so it takes on slightly more importance...) It's not new. Indeed, I've run into this as well. I've also seen the Crash Reporter pop up many times, but as far as I could tell, in all cases the test suite output told me it was expected. Perhaps tests which listen for network connections should also mention that, at least on Macs? Wouldn't hurt. Just requires tracking down which test(s) triggers it (might be more than one and I don't know if answering that popup applies for the rest of the test execution or once per test if you use -j). If anyone starts working on this, let me know if I can help, e.g. trying things on my own Mac. \ I believe that the message has to do with OS X's sandboxing implementation and the setting of the sandbox's entitlement keys. Here's an Apple doc: https://developer.apple.com/library/ios/documentation/Miscellaneous/Referenc e/EntitlementKeyReference/Chapters/EnablingAppSandbox.html#//apple_ref/doc/u id/TP40011195-CH4-SW9 I'm unaware of a way to work around this other than using Apple's code signing or adjusting target build settings in XCode :( If anyone knows a good way to workaround or manually set permission (other than clicking the Allow button), I would be interested. I was reading about this a few weeks ago an recall finding a way to ad-hoc sign the built python executable. Here's a link below. I haven't tried this, though, and don't know if it would work with a python executable rather than a proper OSX app. If it does work, it would be useful to add this as a tool and/or mention it in the developer docs. http://apple.stackexchange.com/a/121010 I believe the issue has to do with the OS X application firewall and not sandboxing, as vanilla Python on OS X is not sandboxed. See: https://support.apple.com/en-us/HT201642 As described there, codesigned applications are automatically authorized to accept inbound connections; that's the workaround proposed in the apple.stackexchange cite. But arbitrarily signing development binaries after every compile is probably not a good idea. Another option is to configure the firewall but that probably only can be made to work with a framework build of Python which launches Python within an app bundle. In any case, please open an issue on the bug tracker so we can follow up on this. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mac popups running make test
On Sun, May 10, 2015 at 5:04 PM, Skip Montanaro skip.montan...@gmail.com wrote: ... I've also seen the Crash Reporter pop up many times, I don't know how to get rid of the popup you mentioned, but Windows had problems with the crash popups for a long time. Eventually it got fixed: https://hg.python.org/cpython/file/default/Lib/test/support/__init__.py#l2202 http://bugs.python.org/issue11732 http://bugs.python.org/issue18948 http://bugs.python.org/issue23314 Perhaps Mac OS has something similar too? Best Regards, Ezio Melotti but as far as I could tell, in all cases the test suite output told me it was expected. Perhaps tests which listen for network connections should also mention that, at least on Macs? Thx, Skip ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PYTHONHTTPSVERIFY env var
On 12 May 2015 at 04:49, M.-A. Lemburg m...@egenix.com wrote: On 11.05.2015 12:15, Nick Coghlan wrote: By contrast, the configuration file shouldn't provide a new attack vector (or simplify any existing attack vector), as if you have the permissions needed to modify the config file, you likely also have the permissions needed to modify the system certificate store, and the latter is a *far* more interesting attack vector than a downgrade attack solely against Python. Thus the environment variable based off switch is neither necessary (as an administrator controlled configuration file can do the job), nor sufficient (it can't handle the -E switch), *and* it represents an increase in the attack surface area relative to a Python implementation without the capability. Whether or not -E will have an effect on the env var depends on the implementation. At the moment, -E only has an effect on the C runtime, while the stdlib happily reads from os.environ without taking the flag into account. I had an off-list discussion with Christian Heimes about that in relation to the OpenSSL flags, and he pointed out the reason -E specifically needs to be a command line switch is because that is the only way to affect how environment variables are processed during interpreter startup. Once an application is up and running, further environment variable sanitisation can be handled at an application level by whitelisting entries in os.environ and deleting everything else. As proposed, the PYTHONHTTPSVERIFY would only affect the ssl module and only be checked when loading this module, i.e. not at Python startup time. Right, the same is true for the configuration file proposal. It would be nice to hear from ActiveState, Enthought Continuum Analytics as well, but if they don't chime in here, I don't see any particular need to go chasing them explicitly. I think the approach to only consider a subset of redistributors as viable targets for such a switch is a bit too narrow. You are leaving out all the parties which use custom Python installations to run their applications, e.g. the Plone and Zope community, the ZenOSS community, the many Windows applications built on Python, etc. No, they already have a solution: monkeypatch (or just plain patch) the SSL module. That's an upstream supported technique, which is why it's documented in the PEP. The problem we (as in Red Hat) ran into was that that technique *doesn't work* for the case of backporting PEP 476 to Python 2.7.5 as an opt-in feature. to fix this regression in 2.7.9. We made the decision when PEP 476 was accepted that this change turned a silent security failure into a noisy one, rather than being a regression in its own right. PEP 493 isn't about disagreeing with that decision, it's about providing a smoother upgrade path in contexts where letting the security failure remain silent is deemed to be preferred in the near term. The change wasn't regression. The missing downgrade path is a regression. It's a shame we don't have -X options in Python 2, as that would be a nice hard-to-attack option (although it wouldn't play well with subprocesses) Some other comments on PEP 493: * I don't think we really want to add the overhead of having to parse an INI file every time Python starts up. Please remember that we just parsing of the sysconfig data not long ago because we wanted to avoid this startup time. Compared to the overhead of reading from the system cert database, reading a config file at ssl module import time should be trivial. * I don't see why the attack surface of using an INI file somewhere in the system should be smaller than e.g. using sitecustomize.py You can put sitecustomize.py in a user directory, and if there's no system wide sitecustomize, Python will read it automatically (unless user site directories are turned off). * If done right, we'd also need a switch to ignore this global config file and recommend using it to reduce the attack surface (for the same reason you explain in the PEP) No, the recommendation there would be to upgrade to a newer version of Python that doesn't offer this downgrade capability. It's a proposal for a transition smoothing technique, not a permanent capability (I did suggest the latter at one point, but the discussion on the issue tracker persuaded me that was a bad idea, with the increased attack surface being a key part of that change of heart). * I don't think a global switch is the right way forward. Many applications on properly configured systems will work fine with the new default. The downgrade option is only needed for those cases, where they don't and you don't have a good way to fix the application. And techniques like chroots and containers let you do that selectively. The key thing I'm after is an agreed technique for backporting to earlier 2.7.x releases that allows PEP 476 to be provided as an opt-in capability,
