Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 1 September 2014 11:10, R. David Murray rdmur...@bitdance.com wrote: It sounds like this would address my concerns as well (I don't really care *how* it is implemented as long as I don't have to touch the code of a third party application when I upgrade my python version to 3.5...remember, the context here is backward compatibility concerns). Does it address the issue of accepting an invalid cert, though? That's actually an interesting question, as the PEP doesn't currently propose adding any new global configuration knobs to the ssl or httplib modules - it just proposes switching httplib from the legacy (but fully backwards compatible) ssl._create_stdlib_context() API to the newer (but potentially backwards incompatible in some environments) ssl.create_default_context() API. Having the ssl module import an sslcustomize module at the end wouldn't be enough unless the appropriate APIs were put in place to allow things to be configured at a process global level. One possible way to do that would be to provide a central context factory mapping that provide a module specific SSL context creator. We'd seed it appropriately for the stdlib modules where we wanted to use the legacy context definition, but it would default to using ssl.create_default_context. Under that kind of model, the first change we would actually make is to make ssl._create_stdlib_context() public under a suitable name, let's say ssl.create_legacy_context() Independenting of any other changes, exposing ssl.create_legacy_context() like that would also make it straightforward for folks to opt in to the old behaviour as an interim hack in a way that is easy to grep for and fix later (it's also something a linter can easily disallow). The second change would be to provide a mapping from arbitrary names to context factories in the ssl module that defaults to ssl.create_default_context: named_contexts = defaultdict((lambda name: create_default_context)) (A more accurate name would be named_context_factory, but I think named_contexts reads better. Folks will learn quickly enough that it actually stores context factories rather than prebuilt context objects) The third change would be to replace all calls to ssl._create_stdlib_context() with calls to ssl.named_contexts[__name__]() instead. The final change would be to seed the context factory map appropriately for the standard library modules where we wanted to keep the *old* default: for modname in (nntplib, poplib, imaplib, ftplib, smtplib, asyncio.selector_events, urllib.request, http.client): named_contexts[modname] = create_legacy_context The list I have above is for *all* current uses of sss._create_stdlib_context. The backwards incompatible part of PEP 476 would then just be about removing names from that list (currently just http.client, but I'd suggest asyncio.selector_events as another candidate, taking advantage of asyncio's provisional API status). The revert to 3.4 behaviour content for sslcustomize.py would then just be: import ssl ssl.named_contexts[http.client] = ssl.create_legacy_context However, someone that wanted to also enforce SSL properly for other standard library modules could go the other way: import ssl for modname in (nntplib, poplib, imaplib, ftplib, smtplib, urllib.request): ssl.named_contexts[modname] = ssl.create_default_context Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 31 August 2014 23:10, Nick Coghlan ncogh...@gmail.com wrote: Assuming sslcustomize was in site-packages rather than the standard library directories, you would also be able to use virtual environments with an appropriate sslcustomize module to disable cert checking even if the application you were running didn't support direct configuration. Would this mean that a malicious package could install a custom sslcustomize.py and so add unwanted certs to the system? I guess we have to assume that installed packages are trusted, but I just wanted to be explicit. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On 31 August 2014 22:38, Victor Stinner victor.stin...@gmail.com wrote: This case is described as the use case #2 in the PEP, so it is supported. As written in the PEP, if you want to be notified of the signal, set a signal handler which raises an exception. For example the default signal handler for SIGINT raises KeyboardInterrupt. Wait - sigint? Does this mean that (unless the application adds a signal handler) keyboard interrupts will be in effect ignored while in a system call? I'm not sure I like that - I'd rather Ctrl-C always interrupted the program. Specifically, in one-off scripts that *don't* take care to handle all errors appropriately and just show the traceback... Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7
In article cadisq7e4zachw4b_nmwewpwtxg6davndc-zwhaoajkq4sa3...@mail.gmail.com, Nick Coghlan ncogh...@gmail.com wrote: On 1 Sep 2014 09:23, Benjamin Peterson benja...@python.org wrote: On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote: On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan ncogh...@gmail.com wrote: That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle. However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers. A much bigger than average +1 I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain. 2.7.x has been and will be alive for so long that will already have to explain that sort thing; i.e. PEP 466 and why different bugfix releases support different versions of dependency libraries. And that is a minor complication compared with the confusion and difficulty of trying to explain to users (stuck with 2.7 for the time being) of how to install third-party packages on each platform (especially Windows) versus the simplicity of the 3.4.x story, thanks to ensurepip. Having pip available as a documented, batteries-included tool for all current releases would be a huge usability improvement. Exactly. LTS is genuinely different from stopping maintenance after the next feature release - it requires considering the stability risk and user experience improvement questions separately. In this case, the problem is that the Python 2 platform *is* still evolving, but the centre of that evolution has moved to PyPI. For standard library only users, Python 2 stopped evolving back in 2010. For PyPI users, by contrast, it's still evolving at a rapid pace. For our Python 3 transition story to be coherent, we need to ensure tools like six, modernize and future are readily available, while still remaining free to evolve independently of the standard library. That means providing a package management utility as easily and as painlessly as possible. Embracing pip upstream for Python 2 as well as Python 3 also sends a powerful signal to redistributors that says your users are going to need this and makes them do additional work to *avoid* providing it. Some of them *will* choose that path, but that becomes a matter for discussion between them and their user base, rather than a problem we need to worry about upstream. FTR, I'm willing to backport the pieces I did for 3.4 and I could do the ensurepip backport, as well. I'll leave the Windows installer changes for someone else, though. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 1 September 2014 16:07, Paul Moore p.f.mo...@gmail.com wrote: On 31 August 2014 23:10, Nick Coghlan ncogh...@gmail.com wrote: Assuming sslcustomize was in site-packages rather than the standard library directories, you would also be able to use virtual environments with an appropriate sslcustomize module to disable cert checking even if the application you were running didn't support direct configuration. Would this mean that a malicious package could install a custom sslcustomize.py and so add unwanted certs to the system? I guess we have to assume that installed packages are trusted, but I just wanted to be explicit. Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the ssl module. The -S and -I switches would need to disable the implied sslcustomize, just as they disable import site. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 01.09.2014 08:44, Nick Coghlan wrote: Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the ssl module. The -S and -I switches would need to disable the implied sslcustomize, just as they disable import site. A malicious package can already play havoc with your installation with a custom ssl module. If somebody is able to sneak in a ssl.py then you are screwed anyway. sslcustomize is not going to make the situation worse. Christian ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
Victor Stinner wrote: Le 1 sept. 2014 00:17, Marko Rauhamaa ma...@pacujo.net mailto:ma...@pacujo.net a écrit : If a signal is received when read() or write() has completed its task partially ( 0 bytes), no EINTR is returned but the partial count. Obviously, Python should take that possibility into account so that raising an exception in the signal handler (as mandated by the PEP) doesn't cause the partial result to be lost on os.read() or os.write(). This case is unrelated to the PEP, the PEP only changes the behaviour when a syscall fails with EINTR. I think there's a problem here, though. As thing stand, a signal handler that doesn't raise an exception can set a flag, and code after the read() can test it. Under the proposed scheme, the signal handler has to be made to raise an exception so that the read will be broken out of in the EINTR case. But what happens if the read returns *without* an EINTR? The signal handler will still raise an exception, which is either going to clobber the partial return value or mess up the code that does something with it. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
No, it's the opposite. The PEP doesn't change the default behaviour of SIGINT: CTRL+C always interrupt the program. Victor Le 1 sept. 2014 08:12, Paul Moore p.f.mo...@gmail.com a écrit : On 31 August 2014 22:38, Victor Stinner victor.stin...@gmail.com wrote: This case is described as the use case #2 in the PEP, so it is supported. As written in the PEP, if you want to be notified of the signal, set a signal handler which raises an exception. For example the default signal handler for SIGINT raises KeyboardInterrupt. Wait - sigint? Does this mean that (unless the application adds a signal handler) keyboard interrupts will be in effect ignored while in a system call? I'm not sure I like that - I'd rather Ctrl-C always interrupted the program. Specifically, in one-off scripts that *don't* take care to handle all errors appropriately and just show the traceback... Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7
On Sep 1, 2014, at 2:22 AM, Ned Deily n...@acm.org wrote: In article cadisq7e4zachw4b_nmwewpwtxg6davndc-zwhaoajkq4sa3...@mail.gmail.com mailto:cadisq7e4zachw4b_nmwewpwtxg6davndc-zwhaoajkq4sa3...@mail.gmail.com, Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: On 1 Sep 2014 09:23, Benjamin Peterson benja...@python.org wrote: On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote: On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan ncogh...@gmail.com wrote: That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle. However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers. A much bigger than average +1 I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain. 2.7.x has been and will be alive for so long that will already have to explain that sort thing; i.e. PEP 466 and why different bugfix releases support different versions of dependency libraries. And that is a minor complication compared with the confusion and difficulty of trying to explain to users (stuck with 2.7 for the time being) of how to install third-party packages on each platform (especially Windows) versus the simplicity of the 3.4.x story, thanks to ensurepip. Having pip available as a documented, batteries-included tool for all current releases would be a huge usability improvement. Yes this is a major driver. I mean I think I probably have an above average knowledge of how to bootstrap pip, and if you dump me on a Windows box I struggle to actually do it the first time around without stumbling around and doing things in the wrong order and the like. (Getting a compiler toolchain is worse, but yay for Wheels). Exactly. LTS is genuinely different from stopping maintenance after the next feature release - it requires considering the stability risk and user experience improvement questions separately. In this case, the problem is that the Python 2 platform *is* still evolving, but the centre of that evolution has moved to PyPI. For standard library only users, Python 2 stopped evolving back in 2010. For PyPI users, by contrast, it's still evolving at a rapid pace. For our Python 3 transition story to be coherent, we need to ensure tools like six, modernize and future are readily available, while still remaining free to evolve independently of the standard library. That means providing a package management utility as easily and as painlessly as possible. Embracing pip upstream for Python 2 as well as Python 3 also sends a powerful signal to redistributors that says your users are going to need this and makes them do additional work to *avoid* providing it. Some of them *will* choose that path, but that becomes a matter for discussion between them and their user base, rather than a problem we need to worry about upstream. FTR, I'm willing to backport the pieces I did for 3.4 and I could do the ensurepip backport, as well. I'll leave the Windows installer changes for someone else, though. Awesome, I’m of course willing to back port ensure pip itself as well. Truthfully it shouldn’t be a very difficult backport. It’s only ~200 SLOC or so and the only real things would be removing a Python3ism here or there. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
Le 1 sept. 2014 02:40, Greg Ewing greg.ew...@canterbury.ac.nz a écrit : Victor Stinner wrote: As written in the PEP, if you want to be notified of the signal, set a signal handler which raises an exception. I'm not convinced that this covers all possible use cases. It might be all right if you have control over the signal handler, but what if you don't? Ok, let's say that a syscall is interrupted by a signal, but rhe signal doesn't raise an exception. So your program can only be interrupted if the signal is received during a syscall, right? I don't think that such program is reliable. Python should not promote such design. It should behave the same if the signal is received during a CPU-bound function. Extract of the PEP: Backward Compatibility: Applications relying on the fact that system calls are interruptedwith ``InterruptedError`` will hang. The authors of this PEP don'tthink that such application exist.If such applications exist, they are not portable and are subject torace conditions (deadlock if the signal comes before the system call).These applications must be fixed to handle signals differently, tohave a reliable behaviour on all platforms and all Python versions.For example, use a signal handler which raises an exception, or use awakeup file descriptor.For applications using event loops, ``signal.set_wakeup_fd()`` is therecommanded option to handle signals. The signal handler writes signalnumbers into the file descriptor and the event loop is awaken to readthem. The event loop can handle these signals without the restrictionof signal handlers. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
Victor Stinner victor.stin...@gmail.com: No, it's the opposite. The PEP doesn't change the default behaviour of SIGINT: CTRL+C always interrupt the program. Which raises an interesting question: what happens to the os.read() return value if SIGINT is received? Marko ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 1 September 2014 17:13, Christian Heimes christ...@python.org wrote: On 01.09.2014 08:44, Nick Coghlan wrote: Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the ssl module. The -S and -I switches would need to disable the implied sslcustomize, just as they disable import site. A malicious package can already play havoc with your installation with a custom ssl module. If somebody is able to sneak in a ssl.py then you are screwed anyway. sslcustomize is not going to make the situation worse. That's not quite true - we're fairly careful about putting the standard library before userspace directories, so aside from the current directory problem, shadowing ssl itself can be tricky to arrange. sslcustomize would be more like sitecustomize - since it wouldn't normally be in the standard library, it can appear anywhere on sys.path, rather than having to be injected ahead of the standard library. I think that's OK though - compared to the security nightmare that is downloading modules from PyPI and running ./setup.py install (or, even worse, sudo ./setup.py install), this would be a rather esoteric attack vector, and the existing -S and -I mechanisms could be used to defend against it :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Bug 19494 ... urllib2.HTTPBasicAuthHandler for GitHub et al.
Hi,
now when the vacations even in Europe are over could I ask for
some movement on http://bugs.python.org/issue19494? Demanding
a half-megabyte amount of packages from PIP ('just use requests'
mentioned by some comments in the thread) or for that matter any
package from PIP (including mine
https://pypi.python.org/pypi/urllib2_prior_auth/) for something
which is essentially a ten-lines patch seems to me rather crazy.
What happened to all those batteries which were supposed to be
included?
Best,
Matěj
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7
On 1 Sep 2014 17:31, Donald Stufft don...@stufft.io wrote: On Sep 1, 2014, at 2:22 AM, Ned Deily n...@acm.org wrote: And that is a minor complication compared with the confusion and difficulty of trying to explain to users (stuck with 2.7 for the time being) of how to install third-party packages on each platform (especially Windows) versus the simplicity of the 3.4.x story, thanks to ensurepip. Having pip available as a documented, batteries-included tool for all current releases would be a huge usability improvement. Yes this is a major driver. I mean I think I probably have an above average knowledge of how to bootstrap pip, and if you dump me on a Windows box I struggle to actually do it the first time around without stumbling around and doing things in the wrong order and the like. (Getting a compiler toolchain is worse, but yay for Wheels). Yeah. I've mentioned it before, but I think it bears repeating that trying to install pip on Windows with both Python 2 3 installed was one of the key things that convinced me to write PEP 453 in the first place. The default settings in both Internet and Windows explorer make it tricky regardless, but parallel installs make it even worse. FTR, I'm willing to backport the pieces I did for 3.4 and I could do the ensurepip backport, as well. I'll leave the Windows installer changes for someone else, though. Awesome, I’m of course willing to back port ensure pip itself as well. Truthfully it shouldn’t be a very difficult backport. It’s only ~200 SLOC or so and the only real things would be removing a Python3ism here or there. Backporting meaningful tests will actually be the annoying part: the current unit tests use unittest.mock, while the current functional tests use pyvenv :) Both of those can be dealt with, but the tests will be a bit of an ugly hack by comparison with their Py3 counterparts :) Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
There's no return value, a KeywordInterrupt exception is raised. The PEP wouldn't change this behavior. As for the general behavior: all programming languages/platforms handle EINTR transparently. It's high time for Python to have a sensible behavior in this regard. 2014-09-01 8:38 GMT+01:00 Marko Rauhamaa ma...@pacujo.net: Victor Stinner victor.stin...@gmail.com: No, it's the opposite. The PEP doesn't change the default behaviour of SIGINT: CTRL+C always interrupt the program. Which raises an interesting question: what happens to the os.read() return value if SIGINT is received? Marko ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/cf.natali%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
Charles-François Natali cf.nat...@gmail.com: Which raises an interesting question: what happens to the os.read() return value if SIGINT is received? There's no return value, a KeywordInterrupt exception is raised. The PEP wouldn't change this behavior. Slightly disconcerting... but I'm sure overriding SIGINT would cure that. You don't want to lose data if you want to continue running. As for the general behavior: all programming languages/platforms handle EINTR transparently. C doesn't. EINTR is there for a purpose. I sure hope Python won't bury it under opaque APIs. The two requirements are: * Allow the application to react to signals immediately in the main flow. * Don't lose information. Marko ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
2014-09-01 12:15 GMT+01:00 Marko Rauhamaa ma...@pacujo.net: Charles-François Natali cf.nat...@gmail.com: Which raises an interesting question: what happens to the os.read() return value if SIGINT is received? There's no return value, a KeywordInterrupt exception is raised. The PEP wouldn't change this behavior. Slightly disconcerting... but I'm sure overriding SIGINT would cure that. You don't want to lose data if you want to continue running. As for the general behavior: all programming languages/platforms handle EINTR transparently. C doesn't. EINTR is there for a purpose. Python is slightly higher level than C, right? I was referring to Java, go, Haskell... Furthermore, that's not true: many operating systems actually restart syscalls by default (including Linux, man 7 signal): Interruption of system calls and library functions by signal handlers If a signal handler is invoked while a system call or library function call is blocked, then either: * the call is automatically restarted after the signal handler returns; or * the call fails with the error EINTR. Which of these two behaviors occurs depends on the interface and whether or not the signal handler was established using the SA_RESTART flag (see sigaction(2)). The details vary across UNIX systems; below, the details for Linux. The reason the interpreter is subject to so many EINTR is that we *explicitly* clear SA_RESTART because the C-level signal handler must be handled by the interpreter to have a chance to run the Python-level handlers from the main loop. There are many aspects of signal handling in Python that make it different from C: if you want C semantics, stick to C. I do not want to have to put all blocking syscalls within a try/except loop: have a look at the stdlib code, you'll see it's really a pain and ugly. And look at the number of EINTR-related syscalls we've had. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
Le 01/09/2014 10:09, Nick Coghlan a écrit : On 1 September 2014 17:13, Christian Heimes christ...@python.org wrote: On 01.09.2014 08:44, Nick Coghlan wrote: Yes, it would have exactly the same security failure modes as sitecustomize, except it would only fire if the application imported the ssl module. The -S and -I switches would need to disable the implied sslcustomize, just as they disable import site. A malicious package can already play havoc with your installation with a custom ssl module. If somebody is able to sneak in a ssl.py then you are screwed anyway. sslcustomize is not going to make the situation worse. That's not quite true - we're fairly careful about putting the standard library before userspace directories, so aside from the current directory problem, shadowing ssl itself can be tricky to arrange. Not sure why. Just put another module named ssl in sys.modules directly. You can also monkeypatch the genuine ssl module. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 01 Sep 2014 19:15:33 +1200 Greg Ewing greg.ew...@canterbury.ac.nz wrote: Victor Stinner wrote: Le 1 sept. 2014 00:17, Marko Rauhamaa ma...@pacujo.net mailto:ma...@pacujo.net a écrit : If a signal is received when read() or write() has completed its task partially ( 0 bytes), no EINTR is returned but the partial count. Obviously, Python should take that possibility into account so that raising an exception in the signal handler (as mandated by the PEP) doesn't cause the partial result to be lost on os.read() or os.write(). This case is unrelated to the PEP, the PEP only changes the behaviour when a syscall fails with EINTR. I think there's a problem here, though. As thing stand, a signal handler that doesn't raise an exception can set a flag, and code after the read() can test it. Under the proposed scheme, the signal handler has to be made to raise an exception so that the read will be broken out of in the EINTR case. The PEP already addresses this remark: Applications relying on the fact that system calls are interrupted with ``InterruptedError`` will hang. The authors of this PEP don't think that such application exist. If such applications exist, they are not portable and are subject to race conditions (deadlock if the signal comes before the system call). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
Hi, I'm +1 on the whole PEP. Writing a signal handler is difficult, only async-signal safe functions can be called. You mean a C signal handler? Python signal handlers are not restricted. Some signals are not interesting and should not interrupt the the application. There are two options to only interrupt an application on some signals: * Raise an exception in the signal handler, like ``KeyboardInterrupt`` for ``SIGINT`` * Use a I/O multiplexing function like ``select()`` with the Python signal wakeup file descriptor: see the function ``signal.set_wakeup_fd()``. This section looks a bit incomplete. Some calls such as os.read() or os.write() will (should) return a partial result when interrupted and they already handled 0 bytes. Perhaps other functions have a similar behaviour? On Unix, the ``asyncio`` module uses the wakeup file descriptor to wake up its event loop. How about Windows? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org wrote: Not sure why. Just put another module named ssl in sys.modules directly. You can also monkeypatch the genuine ssl module. That has to be done inside the same process. But imagine this scenario: You have a program that gets invoked as root (or some other user than yourself), and you're trying to fiddle with what it sees. You don't have root access, but you can manipulate the file system, to the extent that your userid has access. What can you do to affect this other program? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Mon, 1 Sep 2014 23:24:39 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org wrote: Not sure why. Just put another module named ssl in sys.modules directly. You can also monkeypatch the genuine ssl module. That has to be done inside the same process. But imagine this scenario: You have a program that gets invoked as root (or some other user than yourself), and you're trying to fiddle with what it sees. You don't have root access, but you can manipulate the file system, to the extent that your userid has access. What can you do to affect this other program? If you're root you shouldn't run untrusted code. See https://docs.python.org/3/using/cmdline.html#cmdoption-I Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Mon, Sep 1, 2014 at 11:34 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:24:39 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org wrote: Not sure why. Just put another module named ssl in sys.modules directly. You can also monkeypatch the genuine ssl module. That has to be done inside the same process. But imagine this scenario: You have a program that gets invoked as root (or some other user than yourself), and you're trying to fiddle with what it sees. You don't have root access, but you can manipulate the file system, to the extent that your userid has access. What can you do to affect this other program? If you're root you shouldn't run untrusted code. See https://docs.python.org/3/using/cmdline.html#cmdoption-I Right, which is why sslcustomize has to be controlled by that, but the possibility of patching (or monkeypatching) ssl.py isn't as big a deal. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Mon, 1 Sep 2014 23:42:10 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 11:34 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:24:39 +1000 Chris Angelico ros...@gmail.com wrote: On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou anto...@python.org wrote: Not sure why. Just put another module named ssl in sys.modules directly. You can also monkeypatch the genuine ssl module. That has to be done inside the same process. But imagine this scenario: You have a program that gets invoked as root (or some other user than yourself), and you're trying to fiddle with what it sees. You don't have root access, but you can manipulate the file system, to the extent that your userid has access. What can you do to affect this other program? If you're root you shouldn't run untrusted code. See https://docs.python.org/3/using/cmdline.html#cmdoption-I Right, which is why sslcustomize has to be controlled by that, but the possibility of patching (or monkeypatching) ssl.py isn't as big a deal. To be frank I don't understand what you're arguing about. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 2 Sep 2014 00:08, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:42:10 +1000 Chris Angelico ros...@gmail.com wrote: That has to be done inside the same process. But imagine this scenario: You have a program that gets invoked as root (or some other user than yourself), and you're trying to fiddle with what it sees. You don't have root access, but you can manipulate the file system, to the extent that your userid has access. What can you do to affect this other program? If you're root you shouldn't run untrusted code. See https://docs.python.org/3/using/cmdline.html#cmdoption-I Right, which is why sslcustomize has to be controlled by that, but the possibility of patching (or monkeypatching) ssl.py isn't as big a deal. To be frank I don't understand what you're arguing about. When I said shadowing ssl can be tricky to arrange, Chris correctly interpreted it as referring to the filesystem based privilege escalation scenario that isolated mode handles, not to normal in-process monkeypatching or module injection. I don't consider the latter cases to be interesting attack scenarios, as they imply the attacker is *already* running arbitrary Python code inside your CPython process, so you've already lost. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Tue, 2 Sep 2014 00:53:11 +1000 Nick Coghlan ncogh...@gmail.com wrote: On 2 Sep 2014 00:08, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 1 Sep 2014 23:42:10 +1000 Chris Angelico ros...@gmail.com wrote: That has to be done inside the same process. But imagine this scenario: You have a program that gets invoked as root (or some other user than yourself), and you're trying to fiddle with what it sees. You don't have root access, but you can manipulate the file system, to the extent that your userid has access. What can you do to affect this other program? If you're root you shouldn't run untrusted code. See https://docs.python.org/3/using/cmdline.html#cmdoption-I Right, which is why sslcustomize has to be controlled by that, but the possibility of patching (or monkeypatching) ssl.py isn't as big a deal. To be frank I don't understand what you're arguing about. When I said shadowing ssl can be tricky to arrange, Chris correctly interpreted it as referring to the filesystem based privilege escalation scenario that isolated mode handles, not to normal in-process monkeypatching or module injection. There's no actual difference. You can have a sitecustomize.py that does the monkeypatching or the shadowing. There doesn't seem to be anything tricky about that. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 2 Sep 2014 00:59, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 2 Sep 2014 00:53:11 +1000 Nick Coghlan ncogh...@gmail.com wrote: To be frank I don't understand what you're arguing about. When I said shadowing ssl can be tricky to arrange, Chris correctly interpreted it as referring to the filesystem based privilege escalation scenario that isolated mode handles, not to normal in-process monkeypatching or module injection. There's no actual difference. You can have a sitecustomize.py that does the monkeypatching or the shadowing. There doesn't seem to be anything tricky about that. Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize (hence the existence of the related command line options). I missed that you had switched to talking about using that attack vector, rather than trying to shadow stdlib modules directly through the filesystem (which is the only tricky thing I was referring to). Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 01 Sep 2014 08:30:27 +0300, Marko Rauhamaa ma...@pacujo.net wrote: R. David Murray rdmur...@bitdance.com: PS: I recently switched from using selectors to using a timeout on a socket because in that particular application I could, and because reading a socket with a timeout handles EINTR (in recent python versions), whereas reading a non-blocking socket doesn't. Under the hood, a socket with a timeout is a non-blocking socket. Under what circumstances would a nonblocking socket generate an EINTR? Windows. Enough said? The exact error message was: BlockingIOError on my non-blocking socket: 'a non-blocking socket operation could not be completed immediately Needless to say, I was not expecting this, and was about to tear my remaining hair out about having to completely restructure the code in order to be able to handle an EINTR on a read on an FD that I got back from select as ready, until I realized that the way the code had evolved the only thing I still needed the select for was the timeout, and that the EINTR bug in sockets with a timeout had already been fixed (thank goodness I'm able to use python3.4 for this project). I got lucky, but this is clearly a serious problem for writing selectors based code on Windows. This should tell you just about everything you need to know about why we want to fix this problem so that things work cross platform. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 01 Sep 2014 14:15:52 +0300, Marko Rauhamaa ma...@pacujo.net wrote: Charles-François Natali cf.nat...@gmail.com: Which raises an interesting question: what happens to the os.read() return value if SIGINT is received? There's no return value, a KeywordInterrupt exception is raised. The PEP wouldn't change this behavior. Slightly disconcerting... but I'm sure overriding SIGINT would cure that. You don't want to lose data if you want to continue running. As for the general behavior: all programming languages/platforms handle EINTR transparently. C doesn't. EINTR is there for a purpose. I sure hope Python won't bury it under opaque APIs. The two requirements are: * Allow the application to react to signals immediately in the main flow. You don't want to be writing your code in Python then. In Python you *never* get to react immediately to signals. The interpreter sets a flag and calls the python signal handler later. Yes, the call is ASAP, but ASAP is *not* immediately. * Don't lose information. Marko ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rdmurray%40bitdance.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 01 Sep 2014 11:47:07 -0400 R. David Murray rdmur...@bitdance.com wrote: The two requirements are: * Allow the application to react to signals immediately in the main flow. You don't want to be writing your code in Python then. In Python you *never* get to react immediately to signals. The interpreter sets a flag and calls the python signal handler later. Yes, the call is ASAP, but ASAP is *not* immediately. Especially if the signal is delivered to another thread (which is OS-dependent), and the main thread is blocked in *another* system call ;-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
R. David Murray rdmur...@bitdance.com: Windows. Enough said? [...] This should tell you just about everything you need to know about why we want to fix this problem so that things work cross platform. I feel your pain. Well, not really; I just don't want my linux bliss to be taken away. Marko ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
R. David Murray rdmur...@bitdance.com: On Mon, 01 Sep 2014 14:15:52 +0300, Marko Rauhamaa ma...@pacujo.net wrote: * Allow the application to react to signals immediately in the main flow. You don't want to be writing your code in Python then. In Python you *never* get to react immediately to signals. The interpreter sets a flag and calls the python signal handler later. Yes, the call is ASAP, but ASAP is *not* immediately. You don't have to get that philosophical. Immediately means, without delay, without further I/O. Marko ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] cpython and parallel make
Hi, what’s people’s experience with compiling cpython using multiple jobs (e.g., make -j 8)? In my case, I sometimes experience build errors that happen when using -j, whereas the single-job build always works. I haven’t tracked this down in detail, though… here’s an extract from a sample log: 17:42:56 make[1]: Entering directory `/path/to/cpython/build' 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/Python-ast.o Python/Python-ast.c 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/ast.o ../../cpython/Python/ast.c 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/ast.o ../../cpython/Python/ast.c 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/ceval.o ../../cpython/Python/ceval.c 17:42:58 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/compile.o ../../cpython/Python/compile.c 17:43:07 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/graminit.o Python/graminit.c 17:43:07 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/ceval.o ../../cpython/Python/ceval.c 17:43:07 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include-DPy_BUILD_CORE -o Python/compile.o ../../cpython/Python/compile.c 17:43:08 make Include/graminit.h 17:43:08 make[1]: Entering directory `/path/to/cpython/build' 17:43:08 make[1]: `Include/graminit.h' is up to date. 17:43:08 make[1]: Leaving directory `/path/to/cpython/build' 17:43:24 clang -fsanitize=address -o Modules/_freeze_importlib Modules/_freeze_importlib.o Modules/getbuildinfo.o Parser/acceler.o [...] Objects/abstract.o [...] Objects/weakrefobject.o Python/_warnings.o [...] Python/ceval.o Python/compile.o Python/codecs.o [...] Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/xxsubtype.o -ldl -lutil -lm 17:43:24 /usr/bin/ld: error: Python/ceval.o: file is empty 17:43:24 /usr/bin/ld: error: Python/compile.o: file is empty For some reason, some files like Python/ceval.o seem to be built multiple times. I reckon this causes race conditions that result in some empty object files, which causes the linker to fail. Any ideas about where this comes from, and how to fix it? Cheers, Jonas ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Sep 1, 2014, at 11:35 AM, Nick Coghlan ncogh...@gmail.com wrote: On 2 Sep 2014 00:59, Antoine Pitrou solip...@pitrou.net mailto:solip...@pitrou.net wrote: On Tue, 2 Sep 2014 00:53:11 +1000 Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: To be frank I don't understand what you're arguing about. When I said shadowing ssl can be tricky to arrange, Chris correctly interpreted it as referring to the filesystem based privilege escalation scenario that isolated mode handles, not to normal in-process monkeypatching or module injection. There's no actual difference. You can have a sitecustomize.py that does the monkeypatching or the shadowing. There doesn't seem to be anything tricky about that. Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize (hence the existence of the related command line options). I missed that you had switched to talking about using that attack vector, rather than trying to shadow stdlib modules directly through the filesystem (which is the only tricky thing I was referring to). Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io Or you can just install something with easy_install, or you can drop a .pth file and monkey patch there. You can’t stop people from overriding modules, it’s trivial to do. The sys.path ordering just makes it slightly less trivial. — Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 01.09.2014 17:35, Nick Coghlan wrote: Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize (hence the existence of the related command line options). If an attacker is able to place a module like sitecustomize.py in an import directory or any .pth file in a site-packages directory than this Python installation is compromised. .pth files are insidious because they are always loaded and their code is always executed. I don't see how sslcustomize is going to make a difference here. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Sep 1, 2014, at 1:01 PM, Christian Heimes christ...@python.org wrote: On 01.09.2014 17:35, Nick Coghlan wrote: Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize (hence the existence of the related command line options). If an attacker is able to place a module like sitecustomize.py in an import directory or any .pth file in a site-packages directory than this Python installation is compromised. .pth files are insidious because they are always loaded and their code is always executed. I don't see how sslcustomize is going to make a difference here. Right, this is the point I was trying to make. If you’ve installed a malicious package it’s game over. There’s nothing Python can do to help you. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython and parallel make
Hi, My bashrc sets MAKEFLAGS to -j9 and Python compilation works fine on Fedora 20 with GNU make and GCC. My computer has 8 cores (4 physical with hyper threading). It looks like your compiler is Clang. What is your OS and OS version? Can you try to run make in verbose mode and attach the full log to your email? Ex: try make SHELL=bash -x to see executed shell commands. (Run make clean before) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython and parallel make
On Mon, Sep 1, 2014, 15:16 Victor Stinner victor.stin...@gmail.com wrote: Hi, My bashrc sets MAKEFLAGS to -j9 and Python compilation works fine on Fedora 20 with GNU make and GCC. My computer has 8 cores (4 physical with hyper threading). It looks like your compiler is Clang. What is your OS and OS version? I compile with -j8 with Clang on OS X and never have issues. -Brett Can you try to run make in verbose mode and attach the full log to your email? Ex: try make SHELL=bash -x to see executed shell commands. (Run make clean before) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
Victor Stinner victor.stin...@gmail.com wrote: HTML version: http://legacy.python.org/dev/peps/pep-0475/ PEP: 475 Title: Retry system calls failing with EINTR I think the proposed design for how Python should behave is a good one. But I think this proposal needs to be treated in the same way as any other backwards-incompatible change. Applications relying on the fact that system calls are interrupted with ``InterruptedError`` will hang. The authors of this PEP don't think that such application exist. The authors are mistaken here. I have a program still running which was designed around this behaviour. My company won't be inconvenienced by this change because I can't imagine the elderly program ever being ported to Python 3. But I think it's very likely there are other such programs out there. If such applications exist, they are not portable and are subject to race conditions (deadlock if the signal comes before the system call). The program is certainly not portable (which is not any kind of a problem), and as pselect is unavailable there is indeed the usual theoretical race (which has not been a problem in practice in the ten years it's been running). (The program handles SIGTERM so that it can do a bit of cleanup before exiting, and it uses the signal-handler-sets-a-flag technique. The call that might be interrupted is sleep(), so the program doesn't strictly _rely_ on the existing behaviour; it would just become very slow to exit.) -M- ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On 2 Sep 2014 03:08, Donald Stufft don...@stufft.io wrote: On Sep 1, 2014, at 1:01 PM, Christian Heimes christ...@python.org wrote: On 01.09.2014 17:35, Nick Coghlan wrote: Oh, now I get what you mean - yes, sitecustomize already poses the same kind of problem as the proposed sslcustomize (hence the existence of the related command line options). If an attacker is able to place a module like sitecustomize.py in an import directory or any .pth file in a site-packages directory than this Python installation is compromised. .pth files are insidious because they are always loaded and their code is always executed. I don't see how sslcustomize is going to make a difference here. Right, this is the point I was trying to make. If you’ve installed a malicious package it’s game over. There’s nothing Python can do to help you. Yes, that's what I said originally when pointing out that isolated mode and the switch to disable site module processing would need to disable sslcustomize processing as well. Antoine was replying to a side comment about it being tricky to shadow stdlib modules. I left out the qualifier directly in my original comment, and he left out indirectly through sitecustomize in his initial reply, so we were talking past each for a while. Cheers, Nick. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daily reference leaks (640c575ab3e1): sum=151940
Is anyone working on those? On Mon, 01 Sep 2014 10:41:45 +0200 solip...@pitrou.net wrote: results for 640c575ab3e1 on branch default test_codecs leaked [5825, 5825, 5825] references, sum=17475 test_codecs leaked [1172, 1174, 1174] memory blocks, sum=3520 test_collections leaked [0, 2, 0] references, sum=2 test_distutils leaked [37735, 37735, 37735] references, sum=113205 test_distutils leaked [5909, 5911, 5911] memory blocks, sum=17731 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 0, 2] references, sum=2 test_site leaked [0, 0, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogHPiXJp', '-x'] ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daily reference leaks (640c575ab3e1): sum=151940
The codecs one is https://bugs.python.org/issue22166 On Mon, Sep 1, 2014, at 16:16, Antoine Pitrou wrote: Is anyone working on those? On Mon, 01 Sep 2014 10:41:45 +0200 solip...@pitrou.net wrote: results for 640c575ab3e1 on branch default test_codecs leaked [5825, 5825, 5825] references, sum=17475 test_codecs leaked [1172, 1174, 1174] memory blocks, sum=3520 test_collections leaked [0, 2, 0] references, sum=2 test_distutils leaked [37735, 37735, 37735] references, sum=113205 test_distutils leaked [5909, 5911, 5911] memory blocks, sum=17731 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 0, 2] references, sum=2 test_site leaked [0, 0, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogHPiXJp', '-x'] ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/benjamin%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] https:bugs.python.org -- Untrusted Connection (Firefox)
As of today I still am getting untrusted cert thought I would re-ping to see if there is an ETA. On Thu, Aug 21, 2014 at 10:32 PM, Terry Reedy tjre...@udel.edu wrote: On 8/21/2014 7:25 PM, Nick Coghlan wrote: On 22 Aug 2014 04:45, Benjamin Peterson benja...@python.org mailto:benja...@python.org wrote: Perhaps some board members could comment, but I hope the PSF could just pay a few hundred a year for a proper certificate. That's exactly what we're doing - MAL reminded me we reached the same conclusion last time this came up, we'll just track it better this time to make sure it doesn't slip through the cracks again. (And yes, switching to forced HTTPS once this is addressed would also be a good idea - we'll add it to the list) I just switched from a 'low variety' short password of the sort almost crackable with brute force (today, though not several years ago) to a higher variety longer password. People with admin privileges on the tracker might be reminded to recheck. What was adequate 10 years ago is not so now. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/gokoproject%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] https:bugs.python.org -- Untrusted Connection (Firefox)
I got the same in Chrome on my Mac. Skip On Sep 1, 2014 8:00 PM, John Wong gokoproj...@gmail.com wrote: As of today I still am getting untrusted cert thought I would re-ping to see if there is an ETA. On Thu, Aug 21, 2014 at 10:32 PM, Terry Reedy tjre...@udel.edu wrote: On 8/21/2014 7:25 PM, Nick Coghlan wrote: On 22 Aug 2014 04:45, Benjamin Peterson benja...@python.org mailto:benja...@python.org wrote: Perhaps some board members could comment, but I hope the PSF could just pay a few hundred a year for a proper certificate. That's exactly what we're doing - MAL reminded me we reached the same conclusion last time this came up, we'll just track it better this time to make sure it doesn't slip through the cracks again. (And yes, switching to forced HTTPS once this is addressed would also be a good idea - we'll add it to the list) I just switched from a 'low variety' short password of the sort almost crackable with brute force (today, though not several years ago) to a higher variety longer password. People with admin privileges on the tracker might be reminded to recheck. What was adequate 10 years ago is not so now. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/gokoproject%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/skip%40pobox.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
