On Sun, May 19, 2013 at 10:02 AM, Carlos Nepomuceno <carlosnepomuc...@outlook.com> wrote: > I didn't know Python threads aren't preemptive. Seems to be something really > old considering the state of the art on parallel execution on multi-cores. > > What's the catch on making Python threads preemptive? Are there any ongoing > projects to make that?
Preemption isn't really the issue here. On the C level, preemptive vs cooperative usually means the difference between a stalled thread locking everyone else out and not doing so. Preemption is done at a lower level than user code (eg the operating system or the CPU), meaning that user code can't retain control of the CPU. With interpreted code eg in CPython, it's easy to implement preemption in the interpreter. I don't know how it's actually done, but one easy implementation would be "every N bytecode instructions, context switch". It's still done at a lower level than user code (N bytecode instructions might all actually be a single tight loop that the programmer didn't realize was infinite), but it's not at the OS level. But none of that has anything to do with multiple core usage. The problem there is that shared data structures need to be accessed simultaneously, and in CPython, there's a Global Interpreter Lock to simplify that; but the consequence of the GIL is that no two threads can simultaneously execute user-level code. There have been GIL-removal proposals at various times, but the fact remains that a global lock makes a huge amount of sense and gives pretty good performance across the board. There's always multiprocessing when you need multiple CPU-bound threads; it's an explicit way to separate the shared data (what gets transferred) from local (what doesn't). ChrisA -- http://mail.python.org/mailman/listinfo/python-list