Neil Schemenauer wrote: > Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: >> For a student project in a course on virtual machines, we are >> evaluating the possibility to experiment with removing the GIL >> from CPython > > Hi, > > It's great to hear of this kind of project. I think what you want > to do is difficult but possible. The major compilcation would be > that extension modules would have to re-written since they all > assume a reference counting GC. A foreign function interface like > CMU Lisp's "alien" or GHC's FFI is not necessarily any worse but it > does place different demands on extension module authors.
Michael Foord's comment about the way Ironclad does it suggests a possible interim step that would allow existing extensions to be used at the cost of some additional overhead: use free threading in the core, but have an "extension module lock" that cuts things back to a single thread whenever non-core code is invoked. It wouldn't be as fast as a completely free-threaded setup (since the EML would need to operate as a variant of a 'read/write' lock, with the core code mostly leaving it in the multiple-threads-allowed 'read' mode, promoting it to single-thread-only 'write' mode when potentially dispatching to non-core code, and dropping back to the 'read' mode when the dispatch landed back in core code again). Over time, extension modules which were updated to use their own internal locks could also drop back to free-threading mode. In essence, the GIL would still exist, but instead of being a simple mutex the way it is now, it would have an intermediate stage which allowed multiple threads to continue running. Once most extension modules had been converted, the function to promote the EML from multi-threaded mode to single-thread only mode could be removed. A mechanism like that would avoid one of the more subtle problems in removing the GIL: currently, CPython-specific Python code can assume that invocation of a C function from Python code is an atomic operation, since the body of the function executes as a single bytecode (we're talking about the case here where the C function is known not to release the GIL). I've written code which relied on the GIL in exactly that way, and I'm sure I'm far from the only Python programmer to do so. If the GIL was removed without adding an extension module lock, such applications would require additional locking either in the C extension module or in the Python code. A possible starting point for such a project: move the locking mechanisms that are implemented in Python in the threading module to C-based implementations in _thread so that a richer low overhead locking arrangement is available to extensions modules (and the rest of the core). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --------------------------------------------------------------- _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
