On Oct 28, 9:30 am, "Andy O'Meara" <[EMAIL PROTECTED]> wrote: > On Oct 25, 9:46 am, "M.-A. Lemburg" <[EMAIL PROTECTED]> wrote: > > > These discussion pop up every year or so and I think that most of them > > are not really all that necessary, since the GIL isn't all that bad. > > Thing is, if the topic keeps coming up, then that may be an indicator > that change is truly needed. Someone much wiser than me once shared > that a measure of the usefulness and quality of a package (or API) is > how easily it can be added to an application--of any flavors--without > the application needing to change. > > So in the rising world of idle cores and worker threads, I do see an > increasing concern over the GIL. Although I recognize that the debate > is lengthy, heated, and has strong arguments on both sides, my reading > on the issue makes me feel like there's a bias for the pro-GIL side > because of the volume of design and coding work associated with > considering various alternatives (such as Glenn's "Py*" concepts). > And I DO respect and appreciate where the pro-GIL people come from: > who the heck wants to do all that work and recoding so that a tiny > percent of developers can benefit? And my best response is that as > unfortunate as it is, python needs to be more multi-threaded app- > friendly if we hope to attract the next generation of app developers > that want to just drop python into their app (and not have to change > their app around python). For example, Lua has that property, as > evidenced by its rapidly growing presence in commercial software > (Blizzard uses it heavily, for example). > > > > > Furthermore, there are lots of ways to tune the CPython VM to make > > it more or less responsive to thread switches via the various sys.set*() > > functions in the sys module. > > > Most computing or I/O intense C extensions, built-in modules and object > > implementations already release the GIL for you, so it usually doesn't > > get in the way all that often. > > The main issue I take there is that it's often highly useful for C > modules to make subsequent calls back into the interpreter. I suppose > the response to that is to call the GIL before reentry, but it just > seems to be more code and responsibility in scenarios where it's no > necessary. Although that code and protocol may come easy to veteran > CPython developers, let's not forget that an important goal is to > attract new developers and companies to the scene, where they get > their thread-independent code up and running using python without any > unexpected reengineering. Again, why are companies choosing Lua over > Python when it comes to an easy and flexible drop-in interpreter? And > please take my points here to be exploratory, and not hostile or > accusatory, in nature. > > Andy
Okay, here's the bottom line: * This is not about the GIL. This is about *completely* isolated interpreters; most of the time when we want to remove the GIL we want a single interpreter with lots of shared data. * Your use case, although not common, is not extraordinarily rare either. It'd be nice to support. * If CPython had supported it all along we would continue to maintain it. * However, since it's not supported today, it's not worth the time invested, API incompatibility, and general breakage it would imply. * Although it's far more work than just solving your problem, if I were to remove the GIL I'd go all the way and allow shared objects. So there's really only two options here: * get a short-term bodge that works, like hacking the 3rd party library to use your shared-memory allocator. Should be far less work than hacking all of CPython. * invest yourself in solving the *entire* problem (GIL removal with shared python objects). -- http://mail.python.org/mailman/listinfo/python-list