> > What you describe, truly independent interpreters, is not threading at > all: it is processes, emulated at the application level, with all the > memory cost and none of the OS protections. True threading would > involve sharing most objects. > > Your solution depends on what you need: > * Killable "threads" -> OS processes > * multicore usage (GIL removal) -> OS processes or alternative Python > implementations (PyPy/Jython/IronPython) > * Sane shared objects -> safethread
I realize what you're saying, but it's better said there's two issues at hand: 1) Independent interpreters (this is the easier one--and solved, in principle anyway, by PEP 3121, by Martin v. Löwis, but is FAR from being carried through in modules as he pointed out). As you point out, this doesn't directly relate to multi-threading BUT it is intimately tied to the issue because if, in principle, every module used instance data (rather than static data), then python would be WELL on its way to "free threading" (as Jesse Noller calls it), or as I was calling it "true multi-threading". 2) Barriers to "free threading". As Jesse describes, this is simply just the GIL being in place, but of course it's there for a reason. It's there because (1) doesn't hold and there was never any specs/ guidance put forward about what should and shouldn't be done in multi- threaded apps (see my QuickTime API example). Perhaps if we could go back in time, we would not put the GIL in place, strict guidelines regarding multithreaded use would have been established, and PEP 3121 would have been mandatory for C modules. Then again--screw that, if I could go back in time, I'd just go for the lottery tickets!! :^) Anyway, I've been at this issue for quite a while now (we're approaching our 3rd release cycle), so I'm pretty comfortable with the principles at hand. I'd say the theme of your comments share the theme of others here, so perhaps consider where end-user software houses (like us) are coming from. Specifically, developing commercial software for end users imposes some restrictions that open source development communities aren't often as sensitive to, namely: - Performance -- emulation is a no-go (e.g. Jython) - Maturity and Licensing -- experimental/academic projects are no-go (PyPy) - Cross platform support -- love it or hate it, Win32 and OS X are all that matter when you're talking about selling (and supporting) software to the masses. I'm just the messenger here (ie. this is NOT flamebait). We publish for OS X, so IronPython is therefore out. Basically, our company is at a crossroads where we really need light, clean "free threading" as Jesse calls it (e.g. on the iPhone, using our python drawing wrapper to do primary drawing while running python jobs on another thread doing image decoding and processing). In our current iPhone app, we achieve this by using two python bundles (dynamic libs) in the way I described in my initial post. Sure, thus solves our problem, but it's pretty messy, sucks up resources, and has been a pain to maintain. Moving forward, please understand my posts here are also intended to give the CPython dev community a glimpse of the issues that may not be as visible to you guys (as they are for dev houses like us). For example, it'd be pretty cool if Blizzard went with python instead of lua, wouldn't you think? But some of the issues I've raised here no doubt factor in to why end-user dev houses ultimately may have to pass up python in favor of another interpreted language. Bottom line: why give prospective devs any reason to turn down python-- there's just so many great things about python! Regards, Andy -- http://mail.python.org/mailman/listinfo/python-list