On Oct 24, 2:12 am, greg <[EMAIL PROTECTED]> wrote: > Andy wrote: > > 1) Independent interpreters (this is the easier one--and solved, in > > principle anyway, by PEP 3121, by Martin v. Löwis > > Something like that is necessary for independent interpreters, > but not sufficient. There are also all the built-in constants > and type objects to consider. Most of these are statically > allocated at the moment. >
Agreed--I was just trying to speak generally. Or, put another way, there's no hope for independent interpreters without the likes of PEP 3121. Also, as Martin pointed out, there's the issue of module cleanup some guys here may underestimate (and I'm glad Martin pointed out the importance of it). Without the module cleanup, every time a dynamic library using python loads and unloads you've got leaks. This issue is a real problem for us since our software is loaded and unloaded many many times in a host app (iTunes, WMP, etc). I hadn't raised it here yet (and I don't want to turn the discussion to this), but lack of multiple load and unload support has been another painful issue that we didn't expect to encounter when we went with python. > > 2) Barriers to "free threading". As Jesse describes, this is simply > > just the GIL being in place, but of course it's there for a reason. > > It's there because (1) doesn't hold and there was never any specs/ > > guidance put forward about what should and shouldn't be done in multi- > > threaded apps > > No, it's there because it's necessary for acceptable performance > when multiple threads are running in one interpreter. Independent > interpreters wouldn't mean the absence of a GIL; it would only > mean each interpreter having its own GIL. > I see what you're saying, but let's note that what you're talking about at this point is an interpreter containing protection from the client level violating (supposed) direction put forth in python multithreaded guidelines. Glenn Linderman's post really gets at what's at hand here. It's really important to consider that it's not a given that python (or any framework) has to be designed against hazardous use. Again, I refer you to the diagrams and guidelines in the QuickTime API: http://developer.apple.com/technotes/tn/tn2125.html They tell you point-blank what you can and can't do, and it's that's simple. Their engineers can then simply create the implementation around those specs and not weigh any of the implementation down with sync mechanisms. I'm in the camp that simplicity and convention wins the day when it comes to an API. It's safe to say that software engineers expect and assume that a thread that doesn't have contact with other threads (except for explicit, controlled message/object passing) will run unhindered and safely, so I raise an eyebrow at the GIL (or any internal "helper" sync stuff) holding up an thread's performance when the app is designed to not need lower-level global locks. Anyway, let's talk about solutions. My company looking to support python dev community endeavor that allows the following: - an app makes N worker threads (using the OS) - each worker thread makes its own interpreter, pops scripts off a work queue, and manages exporting (and then importing) result data to other parts of the app. Generally, we're talking about CPU-bound work here. - each interpreter has the essentials (e.g. math support, string support, re support, and so on -- I realize this is open-ended, but work with me here). Let's guesstimate about what kind of work we're talking about here and if this is even in the realm of possibility. If we find that it *is* possible, let's figure out what level of work we're talking about. >From there, I can get serious about writing up a PEP/spec, paid support, and so on. Regards, Andy -- http://mail.python.org/mailman/listinfo/python-list
