Jesse, Terry, Martin - First off, thanks again for your time and interest in this matter. It's definitely encouraging to know that time and real effort is being put into the matter and I hope my posts on this subject are hopefully an informative data point for everyone here.
Thanks for that link to Adam Olsen's work, Jesse--I'll definitely look more closely at it. As I mentioned in my previous post, end-user devs like me are programmed to get nervous around new mods but at first glance there definitely seems to be interesting. My initial reaction, as interesting as the project is, goes back to by previous post about putting all the object safety responsibility on the shoulders of the API client. That way, one gets the best of both worlds: free threading and no unnecessary object locking/blocking (ie. the API client will manage moving the synchronization req'd to move objects from one interpreter to another). I could have it wrong, but it seems like safethread inserts some thread-safety features but they come at the cost of performance. I know I keep mentioning it, but I think the QuickTime API (and its documentation) is a great model for how any API should approach threading. Check out their docs to see how they address it; conceptually speaking, there's not a single line of thread safety in QuickTime: http://developer.apple.com/technotes/tn/tn2125.html In short: multiple thread is tricky; it's the responsibility of the API client to not do hazardous things. And for the record: the module multiprocessing is totally great answer for python-level MP stuff--very nice work, Jesse! I'd like to post and discuss more, but I'll pick it up tomorrow... All this stuff is fun and interesting to talk about, but I have to get to some other things and it unfortunately comes down to cost analysis. Sadly, I look at it as I can allocate 2-3 man months (~ $40k) to build our own basic python interpreter implementation that solves our need for free threading and increased performance (we've built various internal interpreters over the years so we have good experience in house, our tools are high performance, and we only use a pretty small subset of python). Or, there's the more attractive approach to work with the python dev community and put that dev expenditure into a form everyone can benefit from. Regards, Andy On Oct 22, 5:21 pm, "Jesse Noller" <[EMAIL PROTECTED]> wrote: > On Wed, Oct 22, 2008 at 12:32 PM, Andy <[EMAIL PROTECTED]> wrote: > > And, yes, I'm aware of the multiprocessing module added in 2.6, but > > that stuff isn't lightweight and isn't suitable at all for many > > environments (including ours). The bottom line is that if you want to > > perform independent processing (in python) on different threads, using > > the machine's multiple cores to the fullest, then you're out of luck > > under python 2. > > So, as the guy-on-the-hook for multiprocessing, I'd like to know what > you might suggest for it to make it more apt for your - and other > environments. > > Additionally, have you looked > at:https://launchpad.net/python-safethreadhttp://code.google.com/p/python-safethread/w/list > (By Adam olsen) > > -jesse -- http://mail.python.org/mailman/listinfo/python-list