On Thu, Apr 04, 2013 at 11:53:01PM -0700, Charles-Fran?ois Natali wrote: > Hello, > > >> async.submit_work(func, args, kwds, callback=None, errback=None) > >> > >> How do you implement arguments passing and return value? > >> > >> e.g. let's say I pass a list as argument: how do you iterate on the > >> list from the worker thread without modifying the backing objects for > >> refcounts (IIUC you use a per-thread heap and don't do any > >> refcounting). > > > > Correct, nothing special is done for the arguments (apart from > > incref'ing them in the main thread before kicking off the parallel > > thread (then decref'ing them in the main thread once we're sure the > > parallel thread has finished)). > > IIUC you incref the argument from the main thread before publishing it > to the worker thread: but what about containers like list? How do you > make sure the refcounts of the elements don't get deallocated while > the worker thread iterates?
Ah, so, all of my examples were missing async.run(). They should have looked like this: async.submit_work(foo) async.submit_work(bar) async.run() async.run() is called from the main thread, with the GIL held, and it blocks until all parallel threads (well, parallel contexts, to be exact) have completed. The parallel 'work' doesn't actually start until async.run() is called either. (That's completely untrue at the moment; async.submit_work(foo) will execute foo() in a parallel thread immediately. Fixing that is on the todo list.) With only parallel threads running, no main-thread objects could ever be deallocated*, as no decref'ing is ever done. [*]: unless you went out of your way to delete/deallocate main thread objects via the @async.call_from_main_thread facility. At the moment, that's firmly in the category of "Don't Do That". (And, thinking about it a little more, I guess I could augment the ceval loop in such a way that in order for the main thread to run things scheduled via @async.call_from_main_thread, all parallel threads need to be suspended. Or I could just freeze/thaw them (although I don't know if there are POSIX counterparts to those Windows methods). That would definitely impede performance, but it would assure data integrity. Perhaps it should be enabled by default, with the option to disable it for consenting adults.) > More generally, how do you deal with non-local objects? Read-only ops against non-local (main-thread) objects from parallel threads are free, which is nice. Things get tricky when you try to mutate main-thread objects from parallel threads. That's where all the context persistence, interlocked data types, object protection etc stuff comes in. Is... that what you mean by how do I deal with non-local objects? I took a guess ;-) Regards, Trent. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com