I edited the post so many times I forgot the subject line! I am a bad person.
Maybe the most condensed question here is: How do you normally manage multiple resources competing with the GIL in a way that could cause deadlocks with it? For the details, see the long post. On Thu, Mar 15, 2012 at 11:59 PM, Adam Preble <adam.pre...@gmail.com> wrote: > I discovered recently that during callbacks to Python objects, sometimes > the interpreter would try to do stuff at the same time, despite the fact I > made a call to ensure the GIL. I read the solution for that kind of thing > I'm doing is calling PyEval_InitThreads(). This worked at first, but like > more race conditions and stuff, all it takes is walking away from the > computer for it all to fall apart. > > What I'm seeing is a rather elaborate deadlock situation revolved around > securing the GIL. I think I need to basically put my interpreter in its > own subsystem, design-wise, and ferret calls to invoke things in the > interpreter to it, in order to ultimately get around this. However, what > I'm asking of the distribution is how they've gotten around this. > > To give something a little more concrete--this is pretty gross: > 1. Main thread starts interpreter and is running a script > 2. The script defines an implementation of callback interface A > 3. The script starts some objects that represent threads in the C++ > runtime. These are threads 1 and 2. > 4. The script starts to create an object that is wrapped from C++ > 5. The object requires a resource from thread 1, where I use promises and > futures enqueue and fulfill the request when thread #1 isn't busy. > 6. Meanwhile, the interpreter thread is waiting for the resource since it > cannot put it off any further > 7. At this point, thread 2 tries to invoke the callback to interface A, > and it needs the interpreter thread. > 8. thread #1 needs thread #2 to complete this critical step before the > get() call will complete for the master interpreter thread > 9. Thread #2 needs thread #1 to finish so it can get the GIL. It's > basically locked up in PyGILState_Ensure(). > > Heck of a deadlock. I am pondering having something else control the > interpreter in its own thread and have everybody enqueue stuff up to run in > it, like with the promises and futures I'm using elsewhere already. The > reason is that thread #2 doesn't really need to steal the interpreter at > that very exact moment. And furthermore, I'm trying to use Stackless, and > it's my understanding there I can link it up so that the interpreter gets > ahold of the Python runtime at controlled intervals--if desired--to invoke > other stuff. > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig@python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig >
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
