freesteel wrote: > Yes, I see that now in the documentation, which to me is quite > confusing. > So, how do you use python in a multithreaded environment, where for > example you want to run some embeded python code from a number of > different C threads? > > This article: http://www.linuxjournal.com/article/3641 is quite good, > but must have been written before these PyGILState_* functions. > > I only used Py_NewInterpreter to have a fresh 'import sys'. > > Somebody enlighten me, please. > > Martin
If you try to replicate the code from the linux journal (http://www.linuxjournal.com/article/3641) and compile with Py_DEBUG defined, when running it you will find that you get an exception and a fatal error message from the python core. The exception is thrown from pystate.c, line 306: Py_FatalError("Invalid thread state for this thread"); The exception is thrown in my understanding of the code because there can only ever be one thread state. If this is true a function to swap thread states seems rather pointless. Now, reading up about how to call C API to python from a C thread I find that with version 2.3 the function pair PyGILState_Ensure/PyGILState_Release was introduced. This allows 'grabbing' the global interpreter lock, calling your embedded python code and release the GIL at the end again. Am I right in my understanding that the use of this PyGILState_* pair is meant to 'replace' the prologue of creating a new thread state from the main thread state, swapping it with the current state, doing your Python/C API calls and then in an epilogue swap the previous thread state back in, and deletie the now obsolete previously created thread state? At least that how I understand the motivation behind the introduction of tPyGILState_*, read here: http://www.python.org/dev/peps/pep-0311/ Now, I tried to use this mechanism, but I am not really successful with it. Basically, in my C thread, I wrap a number of calls to embedded python (a few PyRun_SimpleString calls, nothing really fancy) with PyGILState_Ensure and PyGILState_Release. The first C thread also initializes python as well as initializes threading: Py_Initialize(); PyEval_InitThreads(); (If it helps I can post my whole code here). When I test this I get deadlocks in most cases after the first thread has called PyGILState_Release. All other threads at that point 'freeze' and never return from whatever Python/C API function they are in. I don't understand why. Moreover, this behavious is not always reproducable, sometimes the code works as intended. Must be some kind of race condition? Who can help? I read that the problem of calling python from a multithreaded application per thread is very difficult, but a few projects have managed to solve it, but now we have these PyGILStates and all is much easier? Martin -- http://mail.python.org/mailman/listinfo/python-list
