[Python-Dev] thread issues when embedding Python

2013-12-17 Thread Daniel Pocock
I've successfully embedded Python for a single thread I tried to extend the implementation for multiple threads (a worker thread scenario) and I'm encountering either deadlocks or seg faults depending upon how I got about it. There seems to be some inconsistency between what is covered in the d

[Python-Dev] thread issues when embedding Python

2013-12-17 Thread Daniel Pocock
I've successfully embedded Python for a single thread I tried to extend the implementation for multiple threads (a worker thread scenario) and I'm encountering either deadlocks or seg faults depending upon how I got about it. There seems to be some inconsistency between what is covered in the d

Re: [Python-Dev] thread issues when embedding Python

2013-12-18 Thread Daniel Pocock
Another link that fills in some gaps and finally helped me make this work: http://www.codevate.com/blog/7-concurrency-with-embedded-python-in-a-multi-threaded-c-application In particular, I found that PyGILState_Ensure/PyGILState_Release as described in the Python docs is not sufficient - as des

Re: [Python-Dev] thread issues when embedding Python

2013-12-18 Thread Chris Angelico
On Wed, Dec 18, 2013 at 9:26 PM, Daniel Pocock wrote: > b) when each worker thread starts, call > PyThreadState_New(mInterpreterState) and save the result in a thread > local mPyThreadState > > c) use the mPyThreadState with PyEval_RestoreThread and > PyEval_SaveThread before and after calling Py

Re: [Python-Dev] thread issues when embedding Python

2013-12-18 Thread Daniel Pocock
On 18/12/13 16:02, Chris Angelico wrote: > On Wed, Dec 18, 2013 at 9:26 PM, Daniel Pocock wrote: >> b) when each worker thread starts, call >> PyThreadState_New(mInterpreterState) and save the result in a thread >> local mPyThreadState >> >> c) use the mPyThreadState with PyEval_RestoreThread and

Re: [Python-Dev] thread issues when embedding Python

2013-12-18 Thread Antoine Pitrou
On Wed, 18 Dec 2013 00:19:23 +0100 Daniel Pocock wrote: > > If a main thread does things like importing a module and obtaining a > reference to a Python method, can those things be used by other C++ > threads or do they have to repeat those lookups? The C++ threads must use the PyGILState API to

Re: [Python-Dev] thread issues when embedding Python

2013-12-18 Thread Victor Stinner
2013/12/18 Antoine Pitrou : > You only need to call PyEval_InitThreads() once in the main Python > thread. This is not well documented. For your information, PyGILState_Ensure() now calls PyEval_InitThreads() in Python 3.4, see: http://bugs.python.org/issue19576 Victor ___

Re: [Python-Dev] thread issues when embedding Python

2013-12-18 Thread Daniel Pocock
On 18/12/13 16:29, Victor Stinner wrote: > 2013/12/18 Antoine Pitrou : >> You only need to call PyEval_InitThreads() once in the main Python >> thread. > > This is not well documented. For your information, PyGILState_Ensure() > now calls PyEval_InitThreads() in Python 3.4, see: > http://bugs.py

Re: [Python-Dev] thread issues when embedding Python

2013-12-19 Thread Nick Coghlan
On 19 December 2013 07:58, Daniel Pocock wrote: > > > On 18/12/13 16:29, Victor Stinner wrote: >> 2013/12/18 Antoine Pitrou : >>> You only need to call PyEval_InitThreads() once in the main Python >>> thread. >> >> This is not well documented. For your information, PyGILState_Ensure() >> now calls

Re: [Python-Dev] thread issues when embedding Python

2013-12-19 Thread Daniel Pocock
On 19/12/13 12:22, Nick Coghlan wrote: > On 19 December 2013 07:58, Daniel Pocock wrote: >> >> On 18/12/13 16:29, Victor Stinner wrote: >>> 2013/12/18 Antoine Pitrou : You only need to call PyEval_InitThreads() once in the main Python thread. >>> This is not well documented. For your inf

Re: [Python-Dev] thread issues when embedding Python

2013-12-19 Thread Nick Coghlan
On 19 December 2013 21:28, Daniel Pocock wrote: > On 19/12/13 12:22, Nick Coghlan wrote: >> I don't see anything in your article about how you ensure that the >> main thread of the application *before anything else related to the >> embedded Python happens* calls both Py_Initialize() and >> PyEval