Hi all! I am new to the list and arriving with a concrete problem that I'd like to fix myself.
I am embedding Python (3.6) into my C++ application and I would like to run Python scripts isolated from each other using sub-interpreters. I am not using threads; everything is supposed to run in the application's main thread. I noticed that if I create an interpreter, switch to it and execute code that imports numpy (1.13), my application will hang. ntdll.dll!NtWaitForSingleObject() Unknown KernelBase.dll!WaitForSingleObjectEx() Unknown > python36.dll!_PyCOND_WAIT_MS(_PyCOND_T * cv=0x00000000748a67a0, > _RTL_CRITICAL_SECTION * cs=0x00000000748a6778, unsigned long ms=5) Line 245 C [Inline Frame] python36.dll!PyCOND_TIMEDWAIT(_PyCOND_T *) Line 275 C python36.dll!take_gil(_ts * tstate=0x0000023251cbc260) Line 224 C python36.dll!PyEval_RestoreThread(_ts * tstate=0x0000023251cbc260) Line 370 C python36.dll!PyGILState_Ensure() Line 855 C umath.cp36-win_amd64.pyd!00007ff8c6306ab2() Unknown umath.cp36-win_amd64.pyd!00007ff8c630723c() Unknown umath.cp36-win_amd64.pyd!00007ff8c6303a1d() Unknown umath.cp36-win_amd64.pyd!00007ff8c63077c0() Unknown umath.cp36-win_amd64.pyd!00007ff8c62ff926() Unknown [Inline Frame] python36.dll!_PyObject_FastCallDict(_object *) Line 2316 C [Inline Frame] python36.dll!_PyObject_FastCallKeywords(_object *) Line 2480 C python36.dll!call_function(_object * * * pp_stack=0x00000048be5f5e40, __int64 oparg, _object * kwnames) Line 4822 C Numpy's extension umath calls PyGILState_Ensure(), which in turn calls PyEval_RestoreThread on the (auto) threadstate of the main interpreter. And that's wrong. We are already holding the GIL with the threadstate of our current sub-interpreter, so there's no need to switch. I know that the GIL API is not fully compatible with sub-interpreters, as issues #10915 and #15751 illustrate. But since I need to support calls to PyGILState_Ensure - numpy is the best example -, I am trying to improve the situation here: https://github.com/stephanreiter/cpython/commit/d9d3451b038af2820f500843b6a88f57270e1597 That change may be naive, but it does the trick for my use case. If totally wrong, I don't mind pursuing another alley. Essentially, I'd like to ask for some guidance in how to tackle this problem while keeping the current GIL API unchanged (to avoid breaking modules). I am also wondering how I can test any changes I am proposing. Is there a test suite for interpreters, for example? Thank you very much, Stephan _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com