Boyan,
How critical is this bug with respect to releasing 3.2? I'm heading out
for the weekend in the next couple of hours and was hoping to release
3.2.1beta beforehand. If this is a must fix I'll wait until next week.
(Either way, I'll fix the memory leak get_interpreter which you detail
in MODPYTHON-77).
Regards,
Jim
Jim Gallacher (JIRA) wrote:
[ http://issues.apache.org/jira/browse/MODPYTHON-77?page=comments#action_12320905 ]
Jim Gallacher commented on MODPYTHON-77:
----------------------------------------
I tried your patch and the unit tests fail. Apache fails to start, with the
following message in the error log:
Fatal Python error: PyThreadState_Get: no current thread
Using
GCC 4.0.2
apache mpm-prefork 2.0.54
python 2.3.5
Also, please submit your patches as unified diffs (diff -u) or even better if you are
using code checked out of subversion use "svn diff"
The multiple interpreter concept of mod_python is broken for Python extension
modules since Python 2.3
------------------------------------------------------------------------------------------------------
Key: MODPYTHON-77
URL: http://issues.apache.org/jira/browse/MODPYTHON-77
Project: mod_python
Type: Bug
Components: core
Versions: 3.1.4
Environment: Python >= 2.3
Reporter: Boyan Boyadjiev
Attachments: diff.txt, mod_python.c
The multiple interpreter concept of mod_python is broken for Python extension
modules since Python 2.3 because of the PEP 311 (Simplified Global Interpreter
Lock Acquisition for Extensions):
...
Limitations and Exclusions
This proposal identifies a solution for extension authors with
complex multi-threaded requirements, but that only require a
single "PyInterpreterState". There is no attempt to cater for
extensions that require multiple interpreter states. At the time
of writing, no extension has been identified that requires
multiple PyInterpreterStates, and indeed it is not clear if that
facility works correctly in Python itself.
...
For mod_python this means, that complex Python extensions won't work any more with Python >= 2.3, because they are supposed to work only with the first interpreter state initialized for the current process (a problem we experienced). The first interpreter state is not used by mod_python after the python_init is called.
One solution, which works fine for me, is to save the first interpreter state into the "interpreters" dictionary in the function python_init (MAIN_INTERPRETER is used as a key):
static int python_init(apr_pool_t *p, apr_pool_t *ptemp,
apr_pool_t *plog, server_rec *s)
{
...
/* initialize global Python interpreter if necessary */
if (! Py_IsInitialized())
{
/* initialze the interpreter */
Py_Initialize();
#ifdef WITH_THREAD
/* create and acquire the interpreter lock */
PyEval_InitThreads();
#endif
/* create the obCallBack dictionary */
interpreters = PyDict_New();
if (! interpreters) {
ap_log_error(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, s,
"python_init: PyDict_New() failed! No more memory?");
exit(1);
}
{
/*
Workaround PEP 311 - Simplified Global Interpreter Lock Acquisition
for Extensions
BEGIN
*/
PyObject *p = 0;
interpreterdata * idata = (interpreterdata
*)malloc(sizeof(interpreterdata));
PyThreadState* currentThreadState = PyThreadState_Get();
PyInterpreterState *istate = currentThreadState->interp;
idata->istate = istate;
/* obcallback will be created on first use */
idata->obcallback = NULL;
p = PyCObject_FromVoidPtr((void ) idata, NULL); /*p->refcout = 1*/
PyDict_SetItemString(interpreters, MAIN_INTERPRETER, p);
/*p->refcout = 2*/
Py_DECREF(p); /*p->refcout = 1*/
/*
END
Workaround PEP 311 - Simplified Global Interpreter Lock Acquisition
for Extensions
*/
}
/* Release the thread state because we will never use
* the main interpreter, only sub interpreters created later. */
PyThreadState_Swap(NULL);
#ifdef WITH_THREAD
/* release the lock; now other threads can run */
PyEval_ReleaseLock();
#endif
}
return OK;
}
Another change I've made in the attached file is to Py_DECREF(p) in
get_interpreter, which will remove leaky reference to the PyCObject with the
interpreter data. This was not a real problem, but now I see fewer leaks in
BoundsChecker :-).
