[jira] Commented: (MODPYTHON-217) Python 2.3 and simplified GIL state API still causes problems.
[
https://issues.apache.org/jira/browse/MODPYTHON-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523668
]
Deron Meranda commented on MODPYTHON-217:
-
Another real-world case where this comes up is with the Xapian
search engine (http://xapian.org/). It also is SWIG generated.
See their bug #185,
http://www.xapian.org/cgi-bin/bugzilla/show_bug.cgi?id=185
Python 2.3 and simplified GIL state API still causes problems.
--
Key: MODPYTHON-217
URL: https://issues.apache.org/jira/browse/MODPYTHON-217
Project: mod_python
Issue Type: Bug
Components: core
Affects Versions: 3.3.x, 3.2.10
Reporter: Graham Dumpleton
There are still problems in mod_python when third party extension modules are
used which make use of the Python simplified GIL state API. Specifically, if
C code makes calls to PyGILState_Ensure() when it already holds the Python
GIL and has an active thread state, but the active thread was one which was
originally created outside of Python, such as by Apache with mod_python, then
the call to PyGILState_Ensure() results in a dead lock. This is occurring
because the PyGILState_Ensure() function finds it does not know anything
about the current thread ID and therefore assumes it needs to create a new
thread state for it and make that the active thread state. In doing that
though it tries to acquire the GIL when the thread already holds it,
resulting in the deadlock.
At this stage it is believed this only occurs with Python 2.3 and shouldn't
be a problem with later versions of Python. This is the case as later
versions of Python were modified so that PyThreadState_New() will register
the thread with the underpinnings of the PyGILState_???() mechanism and
because it already knows about it, it will not attempt to create a new thread
state object and so the deadlock will not occur.
When C code is hand crafted it is unlikely that anyone would call
PyGILState_Ensure() when they already know they hold the GIL, ie., when
calling out of Python code, but SWIG when used with the -threads option does
exactly this. For example:
SWIGINTERN PyObject *_wrap_ap_get_server_version(PyObject
*SWIGUNUSEDPARM(self), PyObject *args) { PyObject *resultobj = 0;
char *result = 0 ;
SWIG_PYTHON_THREAD_BEGIN_BLOCK;
if (!PyArg_ParseTuple(args,(char *):ap_get_server_version)) SWIG_fail;
{
SWIG_PYTHON_THREAD_BEGIN_ALLOW;
result = (char *)ap_get_server_version();
SWIG_PYTHON_THREAD_END_ALLOW;
}
resultobj = SWIG_FromCharPtr((const char *)result);
SWIG_PYTHON_THREAD_END_BLOCK;
return resultobj;
fail:
SWIG_PYTHON_THREAD_END_BLOCK;
return NULL;
}
where SWIG_PYTHON_THREAD_BEGIN_BLOCK eventually expands to a call to
PyGILState_Ensure().
The only solution to this problem would be for mod_python to behave
differently when it is acquiring a thread state against the main Python
interpreter, ie. 'main_interpreter'. Specifically, rather than use
PyThreadState_New() etc, it should call PyGILState_Ensure() instead. When
needing to release the main Python interpreter it should call
PyGILState_Release(). It should continue to work as before for all other
interpreters and anyone with GILState code would need to ensure they are
using 'main_interpreter'.
Unfortunately at the moment making this change is not completely straight
forward as when an interpreter is being released it isn't known that it is
the main Python interpreter. Thus some knowledge that it is the main
interpreter would need to be carried into the call. The only other option is
that the list of all active interpreters is traversed and the last one in the
list is assumed to be the main interpreter. Then if the interpreter being
released matches that then act differently.
Note that at present the acquire/release methods are exported as optional
functions so that other Apache modules use them. It is unlikely that anyone
is making use of them, so changing the in/out parameters of the functions
would possibly be acceptable.
Also note that maintaining support for threaded and non threaded Python and
now this GIL state variation is going to make the code even more messy. As
such it might be worthwhile considering dropping support for non threaded
Python. These days Python defaults to being threaded anyway, and highly
unlikely that someone would want to use non thread Python with mod_python
anyway. When ever any development of mod_python is done, no testing is ever
done with a non threaded Python anyway and so it is possibly questionable if
that support still works.
FWIW, this issue came up as a result of queries expressed in:
http://www.modpython.org/pipermail/mod_python/2007-April/023445.html
The GIL state issue has
[jira] Commented: (MODPYTHON-78) No support for Apache 2.2 yet
[ http://issues.apache.org/jira/browse/MODPYTHON-78?page=comments#action_12364671 ] Deron Meranda commented on MODPYTHON-78: One small incompatibility is the call to the APR function apr_sockaddr_port_get(), as introduced in the fix for bug MODPYTHON-64. That function was deprecated in APR 1.0 (which maked it into httpd 2.2). The changelog for APR says: *) The following deprecated interfaces have been removed: apr_sockaddr_port_get- (access directly) After reviewing the APR changes, it appears that the correct fix to mod_python is quite simple. The line in connobject.c: apr_sockaddr_port_get(port, addr); should be replaced with port = addr-port; This is still portable and has no endianness or word-size issues. (as it would if it tried accessing the addr-sa sub-struct directly). No support for Apache 2.2 yet - Key: MODPYTHON-78 URL: http://issues.apache.org/jira/browse/MODPYTHON-78 Project: mod_python Type: Bug Components: core Versions: 3.2 Reporter: Nicolas Lehuen Fix For: 3.3 See http://article.gmane.org/gmane.comp.apache.mod-python.devel/1425 for some remarks by Jorey Bump, raised during the 3.2.1-BETA tests. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-94) Calling APR optional functions provided by mod_ssl
[ http://issues.apache.org/jira/browse/MODPYTHON-94?page=comments#action_12362005 ] Deron Meranda commented on MODPYTHON-94: Keeping the same mod_ssl name for this function is fine with me. Calling APR optional functions provided by mod_ssl -- Key: MODPYTHON-94 URL: http://issues.apache.org/jira/browse/MODPYTHON-94 Project: mod_python Type: New Feature Components: core Environment: Apache 2 Reporter: Deron Meranda Fix For: 3.3 Attachments: modpython4.tex.patch, requestobject.c.patch mod_python is not able to invoke APR Optional Functions. There are some cases however where this could be of great benifit. For example, consider writing an authentication or authorization handler which needs to determine SSL properties (even if to just answer the simple question: is the connection SSL encrypted). The normal way of looking in the subprocess_env for SSL_* variables does not work in those early handler phases because those variables are not set until the fixup phase. The mod_ssl module though does provide both a ssl_is_https() and ssl_var_lookup() optional functions which can be used in earlier phases. For example look at how mod_rewrite calls those; using the APR_DECLARE_OPTIONAL_FN and APR_RETRIEVE_OPTIONAL_FN macros. I can see how it might be very hard to support optional functions in general because of the C type linkage issue, but perhaps a select few could be coded directly into mod_python. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MODPYTHON-94) Calling APR optional functions provided by mod_ssl
[ http://issues.apache.org/jira/browse/MODPYTHON-94?page=all ] Deron Meranda updated MODPYTHON-94: --- Attachment: modpython4.tex.patch This is a documentation patch which goes with the previously attached code patch. Made against 3.2.5b. Calling APR optional functions provided by mod_ssl -- Key: MODPYTHON-94 URL: http://issues.apache.org/jira/browse/MODPYTHON-94 Project: mod_python Type: New Feature Components: core Versions: 3.2 Environment: Apache 2 Reporter: Deron Meranda Attachments: modpython4.tex.patch, requestobject.c.patch mod_python is not able to invoke APR Optional Functions. There are some cases however where this could be of great benifit. For example, consider writing an authentication or authorization handler which needs to determine SSL properties (even if to just answer the simple question: is the connection SSL encrypted). The normal way of looking in the subprocess_env for SSL_* variables does not work in those early handler phases because those variables are not set until the fixup phase. The mod_ssl module though does provide both a ssl_is_https() and ssl_var_lookup() optional functions which can be used in earlier phases. For example look at how mod_rewrite calls those; using the APR_DECLARE_OPTIONAL_FN and APR_RETRIEVE_OPTIONAL_FN macros. I can see how it might be very hard to support optional functions in general because of the C type linkage issue, but perhaps a select few could be coded directly into mod_python. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
