Okay, found the source of the memory leak. The problem goes right back
to 3.1.4 which also has the problem when tested.
The problem code is in python_handler() in 'src/mod_python.c'.
Specifically
the code does:
if (!hle) {
/* create a handler list object from dynamically registered
handlers */
request_obj->hlo = (hlistobject *)MpHList_FromHLEntry
(dynhle, 1);
}
else {
/* create a handler list object */
request_obj->hlo = (hlistobject *)MpHList_FromHLEntry(hle, 1);
/* add dynamically registered handlers, if any */
if (dynhle) {
MpHList_Append(request_obj->hlo, dynhle);
}
}
Problem is that request_obj->hlo can already be set by a prior phase's
handler and by simply assigning to request_obj->hlo you get a memory
leak as it is a Python object and it isn't being decref'd.
Thus, before this 'if' statement, it would appear that the following
should
be inserted.
if (request_obj->hlo)
Py_DECREF(request_obj->hlo);
or:
Py_XDECREF(request_obj->hlo);
Still need to do some more checking, but adding that seems to get rid of
the problem.
Now what do we do about 3.2.10? Given that this thing leaks really badly
when triggered shows that no one must be using multiple handler phases
at the same time, so may be safe to still release 3.2.10 and we fix
it in next
backport release and 3.3.
Comments.
Graham
On 31/07/2006, at 7:24 PM, Graham Dumpleton wrote:
The good news is that this isn't a problem introduced with
mod_python 3.2.10
so doesn't necessarily have to stop that being released.
The bad news though is that is is also in mod_python 3.2.8, so it
has been
there for some time.
The question now is whether anyone else sees it on other platforms
or whether
it is something specific to my machine.
Graham
On 31/07/2006, at 4:53 PM, Graham Dumpleton wrote:
I get it on Apache 2.0.59 as well. :-(
I will thus be interested to see what others get, as appears to be
an existing
mod_python issue.
BTW, this is with worker MPM.
Graham
Graham Dumpleton wrote ..
I am using Apache 2.2.2 and when using mod_python in a certain
way, I am
seeing
significant ongoing increases in memory use by Apache child
processes.
Initially I thought I had stuffed up recent changes in mod_python
3.3 out
of
subversion trunk that I had been making, but I went back to
mod_python
3.2.10
and am seeing the same problem.
Can some one please run the following test and use "top" or some
other
means of
monitoring memory use to see if you can duplicate the problem.
The test
is really
quite simple actually.
# .htaccess
PythonFixupHandler handlers
AddHandler mod_python .py
PythonHandler handlers
# handlers.py
from mod_python import apache
def handler(req):
req.content_type = 'text/plain'
req.write('handler')
return apache.OK
def fixuphandler(req):
return apache.OK
The command I have been using to test is:
ab -c 1 -n 5000 http://localhost:8082/~grahamd/MODPYTHON-155/
hello.py
What is strange is that if I have only the fixup handler or
response handler
invoked and not both, then memory leak isn't present. It is only
when both
are
being invoked that it leaks memory.
This occurs for me on Mac OS X 10.4, Python 2.3.5 and Apache
2.2.2, with
either mod_python 3.2.10 or mod_python 3.3. I haven't yet tested
with older
Apache 2.0.58 as yet as don't have it on my box at present, but
will download
it and see if that makes a difference.
I guess what I am trying to working out is if this is an issue
with Apache
2.2.2
or whether it is mod_python and problem has been there for a
while. Thus
if
people with both Apache 2.0.X and Apache 2.2.X could test it it
would be
great.
Any help appreciated.
Thanks.
Graham
