2009/10/24 msoulier <[email protected]>: > > So I'm looking at moving from mod_python to mod_wsgi because > mod_python leaks memory in apache on graceful restarts, and we have an > automated linux box without a human present most of the time, so this > is unacceptable. > > Unfortunately, mod_wsgi seems to leak too! > > [r...@miketug2 teleworker]# grep -i python /etc/httpd/admin-conf/ > httpd.conf > [r...@miketug2 teleworker]# sv term /service/httpd-admin/ > [r...@miketug2 teleworker]# ps -eo pid,user,fname,vsize --sort -vsize > | grep httpd-ad > 25737 admin httpd-ad 15284 > 25736 root httpd-ad 11960 > [r...@miketug2 teleworker]# ps -eo pid,user,fname,vsize --sort -vsize > | grep httpd-ad > 25737 admin httpd-ad 15284 > 25736 root httpd-ad 11960 > [r...@miketug2 teleworker]# sv 1 /service/httpd-admin > [r...@miketug2 teleworker]# ps -eo pid,user,fname,vsize --sort -vsize > | grep httpd-ad > 25746 admin httpd-ad 16140 > 25736 root httpd-ad 12816 > [r...@miketug2 teleworker]# sv 1 /service/httpd- > admin > [r...@miketug2 teleworker]# ps -eo pid,user,fname,vsize --sort -vsize > | grep httpd-ad > 25751 admin httpd-ad 16976 > 25736 root httpd-ad 13652 > [r...@miketug2 teleworker]# sv 1 /service/httpd- > admin > [r...@miketug2 teleworker]# ps -eo pid,user,fname,vsize --sort -vsize > | grep httpd-ad > 25755 admin httpd-ad 17812 > 25736 root httpd-ad 14488 > > > If I repeat this test with mod_wsgi disabled, the process size stays > constant. > > [r...@miketug2 teleworker]# rpm -q httpd > httpd-2.2.3-31.el5.centos > [r...@miketug2 teleworker]# rpm -q python > python-2.4.3-27.el5 > [r...@miketug2 teleworker]# rpm -q mod_wsgi > mod_wsgi-2.1-2.el5 > > Is this a known problem?
First off, if using mod_wsgi and you still have mod_python you will still suffer the mod_python problem of it leaking memory as it takes precedence as far as managing initialisation of Python when both modules are loaded. Second, you should use at least mod_wsgi 2.4 as it addressed an issue where when mod_wsgi alone loaded it wasn't destroying Python on an Apache restart. This was the same issue that still afflicts mod_python. See details in: http://code.google.com/p/modwsgi/wiki/ChangesInVersion0204 Thirdly, even using that version and even with mod_wsgi correctly destroying Python interpreter on Apache restart, the Python interpreter for some versions of Python still leaks a small amount of memory. Your problems will not be as bad as when using mod_python or older versions of mod_wsgi, but still may be noticeable over time if doing a lot of restarts. If the reason for your restarts is because you want Python code changes to be picked up, then don't use embedded mode for your application. Instead, use daemon mode and touch WSGI script file or install code monitor to perform automatic restarts in code changes. When these restarts occur, only the affected daemon process are restarted and not a full Apache restart, so don't have issue with Apache parent process growing in size. This issue with Python interpreter memory leaks has gotten much worse in Python 3.0 where they now by design don't destroy memory with belief they can reuse it if Python reinitialised in same process. This will not work with Apache as Python library is unloaded by Apache on a restart and references to that memory lost and so cannot be used. Thus, for Python 3.0 and later, lazy initialisation is done. That is, not done in Apache parent process and instead delayed to after fork of child process. This behaviour is implemented in mod_wsgi 3.0 for Python 3.0. Since the last 3.0 release candidate, has also been decided to make lazy initialisation default for Python 2.X as well to avoid the whole problem with Python interpreter leaking memory. If using mod_wsgi trunk from subversion you will thus get that behaviour. If instead using 3.0c5 (release candidate), then you would need to set the WSGILazyInitialization directive to On instead. There is more details about the changes in: http://code.google.com/p/modwsgi/wiki/ChangesInVersion0300 Short answer therefore is to stop using that old version of mod_wsgi and use mod_wsgi 2.6 instead, or 2.5 if using Debian binaries from its repository. Long term, mod_wsgi 3.0 eliminates completely the problem. Graham --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en -~----------~----~----~----~------~----~------~--~---
