David Fraser wrote: > Graham Dumpleton wrote: >> On 18/08/2006, at 7:48 PM, Richard Lewis wrote: >>> On Tuesday 15 August 2006 23:31, Graham Dumpleton wrote: >>>> Richard Lewis wrote .. >>>>> Do you mean that there will be no opportunity to have code run at >>>>> server >>>>> shutdown? >>>> Correct. Reason being that it doesn't actually work most of the time >>>> anyway. >>>> >>>> >>>> Because of how Apache is implemented, there is no reliable/safe way of >>>> implementing this feature. If one can't do it properly, it seems >>>> better not >>>> to attempt it at all. >>> Goodness! So, in my mod_python applications I often acquire database >>> handles >>> and store them in objects outside of the handler() function so that >>> they >>> persist between requests. (This is to avoid the expense of acquiring >>> a new >>> handle for every request). >> And you can still do that. >>> I then use a cleanup() function to release those >>> database handles. >>> >>> The implication of this is that I will no longer be able to release the >>> handles at server shutdown, yes? >> I think you miss what I have been saying. That it doesn't work now. >> If you >> were to put calls to apache.log_error() in your cleanup handler and >> shutdown >> Apache when it is handling a decent amount of traffic, you will probably >> find your cleanup functions aren't actually being called, or if they >> do, they >> might not exit and will hang at some point. What you are more likely >> to see >> in the Apache log is a series of SIGTERM signals being sent and then a >> SIGKILL signal which is forcibly killing off the process. >>> Can anyone suggest an alternative method of >>> doing this? Are there any Python tricks where you can execute code >>> when the >>> interpreter itself is about to stop? >> Python does have means of calling code on application shutdown, but >> because of how Apache uses signals to shutdown processes and how >> the Apache main loop works, with the main loop being managed >> by Apache and not Python they can't be used either. >>> Or could I have a Python script running >>> in another process which "looks in" to the mod_python process and >>> periodically cleanly releases database handles? >> No. >> >> In the greater scheme of things it shouldn't ultimately matter. This >> is because >> your database server is going to notice that the connections to it >> have been >> dropped and will cleanup the resources on its side anyway. It has to >> do this >> as there is nothing to say that Apache or any client process connecting >> to it will not simply crash without cleanly disconnecting. In other >> words, you >> will not get resource leaks. That things didn't get cleanup in the >> Apache child >> process doesn't matter as it has been killed anyway, with all its >> memory being >> released back to operating system. > So you are saying: > 1) There is a mechanism for cleaning up code > 2) This mechanism is not reliable > 3) Since databases have to assume clients are not reliable, they clean > up for them anyway > 4) Therefore we should not even try to clean up > > I'm with you on points 1, 2, and 3, but I think point 4 is taking it a > bit too far... > Surely there must be *some* value in trying to clean up behind > yourself, sometimes? Hi
I thought it would be good to take this across to python-dev. I've read through https://issues.apache.org/jira/browse/MODPYTHON-109?page=all and the discussion in http://www.modpython.org/pipermail/mod_python/2006-January/019865.html http://www.modpython.org/pipermail/mod_python/2006-January/019866.html and http://www.modpython.org/pipermail/mod_python/2006-January/019870.html again, and I'm just not sure about this. Basically, Apache seems to provide some sort of mechanism for child processes to clean themselves up, and for modules to clean up their resources in a particular child. The argument to remove the ability to clean up Python objects seems to be that: A) The finalize method was been called in an awkward place (from inside a signal handler) and other code may be running and have the GIL, so it may not be called at all, even in a graceful shutdown. B) A normal restart will just send a TERM signal, which doesn't give proper opportunity for cleanup C) If the graceful shutdown doesn't work or respond quickly, Apache will just kill the process anyway, so we may as will live with being killed (talk about mixed metaphors...) D) Since databases etc have to deal with the client process being killed, they generally will handle this I accept that problem A with the finalizing methods is a real problem, but wonder if there are alternate solutions that can be provided to allow cleanups to be attempted. I don't think that B or C is a good argument - in that case why would Apache be providing the hooks to clean up anyway? It feels like throwing in the towel... And D just seems impolite - if we can try and clean up we should. Of course, if we can't manage to call finalize methods even in a graceful shutdown none of this may be possible... Trying to find relevant info on this from the Apache docs and other module documentation: http://httpd.apache.org/docs/2.2/stopping.html#gracefulstop talks about advising children to exit after their current request. In this case it would seem the cleanup methods should get called at the end of the request processing, and thus shouldn't be in a signal handler (and there should be no other Python code executing...) http://www.apachetutor.org/dev/pools talks about using pools to allocate/deallocate resources other than memory - could we provide a way to register Python objects that need cleanup using this mechanism? Am I barking up the wrong tree or is this worth investigating further? David
