2009/3/27 [email protected] <[email protected]>: > > I recently tracked down an issue in one of our Django apps where we > were foolishly returning way too much data from the DB to the view. > In some cases, the python view code was allocating over 100 MB of > data. The overall symptoms experienced on the server seemed a bit > odd, hence my message. > > We're running the prefork mpm on Ubuntu 8.04, apache and mod wsgi > installed via apt. When our django view queried all this data, we saw > the apache child's memory usage spike as you might expect. What seems > odd though is the memory never seemed to be released. When that child > handled another request processed by the bad code, even more memory > would get allocated. This seems odd since the code in question is all > completely local to the view function. It's not doing anything crazy > like setting module level variables or anything else that would be > persisted after the view returned. > > The end result was that after a number of these requests were > processed, the entire server would go into panic. All other services > would be unresponsive, including sshd. The only way to regain access > was to power cycle the box. It seems to me that some piece in the > stack should play nicer with the OS when an out of memory situation > comes up. I'd much rather see apache (or modwsgi) segfault and still > have access to my server than the opposite. > > Is there something already built into apache and/or modwsgi to play > nicer with the OS that I'm missing? If not, should there be?
Short on time right now. Make sure you read: http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html Don't use embedded mode, especially with prefork, it is a recipe for disaster if Apache MPM settings not tuned properly. Use daemon mode as someone else already suggested. Setting maximum requests is good fail safe in application is leaking/growing memory. As to fixed memory limit, operating systems don't consistently implement setrlimit for memory and so can't really support it. You need to have an external application that uses OS specific way of find amount of memory in use and SIGTERM the processes when they grow too big. This is easier if using daemon mode as don't have to worry about implications of signalling an Apache child worker process. Finally, make sure that you don't have Django DEBUG flag set in settings file as this causes an accumulation of database information to be kept. Graham --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en -~----------~----~----~----~------~----~------~--~---
