On Nov 26, 7:00 am, Jose C <[EMAIL PROTECTED]> wrote: > > In short, if you are running Django in worker MPM for Apache, you can > > be subject to sudden memory increases when you get concurrent requests > > against resources which chew up a lot of transient memory. Receiving a > > lot of concurrent POST requests with large content data can be one > > trigger if the framework reads such POST content all into memory at > > the same time. > <snip> > > As long as people want to run big Python web applications in memory > > constrained web hosting configurations, where Apache worker MPM with a > > small number of processes is the only viable solution for Apache, in > > contrast to prefork MPM which isn't, this is going to continue to be > > an issue. > > Graham > > Having recently come across Graham's comments on the Apache worker vs > prefork MPM on another framework group, and being a webfaction client, > I checked with webfaction which version they use. They use the > prefork version, so I'd suggest Kenneth look for other possible > reasons for his memory spikes in his app.
Hmmm, are you sure they always use prefork? I'll admit I don't have first had experience with WebFaction, but given that they have a limit on the number of persistent processes you can have, this would be a severe restriction as it would limit the number of concurrent requests your overall application could handle quite drastically, as any spike in requests would require extra Apache child processes to be created. Thus, if you have a steady streams of requests, you would be stuck on your process limit quite quickly and so limit your throughput. I could understand them using prefork for the static file hosting, but for users own applications proxied behind their front end, using prefork doesn't make a great deal of sense to me as far as trying to use memory resources as efficiently as possible. Anyone else able to confirm how they setup the Apache instance which a users application runs in, not the main server used for static files? Irrespective of whether prefork is used instead of worker, in both cases there are a range of other issues that can also cause bloating in memory. With WSGI frameworks which use yield in some way, one such issue is where a user generator doesn't implement close() method for reclaiming resources immediately. As a consequence the reclaiming of resources can be deferred until the next GC cycle to try and break reference cycles between objects. Such reference count cycles can also occur in other situations as well, so not trying to clean up and aid garbage collection, by breaking cycles up front, can result in an accumulation of unreclaimed objects until the next time that GC runs. If other requests come in before GC kicks in, then total memory in use can end up growing more than one might expect. Graham --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
