On Dec 12, 6:21 am, Rajesh Dhawan <[EMAIL PROTECTED]> wrote: > Hi again, > > > 3. I'm using prefork MPM on apache with maxclients set to 1000. > > Apart from Joe's excellent profiling suggestion, I would recommend > reducing maxclients to a much lower value (like say 100) and then > increasing it gradually. > > From your description of the sluggish response of your web server, it > looks like the machine may be swapping to disk. Each of your clients > fires up a Python VM taking up several tens of MB of RAM.
To clarify on this comment. Each of the 'clients' does not fire up a Python VM. The Python framework as a whole is initialised only once per Apache child process. This initialisation as a side effect will always create the main Python interpreter instance for the Apache child process. Unless you use PythonInterpreter directive to force site to always run in the main Python interpreter instance, then by default an additional Python sub interpreter instance will be created per virtual host. The per virtual host sub interpreter instance will be created the first time a request to mod_python for that virtual host is received. When any interpreter instance is created within an Apache child process, that interpreter instance will then persist for the life of that process and will be used in handling all requests which map to that interpreter instance. A Python interpreter instance when first created and before any web application is loaded should only add a few hundred kilobytes, although mod_python adds on top of that with its dispatch code, it is not tens of MBs though. Thus to say that 'Python VM taking up several tens of MB of RAM' isn't accurate. If you are seeing this sort of memory usage from just loading mod_python, then you have a Python installation which doesn't provide a shared library for Python and which also may include debug symbols. Now, when a Django site instance is loaded, typically this can add about 5MB per process. This value will increase over time based on your application and how it uses memory based on database queries and internal operations. Yes this can get up to tens of MBs, but that is from the Django application and not mod_python or Python itself. > Even if each > VM takes up just 20MB (it's probably more than that), a thousand > clients will eat up 20GB, forcing your machine to swap. This can be a problem if using prefork, so dropping max clients to a level more realistic for the amount of memory available is a good idea. If memory is still an issue, then validating that your particular Django application is thread safe would be worthwhile and switch to worker MPM for Apache instead. Although worker MPM uses multithreading, claims that the GIL would be a problem is not the big issue some would say it is. This is because Apache is a multiprocess web server so there is no restriction on making good use of multiple processes and cores. As far as memory overhead goes, mod_wsgi uses less base level memory than mod_python. The per request overhead of mod_wsgi is also less than mod_python, so if you really wanted to squeeze as much as possible out of what you got, mod_wsgi should be better. That said, realistically though the bottlenecks appear to be in the application or database rather than underlying web server adapter. Graham --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
