On 20 November 2010 08:22, Michael Schurter <[email protected]> wrote: > StartServers 2 > MinSpareThreads 25 > MaxSpareThreads 75 > ThreadLimit 64 > ThreadsPerChild 50 > MaxClients 250 > MaxRequestsPerChild 0
A few guidelines. MaxClients should be a multiple of ThreadsPerChild. MinSpareThreads should be a multiple of ThreadsPerChild. MaxSpareThreads should be a multiple of ThreadsPerChild. Apache doesn't actually dynamically manage number of threads within a single process. That is, thread pool size will be fixed at ThreadsPerChild. MinSpareThreads and MaxSpareThreads is thus in practice controlling how many processes are created/destroyed in multiples of ThreadsPerChild. I don't remember what Apache does when those directives aren't a multiple of ThreadsPerChild so better off ensuring though match so have some predictability. In your configuration, only 2 Apache child processes will be started initially. Apache will spin up additional child processes to meet demand. The maximum it will create will be MaxClients/ThreadsPerChild. In your case, maximum number of child processes which will be created is 5. By default (also applies to daemon mode), your WSGI application is only loaded into a process the first time a request arrives. Further, Django default WSGI deployment recipe means that parts of your Django application are also only loaded on first request. Thus first request against each process will be slow. Once loaded though, stay persistent for the life of the process. In daemon mode, unless you have configured it otherwise, the process is persistent and so subsequent requests always hitting memory resident application. In embedded mode, you can have requests some time later be slow if Apache decides to start up a new child process. This is made worse because Apache can decide to shutdown child processes depending on request throughput and the MinSpareThreads and MaxSpareThreads directive settings. Thus, if you are going to use embedded mode and dedicating Apache to this web application, you are better off configuring it to start maximum number of process at the outset rather than allowing them to ramp it up later and incur subsequent slow down. Further, configure MinSpareThreads and MaxSpareThreads to avoid Apache trying to shutdown processes when it thinks they are no longer needed. In respect of number of threads per process, more is not necessarily better due to how GIL locking works as well as poor performance of Python 2.X threading for a process when you have multiple CPUs. Better therefore to use less threads per process and have more processes. A balance obviously needs to be found based on how much memory you have available. For daemon mode, the default is a single process with 15 threads. If using defaults for daemon processes, use same number of threads for comparison. Do that and you will get something closer to the perpetually persistent processes of daemon mode and avoid possible slow downs due to Apache recycling child processes. So, try: StartServers 5 MinSpareThreads 0 MaxSpareThreads 0 ThreadsPerChild 15 MaxClients 75 MaxRequestsPerChild 0 I am not actually sure what MinSpareThreads and MaxSpareThreads should be to stop it from doing dynamic process management, but try that for a start. This would be equivalent to: WSGIDaemonProcess xxx processes=5 WSGIProcessGroup xxx ie., WSGIDaemonProcess xxx processes=5 threads=15 WSGIProcessGroup xxx You mentioned that you had tried to force preloading of web application by doing burst of requests initially. But that small number of requests is no guarantee that Apache would have spun up processes to maximum possible and also doesn't stop Apache from killing processes shortly after before test is run. You might therefore be incurring startup cost later on during test still. As to preloading Django, if you use the Django development server it will preload Django and your installed applications. This doesn't occur with Django WSGI adapter. You can force Django to be preloaded by following recipe for WSGI script described at end of: http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html This can be used for either embedded mode or daemon mode. It ensures that code is preloaded when process starts and not when first actual request for process is being handled. If maximum number of processes is started at outset, as it will by default with daemon mode, and in embedded mode if StartServers equates to MaxClients/ThreadsPerChild, then everything should be preloaded except for stuff which is loaded and cached in context of actual requests. This preloading and using maximum processes from outset isn't just something to make benchmarking easier, but for full on production system where you have adequate memory is good practice. In the main it restricts the intense cpu activity of preloading to when you are starting server and you expect a load spike and affect on traffic. Anyway, hope that helps you to understand. Might be useful if you post what daemon mode process configuration was so can compare to MPM settings you were using. When using daemon mode, you obviously still have to configure MPM settings appropriate to match daemon mode, but with additional capacity to handle static media in main Apache child processes as well as acting as queueing backlog where request load exceeds what daemon process capable of based on its settings. Also need to have extra capacity in MPM settings to handle keep alive socket connections. The latter not being relevant though if you are using nginx front end to Apache. Graham > I tried raising and lowering ThreadLimit and ThreadsPerChild by about > 50%, but my little bit of experimentation only seemed to yield worse > results. > > On Nov 18, 10:43 pm, Graham Dumpleton <[email protected]> > wrote: >> What are the actual MPM settings? >> >> Graham >> >> On Friday, November 19, 2010, Michael Schurter >> >> <[email protected]> wrote: >> > While attempting to tune my Apache and modwsgi settings I could not >> > get embedded mode to out-perform daemon mode. >> >> > I was testing against a Django/Piston RESTful API which ends up >> > hitting Memcached, Cassandra, and Beanstalkd on an Amazon EC2 Medium >> > instance (2 virtual CPUs). Apache is Ubuntu 10.04's (worker mpm) and >> > modwsgi 3.3. >> >> > Daemon mode peaked over 550 requests per second with 2 processes and >> > 16-18 threads. >> >> > Embedded mode peaked just under 475r/s with Apache's default settings. >> > Adjusting thread/client limits up and down only seemed to decrease >> > throughput. >> >> > Tests were done over 10,000 or 20,000 iterations with between 10-100 >> > initial iterations to make sure everything was initialized. >> >> > Is this to be expected? >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "modwsgi" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group >> > athttp://groups.google.com/group/modwsgi?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
