Hmmmm, I think all this is showing is how hard it is to get reliable results from one test run to the next.
You can see for example how the max processing time and transfer rate jump around so much. FWIW, theoretically, the AddHandler method should be slowest. The next quickest should be WSGIScriptAlias. The fastest should be WSGIQuickHandlerScript. Hang on. No, AddHandler inside of .htaccess file would definitely be slower. If AddHandler in main Apache configuration then you are avoiding .htaccess parsing and so possibly no real difference to WSGIScriptAlias or if any difference only ever so slightly slower. WSGIQuickHandlerScript should be quickest because it is using a special hook in Apache that gets to run and is intended to circumvent the normal request processing phases. You can see a depiction of the Apache request phases in: http://www.kuzbass.ru/docs/apache_c_mod_perl/24.htm#BIN68 The ap_quick_handler_hook() comes in even before the post read request phase. The thought in trying this was that the attitude of many Python web developers is that everything should be done in the WSGI application and as such they don't care about what Apache does or can do for them. As such, one could hook in so early that all the extra stuff that Apache does is skipped. By doing this though you are even skipping normal Apache URL matching and so this can only be used in a VirtualHost where absolutely everything is being handled by the WSGI application. That is, can't even serve static files out of the same VirtualHost and would need to use a different VirtualHost for that. Theoretically though, skipping all those extra phases should be that the overhead of Apache request processing is less. In the context of a hello world program this may be noticeable, but still insignificant in the context of a fat Python web application where one is talking about 100s of request/sec vs 1000s for a hello world application. So, there may be some use for this with applications which are very very streamlined and optimised that really need to push for maximum performance, but in general possibly not worth it. Anyway, still have some issues with this to sort out, as possible that by hooking in this early, even the cleanup of REQUEST_URI hasn't been done and so things like repeating slashes and encoded slashes still occur in PATH_INFO. At the same time, some people actually have wanted that. Graham On 24 March 2010 18:48, Clodoaldo Neto <[email protected]> wrote: > 2010/3/24 Clodoaldo Neto <[email protected]>: >> 2010/3/23 Graham Dumpleton <[email protected]>: >>> For whatever reason I often have lots of problems with doing benchmark >>> testing on MacOS X. There are some known issues with MacOS X and >>> Apache, so possibly I am always hitting that when trying to stress >>> test stuff. >>> >>> As such, wandering if any eager people out there who would like to run >>> some tests for me on a non MacOS X platform which may give more >>> repeatable results. UNIX systems preferred as Windows is usually worse >>> than MacOS X for predictability. >>> >>> All I need for the test is a comparative set of 'ab' output for a few >>> different mod_wsgi configurations. The intent here is to compare the >>> different configurations so consistency of results is more important >>> than outright speed. >>> >>> Note that you MUST being using latest mod_wsgi from subversion trunk >>> to do this test. >>> >>> The hello world program should be that as detailed on: >>> >>> http://code.google.com/p/modwsgi/wiki/QuickConfigurationGuide >>> >>> Call the file 'hello.wsgi' and stick it in '/some/directory'. >>> >>> The first configuration is: >>> >>> Alias / /some/directory/ >>> >>> <Directory /some/directory> >>> AddHandler wsgi-script .wsgi >>> Order Deny,Allow >>> Allow from All >>> </Directory> >>> >>> The second configuration is: >>> >>> WSGIScriptAlias / /some/directory/hello.wsgi >>> >>> <Directory /some/directory> >>> Order Deny,Allow >>> Allow from All >>> </Directory> >>> >>> The third configuration is: >>> >>> WSGIQuickHandlerScript /some/directory/hello.wsgi >>> >>> In all cases use URL on the site as '/hello.wsgi'. >>> >>> Am happy with a test such as: >>> >>> ab -n 10000 http://localhost/hello.wsgi >>> >>> The more requests the merrier. Anything below 5000 isn't particularly >>> useful as too variable with respect to Apache start up costs. >>> >>> If you want to try different levels of concurrency or try different >>> tools, that is your call. >>> >>> Average results from some number of runs if that makes you feel >>> happier that getting good results. >>> >>> If using 'ab', always keep an eye on the section: >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 0.1 0 6 >>> Processing: 0 0 0.2 0 7 >>> Waiting: 0 0 0.2 0 7 >>> Total: 0 0 0.2 0 7 >>> >>> From one run to the next, you want to get similar results for that >>> 'max' column and discard anything that shows some out of the ordinary >>> happened. For example, crap like I get on MacOS X such as: >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 2 126.6 0 6935 >>> Processing: 0 0 0.3 0 10 >>> Waiting: 0 0 0.3 0 10 >>> Total: 0 3 126.6 0 6936 >>> >>> Not sure if the DNS here simply sucks or what the issue is. :-( >>> >>> Thanks in advance. >>> >>> Will explain what this is about when I see some results and know if it >>> makes much difference or not. >>> >>> Graham >> >> Alias / /home/www/default/ >> <Directory /home/www/default> >> Options +ExecCGI >> AddHandler wsgi-script .wsgi >> Order Deny,Allow >> Allow from All >> </Directory> >> >> $ ab -n 100000 http://10.1.1.110/hello.wsgi >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking 10.1.1.110 (be patient) >> Completed 10000 requests >> Completed 20000 requests >> Completed 30000 requests >> Completed 40000 requests >> Completed 50000 requests >> Completed 60000 requests >> Completed 70000 requests >> Completed 80000 requests >> Completed 90000 requests >> Completed 100000 requests >> Finished 100000 requests >> >> >> Server Software: Apache/2.2.14 >> Server Hostname: 10.1.1.110 >> Server Port: 80 >> >> Document Path: /hello.wsgi >> Document Length: 12 bytes >> >> Concurrency Level: 1 >> Time taken for tests: 50.235 seconds >> Complete requests: 100000 >> Failed requests: 0 >> Write errors: 0 >> Total transferred: 18000000 bytes >> HTML transferred: 1200000 bytes >> Requests per second: 1990.66 [#/sec] (mean) >> Time per request: 0.502 [ms] (mean) >> Time per request: 0.502 [ms] (mean, across all concurrent requests) >> Transfer rate: 349.92 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.0 0 6 >> Processing: 0 0 0.4 0 38 >> Waiting: 0 0 0.3 0 25 >> Total: 0 0 0.4 0 38 >> >> Percentage of the requests served within a certain time (ms) >> 50% 0 >> 66% 0 >> 75% 1 >> 80% 1 >> 90% 1 >> 95% 1 >> 98% 1 >> 99% 1 >> 100% 38 (longest request) >> ------------------------------------------------- >> >> WSGIScriptAlias / /home/www/default/ >> <Directory /home/www/default> >> Options +ExecCGI >> Order Deny,Allow >> Allow from All >> </Directory> >> >> $ ab -n 100000 http://10.1.1.110/hello.wsgi >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking 10.1.1.110 (be patient) >> Completed 10000 requests >> Completed 20000 requests >> Completed 30000 requests >> Completed 40000 requests >> Completed 50000 requests >> Completed 60000 requests >> Completed 70000 requests >> Completed 80000 requests >> Completed 90000 requests >> Completed 100000 requests >> Finished 100000 requests >> >> >> Server Software: Apache/2.2.14 >> Server Hostname: 10.1.1.110 >> Server Port: 80 >> >> Document Path: /hello.wsgi >> Document Length: 12 bytes >> >> Concurrency Level: 1 >> Time taken for tests: 50.152 seconds >> Complete requests: 100000 >> Failed requests: 0 >> Write errors: 0 >> Total transferred: 18000000 bytes >> HTML transferred: 1200000 bytes >> Requests per second: 1993.96 [#/sec] (mean) >> Time per request: 0.502 [ms] (mean) >> Time per request: 0.502 [ms] (mean, across all concurrent requests) >> Transfer rate: 350.50 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.0 0 7 >> Processing: 0 0 2.1 0 645 >> Waiting: 0 0 2.1 0 645 >> Total: 0 0 2.1 0 645 >> >> Percentage of the requests served within a certain time (ms) >> 50% 0 >> 66% 0 >> 75% 0 >> 80% 1 >> 90% 1 >> 95% 1 >> 98% 1 >> 99% 1 >> 100% 645 (longest request) >> --------------------------------------------------------- >> >> WSGIQuickHandlerScript /home/www/default/ >> <Directory /home/www/default> >> Options +ExecCGI >> Order Deny,Allow >> Allow from All >> </Directory> >> >> $ ab -n 100000 http://10.1.1.110/hello.wsgi >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking 10.1.1.110 (be patient) >> Completed 10000 requests >> Completed 20000 requests >> Completed 30000 requests >> Completed 40000 requests >> Completed 50000 requests >> Completed 60000 requests >> Completed 70000 requests >> Completed 80000 requests >> Completed 90000 requests >> Completed 100000 requests >> Finished 100000 requests >> >> >> Server Software: Apache/2.2.14 >> Server Hostname: 10.1.1.110 >> Server Port: 80 >> >> Document Path: /hello.wsgi >> Document Length: 284 bytes >> >> Concurrency Level: 1 >> Time taken for tests: 36.570 seconds >> Complete requests: 100000 >> Failed requests: 0 >> Write errors: 0 >> Non-2xx responses: 100000 >> Total transferred: 46400000 bytes >> HTML transferred: 28400000 bytes >> Requests per second: 2734.49 [#/sec] (mean) >> Time per request: 0.366 [ms] (mean) >> Time per request: 0.366 [ms] (mean, across all concurrent requests) >> Transfer rate: 1239.07 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.0 0 4 >> Processing: 0 0 2.8 0 882 >> Waiting: 0 0 2.8 0 882 >> Total: 0 0 2.8 0 882 >> >> Percentage of the requests served within a certain time (ms) >> 50% 0 >> 66% 0 >> 75% 0 >> 80% 0 >> 90% 0 >> 95% 0 >> 98% 1 >> 99% 1 >> 100% 882 (longest request) >> >> Regards, Clodoaldo >> > > This second and third were wrongly configured. Forgot the script name. > > WSGIScriptAlias / /home/www/default/hello.wsgi > <Directory /home/www/default> > Options +ExecCGI > Order Deny,Allow > Allow from All > </Directory> > > $ ab -n 100000 http://10.1.1.110/hello.wsgi > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking 10.1.1.110 (be patient) > Completed 10000 requests > Completed 20000 requests > Completed 30000 requests > Completed 40000 requests > Completed 50000 requests > Completed 60000 requests > Completed 70000 requests > Completed 80000 requests > Completed 90000 requests > Completed 100000 requests > Finished 100000 requests > > > Server Software: Apache/2.2.14 > Server Hostname: 10.1.1.110 > Server Port: 80 > > Document Path: /hello.wsgi > Document Length: 12 bytes > > Concurrency Level: 1 > Time taken for tests: 59.074 seconds > Complete requests: 100000 > Failed requests: 0 > Write errors: 0 > Total transferred: 18000000 bytes > HTML transferred: 1200000 bytes > Requests per second: 1692.78 [#/sec] (mean) > Time per request: 0.591 [ms] (mean) > Time per request: 0.591 [ms] (mean, across all concurrent requests) > Transfer rate: 297.56 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.0 0 2 > Processing: 0 1 2.9 0 902 > Waiting: 0 0 2.9 0 901 > Total: 0 1 2.9 0 902 > > Percentage of the requests served within a certain time (ms) > 50% 0 > 66% 1 > 75% 1 > 80% 1 > 90% 1 > 95% 1 > 98% 1 > 99% 1 > 100% 902 (longest request) > --------------------------------------------------------------------- > > WSGIQuickHandlerScript /home/www/default/hello.wsgi > <Directory /home/www/default> > Options +ExecCGI > Order Deny,Allow > Allow from All > </Directory> > > $ ab -n 100000 http://10.1.1.110/hello.wsgi > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking 10.1.1.110 (be patient) > Completed 10000 requests > Completed 20000 requests > Completed 30000 requests > Completed 40000 requests > Completed 50000 requests > Completed 60000 requests > Completed 70000 requests > Completed 80000 requests > Completed 90000 requests > Completed 100000 requests > Finished 100000 requests > > > Server Software: Apache/2.2.14 > Server Hostname: 10.1.1.110 > Server Port: 80 > > Document Path: /hello.wsgi > Document Length: 12 bytes > > Concurrency Level: 1 > Time taken for tests: 46.400 seconds > Complete requests: 100000 > Failed requests: 0 > Write errors: 0 > Total transferred: 18000000 bytes > HTML transferred: 1200000 bytes > Requests per second: 2155.16 [#/sec] (mean) > Time per request: 0.464 [ms] (mean) > Time per request: 0.464 [ms] (mean, across all concurrent requests) > Transfer rate: 378.84 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.0 0 4 > Processing: 0 0 0.4 0 72 > Waiting: 0 0 0.4 0 71 > Total: 0 0 0.4 0 72 > > Percentage of the requests served within a certain time (ms) > 50% 0 > 66% 0 > 75% 0 > 80% 0 > 90% 1 > 95% 1 > 98% 1 > 99% 1 > 100% 72 (longest request) > > Sorry for the noise, Clodoaldo > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
