Bill
[EMAIL PROTECTED] wrote:
I'm interested in doing some scalability testing with worker on Linux to see what the O(1) scheduler and new pthread library buys us, and what happens with different values for ThreadsPerChild. I decided to use a simple handler that just nanosleep()s for a variable amount of time controlled by the client. I ran oprofile and saw that the kernel's directory lookup function was the top CPU eater. An strace revealed that we were doing a directory walk, resulting in a stat() and 7 lstat()s per request.
My config to enable the handler is
<Location /silly> SetHandler s2-handler </Location>
...which is the same way we enable mod_status and mod_info. The key thing here is that the URIs to access a Location enabled handler do not map to the filesystem, so the directory walk is a waste of cycles. So what can we do about it? Maybe detect that the current URI matches such a block, and skip the filesystem mapping if so. That's exactly what this patch does.
Throughput measured by ab increases from 3053 requests/sec to 3984 with this patch, slightly more than 30% (dual 400MHz PII, both pegged, 256M RAM, RH 7.3). This should make it easier to see how the OS changes etc affect our scalability.
I really don't grok how the cache works in ap_location_walk so I wouldn't be shocked if I haven't implemented this optimally. But it seems to do the Right Thing with the nanosleep handler, mod_status, and static files. I also confirmed that we do get a full directory walk if I type in a garbage URL.
Comments? Greg
------------------------------------------------------------------------
Index: include/http_core.h
===================================================================
RCS file: /home/cvs/httpd-2.0/include/http_core.h,v
retrieving revision 1.77
diff -u -d -b -r1.77 http_core.h
--- include/http_core.h 1 Jan 2004 13:26:16 -0000 1.77
+++ include/http_core.h 14 Jan 2004 22:48:09 -0000
@@ -367,6 +367,8 @@
* won't actually be delivered as the response for the non-GET method.
*/
int deliver_script;
+ /* The URI matches a <Location > block containing a SetHandler */
+ int uri_outside_filesystem;
} core_request_config;
/* Standard entries that are guaranteed to be accessible via
Index: server/core.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/core.c,v
retrieving revision 1.256
diff -u -d -b -r1.256 core.c
--- server/core.c 8 Jan 2004 18:56:20 -0000 1.256
+++ server/core.c 14 Jan 2004 22:48:09 -0000
@@ -3349,6 +3349,9 @@
static int core_map_to_storage(request_rec *r)
{
int access_status;
+ core_request_config *r_conf = ap_get_module_config(r->request_config,
+ &core_module);
+ if (!r_conf->uri_outside_filesystem) {
if ((access_status = ap_directory_walk(r))) {
return access_status;
@@ -3356,6 +3359,7 @@
if ((access_status = ap_file_walk(r))) {
return access_status;
+ }
}
return OK;
Index: server/request.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/request.c,v
retrieving revision 1.132
diff -u -d -b -r1.132 request.c
--- server/request.c 1 Jan 2004 13:26:23 -0000 1.132
+++ server/request.c 14 Jan 2004 22:48:09 -0000
@@ -1330,9 +1330,22 @@
* and note the end result to (potentially) skip this step next time.
*/
if (now_merged) {
+ core_dir_config *core_dir_conf = ap_get_module_config(now_merged,
+ &core_module);
+ core_request_config *r_conf = ap_get_module_config(r->request_config,
+ &core_module);
+
r->per_dir_config = ap_merge_per_dir_configs(r->pool,
r->per_dir_config,
now_merged);
+ if (core_dir_conf->handler && strcmp(core_dir_conf->handler,
+ "none")) {
+ /* this location block has a handler. A pretty strong hint
+ * that this URI doesn't live in the filesystem
+ */
+ r_conf->uri_outside_filesystem = 1; + }
+ }
cache->per_dir_result = r->per_dir_config;
