dean gaudet wrote:
>
> On Tue, 31 Oct 2000, Andrew Morton wrote:
>
> > Dean, it looks like the same problem will occur with flock()-based
> > serialisation. Does Apache/Linux ever use that option?
>
> from apache/src/include/ap_config.h in the linux section there's
> this:
>
> /* flock is faster ... but hasn't been tested on 1.x systems */
> /* PR#3531 indicates flock() may not be stable, probably depends on
> * kernel version. Go back to using fcntl, but provide a way for
> * folks to tweak their Configuration to get flock.
> */
> #ifndef USE_FLOCK_SERIALIZED_ACCEPT
> #define USE_FCNTL_SERIALIZED_ACCEPT
> #endif
>
> so you should be able to -DUSE_FLOCK_SERIALIZED_ACCEPT to try it.
>
Dean,
neither flock() nor fcntl() serialisation are effective
on linux 2.2 or linux 2.4. This is because the file
locking code still wakes up _all_ waiters. In my testing
with fcntl serialisation I have seen a single Apache
instance get woken and put back to sleep 1,500 times
before the poor thing actually got to service a request.
For kernel 2.2 I recommend that Apache consider using
sysv semaphores for serialisation. They use wake-one.
For kernel 2.4 I recommend that Apache use unserialised
accept.
This means that you'll need to make a runtime decision
on whether to use unserialised, serialised with sysv or
serialised with fcntl (if sysv IPC isn't installed).
In my testing I launched 3, 10, 30 or 150 Apache instances and then used
httperf --num-conns=2000 --num-calls=1 --uri=/index.html
to open, use and close 2000 connections.
Here are the (terrible) results on 2.4 SMP with fcntl
serialisation:
fcntl accept, 3 servers, vanilla: 938.0 req/s
fcntl accept, 30 servers, vanilla: 697.1 req/s
fcntl accept, 150 servers, vanilla: 99.9 req/s (sic)
2.4 SMP with no serialisation:
unserialised accept, 3 servers, vanilla: 1049.0 req/s
unserialised accept, 10 servers, vanilla: 968.8 req/s
unserialised accept, 30 servers, vanilla: 1040.2 req/s
unserialised accept, 150 servers, vanilla: 1091.4 req/s
2.4 SMP with no serialisation and my patch to the
wakeup and waitqueue code:
unserialised accept, 3 servers, task_exclusive: 1117.4 req/s
unserialised accept, 10 servers, task_exclusive: 1118.6 req/s
unserialised accept, 30 servers, task_exclusive: 1105.6 req/s
unserialised accept, 150 servers, task_exclusive: 1077.1 req/s
2.4 SMP with sysv semaphore serialisation:
sysvsem accept, 3 servers: 1001.2 req/s
sysvsem accept, 10 servers: 1061.0 req/s
sysvsem accept, 30 servers: 1021.2 req/s
sysvsem accept, 150 servers: 943.6 req/s
2.2.14 SMP with fcntl serialisation:
fcntl accept, 3 servers: 1053.8 req/s
fcntl accept, 10 servers: 996.2 req/s
fcntl accept, 30 servers: 934.3 req/s
fcntl accept, 150 servers: 141.4 req/s (sic)
2.2.14 SMP with no serialisation:
unserialised accept, 3 servers: 1039.9 req/s
unserialised accept, 10 servers: 983.1 req/s
unserialised accept, 30 servers: 775.7 req/s
unserialised accept, 150 servers: 220.7 req/s (sic)
2.2.14 SMP with sysv sem serialisation:
sysv accept, 3 servers: 932.2 req/s
sysv accept, 10 servers: 910.6 req/s
sysv accept, 30 servers: 1026.6 req/s
sysv accept, 150 servers: 927.2 req/s
Note that the first test (2.4 with fcntl serialisation) was
with an unpatched 2.4.0-test10-pre5. Once the simple
flock.patch is applied, the performance with 150 servers
doubles. But it's still sucky. The flock.patch change
is effective in increasing scalability wiht a large number
of CPUs, not a large number of httpd's.
Here's the silly patch I used to turn on sysv sem serialisation
in Apache. There's probably a better way than this :)
--- apache_1.3.14.orig/src/main/http_main.c Fri Sep 29 00:32:36 2000
+++ apache_1.3.14/src/main/http_main.c Sat Nov 4 15:01:41 2000
@@ -172,6 +172,13 @@
#include "explain.h"
+/* AKPM */
+#if 1
+#define NEED_UNION_SEMUN
+#define USE_SYSVSEM_SERIALIZED_ACCEPT
+#define USE_FCNTL_SERIALIZED_ACCEPT
+#endif
+
#if !defined(max)
#define max(a,b) (a > b ? a : b)
#endif
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
