Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Tue, Aug 30, 2005 at 10:37:07PM +0100, Colm MacCarthaigh wrote: On Tue, Aug 30, 2005 at 06:00:47PM +0100, Colm MacCarthaigh wrote: On Tue, Aug 30, 2005 at 05:25:07PM +0100, Joe Orton wrote: So is mod_cgid still the default CGI module for worker because there once existed some ye-olde-Unixes which had an interesting fork() implementation? Given that POSIX has mandated since 2001 or whatever that fork() duplicates only the calling thread, is there any disadvantage to using mod_cgi in all MPMs on modern platforms? Can't think of any, and performance is easy to measure. Just some comparitive ab's, all with Linux 2.6.13, NPTL, ab -n 5000 -c 10 http://localhost/cgi-bin/test-cgi MPM cgidcgi worker 594.28 588.69 req/sec. event 665.11 661.28 req/sec. leader 598.14 592.86 req/sec. threadpool 594.94 591.87 req/sec. There seems to be a very consistent .1% advantage to using mod_cgid, for me at least. Of course, cgid eats a process table entry and some memory, but fork()'ing a plain process does seem to be less overhead. Apologies for the previous empty mail ;) If the performance difference is that small then mod_cgi is definitely a better default IMO: it has much better stderr handling (the CGI bucket stuff) and it will log to the correct vhost-specific logs. joe
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Wed, Aug 31, 2005 at 10:24:49AM +0100, Joe Orton wrote: Apologies for the previous empty mail ;) If the performance difference is that small then mod_cgi is definitely a better default IMO: it has much better stderr handling (the CGI bucket stuff) and it will log to the correct vhost-specific logs. +1 on changing the default, cgid can remain available for .1% freaks. -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On 8/31/05, Colm MacCarthaigh [EMAIL PROTECTED] wrote: On Wed, Aug 31, 2005 at 10:24:49AM +0100, Joe Orton wrote: Apologies for the previous empty mail ;) If the performance difference is that small then mod_cgi is definitely a better default IMO: it has much better stderr handling (the CGI bucket stuff) and it will log to the correct vhost-specific logs. +1 on changing the default, cgid can remain available for .1% freaks. I keep a quote from Ulrich Drepper on the wall behind my monitor (somewhere between the TCP state transition diagram and how to write a DVD on Linux): No threaded program must use anything but _exit or exec after fork. apr_proc_create() does a lot more than that. It isn't that I believe the statement to be the absolute truth given infinite programming time and highly constrained web server APIs (neither of which we have), but I believe it is wise to follow anyway. Sadly, mod_ext_filter calls apr_proc_create() similar to mod_cgi and is subject to race conditions in threaded servers under heavy load, with pipe handles open in the wrong processes which prevent the children from seeing EOF on their stdin (due to magic of fork, handles intended for a particular child get duped in another child). A lock to be held by any code dorking with file descriptors or forking is needed to prevent file descriptors from being duplicated unexpectedly in children. What about mutexes? Does the forked, held mutex get released automatically by the new child when it does exec()? Does it depend on which type of mutex? Is APR going to have to use atfork handlers to clean up its own mutexes to prepare for this scenario? (which wouldn't help the typical commercial third-party module which has its own portability layer to support deployment in Apache, IIS, SunOne, etc.) (I keep the idea of a fork service in the back of my mind, which generalizes the daemon feature of mod_cgid for use by any Apache modules which need to create child processes from MPM threads.)
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Wed, Aug 31, 2005 at 06:54:03AM -0400, Jeff Trawick wrote: On 8/31/05, Colm MacCarthaigh [EMAIL PROTECTED] wrote: On Wed, Aug 31, 2005 at 10:24:49AM +0100, Joe Orton wrote: Apologies for the previous empty mail ;) If the performance difference is that small then mod_cgi is definitely a better default IMO: it has much better stderr handling (the CGI bucket stuff) and it will log to the correct vhost-specific logs. +1 on changing the default, cgid can remain available for .1% freaks. I keep a quote from Ulrich Drepper on the wall behind my monitor (somewhere between the TCP state transition diagram and how to write a DVD on Linux): No threaded program must use anything but _exit or exec after fork. apr_proc_create() does a lot more than that. It isn't that I believe the statement to be the absolute truth given infinite programming time and highly constrained web server APIs (neither of which we have), but I believe it is wise to follow anyway. must not use anything but async-signal-safe functions is actually what the standard requires FWIW. Sadly, mod_ext_filter calls apr_proc_create() similar to mod_cgi and is subject to race conditions in threaded servers under heavy load, with pipe handles open in the wrong processes which prevent the children from seeing EOF on their stdin (due to magic of fork, handles intended for a particular child get duped in another child). A lock to be held by any code dorking with file descriptors or forking is needed to prevent file descriptors from being duplicated unexpectedly in children. Ah, I always forget about this people using threaded servers are doomed scenario. I guess all the people voting to make worker the default MPM do too... Regardless of cgid vs cgi, how much of this is it feasible to fix? APR has to require that all child_cleanup functions are async-signal-safe; that's probably doable. Can apr_proc_create just do some close all fds 3 in the forked child? I guess it all gets too icky. joe
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Tue, Aug 30, 2005 at 08:24:17AM -, [EMAIL PROTECTED] wrote: Modified: httpd/httpd/trunk/docs/manual/stopping.xml URL: http://svn.apache.org/viewcvs/httpd/httpd/trunk/docs/manual/stopping.xml?rev=264737r1=264736r2=264737view=diff == --- httpd/httpd/trunk/docs/manual/stopping.xml (original) +++ httpd/httpd/trunk/docs/manual/stopping.xml Tue Aug 30 01:24:14 2005 @@ -190,13 +190,15 @@ pThe codeWINCH/code or codegraceful-stop/code signal causes the parent process to emadvise/em the children to exit after their current request (or to exit immediately if they're not -serving anything). The parent will then remove it's directive +serving anything). The parent will then remove its directive erk :) module=mpm_commonPidFile/directive and cease listening on all ports. The parent will continue to run, and monitor children which are handling requests. Once all children have finalised and exited or the timeout specified by the directive module=mpm_commonGracefulShutdownTimeout/directive has been -reached, the parent will also exit./p +reached, the parent will also exit. If the timeout is reached, +any remaining children will be sent the codeTERM/code signal +to force them to exit./p This is only the case on prefork, on the threaded MPM's its more complicated that that. The children are told via the pod to exit, but TERM is not sent explicitly, instead the children will get sent HUP due to parent exit. This has all sorts of consequences, the most annoying of which is that both stop and graceful-stop actually won't kill CGI processes when using a threaded MPM/cgid. -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Tue, Aug 30, 2005 at 02:27:20PM +0100, Colm MacCarthaigh wrote: On Tue, Aug 30, 2005 at 08:24:17AM -, [EMAIL PROTECTED] wrote: module=mpm_commonPidFile/directive and cease listening on all ports. The parent will continue to run, and monitor children which are handling requests. Once all children have finalised and exited or the timeout specified by the directive module=mpm_commonGracefulShutdownTimeout/directive has been -reached, the parent will also exit./p +reached, the parent will also exit. If the timeout is reached, +any remaining children will be sent the codeTERM/code signal +to force them to exit./p This is only the case on prefork, on the threaded MPM's its more complicated that that. The children are told via the pod to exit, but TERM is not sent explicitly, instead the children will get sent HUP due to parent exit. Ah right, thanks. I guess I'll just change this to have a behaviour may vary across MPMs disclaimer... This has all sorts of consequences, the most annoying of which is that both stop and graceful-stop actually won't kill CGI processes when using a threaded MPM/cgid. So is mod_cgid still the default CGI module for worker because there once existed some ye-olde-Unixes which had an interesting fork() implementation? Given that POSIX has mandated since 2001 or whatever that fork() duplicates only the calling thread, is there any disadvantage to using mod_cgi in all MPMs on modern platforms? joe
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Tue, Aug 30, 2005 at 05:25:07PM +0100, Joe Orton wrote: This has all sorts of consequences, the most annoying of which is that both stop and graceful-stop actually won't kill CGI processes when using a threaded MPM/cgid. So is mod_cgid still the default CGI module for worker because there once existed some ye-olde-Unixes which had an interesting fork() implementation? Given that POSIX has mandated since 2001 or whatever that fork() duplicates only the calling thread, is there any disadvantage to using mod_cgi in all MPMs on modern platforms? Can't think of any, and performance is easy to measure. Either way, might be worth doing something like; http://people.apache.org/~colm/cgid-kill-children.patch in cgid itself, to help fix the immediate problem. Of course that patch won't help with suexec CGI's, since the daemon won't have permission to send the child a signal. -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Tue, Aug 30, 2005 at 06:00:47PM +0100, Colm MacCarthaigh wrote: On Tue, Aug 30, 2005 at 05:25:07PM +0100, Joe Orton wrote: This has all sorts of consequences, the most annoying of which is that both stop and graceful-stop actually won't kill CGI processes when using a threaded MPM/cgid. So is mod_cgid still the default CGI module for worker because there once existed some ye-olde-Unixes which had an interesting fork() implementation? Given that POSIX has mandated since 2001 or whatever that fork() duplicates only the calling thread, is there any disadvantage to using mod_cgi in all MPMs on modern platforms? Can't think of any, and performance is easy to measure. Just some comparitive ab's, all with Linux 2.6.13, NPTL, ab -n 5000 -c 10 http://localhost/cgi-bin/test-cgi MPM cgidcgi worker 594.28 588.69 req/sec. event 665.11 661.28 req/sec. leader 598.14 592.86 req/sec. threadpool 594.94 591.87 req/sec. There seems to be a very consistent .1% advantage to using mod_cgid, for me at least. Of course, cgid eats a process table entry and some memory, but fork()'ing a plain process does seem to be less overhead. -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
Re: svn commit: r264737 - /httpd/httpd/trunk/docs/manual/stopping.xml
On Tue, Aug 30, 2005 at 10:37:07PM +0100, Colm MacCarthaigh wrote: On Tue, Aug 30, 2005 at 06:00:47PM +0100, Colm MacCarthaigh wrote: On Tue, Aug 30, 2005 at 05:25:07PM +0100, Joe Orton wrote: This has all sorts of consequences, the most annoying of which is that both stop and graceful-stop actually won't kill CGI processes when using a threaded MPM/cgid. So is mod_cgid still the default CGI module for worker because there once existed some ye-olde-Unixes which had an interesting fork() implementation? Given that POSIX has mandated since 2001 or whatever that fork() duplicates only the calling thread, is there any disadvantage to using mod_cgi in all MPMs on modern platforms? Can't think of any, and performance is easy to measure. Just some comparitive ab's, all with Linux 2.6.13, NPTL, ab -n 5000 -c 10 http://localhost/cgi-bin/test-cgi MPM cgidcgi worker 594.28 588.69 req/sec. event 665.11 661.28 req/sec. leader 598.14 592.86 req/sec. threadpool 594.94 591.87 req/sec. There seems to be a very consistent .1% advantage to using mod_cgid, for me at least. Of course, cgid eats a process table entry and some memory, but fork()'ing a plain process does seem to be less overhead. -- Colm MacC�rthaighPublic Key: [EMAIL PROTECTED]
