Re: cvs commit: httpd-2.0/server listen.c
[EMAIL PROTECTED] writes: wrowe 02/03/19 23:03:07 Modified:server listen.c Log: Here's the patch that really sucks. old_listeners points to an array of apr_socket objects already destroyed by their cleanups, and in any case they now live in invalid memory. Extend their lifetimes. This implies that the process pool grows on every restart for no good reason. One possible solution is to let the old pconf survive until the new pconf is alive. Another is to create the listeners in a subpool of process-pool, destroyed after the old_listeners are closed. Either which way, a better solution exists, but this closes the immediate bug. [How haven't we been segfaulting in unix on restarts before this patch, gurus?] good question... even more interesting (to me) is the question how haven't we been segfaulting in unix on restarts even when we build with electric fence and pool debug, time after time after time :) maybe somebody can make us both happy with the answer -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
RE: cvs commit: httpd-2.0/server listen.c
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Jeff Trawick Sent: 20 March 2002 11:48 [EMAIL PROTECTED] writes: wrowe 02/03/19 23:03:07 Modified:server listen.c Log: Here's the patch that really sucks. old_listeners points to an array of apr_socket objects already destroyed by their cleanups, and in any case they now live in invalid memory. Extend their lifetimes. This implies that the process pool grows on every restart for no good reason. One possible solution is to let the old pconf survive until the new pconf is alive. Another is to create the listeners in a subpool of process-pool, destroyed after the old_listeners are closed. Either which way, a better solution exists, but this closes the immediate bug. [How haven't we been segfaulting in unix on restarts before this patch, gurus?] good question... even more interesting (to me) is the question how haven't we been segfaulting in unix on restarts even when we build with electric fence and pool debug, time after time after time :) maybe somebody can make us both happy with the answer Maybe the memory wasn't being accessed on unix?? Sander 'with a puzzled look on his face'
questions about Apache 2.0 on Solaris 7
Can anybody verify the following on Solaris 7? a) APR recognizes and deems usable the OS's IPv6 support (i.e., it sets APR_HAVE_IPV6) b) Apache IPv6 support doesn't interfere with normal operation on IPv4-only host c) Apache is actually able to exploit IPv6 without any apparent problems Thanks! -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
[EMAIL PROTECTED] writes:
wrowe 02/03/19 21:58:21
Modified:server/mpm/beos beos.c
server/mpm/netware mpm_netware.c
server/mpm/perchild perchild.c
server/mpm/prefork prefork.c
server/mpm/winnt mpm_winnt.c
server/mpm/worker worker.c
Log:
The pre_mpm hook creates server-lifetime objects (or at least, for the
generations across graceful restarts.) They should use the process pool.
The pre_mpm hook doesn't seem generically useful. It seems to be tied
to our current idea of how the scoreboard should be managed.
Index: beos.c
===
RCS file: /home/cvs/httpd-2.0/server/mpm/beos/beos.c,v
retrieving revision 1.88
retrieving revision 1.89
diff -u -r1.88 -r1.89
--- beos.c 15 Mar 2002 00:50:31 - 1.88
+++ beos.c 20 Mar 2002 05:58:20 - 1.89
@@ -839,7 +839,7 @@
if (!is_graceful) {
/* setup the scoreboard shared memory */
-if (ap_run_pre_mpm(pconf, SB_SHARED) != OK) {
+if (ap_run_pre_mpm(s-process-pool, SB_SHARED) != OK) {
return 1;
}
Now the scoreboard lives forever and it is safe to call
ap_run_pre_mpm() at any restart. One piece I think you have missed is
proper initialization of the scoreboard after a non-graceful restart.
For a graceful restart we don't want cleared scoreboard values (and
we don't).
For a non-graceful restart we do want a fresh (or at least cleared)
scoreboard (and we don't get that any more).
Isn't non-graceful restart now broken? (There is plenty of
opportunity for me to be confused here.)
Can you remind me what was broken before this commit?
--
Jeff Trawick | [EMAIL PROTECTED]
Born in Roswell... married an alien...
Re: Win32 Segfaults at shutdown ... what happened.
What shutdown problems are you referring to? I identified the only one I have been seeing (in APR). Bill - Original Message - From: William A. Rowe, Jr. [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, March 20, 2002 12:49 AM Subject: Win32 Segfaults at shutdown ... what happened. Ok folks, hope I can offer a reasonable suggestion of why we 'suddenly' discovered all these segfaults. The old MPM code, in the process of shutting down by Win32's request, would tell the Win32 SCM that it had 'shut down', before all of the cleanup code had run. It's my suspicion that this code caused the SCM to terminate our process, before we had finished running our own cleanups. That left us wide open for the ordering faults we are seeing today. Some of that may be in the MPM itself - I will be vetting and fixing the pools selected for each call, based on the initialization phase and scope of the object created One huge change that affects all MPMs is the create scoreboard code. We pass pconf through ap_run_pre_mpm ... but pre_mpm is only invoked on initial starts and graceless restarts. It MUST be passed a process-pool that will outlive that generation. I'll pass on discoveries, but I think that the 'good' shutdown code has just opened up the can, and the worms are wiggling. Bill
Re: new compile errors on 2.0 cvs head
I am on it. Funny thing is, I developed the initial cut of the patch expecting to see this warning. My compiler didn't issue the warning (gotta find out why), I sat on the patch for a few days did a recompile before committing the patch, didn;t see any new warnings so committed it. I must have a setting off in my VC++ 6.0 configuration... Bill - Original Message - From: William A. Rowe, Jr. [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, March 20, 2002 2:19 AM Subject: new compile errors on 2.0 cvs head http_request.c C:\clean\httpd-2.0\modules\http\http_request.c(389) : warning C4090: 'function' : different 'const' qualifiers C:\clean\httpd-2.0\modules\http\http_request.c(389) : warning C4024: 'ap_rset_content_type' : different types for formal and actual parameter 1 C:\clean\httpd-2.0\modules\http\http_request.c(431) : warning C4090: 'function' : different 'const' qualifiers C:\clean\httpd-2.0\modules\http\http_request.c(431) : warning C4024: 'ap_rset_content_type' : different types for formal and actual parameter 1 mod_rewrite.c C:\clean\httpd-2.0\modules\mappers\mod_rewrite.c(1314) : warning C4090: 'function' : different 'const' qualifiers C:\clean\httpd-2.0\modules\mappers\mod_rewrite.c(1314) : warning C4024: 'ap_rset_content_type' : different types for formal and actual parameter 1 proxy_ftp.c C:\clean\httpd-2.0\modules\proxy\proxy_ftp.c(1721) : warning C4090: 'function' : different 'const' qualifiers C:\clean\httpd-2.0\modules\proxy\proxy_ftp.c(1721) : warning C4024: 'ap_rset_content_type' : different types for formal and actual parameter 1
Re: new compile errors on 2.0 cvs head
Bill Stoddard [EMAIL PROTECTED] writes: I am on it. Funny thing is, I developed the initial cut of the patch expecting to see this warning. My compiler didn't issue the warning (gotta find out why), I sat on the patch for a few days did a recompile before committing the patch, didn;t see any new warnings so committed it. I must have a setting off in my VC++ 6.0 configuration... try grabbing my very recent commit to http_protocol.c and http_protocol.h... I made the parm to ap_rset_content_type() const char * instead of char * -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: cvs commit: httpd-2.0/server/mpm/winnt mpm_winnt.c
[EMAIL PROTECTED] writes:
On 20 Mar 2002 [EMAIL PROTECTED] wrote:
wrowe 02/03/19 20:29:55
Modified:server/mpm/winnt mpm_winnt.c
Log:
When restarting [always graceful on Win32], we don't repeat pre_mpm
(Unix doesn't, we shouldn't either.) [Ryan Bloom]
-if ((parent_pid == my_pid) || one_process) {
+/* ### If non-graceful restarts are ever introduced - we need to rerun
+ * the pre_mpm hook on subsequent non-graceful restarts. But Win32
+ * has only graceful style restarts - and we need this hook to act
+ * the same on Win32 as on Unix.
+ */
+if (!restart ((parent_pid == my_pid) || one_process)) {
/* Set up the scoreboard. */
if (ap_run_pre_mpm(pconf, SB_SHARED) != OK) {
return 1;
While I agree with this patch, you also need to kill the cleanup on the
scoreboard, so that it isn't set to NULL when pconf is cleared.
Actually, the cleanup never runs because it never finds the data in
the pconf pool :) But yes it needs to be dealt with.
I was cleaning this issue up this morning (removing cleanups, removing
logic I added recently to restore the proper running_generation field)
and realized (or was confused into thinking) that we aren't handling
non-graceful restart anymore, in that we don't start back with a fresh
scoreboard (fresh except for the running_generation field). Now,
whatever was in the scoreboard from the previous, whacked generation
of children is still there.
For now I've set aside the cleanup changes until there is some
resolution on how it should be handled (should MPM have to clean up,
should scoreboard-create clean up when it realizes that it doesn't
have to reallocate, etc.).
--
Jeff Trawick | [EMAIL PROTECTED]
Born in Roswell... married an alien...
Re: cvs commit: httpd-2.0/server/mpm/winnt mpm_winnt.c
Jeff Trawick [EMAIL PROTECTED] writes:
[EMAIL PROTECTED] writes:
On 20 Mar 2002 [EMAIL PROTECTED] wrote:
wrowe 02/03/19 20:29:55
Modified:server/mpm/winnt mpm_winnt.c
Log:
When restarting [always graceful on Win32], we don't repeat pre_mpm
(Unix doesn't, we shouldn't either.) [Ryan Bloom]
-if ((parent_pid == my_pid) || one_process) {
+/* ### If non-graceful restarts are ever introduced - we need to rerun
+ * the pre_mpm hook on subsequent non-graceful restarts. But Win32
+ * has only graceful style restarts - and we need this hook to act
+ * the same on Win32 as on Unix.
+ */
+if (!restart ((parent_pid == my_pid) || one_process)) {
/* Set up the scoreboard. */
if (ap_run_pre_mpm(pconf, SB_SHARED) != OK) {
return 1;
While I agree with this patch, you also need to kill the cleanup on the
scoreboard, so that it isn't set to NULL when pconf is cleared.
Actually, the cleanup never runs because it never finds the data in
the pconf pool :) But yes it needs to be dealt with.
wrong... let me restate... we currently have logic to kill the
cleanup for graceful restart but that call to cleanup_kill is bogus in
that 1) it searches the wrong pool and 2) if OtherBill's changes to
keep using the same scoreboard forever and ever then we don't need to
kill the cleanup anyway
--
Jeff Trawick | [EMAIL PROTECTED]
Born in Roswell... married an alien...
RE: cvs commit: httpd-2.0/server/mpm/winnt mpm_winnt.c
And, you need to find some way to pass the scoreboard back to the child process because you are about to start a new one. What are you talking about, we killed Kenny :0/ This next generation we have new offspring. yeah, but the new child process needs the same information as the old one did. Won't you need to pass the scoreboard to the new child, and isn't that done by the pre_mpm hook? I could very easily be wrong about which hook does that work, but the last time I looked, I thought it was pre_mpm. Ryan
RE: cvs commit: httpd-2.0/server/mpm/worker worker.c
Log: The pre_mpm hook creates server-lifetime objects (or at least, for the generations across graceful restarts.) They should use the process pool. The pre_mpm hook doesn't seem generically useful. It seems to be tied to our current idea of how the scoreboard should be managed. The per_mpm phase was originally written to provide a place to create shared memory that the server can use. Because our best map to how shared memory should work is the scoreboard, that is how the hook was implemented. Ryan
Mod_rewrite + VirtualScriptAlias
Hello list! I´m using Mod_vhost_alias + Mod_rewrite... and trying to configure the VirtualScriptAlias /cgi-bin/ to get inside the SubDomains/cgi-bin directory; Take a look: For example, my domain: http://test.mydomain.com.br For: VirtualScriptAlias /opt/vhosts/t/e/test/cgi-bin I´m using: VirtualScriptAlias /opt/vhosts/%1.1/%1.2/%1/cgi-bin Directory /home/vhosts Options FollowSymLinks ExecCGI AllowOverride None /Directory I can´t execute any file inside this directory... could someone explain to me why this is happening? I really appreciate. LOG_FILE: [Wed Mar 20 11:58:14 2002] [error] [client xxx.xxx.xxx.xxx] file permissions deny server execution: /opt/vhosts/t/e/teste/cgi-bin/formmail.cgi Tks Dan
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
[EMAIL PROTECTED] wrote: +/* XXX unfortunate issue: + * with apachectl restart (not graceful), previous generation dies + * abruptly with no chance to clean up scoreboard entries; when new + * generation is started, processes can loop forever in start_threads() + * waiting for scoreboard entries for threads of prior generation to + * get cleaned up... yikes! Did that break when the scoreboard moved to the process pool? Looks like ap_cleanup_scoreboard won't run any more. Greg
Re: 404 .var file served as text/plain revisited
Cliff Woolley wrote: I know this has been brought up before, but what did we decide was the fix for this problem? I have a test install that's almost identical to httpd-std.conf, and yet my 404 .var file gets served as text/plain. HEAD /server-status/ HTTP/1.0 Host: localhost HTTP/1.1 404 Not Found Date: Mon, 18 Mar 2002 21:32:18 GMT Server: Apache/2.0.34-dev (Unix) mod_ssl/3.0a0 OpenSSL/0.9.6c DAV/2 Vary: accept-language Content-Length: 790 Connection: close Content-Type: text/plain; charset=ISO-8859-1 [discovered because I neglected to configure mod_status ;] Thanks, --Cliff -- Cliff Woolley [EMAIL PROTECTED] Charlottesville, VA I have so more funny things related to the topic today: +++ HEAD /toto/ HTTP/1.0 Host: vtxrm2 HTTP/1.1 501 Method Not Implemented Date: Wed, 20 Mar 2002 15:06:14 GMT Server: Apache/2.0.34-dev (Unix) DAV/2 Allow: GET,HEAD,POST,OPTIONS,TRACE Content-Length: 326 Connection: close Content-Type: text/html; charset=iso-8859-1 +++ That is without mod_include (with mod_include I get the text/plain 501*.var file). Cheers Jean-frederic
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Greg Ames wrote: [EMAIL PROTECTED] wrote: +/* XXX unfortunate issue: + * with apachectl restart (not graceful), previous generation dies + * abruptly with no chance to clean up scoreboard entries; when new + * generation is started, processes can loop forever in start_threads() + * waiting for scoreboard entries for threads of prior generation to + * get cleaned up... yikes! Did that break when the scoreboard moved to the process pool? Looks like ap_cleanup_scoreboard won't run any more. ooops, sorry. I see that your comment preceded the process pool change. Maybe it was the shmem change that broke it? In any case, we need to design a reliable way to fix the problem you described. Greg
Re: Win32 build broken
Sebastian Bergmann wrote: Brian Pane wrote: Those look like some of the symbols introduced in the new version of PCRE that I just imported. They're defined in config.h, so they'll likely require a buildconf;configure;make clean. I'm using MSVC :) somone will need to add/edit the config.hw file to put the window specific definitions in there for the missing vars
Re: questions about Apache 2.0 on Solaris 7
On 20 Mar 2002, Jeff Trawick wrote: | Can anybody verify the following on Solaris 7? | | a) APR recognizes and deems usable the OS's IPv6 support (i.e., it |sets APR_HAVE_IPV6) | | b) Apache IPv6 support doesn't interfere with normal operation on |IPv4-only host | | c) Apache is actually able to exploit IPv6 without any apparent |problems I no-longer have a sol7 box on hand, but I do know that Solaris 7 does not support IPv6... APR_HAVE_IPV6 should be undef'd. IPv6 support in Solaris 7 was a internal Sun project, and the patches you had to apply to sol7 to get IPv6 support could be applied only to Solaris 7 FCS (the first release, non-patched). /dale
RE: cvs commit: httpd-2.0/server/mpm/winnt mpm_winnt.c
At 08:41 AM 3/20/2002, you wrote: yeah, but the new child process needs the same information as the old one did. Won't you need to pass the scoreboard to the new child, and isn't that done by the pre_mpm hook? I could very easily be wrong about which hook does that work, but the last time I looked, I thought it was pre_mpm. No, pre_mpm doesn't pass anything from parent-child [that would be broken already by the pre_mpm-only-once patch.] The create_process logic within mpm_winnt calls two passing fns, one of handles in general, one for the listeners. It's invoked every time we create a child. We have two matching fn's called by child_main() that slurp those general handlers and listeners, respectively. Whatever the ap_scoreboard_shm's os_handle, we pass it. We call ap_scoreboard_init, not to init the score, but to init the local storage, e.g. array pointers into the scoreboard memory. The memory isn't passed to the child, the handle to this memory is passed; the child reopens the scoreboard by its handle. Bill
RE: cvs commit: httpd-2.0/server/mpm/winnt mpm_winnt.c
Cool, thanks for the info. This is much clearer now. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 20, 2002 8:28 AM To: [EMAIL PROTECTED] Subject: RE: cvs commit: httpd-2.0/server/mpm/winnt mpm_winnt.c At 08:41 AM 3/20/2002, you wrote: yeah, but the new child process needs the same information as the old one did. Won't you need to pass the scoreboard to the new child, and isn't that done by the pre_mpm hook? I could very easily be wrong about which hook does that work, but the last time I looked, I thought it was pre_mpm. No, pre_mpm doesn't pass anything from parent-child [that would be broken already by the pre_mpm-only-once patch.] The create_process logic within mpm_winnt calls two passing fns, one of handles in general, one for the listeners. It's invoked every time we create a child. We have two matching fn's called by child_main() that slurp those general handlers and listeners, respectively. Whatever the ap_scoreboard_shm's os_handle, we pass it. We call ap_scoreboard_init, not to init the score, but to init the local storage, e.g. array pointers into the scoreboard memory. The memory isn't passed to the child, the handle to this memory is passed; the child reopens the scoreboard by its handle. Bill
Re: questions about Apache 2.0 on Solaris 7
Dale Ghent [EMAIL PROTECTED] writes: I no-longer have a sol7 box on hand, but I do know that Solaris 7 does not support IPv6... APR_HAVE_IPV6 should be undef'd. IPv6 support in Solaris 7 was a internal Sun project, and the patches you had to apply to sol7 to get IPv6 support could be applied only to Solaris 7 FCS (the first release, non-patched). Ouch! Thanks for the info. I found a web page at sun for these patches. The words they use aren't inspiring (this prototype is not officially supported). It sounds like you should upgrade to Solaris 8 if you really want IPv6 code that has been updated recently. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: Win32 build broken
It seems that NetWare build is also broken for the same reason, so it's needed to do a similar thing for this platform, e.g. add the config.hnw file under .../srclib/pcre/. Thanks, Pavel Ian Holsman wrote: Sebastian Bergmann wrote: Brian Pane wrote: Those look like some of the symbols introduced in the new version of PCRE that I just imported. They're defined in config.h, so they'll likely require a buildconf;configure;make clean. I'm using MSVC :) somone will need to add/edit the config.hw file to put the window specific definitions in there for the missing vars
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
[EMAIL PROTECTED] writes: jim 02/03/20 08:44:13 Modified:.CHANGES include mpm_common.h server mpm_common.c server/mpm/mpmt_os2 mpmt_os2.c server/mpm/netware mpm_netware.c server/mpm/perchild perchild.c server/mpm/prefork prefork.c server/mpm/worker worker.c Log: Bring 2.0 up to parity, a bit, with how much info we provide to the admin regarding valid values for AcceptMutex. Should also tell 'em what default actually maps to, but that can wait. I don't think this works as you desired. I just did an update (no diffs in this tree) and for a config with no AcceptMutex I get this at startup: [notice] AcceptMutex: Valid accept mutexes for this platform and MPM are: default, fcntl, sysvsem, pthread. Isn't that the message for httpd -L output? The 1.3 startup message is something like this: [notice] Accept mutex: fcntl (Default: fcntl) much more appropriate -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: Win32 build broken
On Wed, 20 Mar 2002, Pavel Novy wrote: somone will need to add/edit the config.hw file to put the window specific definitions in there for the missing vars I'm on it. --Cliff -- Cliff Woolley [EMAIL PROTECTED] Charlottesville, VA
Re: Win32 build broken
This *should* fix it, but I'm eyeballing it from Unix... if no one beats me to it, I'll dig up a win32 box and test it after I get back from lunch. --Cliff Index: config.hw === RCS file: /home/cvs/httpd-2.0/srclib/pcre/config.hw,v retrieving revision 1.1 diff -u -d -r1.1 config.hw --- config.hw 6 May 2000 15:02:12 - 1.1 +++ config.hw 20 Mar 2002 17:38:53 - @@ -3,9 +3,13 @@ written in Standard C, but there are a few non-standard things it can cope with, allowing it to run on SunOS4 and other close to standard systems. -On a non-Unix system you should just copy this file into config.h and change -the definitions of HAVE_STRERROR and HAVE_MEMMOVE to 1. Unfortunately, because -of the way autoconf works, these cannot be made the defaults. */ +On a non-Unix system you should just copy this file into config.h, and set up +the macros the way you need them. You should normally change the definitions of +HAVE_STRERROR and HAVE_MEMMOVE to 1. Unfortunately, because of the way autoconf +works, these cannot be made the defaults. If your system has bcopy() and not +memmove(), change the definition of HAVE_BCOPY instead of HAVE_MEMMOVE. If your +system has neither bcopy() nor memmove(), leave them both as 0; an emulation +function will be used. */ /* Define to empty if the keyword does not work. */ @@ -24,5 +28,19 @@ #define HAVE_STRERROR 1 #define HAVE_MEMMOVE 1 + +/* There are some non-Unix systems that don't even have bcopy(). If this macro +is false, an emulation is used. If HAVE_MEMMOVE is set to 1, the value of +HAVE_BCOPY is not relevant. */ + +#define HAVE_BCOPY 0 + +/* The value of NEWLINE determines the newline character. The default is to +leave it up to the compiler, but some sites want to force a particular value. +On Unix systems, configure can be used to override this default. */ + +#ifndef NEWLINE +#define NEWLINE '\n' +#endif /* End */ Index: pcre.hw === RCS file: /home/cvs/httpd-2.0/srclib/pcre/pcre.hw,v retrieving revision 1.2 diff -u -d -r1.2 pcre.hw --- pcre.hw 8 Oct 2000 23:59:45 - 1.2 +++ pcre.hw 20 Mar 2002 17:38:53 - @@ -2,7 +2,7 @@ * Perl-Compatible Regular Expressions * */ -/* Copyright (c) 1997-2000 University of Cambridge */ +/* Copyright (c) 1997-2001 University of Cambridge */ #ifndef _PCRE_H #define _PCRE_H @@ -11,8 +11,8 @@ Do not edit it; instead make changes to pcre.in and/or pcre.hw */ #define PCRE_MAJOR 3 -#define PCRE_MINOR 2 -#define PCRE_DATE 12-May-2000 +#define PCRE_MINOR 9 +#define PCRE_DATE 02-Jan-2002 /* Win32 uses DLL by default */ @@ -50,6 +50,7 @@ #define PCRE_NOTEOL 0x0100 #define PCRE_UNGREEDY0x0200 #define PCRE_NOTEMPTY0x0400 +#define PCRE_UTF80x0800 /* Exec-time and get-time error codes */ @@ -73,8 +74,11 @@ /* Types */ -typedef void pcre; -typedef void pcre_extra; +struct real_pcre;/* declaration; the definition is private */ +struct real_pcre_extra; /* declaration; the definition is private */ + +typedef struct real_pcre pcre; +typedef struct real_pcre_extra pcre_extra; /* Store get and free functions. These can be set to alternative malloc/free functions if required. Some magic is required for Win32 DLL; it is null on @@ -92,11 +96,13 @@ extern int pcre_copy_substring(const char *, int *, int, int, char *, int); extern int pcre_exec(const pcre *, const pcre_extra *, const char *, int, int, int, int *, int); +extern void pcre_free_substring(const char *); +extern void pcre_free_substring_list(const char **); extern int pcre_get_substring(const char *, int *, int, int, const char **); extern int pcre_get_substring_list(const char *, int *, int, const char ***); extern int pcre_info(const pcre *, int *, int *); extern int pcre_fullinfo(const pcre *, const pcre_extra *, int, void *); -extern unsigned const char *pcre_maketables(void); +extern const unsigned char *pcre_maketables(void); extern pcre_extra *pcre_study(const pcre *, int, const char **); extern const char *pcre_version(void); -- Cliff Woolley [EMAIL PROTECTED] Charlottesville, VA
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Jeff Trawick wrote: Bring 2.0 up to parity, a bit, with how much info we provide to the admin regarding valid values for AcceptMutex. Should also tell 'em what default actually maps to, but that can wait. I don't think this works as you desired. I just did an update (no diffs in this tree) and for a config with no AcceptMutex I get this at startup: [notice] AcceptMutex: Valid accept mutexes for this platform and MPM are: default, fcntl, sysvsem, pthread. Isn't that the message for httpd -L output? The 1.3 startup message is something like this: [notice] Accept mutex: fcntl (Default: fcntl) much more appropriate -- Actually, it does exactly what I intended, for the present. :) Agreed, and this is part of what I alluded to above about tell 'em what default is... For right now, since TAKE1 is a macro, you can't use normal preprocessor foo bar concats to build up the supported string, so the info message for AllowMutex is hard to be very 1.3-like. So, the place to put it is in error log (we need *some* way of noting valid mutexes). Ideally the solution is something that reports what default maps to, but it requires some APR work and some consolidation since the MPM's make use of different locking calls (apr_proc_mutex_lock, apr_lock_acquire) and thus need a good solution for a common function for something like ap_mutex_method_name() with the addition of the name to the actual lock structs... -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
APR_LOCK_DEFAULT ordering
Are people happy with the priority order of the accept mutex? Right now it's flock - sysvsem - fcntl - pthread. I think it should be pthread - sysvsem - fcntl - flock, which is what 1.3 has... -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
this a.m. how do we get a cleared scoreboard after apachectl restart?
If there is no such logic right now: Is the WinNT MPM gonna puke if ap_create_scoreboard() clears it (but preserving running_generation) in the already-have-a-scoreboard path? -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: APR_LOCK_DEFAULT ordering
On Wed, Mar 20, 2002 at 12:49:32PM -0500, Jim Jagielski wrote: Are people happy with the priority order of the accept mutex? Right now it's flock - sysvsem - fcntl - pthread. I think it should be pthread - sysvsem - fcntl - flock, which is what 1.3 has... Now that we're more confident in pthread, I'm a huge +1 for this second ordering. -aaron
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Jim Jagielski [EMAIL PROTECTED] writes: Jeff Trawick wrote: Bring 2.0 up to parity, a bit, with how much info we provide to the admin regarding valid values for AcceptMutex. Should also tell 'em what default actually maps to, but that can wait. I don't think this works as you desired. I just did an update (no diffs in this tree) and for a config with no AcceptMutex I get this at startup: [notice] AcceptMutex: Valid accept mutexes for this platform and MPM are: default, fcntl, sysvsem, pthread. Isn't that the message for httpd -L output? The 1.3 startup message is something like this: [notice] Accept mutex: fcntl (Default: fcntl) much more appropriate -- Actually, it does exactly what I intended, for the present. :) It doesn't look too cool :) Agreed, and this is part of what I alluded to above about tell 'em what default is... For right now, since TAKE1 is a macro, you can't use normal preprocessor foo bar concats to build up the supported string, so the info message for AllowMutex is hard to be very 1.3-like. You don't have to use the macro (of course that extra type-checking via the designated initializers is nice)... If you use the macro, couldn't you pass a variable to the macro, and use the preprocessor to build the string pointed to by that variable? static const char *accept_mutex_help = Valid accept mutexes for this platform and MPM are: default #if APR_HAS_FLOCK_SERIALIZE , flock #endif #if APR_HAS_FCNTL_SERIALIZE , fcntl #endif #if APR_HAS_SYSVSEM_SERIALIZE !defined(PERCHILD_MPM) , sysvsem #endif #if APR_HAS_PROC_PTHREAD_SERIALIZE , pthread #endif . then specify accept_mutex_help as the last parm to AP_INIT_TAKE1()? (untested :) ) -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
On Wed, Mar 20, 2002 at 12:41:21PM -0500, Jim Jagielski wrote: Actually, it does exactly what I intended, for the present. :) Agreed, and this is part of what I alluded to above about tell 'em what default is... For right now, since TAKE1 is a macro, you can't use normal preprocessor foo bar concats to build up the supported string, so the info message for AllowMutex is hard to be very 1.3-like. So, the place to put it is in error log (we need *some* way of noting valid mutexes). Ideally the solution is something that reports what default maps to, but it requires some APR work and some consolidation since the MPM's make use of different locking calls (apr_proc_mutex_lock, apr_lock_acquire) and thus need a good solution for a common function for something like ap_mutex_method_name() with the addition of the name to the actual lock structs... Accept mutexes should all be apr_proc_mutex_t types, since the apr_lock_t type is being phased out and will be soon removed from APR. Personally I'd like to see the default set in autoconf and dictated to APR. That would give us the side-effect benefit of having the default defined in a cpp symbol. -aaron
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Aaron Bannert wrote: Accept mutexes should all be apr_proc_mutex_t types, since the apr_lock_t type is being phased out and will be soon removed from APR. And that's the other issue as well... Let that settle down first :) -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Jeff Trawick wrote: It doesn't look too cool :) I agree! :) You don't have to use the macro (of course that extra type-checking via the designated initializers is nice)... If you use the macro, couldn't you pass a variable to the macro, and use the preprocessor to build the string pointed to by that variable? static const char *accept_mutex_help = Valid accept mutexes for this platform and MPM are: default #if APR_HAS_FLOCK_SERIALIZE , flock #endif #if APR_HAS_FCNTL_SERIALIZE , fcntl #endif #if APR_HAS_SYSVSEM_SERIALIZE !defined(PERCHILD_MPM) , sysvsem #endif #if APR_HAS_PROC_PTHREAD_SERIALIZE , pthread #endif . then specify accept_mutex_help as the last parm to AP_INIT_TAKE1()? I'll be looking at that ;) Not sure if it will actually take a variable though... -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
On Wed, Mar 20, 2002 at 01:01:07PM -0500, Jim Jagielski wrote: Accept mutexes should all be apr_proc_mutex_t types, since the apr_lock_t type is being phased out and will be soon removed from APR. And that's the other issue as well... Let that settle down first :) I made an attempt to convert all the remaining uses of apr_lock_t a few weeks ago, and the only places where it is left will be in pieces of code that don't compile for me (mod_auth_digest, perchild mpm). I can't remember if I posted a patch for the beos MPM, but that should be trivial as well. Prefork, Worker, and WinNT are converted. -aaron
Re: PCRE status?
Roy T. Fielding wrote: On Tue, Mar 19, 2002 at 06:07:05PM -0800, Brian Pane wrote: Is the copy of PCRE within httpd-2.0 a separately maintained fork of PCRE, or is it supposed to be an unmodified copy? (The one in the httpd tree appears to be a few releases out of date.) It is supposed to be maintained up to date with the source, but has no current maintainer. It should not be a fork. The reason I ask is that I want to fix a performance problem in PCRE's regexec() function... Do a vendor import and merge the latest stuff first -- it needs to be done anyway for licensing reasons. Roy We're upgraded to the latest PCRE now (thanks for Cliff to fixing the Win32 build). I checked with the PCRE maintainer and learned that the next release is several months away. In the meantime, that leaves me with two options for speeding up ap_regexec(): * Commit a change to the PCRE regexec() function (the same change that I've submitted for the next release of PCRE) into the Apache copy of PCRE for now. * Or change ap_regexec() to bypass regexec() and call the PCRE native regexp exec function directly. (The PCRE regexec() is a thin wrapper around pcre_exec(), so this shouldn't be difficult.) Anybody have a strong preference for one of those alternatives vs. the other? I like the latter. --Brian
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Yes, the perchild one was the main one that concerned me... Aaron Bannert wrote: On Wed, Mar 20, 2002 at 01:01:07PM -0500, Jim Jagielski wrote: Accept mutexes should all be apr_proc_mutex_t types, since the apr_lock_t type is being phased out and will be soon removed from APR. And that's the other issue as well... Let that settle down first :) I made an attempt to convert all the remaining uses of apr_lock_t a few weeks ago, and the only places where it is left will be in pieces of code that don't compile for me (mod_auth_digest, perchild mpm). I can't remember if I posted a patch for the beos MPM, but that should be trivial as well. Prefork, Worker, and WinNT are converted. -aaron -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: APR_LOCK_DEFAULT ordering
Jim Jagielski [EMAIL PROTECTED] writes: Are people happy with the priority order of the accept mutex? Right now it's flock - sysvsem - fcntl - pthread. I think it should be pthread - sysvsem - fcntl - flock, which is what 1.3 has... I realize everybody has jumped in and +1-ed you, but I don't see how you can compare the two orders which you stated above. I would express the current APR default selection (configure.in, line 1303) this way: 1st choice : fcntl 2nd: pthread 3rd: flock 4th: SysV sem I guess I'd vote for this order on systems where we don't have specific knowledge (e.g., use sysvsem for myOS = x): 1st choice: pthread 2nd choice: fcntl 3rd choice: SysV sem 4th choice: flock Also, in 1.3, I think there is a lot more explicit selection (i.e., case myOS use foo) that needs to be brought forward to APR so that these priorities are irrelevant (they should only be used when we don't have specific OS knowledge). Even if we think we ave the priorities the same between 2.0 and 1.3 it isn't really the same if we don't bring forward the cases where the priorities aren't even used. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: PCRE status?
On Wed, Mar 20, 2002 at 10:11:08AM -0800, Brian Pane wrote: We're upgraded to the latest PCRE now (thanks for Cliff to fixing the Win32 build). I checked with the PCRE maintainer and learned that the next release is several months away. In the meantime, that leaves me with two options for speeding up ap_regexec(): * Commit a change to the PCRE regexec() function (the same change that I've submitted for the next release of PCRE) into the Apache copy of PCRE for now. * Or change ap_regexec() to bypass regexec() and call the PCRE native regexp exec function directly. (The PCRE regexec() is a thin wrapper around pcre_exec(), so this shouldn't be difficult.) Anybody have a strong preference for one of those alternatives vs. the other? I like the latter. I'd prefer option one, but not violently so. -- justin
Re: performance: using mlock(2) on httpd parent process
Hi Scott, On Wed, 20 Mar 2002, Stas Bekman wrote: mod_perl child processes save a lot of memory when they can share memory with the parent process and quite often we get reports from people that they lose that shared memory when the system decides to page out the parent's memory pages because they are LRU (least recently used, the algorithm used by many memory managers). I'm fairly certain that this is not an issue. If a page was shared COW before being paged out, I expect it will be shared COW when paged back in, at least for any modern OS. But if the system needs to page things out, most of the parent process's pages will be scheduled to go first, no? So we are talking about a constant page-in/page-out from/to the parent process as a performance degradation rather than memory unsharing. Am I correct? [To verify that I wasn't talking through my hat, here, I just verified this using RedHat 7.2 running kernel 2.4.9-21. If you're interested in my methodology, drop me an email.] I suppose that this could vary from one kernel version to another. I'm just repeating the reports posted to the mod_perl list. I've never seen such a problem myself, since I try hard to have close to zero swap usage. [Yes, please let me know your methodology for testing this] I believe that this applies to all httpd modules and httpd itself, the more we can share the less memory resources are needed, and usually it leads to a better performance. I'm absolutely _certain_ that unmodified pages from executable files will be backed by the executable, and will thus be shared by default. Therefore my question is there any reason for not using mlockall(2) in the parent process on systems that support it and when the parent httpd is started as root (mlock* works only within root owned processes). I don't think mlockall is appropriate for something with the heft of mod_perl. Why are the pages being swapped out in the first place? Presumably there's a valid reason. Well, the system coming close to zero of real memory available. The parent process starts swapping like crazy because most of its pages are LRU, slowing the whole system down and if the load doesn't go away, the system takes a whirl down to a halt. Doing mlockall on your mod_perl would result in restricting the memory available to the rest of the system. Whatever is causing mod_perl to page out would then start thrashing. Worse, since mlockall will lock down mod_perl pages indiscriminately, the resulting thrashing will probably be even worse than what they're seeing right now. Possible, I've never tried this myself and therefore asked. Someone has suggested using P_SYSTEM flag which is supposed to tell the system not to page out certain pages, but seems to be a *BSD thingy. I've also seen madvise(2) - give advice about use of memory but again no personal experience here. _ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://ticketmaster.com http://apacheweek.com http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: APR_LOCK_DEFAULT ordering
Jeff Trawick wrote: Jim Jagielski [EMAIL PROTECTED] writes: Are people happy with the priority order of the accept mutex? Right now it's flock - sysvsem - fcntl - pthread. I think it should be pthread - sysvsem - fcntl - flock, which is what 1.3 has... I realize everybody has jumped in and +1-ed you, but I don't see how you can compare the two orders which you stated above. I would express the current APR default selection (configure.in, line 1303) this way: 1st choice : fcntl 2nd: pthread 3rd: flock 4th: SysV sem But proc_mutex.c has the below: case APR_LOCK_DEFAULT: #if APR_USE_FLOCK_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_flock_methods; #elif APR_USE_SYSVSEM_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_sysv_methods; #elif APR_USE_FCNTL_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_fcntl_methods; #elif APR_USE_PROC_PTHREAD_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_proc_pthread_methods; #else return APR_ENOTIMPL; #endif And APR_LOCK_DEFAULT is what ap_accept_lock_mech is set to in mpm_common... -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: APR_LOCK_DEFAULT ordering
Jim Jagielski [EMAIL PROTECTED] writes: Jeff Trawick wrote: Jim Jagielski [EMAIL PROTECTED] writes: Are people happy with the priority order of the accept mutex? Right now it's flock - sysvsem - fcntl - pthread. I think it should be pthread - sysvsem - fcntl - flock, which is what 1.3 has... I realize everybody has jumped in and +1-ed you, but I don't see how you can compare the two orders which you stated above. I would express the current APR default selection (configure.in, line 1303) this way: 1st choice : fcntl 2nd: pthread 3rd: flock 4th: SysV sem But proc_mutex.c has the below: case APR_LOCK_DEFAULT: #if APR_USE_FLOCK_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_flock_methods; #elif APR_USE_SYSVSEM_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_sysv_methods; If that were #if apr_HAVE_flock_serialize instead of apr_USE_flock_serialize then I'd believe you. But since it is referencing apr_USE_flock_serialize it is making use of the choices made at configure time, and thus the order of the logic here under APR_LOCK_DEFAULT is irrelevant. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: performance: using mlock(2) on httpd parent process
On Thu, Mar 21, 2002 at 02:14:04AM +0800, Stas Bekman wrote: But if the system needs to page things out, most of the parent process's pages will be scheduled to go first, no? So we are talking about a constant page-in/page-out from/to the parent process as a performance degradation rather than memory unsharing. Am I correct? [...] Well, the system coming close to zero of real memory available. The parent process starts swapping like crazy because most of its pages are LRU, slowing the whole system down and if the load doesn't go away, the system takes a whirl down to a halt. FWIW, Solaris 8 introduced priority paging, which basicly means that pages that are non-executable will be swapped out earlier than higher-priority executable pages. This of course introduces nasty little tricks like marking a data file as executable just to ensure that it gets priority paging. I'm not sure how this affects mod_perl and it's data is runtime-compiled bytecode situation, but it means that we get it for free w/ httpd and all its dependent libraries (and DSOs). -aaron p.s. It can be enabled on Solaris 7, it's just not on by default.
Re: APR_LOCK_DEFAULT ordering
Jeff Trawick wrote: But proc_mutex.c has the below: case APR_LOCK_DEFAULT: #if APR_USE_FLOCK_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_flock_methods; #elif APR_USE_SYSVSEM_SERIALIZE new_mutex-inter_meth = apr_proc_mutex_unix_sysv_methods; If that were #if apr_HAVE_flock_serialize instead of apr_USE_flock_serialize then I'd believe you. By gum, you're right. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Jeff Trawick [EMAIL PROTECTED] writes:
Jim Jagielski [EMAIL PROTECTED] writes:
Jeff Trawick wrote:
Bring 2.0 up to parity, a bit, with how much info we provide to
the admin regarding valid values for AcceptMutex. Should also
tell 'em what default actually maps to, but that can wait.
I don't think this works as you desired.
I just did an update (no diffs in this tree) and for a config with no
AcceptMutex I get this at startup:
[notice] AcceptMutex: Valid accept mutexes for this platform and MPM
are: default, fcntl, sysvsem, pthread.
Here is a prototype showing the help text displayed via httpd -L,
keeping the use of AP_INIT_TAKE1.
It compiles/runs with no warnings or errors with gcc -Wall and xlc
(IBM's fairly anal compiler for AIX).
Index: server/core.c
===
RCS file: /home/cvspublic/httpd-2.0/server/core.c,v
retrieving revision 1.164
diff -u -r1.164 core.c
--- server/core.c 20 Mar 2002 02:05:42 - 1.164
+++ server/core.c 20 Mar 2002 18:39:39 -
@@ -2802,6 +2802,18 @@
* The AllowOverride of Fileinfo allows webmasters to turn it off
*/
+#ifdef AP_MPM_WANT_SET_ACCEPT_LOCK_MECH
+static const char accept_mutex_help_string[] =
+default
+#if APR_HAS_SYSVSEM_SERIALIZE
+, sysvsem
+#endif
+#if APR_HAS_FCNTL_SERIALIZE
+, fcntl
+#endif
+;
+#endif
+
static const command_rec core_cmds[] = {
/* Old access config file commands */
@@ -3003,7 +3015,7 @@
#endif
#ifdef AP_MPM_WANT_SET_ACCEPT_LOCK_MECH
AP_INIT_TAKE1(AcceptMutex, ap_mpm_set_accept_lock_mech, NULL, RSRC_CONF, \
- The system mutex implementation to use for the accept mutex),
+ accept_mutex_help_string),
#endif
{ NULL }
};
--
Jeff Trawick | [EMAIL PROTECTED]
Born in Roswell... married an alien...
Re: cvs commit: httpd-2.0/srclib/pcre config.hw pcre.hw
On 20 Mar 2002 [EMAIL PROTECTED] wrote: jwoolley02/03/20 11:04:25 Modified:srclib/pcre config.hw pcre.hw Log: Fix the build on Win32 by syncing up the .hw headers This seems to fix the build as far as I can tell... I imagine someone will yell if it's still broken. :) Thanks, Cliff -- Cliff Woolley [EMAIL PROTECTED] Charlottesville, VA
Re: performance: using mlock(2) on httpd parent process
Aaron Bannert wrote: On Thu, Mar 21, 2002 at 02:14:04AM +0800, Stas Bekman wrote: But if the system needs to page things out, most of the parent process's pages will be scheduled to go first, no? So we are talking about a constant page-in/page-out from/to the parent process as a performance degradation rather than memory unsharing. Am I correct? [...] Well, the system coming close to zero of real memory available. The parent process starts swapping like crazy because most of its pages are LRU, slowing the whole system down and if the load doesn't go away, the system takes a whirl down to a halt. FWIW, Solaris 8 introduced priority paging, which basicly means that pages that are non-executable will be swapped out earlier than higher-priority executable pages. This of course introduces nasty little tricks like marking a data file as executable just to ensure that it gets priority paging. priority paging is more to do with the filesystem cache than anything else. in low memory conditions it start freeing pages used by the file cache first. without it you could swap your machine to death by copying a 1 gig file around the place. I'm not sure how this affects mod_perl and it's data is runtime-compiled bytecode situation, but it means that we get it for free w/ httpd and all its dependent libraries (and DSOs). -aaron p.s. It can be enabled on Solaris 7, it's just not on by default.
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
I think I have a version that addresses the concerns... running out to a quick meeting, so I'll test and post a bit later... -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Jim Jagielski [EMAIL PROTECTED] writes: Ideally, this should be localized... other entities may want that info and we don't want to have implementations spread out... But the code passes check here as well. I'll add this in once the lock stuff settles, unless it bothers you enough that you'd like it folded in now :) No big hurry... I just didn't think it was intended to be the way it is currently committed. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: performance: using mlock(2) on httpd parent process
On Wed, Mar 20, 2002 at 11:19:44AM -0800, Ian Holsman wrote: FWIW, Solaris 8 introduced priority paging, which basicly means that pages that are non-executable will be swapped out earlier than higher-priority executable pages. This of course introduces nasty little tricks like marking a data file as executable just to ensure that it gets priority paging. priority paging is more to do with the filesystem cache than anything else. in low memory conditions it start freeing pages used by the file cache first. without it you could swap your machine to death by copying a 1 gig file around the place. We are talking about a low memory condition, when the scanner is looking for a page to swap out for whatever reason (including filesystem activity). Before priority paging it would select a LRU page, which could very likely be the program that is triggering the page fault. With priority paging enabled, a non-executable page will be selected for swapping before an executable one. Therefore the httpd binary will remain in hard-wired memory longer than the anonymous pages it is operating on (including the COW pages inherited in the children and any filesystem cache pages). Without priority paging an httpd process that required access to more memory than physically present on the system could potentially cause itself to be paged out and then immediate back in as soon as the next instruction tripped a page fault. This would result in a huge amount of thrashing, for even something as simple as copying a file (as with your example). So my point is we don't have to do mlock() on solaris8 (or 7 with p.p. enabled) since we get similiar or sufficient behavior already. -aaron
Re: this a.m. how do we get a cleared scoreboard after apachectl restart?
Jeff Trawick wrote: If there is no such logic right now: well, there is ap_cleanup_scoreboard. But it seems to have two problems w.r.t. non-graceful restart: * it won't run on a restart with the pool change. But code in the MPMs kills the cleanup on non-graceful. Will that make things break at shutdown time? * for some period of time before the pool change, it wasn't always effective in preventing the new generation from seeing old generation data, as you reported. Is the WinNT MPM gonna puke if ap_create_scoreboard() clears it (but preserving running_generation) in the already-have-a-scoreboard path? dunno. Aren't you concerned about the Unix MPMs? What would prevent about-to-die old generation processes from writing into the scoreboard after it is cleared? Greg
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
[EMAIL PROTECTED] writes: trawick 02/03/20 11:53:19 Modified:server/mpm/worker worker.c Log: write a debug message to the log when we're stuck in the sicko state of trying to take over scoreboard slots that aren't going to be released (we could also be stalled while taking over slots if a thread in child gracefully terminating is serving a long-running request) note that this sicko state is really easy to get to today given that the scoreboard isn't cleared on a non-graceful restart... just do apachectl restart... but that will be fixed soon enough and we're left with something more subtle that seems to have been present for a while -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: this a.m. how do we get a cleared scoreboard after apachectl restart?
Greg Ames [EMAIL PROTECTED] writes: If there is no such logic right now: well, there is ap_cleanup_scoreboard. But it seems to have two problems w.r.t. non-graceful restart: * it won't run on a restart with the pool change. But code in the MPMs kills the cleanup on non-graceful. Will that make things break at shutdown time? Killing the cleanup is currently a no-op since it is searching the wrong pool... If the scoreboard is going to be associated with the process pool then we shouldn't be killing the cleanup anyway, right? * for some period of time before the pool change, it wasn't always effective in preventing the new generation from seeing old generation data, as you reported. Is the WinNT MPM gonna puke if ap_create_scoreboard() clears it (but preserving running_generation) in the already-have-a-scoreboard path? dunno. Aren't you concerned about the Unix MPMs? What would prevent about-to-die old generation processes from writing into the scoreboard after it is cleared? Since I don't see any answers on WinNT MPM I'm going to forget about it and assume that clearing the scoreboard in ap_create_scoreboard() is good for everybody. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
anybody seeing socket-related segfaults today?
Throughout today I've been seeing very intermittent regression failures on AIX. The segfault happens when trying to get the IP address string from a socket addr. core_create_conn() calls apr_socket_addr_get(), which returns APR_SUCCESS. But somehow we have NULL for the returned socket address so apr_sockaddr_ip_get() bombs. It is intermittent, doesn't seem to matter what kind of request, and I've only seen it on a couple of AIX boxes. Probably a pool misuse of some sort :) The earliest I saw it happen was 8:00 EST today, but prior to that the server wouldn't build on AIX for some hours, so I don't know when the problem was introduced/exposed. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
s-server_hostname null after WinNT MPM restart?
Is anyone else seeing this? -aaron
Re: anybody seeing socket-related segfaults today?
Could this change have interfered with Unix? Modified:server listen.c Log: Here's the patch that really sucks. old_listeners points to an array of apr_socket objects already destroyed by their cleanups, and in any case they now live in invalid memory. Extend their lifetimes. This implies that the process pool grows on every restart for no good reason. One possible solution is to let the old pconf survive until the new pconf is alive. Another is to create the listeners in a subpool of process-pool, destroyed after the old_listeners are closed. Either which way, a better solution exists, but this closes the immediate bug. [How haven't we been segfaulting in unix on restarts before this patch, gurus?] Revision ChangesPath 1.77 +4 -5 httpd-2.0/server/listen.c -aaron On Wed, Mar 20, 2002 at 04:07:23PM -0500, Jeff Trawick wrote: Throughout today I've been seeing very intermittent regression failures on AIX. The segfault happens when trying to get the IP address string from a socket addr. core_create_conn() calls apr_socket_addr_get(), which returns APR_SUCCESS. But somehow we have NULL for the returned socket address so apr_sockaddr_ip_get() bombs. It is intermittent, doesn't seem to matter what kind of request, and I've only seen it on a couple of AIX boxes. Probably a pool misuse of some sort :) The earliest I saw it happen was 8:00 EST today, but prior to that the server wouldn't build on AIX for some hours, so I don't know when the problem was introduced/exposed.
Re: cvs commit: httpd-2.0/server/mpm/worker worker.c
Jeff Trawick wrote: No big hurry... I just didn't think it was intended to be the way it is currently committed. We needed some way to indicate valid mutex methods, and the change in TAKE1 didn't allow the quote concats. Using a const char array is a good idea, but I was loath to use a nasty global. But to allow for it in help, that's what we got. I'll ignore perchild for the moment and work on adding the mutex name stuff to the struct and a mechanism to grab it. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: s-server_hostname null after WinNT MPM restart?
At 03:13 PM 3/20/2002, you wrote: Is anyone else seeing this? That makes perfect sense, although it 'failed to fail' for me when coding yesterday. In fact, early, it occured to me that this could occur - but I was so d*mned tired it then slipped my mind. Consider; we don't close the listeners now [with the pool cleanup] because we moved them to the process pool. They are alert and listening. Now we try to grab the sockets in the server restart open_logs hook, and we don't set the socket as sharable [on Win32, specificially] until we successfully grab them for our exclusive use. [That was the old bug that we could share IIS or anyone elses' sockets due to Win32's lame security model.] Well that patch has come back to bite us [on win32.] I suspect that in the Win32 case, we don't comment out the first SO_REUSEADDR block, but test for (old_listeners) - and set the listeners reuse up front when this is a restart case. If we find (!old_listeners) - then postpone the SO_REUSEADDR flag until after our successful first socket-grab attempt. BTW - there is a not-so-subtle race condition here with a traditional server restart [the Service Control Applet's stop-start button.] The main server stops, but the child is still in it's death throughs. The new parent process can't grab the socket until the child -really- utters it's last whimpers and expires. So an ungraceful-restart, e.g. net stop apache2 net start apache2 can fail today, and I don't have a trivial solution. Bill
Re: PCRE status?
We're upgraded to the latest PCRE now (thanks for Cliff to fixing the Win32 build). Thanks. I checked with the PCRE maintainer and learned that the next release is several months away. In the meantime, that leaves me with two options for speeding up ap_regexec(): * Commit a change to the PCRE regexec() function (the same change that I've submitted for the next release of PCRE) into the Apache copy of PCRE for now. Yeah, do that, but surround it with comments saying that it is a change that has been submitted to the maintainer. * Or change ap_regexec() to bypass regexec() and call the PCRE native regexp exec function directly. (The PCRE regexec() is a thin wrapper around pcre_exec(), so this shouldn't be difficult.) Ummm, do we always use our PCRE for the regexec library? I was under the impression that it is configurable. Roy
Re: anybody seeing socket-related segfaults today?
Jeff Trawick [EMAIL PROTECTED] writes: Throughout today I've been seeing very intermittent regression failures on AIX. The segfault happens when trying to get the IP address string from a socket addr. core_create_conn() calls apr_socket_addr_get(), which returns APR_SUCCESS. But somehow we have NULL for the returned socket address so apr_sockaddr_ip_get() bombs. The immediate cause of the problem is that ap_queue_pop() is returning EINVAL and worker_thread() didn't react to that and instead tried to process the would-be socket. I suspect that the EINVAL from ap_queue_pop() is from trying to use an invalid (cleaned up?) pthread mutex. AIX tends to notice errors on mutexes and fail the call rather than venturing into unpredictable behavior. I just committed a change to worker to not process the socket if rv != APR_SUCCESS. Previously we avoided processing the socket if rv == APR_EINTR or csd is NULL. (But no logic in ap_queue_pop() or caller to set csd to NULL on the EINVAL error!) I recall the fix to check for csd == NULL being very helpful a couple of months back. I hope rv was non-zero in that case (i.e., I hope that problem is still fixed)! -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: 1.3.24-dev new warning...
OK, I'll fix and commit this tonight. Chuck On Friday, April 19, 2002, at 02:07 PM, William A. Rowe, Jr. wrote: At 10:03 AM 3/19/2002, you wrote: At 10:42 PM -0600 3/7/02, William A. Rowe, Jr. wrote: C:\clean\apache-1.3\src\modules\proxy\proxy_util.c(565) : warning C4018: '' : signed/unsigned mismatch n = ap_bread(f, buf, MIN(buf_size, len - total_bytes_rcvd)); I had posted a patch regarding this a bit ago... Any feedback? I was not able to recreate the exact error messages locally on any of my machines, so I couldn't see if it solved the specific problem (without creating more :) ). Doh! MIN() is implemented in terms of a terniary, and those are very broken in terms of the result type on win32. Casting the MIN() back to the arg type of ap_bread() should be sufficient.
Re: 1.3.24-dev new warning...
There are two other places in mod_proxy the MIN macro gets used. Shouldn't those get cast to the desired return type as well? This one's a time_t: proxy_cache.c line 1175 maxage = MIN(maxage_req, maxage_cresp); and this one's a size_t: proxy_util.c line 734 strncpy(valbuf, list, MIN(p-list, sizeof(valbuf)-1)); Chuck On Friday, April 19, 2002, at 02:07 PM, William A. Rowe, Jr. wrote: At 10:03 AM 3/19/2002, you wrote: At 10:42 PM -0600 3/7/02, William A. Rowe, Jr. wrote: C:\clean\apache-1.3\src\modules\proxy\proxy_util.c(565) : warning C4018: '' : signed/unsigned mismatch n = ap_bread(f, buf, MIN(buf_size, len - total_bytes_rcvd)); I had posted a patch regarding this a bit ago... Any feedback? I was not able to recreate the exact error messages locally on any of my machines, so I couldn't see if it solved the specific problem (without creating more :) ). Doh! MIN() is implemented in terms of a terniary, and those are very broken in terms of the result type on win32. Casting the MIN() back to the arg type of ap_bread() should be sufficient.
Re: performance: using mlock(2) on httpd parent process
Scott Hess wrote: On Thu, 21 Mar 2002, Stas Bekman wrote: On Wed, 20 Mar 2002, Stas Bekman wrote: mod_perl child processes save a lot of memory when they can share memory with the parent process and quite often we get reports from people that they lose that shared memory when the system decides to page out the parent's memory pages because they are LRU (least recently used, the algorithm used by many memory managers). I'm fairly certain that this is not an issue. If a page was shared COW before being paged out, I expect it will be shared COW when paged back in, at least for any modern OS. But if the system needs to page things out, most of the parent process's pages will be scheduled to go first, no? So we are talking about a constant page-in/page-out from/to the parent process as a performance degradation rather than memory unsharing. Am I correct? The system is going to page out an approximation of the least-recently-used pages. If the children are using those pages, then they won't be paged out, regardless of what the parent is doing. [If the children _aren't_ using those pages, then who cares?] mod_perl users do care. let's say you've preloaded a bunch of modules which aren't in use now but may be used any moment. Ideally we don't want any of the pages to be swapped out. Of course this is all possible if the demand does go over the supply. I'm just repeating the reports posted to the mod_perl list. I've never seen such a problem myself, since I try hard to have close to zero swap usage. :-). In my experience, you can get some really weird stuff happening when you start swapping mod_perl. It seems to be stable in memory usage, though, so long as you have MaxClients set low enough that your maximum amount of committed memory is appropriate. Also, I've seen people run other heavyweight processes, like mysql, on the same system, so that when the volume spikes, mod_perl spikes AND mysql spikes. A sure recipe for disaster. Well, we (mod_perl users) use tools like Apache::GTopLimit and Apache::SizeLimit which monitor the child process' memory and will kill the process after a request if the memory size crossed a total memory used limit, or lost too much of a shared memory. I think that's where the problem lays, the memory usage tools report that the sharing went down and the process gets killed when the parent's pages are getting paged out. Even if in reality the sharing is still there. The memory reporting tools don't reflect the reality. [Yes, please let me know your methodology for testing this] OK, two programs. bigshare.c: thanks Scott, I've forwarded this code to the folks who are having the trouble with. and it'll stay in the archives for others to re-use. Therefore my question is there any reason for not using mlockall(2) in the parent process on systems that support it and when the parent httpd is started as root (mlock* works only within root owned processes). I don't think mlockall is appropriate for something with the heft of mod_perl. Why are the pages being swapped out in the first place? Presumably there's a valid reason. Well, the system coming close to zero of real memory available. The parent process starts swapping like crazy because most of its pages are LRU, slowing the whole system down and if the load doesn't go away, the system takes a whirl down to a halt. I can think of a couple possible causes. One is the MaxClients setting. Say you have MaxClients set to 50, but on a standard day you never need more than 15 servers. Then you get listed on slashdot. As you pass, oh, 30 simultaneous servers, you start thrashing, so requests take longer to process, so you immediately spike to your MaxClients of 50 and the server goes right down the drain. If you shut things down and start them back up, it's likely you'll immediately spike to 50 again, and back down the drain it goes. I've found that it's _super_ important to make certain you've pushed mod_perl to MaxClients under your standard conditions. Once you start swapping, you're doomed, unless the traffic was a very short spike. Another possible cause is that you have another heavyweight server running on the same server. As I indicated above, I've seen people do this with things like mysql. Since high mod_perl traffic implies high mysql traffic, it's just like having MaxClients set too high, but twice as bad! Another possible cause is that the OS is aggressively grabbing pages for the filesystem cache. It's possible that tuning down the size of the filesystem cache would be appropriate - many webservers have a very large maximum amount of data they might server, but a very small working set. Really, though, all of this really tends to come down to MaxClients. The database load is proportional to the number of clients, the filesystem load is proportional to the number of clients, everything is proportional to the number of
Re: anybody seeing socket-related segfaults today?
Jeff Trawick [EMAIL PROTECTED] writes: Jeff Trawick [EMAIL PROTECTED] writes: Throughout today I've been seeing very intermittent regression failures on AIX. The segfault happens when trying to get the IP address string from a socket addr. core_create_conn() calls apr_socket_addr_get(), which returns APR_SUCCESS. But somehow we have NULL for the returned socket address so apr_sockaddr_ip_get() bombs. The immediate cause of the problem is that ap_queue_pop() is returning EINVAL and worker_thread() didn't react to that and instead tried to process the would-be socket. I suspect that the EINVAL from ap_queue_pop() is from trying to use an invalid (cleaned up?) pthread mutex. AIX tends to notice errors on mutexes and fail the call rather than venturing into unpredictable behavior. Yep, the mutex has already been cleaned up. It is the mutex unlock operation that fails. This is termination (ungraceful). We don't wait for worker threads to terminate; sometimes the main thread has cleaned up pchild and bailed by the time the worker threads get dispatched from the interrupt-all and then release the mutex. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: dumb questions on a couple of the current 2.0 showstoppers
Paul J. Reder wrote: Cliff, I know you are busy, is there something I can do to help with this since the pools stuff is in now? Does it still just need to have the malloc replaced and the code tested? I'm happy to help in any way possible. Me too. Cliff, what's your recommendation on how/when/if we can help get the bucket free lists into the code base? Thanks, --Brian
[STATUS] (apache-1.3) Wed Mar 20 23:45:04 EST 2002
APACHE 1.3 STATUS: -*-text-*- Last modified at [$Date: 2002/03/01 23:35:27 $] Release: 1.3.24-dev: In development. Jim proposes to TR around Mar 04 or so because of the Solaris pthread mutex fix and other fixes. 1.3.23: Tagged Jan 21, 2002. 1.3.22: Tagged Oct 8, 2001. Announced Oct 12, 2001. 1.3.21: Not released. (Pulled for htdocs/manual config mismatch. t/r Oct 5, 2001) 1.3.20: Tagged and rolled May 15, 2001. Announced May 21, 2001. 1.3.19: Tagged and rolled Feb 26, 2001. Announced Mar 01, 2001. 1.3.18: Tagged and rolled Not released. (Pulled because of an incorrect unescaping fix. t/r Feb 19, 2001) 1.3.17: Tagged and rolled Jan 26, 2001. Announced Jan 29, 2001. 1.3.16: Not released. (Pulled because of vhosting bug. t/r Jan 20, 2001) 1.3.15: Not released. (Pulled due to CVS dumping core during the tagging when it reached src/os/win32/) 1.3.14: Tagged and Rolled Oct 10, 2000. Released/announced on the 13th. 1.3.13: Not released. (Pulled in the first minutes due to a Netware build bug) 1.3.12: Tagged and rolled Feb. 23, 2000. Released/announced on the 25th. 1.3.11: Tagged and rolled Jan. 19, 2000. Released/announced on the 21st. 1.3.10: Not released. (Pulled at last minute due to a build bug in the MPE port) 1.3.9: Tagged and rolled on Aug. 16. Released and announced on 19th. 1.3.8: Not released. 1.3.7: Not released. 1.3.6: Tagged and rolled on Mar. 22. Released and announced on 24th. 1.3.5: Not released. 1.3.4: Tagged and rolled on Jan. 9. Released on 11th, announced on 12th. 1.3.3: Tagged and rolled on Oct. 7. Released on 9th, announced on 10th. 1.3.2: Tagged and rolled on Sep. 21. Announced and released on 23rd. 1.3.1: Tagged and rolled on July 19. Announced and released. 1.3.0: Tagged and rolled on June 1. Announced and released on the 6th. 2.0 : In alpha development, see httpd-2.0 repository RELEASE SHOWSTOPPERS: RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP: * htpasswd.c and htdigest.c use tmpnam()... consider using mkstemp() when available. Message-ID: [EMAIL PROTECTED] Status: * Dean's unescaping hell (unescaping the various URI components at the right time and place, esp. unescaping the host name). Message-ID: [EMAIL PROTECTED] Status: * Martin observed a core dump because a ipaddr_chain struct contains a NULL-server pointer when being dereferenced by invoking httpd -S. Message-ID: [EMAIL PROTECTED] Status: Workaround enabled. Clean solution can come after 1.3.19 * long pathnames with many components and no AllowOverride None Workaround is to define Directory / with AllowOverride None, which is something all sites should do in any case. Status: Marc was looking at it. * Ronald Tschalär's patch to mod_proxy to allow other modules to set headers too (needed by mod_auth_digest) Message-ID: [EMAIL PROTECTED] Status: Available Patches (Most likely, will be ported to 2.0 as appropriate): * Support for daemontools (already in 2.0): http://www.apache.org/dist/httpd/contrib/patches/1.3/daemontools.patch Message-ID: [EMAIL PROTECTED] Status: * A rewrite of ap_unparse_uri_components() by Jeffrey W. Baker [EMAIL PROTECTED] to more fully close some segfault potential. Message-ID: Pine.LNX.4.21.0102102350060.6815-20@desktop Status: Jim +1 (for 1.3.19), Martin +0 * Patch from C. Bottelier [EMAIL PROTECTED] to run Apache without daemonizing the parent process. PR#7040 Status: fanf +1 (except it needs docs) * Andrew Ford's patch (1999/12/05) to add absolute times to mod_expires Message-ID: [EMAIL PROTECTED] Status: Martin +1, Jim +1, Ken +1 (on concept) * Raymond S Brand's path to mod_autoindex to fix the header/readme include processing so the envariables are correct for the included documents. (Actually, there are two variants in the patch message, for two different ways of doing it.) Message-ID: [EMAIL PROTECTED] Status: Martin +1(concept) * Jayaram's patch (10/27/99) for changes to mod_autoindex Problem 1: AddIcon (alttext,icon) ^^DIRECTORY^^ and AddIcon (alttext,icon) ^^BLANKICON^^ should be able to set the alternate text and icon file for any directory/blankicon in a directory listing. This was not happening because the alternate text for ^^DIRECTORY^^ and ^^BLANKICON^^ were hardcoded to DIR and respectively. Status: resolved in Apache 2.0 Problem 2: - IndexIgnore
[STATUS] (httpd-2.0) Wed Mar 20 23:45:07 EST 2002
APACHE 2.0 STATUS: -*-text-*- Last modified at [$Date: 2002/03/20 16:19:24 $] Release: 2.0.34 : in development 2.0.33 : tagged March 6, 2002. 2.0.32 : released Feburary 16, 2002. 2.0.31 : rolled Feburary 1, 2002. not released. 2.0.30 : tagged January 8, 2002. not rolled. 2.0.29 : tagged November 27, 2001. not rolled. 2.0.28 : released November 13, 2001 2.0.27 : rolled November 6, 2001 2.0.26 : tagged October 16, 2001. not rolled. 2.0.25 : rolled August 29, 2001 2.0.24 : rolled August 18, 2001 2.0.23 : rolled August 9, 2001 2.0.22 : rolled July 29, 2001 2.0.21 : rolled July 20, 2001 2.0.20 : rolled July 8, 2001 2.0.19 : rolled June 27, 2001 2.0.18 : rolled May 18, 2001 2.0.17 : rolled April 17, 2001 2.0.16 : rolled April 4, 2001 2.0.15 : rolled March 21, 2001 2.0.14 : rolled March 7, 2001 2.0a9 : released December 12, 2000 2.0a8 : released November 20, 2000 2.0a7 : released October 8, 2000 2.0a6 : released August 18, 2000 2.0a5 : released August 4, 2000 2.0a4 : released June 7, 2000 2.0a3 : released April 28, 2000 2.0a2 : released March 31, 2000 2.0a1 : released March 10, 2000 Please consult the following STATUS files for information on related projects: * srclib/apr/STATUS * srclib/apr-util/STATUS * docs/STATUS CURRENT RELEASE NOTES: * 34 status: Let's get all API changes and showstoppers in this one. Please. FINAL RELEASE SHOWSTOPPERS: * If any request gets to the core handler, without a flag that this r-filename was tested by dir/file_walk, we need to 500 at the very end of the ap_process_request_internal() processing. This provides authors of older modules better compatibility, while still improving the security and robustness of 2.0. Status: still need to decide where this goes, OtherBill comments... Message-ID: 065701c14526$495203b0$[EMAIL PROTECTED] we need to look at halting this in the 'default handler' case, and that implies pushing the 'handler election' into the request internal processing phase from the run request phase. Jim asks: would a stopgap be something bogus like adding another flag to request_rec ala eos_sent and before we OK, if not set force 500? Jeff says: reviewing the original message and the one follow-up (also from OtherBill) it looks like OtherBill had a good handle on the problem, though I wonder why not just put a simple check in default_handler to see if dir/file_walk has been done (a footprint left by dir/file_walk doesn't have to be in request_rec; a better place is core_request_config) * We need to find out the right place to add the AddOutputFilterByType directive. mod_mime? No. core with fixups? That's what it is now. ap_pass_brigade() hook? As a filter that runs at HTTP_HEADER stage? This needs to be resolved before the next public release. Message-ID: 018701c1b99c$f0f24310$0a01230a@KOJ - First cut of this code has been committed. Needs to be reviewed and documented. Any module that does not use the accessor function will break the server. * API changes planned for 2.0 that should happen before the GA release: * Free lists for bucket allocation * We do not properly substitute the prefix-variables in the configuration scripts or generated-configs. (i.e. if sysconfdir is etc, httpd-std.conf points to conf.) Aaron says: This is not a showstopper, these problems have existed for as long as I can remember. It would be nice to fix them but they are not new. * Address popular PRs * Win32 doesn't install as service correctly [9863, 9914, 9961] * Don't forget to reversion mod_ssl, which is currently listed as mod_ssl/3.0a0. CURRENT VOTES: * Should we always build binaries statically unless otherwise indicated? Message-ID: [EMAIL PROTECTED] +1: Ken -1: Justin, Ian * If the parent process dies, should the remaining child processes gracefully self-terminate. Or maybe we should make it a runtime option, or have a concept of 2 parent processes (one being a hot spare). See: Message-ID: [EMAIL PROTECTED] Self-destruct: Ken, Martin Not self-destruct: BrianP, Ian, Cliff, BillS Make it runtime configurable: Aaron, Jim, Justin Have 2 parents: +1: Jim -1: Justin +0: Martin (while standing by, could it do something useful?) * Make the worker MPM the default MPM for threaded Unix boxes. +1: Justin, Ian, Cliff -0: Aaron (premature decision,
Re: performance: using mlock(2) on httpd parent process
On Wed, 20 Mar 2002, Stas Bekman wrote: mod_perl child processes save a lot of memory when they can share memory with the parent process and quite often we get reports from people that they lose that shared memory when the system decides to page out the parent's memory pages because they are LRU (least recently used, the algorithm used by many memory managers). I'm fairly certain that this is not an issue. If a page was shared COW before being paged out, I expect it will be shared COW when paged back in, at least for any modern OS. [To verify that I wasn't talking through my hat, here, I just verified this using RedHat 7.2 running kernel 2.4.9-21. If you're interested in my methodology, drop me an email.] I believe that this applies to all httpd modules and httpd itself, the more we can share the less memory resources are needed, and usually it leads to a better performance. I'm absolutely _certain_ that unmodified pages from executable files will be backed by the executable, and will thus be shared by default. Therefore my question is there any reason for not using mlockall(2) in the parent process on systems that support it and when the parent httpd is started as root (mlock* works only within root owned processes). I don't think mlockall is appropriate for something with the heft of mod_perl. Why are the pages being swapped out in the first place? Presumably there's a valid reason. Doing mlockall on your mod_perl would result in restricting the memory available to the rest of the system. Whatever is causing mod_perl to page out would then start thrashing. Worse, since mlockall will lock down mod_perl pages indiscriminately, the resulting thrashing will probably be even worse than what they're seeing right now. Later, scott
