Re: double mod_perl initialization
Been away from my computer for several days :( Doesn't look like anyone is going to answer this. Maybe you missed some of the emails that Clinton Gormley and I were exchanging a while back. I've been using 'sudo apachectl graceful' on apache 2.2.11 and 2.2.13 with mod_perl 2.0.4 and 2.0.5 SVN for many months now and have not noticed any problems. Of course you might be using some feature or module or ??? that my site doesn't, which would cause some problems. Fancy SSL use and changing certificates were mentioned as possibilities. If you look thru the change logs for the various apache versions you can see quite a few improvements for graceful and other restarts (there was one in 2.2.12). So why not give them a chance, and if you have problems come back here and tell us? Or look into them a little and come back and tell us even more? And if graceful restart works OK for you, please send an email to say that. The elimination of superstition is an absolute good. cmac On Oct 2, 2009, at 3:38 PM, Jonathan Swartz wrote: On Oct 2, 2009, at 3:25 PM, Perrin Harkins wrote: On Fri, Oct 2, 2009 at 5:54 PM, Jonathan Swartz wrote: Ok, one more question. Disregarding graceful for the moment - is HUP completely reliable with mod_perl at this point, or is there still a reason (as there once supposedly was) to do a full server stop and start? The problem, at least in the 1.x series, was that the parent process and the perl interpreter never restart. It just re-reads the config file and runs through startup again. Right. Ok, in a *relatively modern Apache/mod_perl 2.x*, is there still a reason to do a full server stop and start? Thanks Jon
Re: double mod_perl initialization
On Oct 2, 2009, at 3:25 PM, Perrin Harkins wrote: On Fri, Oct 2, 2009 at 5:54 PM, Jonathan Swartz wrote: Ok, one more question. Disregarding graceful for the moment - is HUP completely reliable with mod_perl at this point, or is there still a reason (as there once supposedly was) to do a full server stop and start? The problem, at least in the 1.x series, was that the parent process and the perl interpreter never restart. It just re-reads the config file and runs through startup again. Right. Ok, in a *relatively modern Apache/mod_perl 2.x*, is there still a reason to do a full server stop and start? Thanks Jon
Re: double mod_perl initialization
On Fri, Oct 2, 2009 at 5:54 PM, Jonathan Swartz wrote: > Ok, one more question. Disregarding graceful for the moment - is HUP > completely reliable with mod_perl at this point, or is there still a reason > (as there once supposedly was) to do a full server stop and start? The problem, at least in the 1.x series, was that the parent process and the perl interpreter never restart. It just re-reads the config file and runs through startup again. - Perrin
Re: double mod_perl initialization
On Sep 25, 2009, at 10:48 AM, Clinton Gormley wrote: If I don't ever plan to use graceful restarts, and I believe that smaller restart times are an unqualified Good, is there any reason why I shouldn't ALWAYS use a script like the above? And is there any way to avoid PerlModule modules from being loaded twice? I do something pretty similar. I do the double load on 'start', single load on 'restart', and I don't load the app on 'stop' / 'graceful- stop'. Ok, one more question. Disregarding graceful for the moment - is HUP completely reliable with mod_perl at this point, or is there still a reason (as there once supposedly was) to do a full server stop and start? I've been doing stop and start reflexively for years. If it's unnecessary, I wish I could have those seconds back. :) Jon
Re: double mod_perl initialization
You can find changes about graceful restarts in many of the version release notes in apache2. My memory is that 2.2.11 was OK for me. Letter -Os means "optimize for size": http://gcc.gnu.org/onlinedocs/gcc-2.95.3/gcc_2.html#SEC10 is the oldest gcc manual I could find, so the option has been around for a long time. On many compilers (including apparently gcc-x86) it also does better on speed, unless you use -O3 which can inline your memory size up. You want to get a compiler guy annoyed, suggest that his compiler's "optimize for size" is faster than its "opt for speed". For building perl, you add '-Doptimize=-Os' to the Configure command line. For building Apache, you put '-Os' in CFLAGS at configure time (possibly with other CFLAGS you may like). mod_perl inherits it from perl or apache. Other XS (C) modules inherit it from perl. You can see either -O or -g (debugging) in most cc commands. I tried -Os last week on my one CPAN module (ExtUtils::MakeMaker suggests it). It took the size down between 10% and 15%. The perl binary shrank in that range too. The impact on the httpd process size is less, but there's lots of data in there. So far there's also less in my error_log, but that's probably coincidence :-) Enjoy, cmac On Sep 26, 2009, at 10:36 AM, Clinton Gormley wrote: You are 4 releases behind. Download 2.2.13 and I bet graceful will work for you. (Bet's off if you have something systemically difficult w/r/t ssl.) It appears that you may be right: Changes with Apache 2.2.12 *) prefork: Fix child process hang during graceful restart/stop in configurations with multiple listening sockets. PR 42829. [Joe Orton, Jeff Trawick] Nice! Will try it out when I have a chance. Throw in a mod_perl 2.0.5 from SVN: I've never seen such performance as the system I built with 'optimize -Os' last night. What does this do and how are you specifying it? I don't see any mention of it in the docs for mod_perl or perl. Is that zero-s or oh-s?
Re: double mod_perl initialization
> You are 4 releases behind. Download 2.2.13 and I bet graceful will > work for you. (Bet's off if you have something systemically difficult > w/r/t ssl.) It appears that you may be right: Changes with Apache 2.2.12 *) prefork: Fix child process hang during graceful restart/stop in configurations with multiple listening sockets. PR 42829. [Joe Orton, Jeff Trawick] Nice! Will try it out when I have a chance. > > Throw in a mod_perl 2.0.5 from SVN: I've never seen such performance > as the system I built with 'optimize -Os' last night. What does this do and how are you specifying it? I don't see any mention of it in the docs for mod_perl or perl. Is that zero-s or oh-s? ta clint
Re: double mod_perl initialization
On Sep 26, 2009, at 4:17 AM, Clinton Gormley wrote: I'm using apache 2.2.9, mod_ssl 2.2.9 and mod_perl 2.04, and I still have problems with graceful-restart. You are 4 releases behind. Download 2.2.13 and I bet graceful will work for you. (Bet's off if you have something systemically difficult w/r/t ssl.) Throw in a mod_perl 2.0.5 from SVN: I've never seen such performance as the system I built with 'optimize -Os' last night. In my setup, I know that it can do this, and so I'd rather avoid that double init. I would too. I was defending graceful restart, not double init. cmac
Re: double mod_perl initialization
> But at some point, several iterative fixes by the Apache crew had > succeeded such that 'apachectl graceful' worked better than my > 'graceful' script in some way. > > It was sufficiently long ago that I've forgotten in what way, but the > memory is clear enough that I think Messrs. Swartz and Clint are > beating a dead bug. > I'm using apache 2.2.9, mod_ssl 2.2.9 and mod_perl 2.04, and I still have problems with graceful-restart. It works under certain conditions, but (i think) combined with mod_ssl, my httpd children hang around indefinitely, unless I set GracefulShutdownTimeout. For this reason I use: apachectl graceful-stop; apachectl start > Double initialization does not prevent problems with restart > (graceful or otherwise) for most configuration or perl problems. > Isn't that because by the time the root httpd starts the first (re) > initialization it has already notified the children (that use the old > configuration and code) to go away? So if restart is the reason for > double initialization, by all means let's encourage the Apache folks > to reconsider the idea, and work around it until they do. Because apache (normally) starts up as root, then drops privileges, it tries to confirm that it can restart as not-root. In my setup, I know that it can do this, and so I'd rather avoid that double init. Also, I'd rather avoid setting up my whole application just to shut down apache. So, YMMV, but it works well for me. clint
Re: double mod_perl initialization
On Sep 25, 2009, at 10:36 AM, Jonathan Swartz wrote: As I understand it, the sole purpose of this double initialization is to make sure that graceful restarts will work. However, since I was a young mod_perl lad I've been taught never to rely on graceful restarts, and always to stop/start. This happens right along: * a bug is found in a piece of software, * the bug gets publicized and responded to and remembered by many people, * the people maintaining the software fix the bug, * but for months and years into the future, people remember the bug and keep working around it. Apache's graceful restart worked fine years ago, but started losing badly at some point. I probably wasn't the only person to write a script called 'graceful' to use instead of 'apachectl graceful'. My log rotation script still uses a variant of 'graceful', but that's mostly because it determines when it's OK to process the log. But at some point, several iterative fixes by the Apache crew had succeeded such that 'apachectl graceful' worked better than my 'graceful' script in some way. It was sufficiently long ago that I've forgotten in what way, but the memory is clear enough that I think Messrs. Swartz and Clint are beating a dead bug. I use 'apachectl graceful' a lot, and can't remember a problem for a long time. Of course problems still exist with older versions of apache2. If they work, graceful restarts are a good thing in that they're as nice as possible to users on a production web site. Double initialization does not prevent problems with restart (graceful or otherwise) for most configuration or perl problems. Isn't that because by the time the root httpd starts the first (re) initialization it has already notified the children (that use the old configuration and code) to go away? So if restart is the reason for double initialization, by all means let's encourage the Apache folks to reconsider the idea, and work around it until they do. cmac P.S. sudo's were omitted for brevity.
Re: double mod_perl initialization
Nope, I still get the error in the logs. It just takes a few seconds to show up. On Sep 25, 2009, at 11:20 AM, Clinton Gormley wrote: Hm. No compile errors would be bad. But I put an error in one of my modules (that only gets loaded the second time) and started apache, and got error log output. I wonder what we're doing differently? Shut down apache, then do "apachectl restart" Jon
Re: double mod_perl initialization
> Hm. No compile errors would be bad. But I put an error in one of my > modules (that only gets loaded the second time) and started apache, > and got error log output. I wonder what we're doing differently? Shut down apache, then do "apachectl restart" > > Jon >
Re: double mod_perl initialization
There's only one small gotcha that I've found, and that occurs in this situation: - apache isn't running - you do : apachectl restart - there is a compile time bug, and apache dies The gotcha is that you don't get any STDERR or log output in this situation. However, you're only ever likely to encounter this in development, and you soon recognise the issue. Hm. No compile errors would be bad. But I put an error in one of my modules (that only gets loaded the second time) and started apache, and got error log output. I wonder what we're doing differently? Jon
Re: double mod_perl initialization
> If I don't ever plan to use graceful restarts, and I believe that
> smaller restart times are an unqualified Good, is there any reason why
> I shouldn't ALWAYS use a script like the above? And is there any way
> to avoid PerlModule modules from being loaded twice?
I do something pretty similar. I do the double load on 'start', single
load on 'restart', and I don't load the app on 'stop' / 'graceful-stop'.
I've altered apachectl to set $ENV{APACHE_MODE} to whatever 'command'
was given, eg start / stop / restart etc.
Then my startup script looks like this:
my $apache_mode = $ENV{APACHE_MODE} || '';
die "'graceful-restart' is buggy. "
. "Use 'apachectl graceful-stop; apachectl start' instead."
if $apache_mode eq 'graceful-restart';
return 1
if $apache_mode
&& ( $apache_mode eq 'restart'
&& Apache2::ServerUtil::restart_count() == 1
|| $apache_mode !~ /start/ );
setup_my_application();
There's only one small gotcha that I've found, and that occurs in this
situation:
- apache isn't running
- you do : apachectl restart
- there is a compile time bug, and apache dies
The gotcha is that you don't get any STDERR or log output in this
situation. However, you're only ever likely to encounter this in
development, and you soon recognise the issue.
clint
double mod_perl initialization
I was looking into why our server's restarts take so long, and I
finally remembered that Apache runs its initialization step twice on
startup (http://tinyurl.com/krr25). This means that my startup.pl is
loaded twice, along with any modules that it loads.
So I moved startup.pl to startup_real.pl and put this in startup.pl:
# Only run our real startup script the second time.
#
use Apache2::ServerUtil ();
if (Apache2::ServerUtil::restart_count() > 1) {
require 'startup_real.pl';
}
And it shaved a bunch of time off our restart.
As I understand it, the sole purpose of this double initialization is
to make sure that graceful restarts will work. However, since I was a
young mod_perl lad I've been taught never to rely on graceful
restarts, and always to stop/start.
If I don't ever plan to use graceful restarts, and I believe that
smaller restart times are an unqualified Good, is there any reason why
I shouldn't ALWAYS use a script like the above? And is there any way
to avoid PerlModule modules from being loaded twice?
Thanks
Jon
