Re: 302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
The good news is that Mr. Mackenna got it! If I set KeepAlive Off in apache2.conf then it all works fine. Below is a functioning long process thingy which works with KeepAlive On and Firefox. I just hope it works with MSIE ... #!/usr/bin/perl use strict; use Apache2::Const -compile = qw(:conn_keepalive); use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers use Apache2::Connection(); use Apache2::RequestRec(); my $r = shift; my $c = $r-connection(); $c-keepalive(Apache2::Const::CONN_CLOSE); $r-push_handlers(PerlCleanupHandler = \cleanup ); $r-err_headers_out-set(Location = 'http://...index.pl'); $r-status(Apache2::Const::REDIRECT); return Apache2::Const::REDIRECT; sub cleanup { my ($r) = @_; $r-warn(Starting cleanup); foreach my $num (1..5) { $r-warn(Number is $num); sleep 2; } return Apache2::Const::OK; } So, thanks again, and as for the warning from William T., well my spin-off processes are maybe 10s-10m long and not a driving feature, but I will have to keep my eye on them. Tosh macke...@animalhead.com wrote: The warning from William T. made me think to ask: Does your site have KeepAlive On in httpd.conf? (If not I can't think of anything to suggest...) If so, try adding this as part of the redirect: use Apache2::Connection(); use Apache2::RequestRec(); ... my $c = $r-connection(); $c-keepalive(Apache2::Const::CONN_CLOSE); This will keep your process (which is about to do a long cleanup) from automatically getting the redirected request from the browser. Hopefully the root httpd will know that this redirecting child has not finished the complete cycle, and will launch other children if needed to process the redirected request plus any other requests. Of course William is right that if lots of requests are arriving that need such cleanup, and the cleanup really does take a long time on average, you are likely to pile up more children than your household income (I'm sorry I meant to say your server :-) can support. Good Luck, cmac On Jan 26, 2010, at 2:27 PM, Tosh Cooey wrote: So this works almost perfectly... Almost because: #!/usr/bin/perl use strict; use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers my $r = shift; $r-push_handlers(PerlCleanupHandler = \cleanup ); $r-headers_out-set(Location = 'http://...index.pl'); $r-status(Apache2::Const::REDIRECT); return Apache2::Const::REDIRECT; sub cleanup { my ($r) = @_; $r-warn(Starting cleanup); foreach my $num (1..5) { $r-warn(Number is $num); sleep 2; } return Apache2::Const::OK; } test.pl It seems if you take the above program and hit it with Firefox (3.5.7 and 3.6) it may take 10 seconds (5 x sleep 2) before Firefox does the redirect. Safari 4.0.4 seems fine. curl works as well :) I said may above because it's not consistent. If you launch Firefox fresh and hit the above program it may redirect instantly, but then subsequent hits will illustrate the delay. I'm also seeing varying behaviour on a different server that has no Basic Auth, but always the problem is there. Can anyone else reproduce this? Thank-you! Tosh macke...@animalhead.com wrote: at(1) is a Unix command to start a process. Assuming you're on a Unix/Linux box, type man at to get the story. A cleanup handler is more pleasant than a prostate exam. You can spend your life waiting for others. Just write a routine called cleanup and have it do something like make a log entry. use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers ... sub cleanup { my ($r) = @_; $r-warn(cleanup was here); return Apache2::Const::OK; } Then put a call like the one below in your ModPerl::Registry routine. If the log entry shows up in error_log, you're on your way... Good Luck, cmac P.S. Google doesn't index some sites well. Look at http://perl.apache.org/docs/2.0/ particularly its API link. On Jan 25, 2010, at 5:49 PM, Tosh Cooey wrote: Sorry, I couldn't figure out what at(1) meant (or maybe ap(1) which you say below) is that an abbreviation for something? And Perrin saying cleanup handler is right up there with prostate exam in my list of things to get into, both scare me! Of course at some point a man needs to do both... So... If this magic: $r-push_handlers(PerlCleanupHandler = \cleanup); is available in ModPerl::Registry context then I will attempt to force all my forks into early retirement and work the problem out that way. Unfortunately Google doesn't return an easy answer, anybody know this before I spend all day tomorrow in my struggle? Thank-you all again, Tosh macke...@animalhead.com wrote: You made no comment on the links I sent you earlier today. They had lots of good advice. Particularly the first one suggested not forking the Apache
Re: 302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
Would this problem be any different in a normal CGI context with the program doing forks? I don't imagine it would be, which is why I see the ultimate wisdom in spawning an external program to handle long-running tasks, or just cron something. Oh well, live and learn. Tosh William T wrote: Caveat Lector: Long Cleanups done inline on the Apache children can cause problems. If you get a situation where the CleanUp takes to long OR you get enough traffic to the page(s) which engage the CleanUp then you will encounter a tipping point, and soon after your website will be almost completely unavailable. This will occur because the Apache Children aren't processing requests fast enough to handle the rate at which the come in; because they are busy in CleanUp. The reason I bring this up is encountering the failure is usually a catastrophic event. The website is almost always entirely down, and workarounds can be hard to come by. This may or may not apply to you depending on your traffic characteristics and how long your cleanup takes, BUT it is something you should be aware of. -wjt -- McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/
Re: 302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
Just wanted to note that since you've put the CONN_CLOSE in the redirect code, it's not necessary (nor desirable) to put KeepAlive Off in apache2.conf With the CONN_CLOSE call you turn KA off just when you need it to be off. So what's the bad news? cmac On Jan 27, 2010, at 5:08 AM, Tosh Cooey wrote: The good news is that Mr. Mackenna got it! If I set KeepAlive Off in apache2.conf then it all works fine. Below is a functioning long process thingy which works with KeepAlive On and Firefox. I just hope it works with MSIE ... #!/usr/bin/perl use strict; use Apache2::Const -compile = qw(:conn_keepalive); use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers use Apache2::Connection(); use Apache2::RequestRec(); my $r = shift; my $c = $r-connection(); $c-keepalive(Apache2::Const::CONN_CLOSE); $r-push_handlers(PerlCleanupHandler = \cleanup ); $r-err_headers_out-set(Location = 'http://...index.pl'); $r-status(Apache2::Const::REDIRECT); return Apache2::Const::REDIRECT; sub cleanup { my ($r) = @_; $r-warn(Starting cleanup); foreach my $num (1..5) { $r-warn(Number is $num); sleep 2; } return Apache2::Const::OK; } So, thanks again, and as for the warning from William T., well my spin-off processes are maybe 10s-10m long and not a driving feature, but I will have to keep my eye on them. Tosh macke...@animalhead.com wrote: The warning from William T. made me think to ask: Does your site have KeepAlive On in httpd.conf? (If not I can't think of anything to suggest...) If so, try adding this as part of the redirect: use Apache2::Connection(); use Apache2::RequestRec(); ... my $c = $r-connection(); $c-keepalive(Apache2::Const::CONN_CLOSE); This will keep your process (which is about to do a long cleanup) from automatically getting the redirected request from the browser. Hopefully the root httpd will know that this redirecting child has not finished the complete cycle, and will launch other children if needed to process the redirected request plus any other requests. Of course William is right that if lots of requests are arriving that need such cleanup, and the cleanup really does take a long time on average, you are likely to pile up more children than your household income (I'm sorry I meant to say your server :-) can support. Good Luck, cmac On Jan 26, 2010, at 2:27 PM, Tosh Cooey wrote: So this works almost perfectly... Almost because: #!/usr/bin/perl use strict; use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers my $r = shift; $r-push_handlers(PerlCleanupHandler = \cleanup ); $r-headers_out-set(Location = 'http://...index.pl'); $r-status(Apache2::Const::REDIRECT); return Apache2::Const::REDIRECT; sub cleanup { my ($r) = @_; $r-warn(Starting cleanup); foreach my $num (1..5) { $r-warn(Number is $num); sleep 2; } return Apache2::Const::OK; } test.pl It seems if you take the above program and hit it with Firefox (3.5.7 and 3.6) it may take 10 seconds (5 x sleep 2) before Firefox does the redirect. Safari 4.0.4 seems fine. curl works as well :) I said may above because it's not consistent. If you launch Firefox fresh and hit the above program it may redirect instantly, but then subsequent hits will illustrate the delay. I'm also seeing varying behaviour on a different server that has no Basic Auth, but always the problem is there. Can anyone else reproduce this? Thank-you! Tosh macke...@animalhead.com wrote: at(1) is a Unix command to start a process. Assuming you're on a Unix/Linux box, type man at to get the story. A cleanup handler is more pleasant than a prostate exam. You can spend your life waiting for others. Just write a routine called cleanup and have it do something like make a log entry. use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers ... sub cleanup { my ($r) = @_; $r-warn(cleanup was here); return Apache2::Const::OK; } Then put a call like the one below in your ModPerl::Registry routine. If the log entry shows up in error_log, you're on your way... Good Luck, cmac P.S. Google doesn't index some sites well. Look at http://perl.apache.org/docs/2.0/ particularly its API link. On Jan 25, 2010, at 5:49 PM, Tosh Cooey wrote: Sorry, I couldn't figure out what at(1) meant (or maybe ap(1) which you say below) is that an abbreviation for something? And Perrin saying cleanup handler is right up there with prostate exam in my list of things to get into, both scare me! Of course at some point a man needs to do both... So... If this magic: $r-push_handlers(PerlCleanupHandler = \cleanup); is available in ModPerl::Registry context then I will attempt to force all my forks into early retirement and work the problem out that way.
302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
So this works almost perfectly... Almost because: #!/usr/bin/perl use strict; use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers my $r = shift; $r-push_handlers(PerlCleanupHandler = \cleanup ); $r-headers_out-set(Location = 'http://...index.pl'); $r-status(Apache2::Const::REDIRECT); return Apache2::Const::REDIRECT; sub cleanup { my ($r) = @_; $r-warn(Starting cleanup); foreach my $num (1..5) { $r-warn(Number is $num); sleep 2; } return Apache2::Const::OK; } test.pl It seems if you take the above program and hit it with Firefox (3.5.7 and 3.6) it may take 10 seconds (5 x sleep 2) before Firefox does the redirect. Safari 4.0.4 seems fine. curl works as well :) I said may above because it's not consistent. If you launch Firefox fresh and hit the above program it may redirect instantly, but then subsequent hits will illustrate the delay. I'm also seeing varying behaviour on a different server that has no Basic Auth, but always the problem is there. Can anyone else reproduce this? Thank-you! Tosh macke...@animalhead.com wrote: at(1) is a Unix command to start a process. Assuming you're on a Unix/Linux box, type man at to get the story. A cleanup handler is more pleasant than a prostate exam. You can spend your life waiting for others. Just write a routine called cleanup and have it do something like make a log entry. use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers ... sub cleanup { my ($r) = @_; $r-warn(cleanup was here); return Apache2::Const::OK; } Then put a call like the one below in your ModPerl::Registry routine. If the log entry shows up in error_log, you're on your way... Good Luck, cmac P.S. Google doesn't index some sites well. Look at http://perl.apache.org/docs/2.0/ particularly its API link. On Jan 25, 2010, at 5:49 PM, Tosh Cooey wrote: Sorry, I couldn't figure out what at(1) meant (or maybe ap(1) which you say below) is that an abbreviation for something? And Perrin saying cleanup handler is right up there with prostate exam in my list of things to get into, both scare me! Of course at some point a man needs to do both... So... If this magic: $r-push_handlers(PerlCleanupHandler = \cleanup); is available in ModPerl::Registry context then I will attempt to force all my forks into early retirement and work the problem out that way. Unfortunately Google doesn't return an easy answer, anybody know this before I spend all day tomorrow in my struggle? Thank-you all again, Tosh macke...@animalhead.com wrote: You made no comment on the links I sent you earlier today. They had lots of good advice. Particularly the first one suggested not forking the Apache process, but using an ap(1) call to start a process to do the additional processing. OK, the ap(1) alternative was a bit light on details. How about the alternative offered by Perrin Hawkins in the same thread, of using a cleanup handler to do the follow-up processing rather than a forked process. From p. 107 of mod_per2 User's Guide: $r-push_handlers(PerlCleanupHandler = \cleanup); print $in-redirect... # to redirect the browser Now cleanup (which receives $r as its operand) can do whatever slow stuff you need to, can probably use DBI without all the pain you have below, and can access the request to find out what to do. In some past context you may have learned how to get hold of a $r to use in these calls, and hopefully you're no longer scared of $r. But there does remain the question of whether a ModPerl::Registry module can do such calls. Hopefully someone who knows can chime in on this. If not, for me it would be worth the editing of getting the module out from under ModPerl::Registry and into the native mode of SetHandler modperl. Best of luck, cmac On Jan 25, 2010, at 1:54 PM, Tosh Cooey wrote: Ok, then maybe I need to supply some code here to try and get clarification: mailfile.pl ### use strict; ... use POSIX; #gather needed modules and objects my $fileOBJ = new MyOBJS::FILE($in-param('id')); my $clientOBJ = new ... my $userOBJ = new ... # All OBJjects have a {DBH} property which is their DB handle # I hear I have to disconnect these first, do I have to disconnect ALL? $fileOBJ-{DBH}-disconnect; $SIG{CHLD} = 'IGNORE'; my $pid; if ($pid = fork) { warn Pid = $pid; } elsif (defined $pid) { close(STDOUT); close(STDIN); close(STDERR); # chdir to /, stops the process from preventing an unmount chdir '/' or die Can't chdir to /: $!; # dump our STDIN and STDOUT handles open STDIN, '/dev/null' or die Can't read /dev/null: $!; open STDOUT, '/dev/null' or die Can't write to /dev/null: $!; # redirect for logging open STDERR, '/tmp/stderr' or die Can't write to /tmp/stderr: $!; # Prevent locking to apache process
Re: 302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
Caveat Lector: Long Cleanups done inline on the Apache children can cause problems. If you get a situation where the CleanUp takes to long OR you get enough traffic to the page(s) which engage the CleanUp then you will encounter a tipping point, and soon after your website will be almost completely unavailable. This will occur because the Apache Children aren't processing requests fast enough to handle the rate at which the come in; because they are busy in CleanUp. The reason I bring this up is encountering the failure is usually a catastrophic event. The website is almost always entirely down, and workarounds can be hard to come by. This may or may not apply to you depending on your traffic characteristics and how long your cleanup takes, BUT it is something you should be aware of. -wjt
Re: 302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
The warning from William T. made me think to ask: Does your site have KeepAlive On in httpd.conf? (If not I can't think of anything to suggest...) If so, try adding this as part of the redirect: use Apache2::Connection(); use Apache2::RequestRec(); ... my $c = $r-connection(); $c-keepalive(Apache2::Const::CONN_CLOSE); This will keep your process (which is about to do a long cleanup) from automatically getting the redirected request from the browser. Hopefully the root httpd will know that this redirecting child has not finished the complete cycle, and will launch other children if needed to process the redirected request plus any other requests. Of course William is right that if lots of requests are arriving that need such cleanup, and the cleanup really does take a long time on average, you are likely to pile up more children than your household income (I'm sorry I meant to say your server :-) can support. Good Luck, cmac On Jan 26, 2010, at 2:27 PM, Tosh Cooey wrote: So this works almost perfectly... Almost because: #!/usr/bin/perl use strict; use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers my $r = shift; $r-push_handlers(PerlCleanupHandler = \cleanup ); $r-headers_out-set(Location = 'http://...index.pl'); $r-status(Apache2::Const::REDIRECT); return Apache2::Const::REDIRECT; sub cleanup { my ($r) = @_; $r-warn(Starting cleanup); foreach my $num (1..5) { $r-warn(Number is $num); sleep 2; } return Apache2::Const::OK; } test.pl It seems if you take the above program and hit it with Firefox (3.5.7 and 3.6) it may take 10 seconds (5 x sleep 2) before Firefox does the redirect. Safari 4.0.4 seems fine. curl works as well :) I said may above because it's not consistent. If you launch Firefox fresh and hit the above program it may redirect instantly, but then subsequent hits will illustrate the delay. I'm also seeing varying behaviour on a different server that has no Basic Auth, but always the problem is there. Can anyone else reproduce this? Thank-you! Tosh macke...@animalhead.com wrote: at(1) is a Unix command to start a process. Assuming you're on a Unix/Linux box, type man at to get the story. A cleanup handler is more pleasant than a prostate exam. You can spend your life waiting for others. Just write a routine called cleanup and have it do something like make a log entry. use Apache2::Const();# defines OK use Apache2::Log();# defines warn use Apache2::RequestUtil();# defines push_handlers ... sub cleanup { my ($r) = @_; $r-warn(cleanup was here); return Apache2::Const::OK; } Then put a call like the one below in your ModPerl::Registry routine. If the log entry shows up in error_log, you're on your way... Good Luck, cmac P.S. Google doesn't index some sites well. Look at http://perl.apache.org/docs/2.0/ particularly its API link. On Jan 25, 2010, at 5:49 PM, Tosh Cooey wrote: Sorry, I couldn't figure out what at(1) meant (or maybe ap(1) which you say below) is that an abbreviation for something? And Perrin saying cleanup handler is right up there with prostate exam in my list of things to get into, both scare me! Of course at some point a man needs to do both... So... If this magic: $r-push_handlers(PerlCleanupHandler = \cleanup); is available in ModPerl::Registry context then I will attempt to force all my forks into early retirement and work the problem out that way. Unfortunately Google doesn't return an easy answer, anybody know this before I spend all day tomorrow in my struggle? Thank-you all again, Tosh macke...@animalhead.com wrote: You made no comment on the links I sent you earlier today. They had lots of good advice. Particularly the first one suggested not forking the Apache process, but using an ap(1) call to start a process to do the additional processing. OK, the ap(1) alternative was a bit light on details. How about the alternative offered by Perrin Hawkins in the same thread, of using a cleanup handler to do the follow-up processing rather than a forked process. From p. 107 of mod_per2 User's Guide: $r-push_handlers(PerlCleanupHandler = \cleanup); print $in-redirect... # to redirect the browser Now cleanup (which receives $r as its operand) can do whatever slow stuff you need to, can probably use DBI without all the pain you have below, and can access the request to find out what to do. In some past context you may have learned how to get hold of a $r to use in these calls, and hopefully you're no longer scared of $r. But there does remain the question of whether a ModPerl::Registry module can do such calls. Hopefully someone who knows can chime in on this. If not, for me it would be worth the editing of getting the module out from under ModPerl::Registry and into the native mode of SetHandler modperl. Best of luck, cmac On Jan 25, 2010, at 1:54 PM,