Re: Apache2::SubProcess sucks
On Feb 20, 2010, at 7:01 AM, Tosh Cooey wrote: Anyway, the solution, at least so far until I run into other problems, seems to be to just make a system() call and the called program uses Proc::Daemon and things *seem* to work fine in testing, we'll see when it hits production... Tosh Doesn't a process that calls 'system' wait for the child process to complete? If the parent process is an Apache child, is that in keeping with what you're trying to do? cmac
Re: global variable
I rewrote IPC::MMA from an earlier CPAN module so that I could use shared memory among Apache children. You can read about it at http://search.cpan.org/~mackenna/IPC-MMA-0.6/MMA.pod On Feb 2, 2010, at 9:45 PM, m...@normalperson.e4ward.com wrote: Hello, Is there a method to setup a global variable for all modperl child processes? Also this variable will be updated sometime, when it get updated, all processes will know it. Thanks.
Re: 302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
Just wanted to note that since you've put the CONN_CLOSE in the
redirect code, it's not necessary (nor desirable) to put
KeepAlive Off in apache2.conf
With the CONN_CLOSE call you turn KA off just when you need it to be
off.
So what's the bad news?
cmac
On Jan 27, 2010, at 5:08 AM, Tosh Cooey wrote:
The good news is that Mr. Mackenna got it! If I set KeepAlive
Off in apache2.conf then it all works fine. Below is a
functioning long process thingy which works with KeepAlive On and
Firefox. I just hope it works with MSIE ...
#!/usr/bin/perl
use strict;
use Apache2::Const -compile = qw(:conn_keepalive);
use Apache2::Log();# defines warn
use Apache2::RequestUtil();# defines push_handlers
use Apache2::Connection();
use Apache2::RequestRec();
my $r = shift;
my $c = $r-connection();
$c-keepalive(Apache2::Const::CONN_CLOSE);
$r-push_handlers(PerlCleanupHandler = \cleanup );
$r-err_headers_out-set(Location = 'http://...index.pl');
$r-status(Apache2::Const::REDIRECT);
return Apache2::Const::REDIRECT;
sub cleanup {
my ($r) = @_;
$r-warn(Starting cleanup);
foreach my $num (1..5) {
$r-warn(Number is $num);
sleep 2;
}
return Apache2::Const::OK;
}
So, thanks again, and as for the warning from William T., well my
spin-off processes are maybe 10s-10m long and not a driving
feature, but I will have to keep my eye on them.
Tosh
macke...@animalhead.com wrote:
The warning from William T. made me think to ask:
Does your site have KeepAlive On in httpd.conf?
(If not I can't think of anything to suggest...)
If so, try adding this as part of the redirect:
use Apache2::Connection();
use Apache2::RequestRec();
...
my $c = $r-connection();
$c-keepalive(Apache2::Const::CONN_CLOSE);
This will keep your process (which is about to do a long
cleanup) from automatically getting the redirected request
from the browser.
Hopefully the root httpd will know that this redirecting
child has not finished the complete cycle, and will launch
other children if needed to process the redirected request
plus any other requests.
Of course William is right that if lots of requests are
arriving that need such cleanup, and the cleanup really does
take a long time on average, you are likely to pile up
more children than your household income (I'm sorry I meant
to say your server :-) can support.
Good Luck,
cmac
On Jan 26, 2010, at 2:27 PM, Tosh Cooey wrote:
So this works almost perfectly... Almost because:
#!/usr/bin/perl
use strict;
use Apache2::Const();# defines OK
use Apache2::Log();# defines warn
use Apache2::RequestUtil();# defines push_handlers
my $r = shift;
$r-push_handlers(PerlCleanupHandler = \cleanup );
$r-headers_out-set(Location = 'http://...index.pl');
$r-status(Apache2::Const::REDIRECT);
return Apache2::Const::REDIRECT;
sub cleanup {
my ($r) = @_;
$r-warn(Starting cleanup);
foreach my $num (1..5) {
$r-warn(Number is $num);
sleep 2;
}
return Apache2::Const::OK;
}
test.pl
It seems if you take the above program and hit it with Firefox
(3.5.7 and 3.6) it may take 10 seconds (5 x sleep 2) before
Firefox does the redirect. Safari 4.0.4 seems fine. curl works
as well :)
I said may above because it's not consistent. If you launch
Firefox fresh and hit the above program it may redirect
instantly, but then subsequent hits will illustrate the delay.
I'm also seeing varying behaviour on a different server that has
no Basic Auth, but always the problem is there.
Can anyone else reproduce this?
Thank-you!
Tosh
macke...@animalhead.com wrote:
at(1) is a Unix command to start a process.
Assuming you're on a Unix/Linux box, type man at to get the
story.
A cleanup handler is more pleasant than a prostate exam.
You can spend your life waiting for others. Just write a
routine called cleanup and have it do something like
make a log entry.
use Apache2::Const();# defines OK
use Apache2::Log();# defines warn
use Apache2::RequestUtil();# defines push_handlers
...
sub cleanup {
my ($r) = @_;
$r-warn(cleanup was here);
return Apache2::Const::OK;
}
Then put a call like the one below in your ModPerl::Registry
routine.
If the log entry shows up in error_log, you're on your way...
Good Luck,
cmac
P.S. Google doesn't index some sites well.
Look at http://perl.apache.org/docs/2.0/
particularly its API link.
On Jan 25, 2010, at 5:49 PM, Tosh Cooey wrote:
Sorry, I couldn't figure out what at(1) meant (or maybe ap(1)
which you say below) is that an abbreviation for something?
And Perrin saying cleanup handler is right up there with
prostate exam in my list of things to get into, both scare me!
Of course at some point a man needs to do both...
So... If this magic: $r-push_handlers(PerlCleanupHandler =
\cleanup); is available in ModPerl::Registry context then I
will attempt to force all my forks into early retirement and
work the problem out that way
Re: 302 Redirect not working as expected with PerlCleanupHandler and Firefox under ModPerl::Registry
The warning from William T. made me think to ask:
Does your site have KeepAlive On in httpd.conf?
(If not I can't think of anything to suggest...)
If so, try adding this as part of the redirect:
use Apache2::Connection();
use Apache2::RequestRec();
...
my $c = $r-connection();
$c-keepalive(Apache2::Const::CONN_CLOSE);
This will keep your process (which is about to do a long
cleanup) from automatically getting the redirected request
from the browser.
Hopefully the root httpd will know that this redirecting
child has not finished the complete cycle, and will launch
other children if needed to process the redirected request
plus any other requests.
Of course William is right that if lots of requests are
arriving that need such cleanup, and the cleanup really does
take a long time on average, you are likely to pile up
more children than your household income (I'm sorry I meant
to say your server :-) can support.
Good Luck,
cmac
On Jan 26, 2010, at 2:27 PM, Tosh Cooey wrote:
So this works almost perfectly... Almost because:
#!/usr/bin/perl
use strict;
use Apache2::Const();# defines OK
use Apache2::Log();# defines warn
use Apache2::RequestUtil();# defines push_handlers
my $r = shift;
$r-push_handlers(PerlCleanupHandler = \cleanup );
$r-headers_out-set(Location = 'http://...index.pl');
$r-status(Apache2::Const::REDIRECT);
return Apache2::Const::REDIRECT;
sub cleanup {
my ($r) = @_;
$r-warn(Starting cleanup);
foreach my $num (1..5) {
$r-warn(Number is $num);
sleep 2;
}
return Apache2::Const::OK;
}
test.pl
It seems if you take the above program and hit it with Firefox
(3.5.7 and 3.6) it may take 10 seconds (5 x sleep 2) before Firefox
does the redirect. Safari 4.0.4 seems fine. curl works as well :)
I said may above because it's not consistent. If you launch
Firefox fresh and hit the above program it may redirect instantly,
but then subsequent hits will illustrate the delay. I'm also
seeing varying behaviour on a different server that has no Basic
Auth, but always the problem is there.
Can anyone else reproduce this?
Thank-you!
Tosh
macke...@animalhead.com wrote:
at(1) is a Unix command to start a process.
Assuming you're on a Unix/Linux box, type man at to get the story.
A cleanup handler is more pleasant than a prostate exam.
You can spend your life waiting for others. Just write a
routine called cleanup and have it do something like
make a log entry.
use Apache2::Const();# defines OK
use Apache2::Log();# defines warn
use Apache2::RequestUtil();# defines push_handlers
...
sub cleanup {
my ($r) = @_;
$r-warn(cleanup was here);
return Apache2::Const::OK;
}
Then put a call like the one below in your ModPerl::Registry routine.
If the log entry shows up in error_log, you're on your way...
Good Luck,
cmac
P.S. Google doesn't index some sites well.
Look at http://perl.apache.org/docs/2.0/
particularly its API link.
On Jan 25, 2010, at 5:49 PM, Tosh Cooey wrote:
Sorry, I couldn't figure out what at(1) meant (or maybe ap(1)
which you say below) is that an abbreviation for something?
And Perrin saying cleanup handler is right up there with
prostate exam in my list of things to get into, both scare me!
Of course at some point a man needs to do both...
So... If this magic: $r-push_handlers(PerlCleanupHandler =
\cleanup); is available in ModPerl::Registry context then I will
attempt to force all my forks into early retirement and work the
problem out that way.
Unfortunately Google doesn't return an easy answer, anybody know
this before I spend all day tomorrow in my struggle?
Thank-you all again,
Tosh
macke...@animalhead.com wrote:
You made no comment on the links I sent you earlier today.
They had lots of good advice. Particularly the first one
suggested not forking the Apache process, but using an
ap(1) call to start a process to do the additional processing.
OK, the ap(1) alternative was a bit light on details.
How about the alternative offered by Perrin Hawkins in the
same thread, of using a cleanup handler to do the follow-up
processing rather than a forked process.
From p. 107 of mod_per2 User's Guide:
$r-push_handlers(PerlCleanupHandler = \cleanup);
print $in-redirect... # to redirect the browser
Now cleanup (which receives $r as its operand) can do
whatever slow stuff you need to, can probably use DBI
without all the pain you have below, and can access the
request to find out what to do.
In some past context you may have learned how to get hold of
a $r to use in these calls, and hopefully you're no longer
scared of $r. But there does remain the question of whether
a ModPerl::Registry module can do such calls.
Hopefully someone who knows can chime in on this.
If not, for me it would be worth the editing of getting the
module out from under ModPerl::Registry and into the native
mode of SetHandler modperl.
Best of luck,
cmac
On Jan 25, 2010, at 1:54 PM,
Re: mod_perl2 + fork + DBI = Chaos
Have you been through these threads already?
The first one seems quite like what you're doing.
http://mail-archives.apache.org/mod_mbox/perl-modperl/200908.mbox/%
3c59a07310908250820m1f789901g22eece5b2897c...@mail.gmail.com%3e
http://www.gossamer-threads.com/lists/modperl/modperl/100099
Is the time for the SQL inquiry so bad that you can't
just do the inquiry before the fork-and-exec bit?
Best of Luck,
cmac
On Jan 25, 2010, at 8:07 AM, Tosh Cooey wrote:
My application-from-hell is doing odd things probably related to
the above storms and culminating in a perfect storm.
Can anyone point me to resources outlining the best way to use the
combination in the subject line?
There seems to be many ways of accomplishing specifics but very
little generic methodologies which can be adapted.
What I'm trying to do is in a nutshell:
##
my $object_with_DBH = new My::DataObject;
... do stuff...
if ($pid = fork) {
... parent things ...
} elsif (defined $pid) {
... close STDhandles ...
my $data = $object_with_DBH-SQL('do SQL stuff with old
connection');
exec 'things from $data';
CORE::exit(0);
}
##
I've tried Stas' code but that's for MP1, and with $DBI-clone(),
and playing with setsid from POSIX and on... There seems to be no
love between mod_perl2, fork, and DBI.
On a side, but possibly related note, my browser is sending a POST
request which results in the FORK above and the parent then sends a
302 redirect with some status variables. This works fine, except
between the POST and the 302 GET is another POST to the same URI
which results in a 500 error, here's the log:
POST /tosh/mailfile.pl HTTP/1.1 302 282
POST /tosh/mailfile.pl HTTP/1.1 500 424
GET /tosh/index.pl?message=Notice%20sent HTTP/1.1 200 4309
The browser only seems to see the middle one, the 500. I'm certain
this is related to my poor forking above but I have no idea why.
Are there any tools for Firefox that will let me track every
connection it's making and why?
Thanks again for any help with my Frankenstein monster...
Tosh
--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/
Re: mod_perl2 + fork + DBI = Chaos
You made no comment on the links I sent you earlier today.
They had lots of good advice. Particularly the first one
suggested not forking the Apache process, but using an
ap(1) call to start a process to do the additional processing.
OK, the ap(1) alternative was a bit light on details.
How about the alternative offered by Perrin Hawkins in the
same thread, of using a cleanup handler to do the follow-up
processing rather than a forked process.
From p. 107 of mod_per2 User's Guide:
$r-push_handlers(PerlCleanupHandler = \cleanup);
print $in-redirect... # to redirect the browser
Now cleanup (which receives $r as its operand) can do
whatever slow stuff you need to, can probably use DBI
without all the pain you have below, and can access the
request to find out what to do.
In some past context you may have learned how to get hold of
a $r to use in these calls, and hopefully you're no longer
scared of $r. But there does remain the question of whether
a ModPerl::Registry module can do such calls.
Hopefully someone who knows can chime in on this.
If not, for me it would be worth the editing of getting the
module out from under ModPerl::Registry and into the native
mode of SetHandler modperl.
Best of luck,
cmac
On Jan 25, 2010, at 1:54 PM, Tosh Cooey wrote:
Ok, then maybe I need to supply some code here to try and get
clarification:
mailfile.pl
###
use strict;
...
use POSIX;
#gather needed modules and objects
my $fileOBJ = new MyOBJS::FILE($in-param('id'));
my $clientOBJ = new ...
my $userOBJ = new ...
# All OBJjects have a {DBH} property which is their DB handle
# I hear I have to disconnect these first, do I have to disconnect
ALL?
$fileOBJ-{DBH}-disconnect;
$SIG{CHLD} = 'IGNORE';
my $pid;
if ($pid = fork) {
warn Pid = $pid;
} elsif (defined $pid) {
close(STDOUT);
close(STDIN);
close(STDERR);
# chdir to /, stops the process from preventing an unmount
chdir '/' or die Can't chdir to /: $!;
# dump our STDIN and STDOUT handles
open STDIN, '/dev/null' or die Can't read /dev/null: $!;
open STDOUT, '/dev/null' or die Can't write to /dev/null: $!;
# redirect for logging
open STDERR, '/tmp/stderr' or die Can't write to /tmp/stderr: $!;
# Prevent locking to apache process
setsid or die Can't start a new session: $!;
# Create file download link email
my $mailSTR = ...
# Send the mail possibly to many people
foreach my $person (@people) {
open(MAIL, '|' . cfg('sendmail_location') . ' -t');
print MAIL $mailSTR;
close(MAIL);
}
# Need to recreate the DBI connection on the $fileOBJ I hear
$fileOBJ = new MyOBJS::FILE($in-param('id'));
# Do some SQL to update the $fileOBJ status based on mailout
$fileOBJ-sql
# create LOGGING Objects to log stuff
my $logOBJ = new ...
$logOBJ-sql...
CORE::exit(0);
}
print $in-redirect... # For the parent to redirect the browser
# Done.
Is there a glaring mistake in the above?
The parent does no more DB stuff, it just sends a redirect.
This runs under ModPerl::Registry.
I'd like to get at least one thing working tonight, either the
forking or the DBI, I'll be happy!
Tosh
Perrin Harkins wrote:
On Mon, Jan 25, 2010 at 3:48 PM, Tosh Cooey t...@1200group.com
wrote:
Thanks Perrin, the forking, my child got a PID of 30033 and then
afterwards
when I checked the processes (ps) for 30033 I see:
[apache2] defunct
Is that what's supposed to happen?
After you call exit? No. It should be gone. That's a zombie
process.
That PM thread seems to indicate that I must disconnect every
DBH, not just
the ones that I will use.
Either that, or you need to set InactiveDestroy on all of them in the
child process. Otherwise, when the child exits, it messes up all of
them for the parent.
Are you also suggesting the use of
Parallel::ForkManager for forks?
No. The DBI stuff is the same with either.
- Perrin
--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/
Re: mod_perl2 + fork + DBI = Chaos
at(1) is a Unix command to start a process.
Assuming you're on a Unix/Linux box, type man at to get the story.
A cleanup handler is more pleasant than a prostate exam.
You can spend your life waiting for others. Just write a
routine called cleanup and have it do something like
make a log entry.
use Apache2::Const(); # defines OK
use Apache2::Log(); # defines warn
use Apache2::RequestUtil(); # defines push_handlers
...
sub cleanup {
my ($r) = @_;
$r-warn(cleanup was here);
return Apache2::Const::OK;
}
Then put a call like the one below in your ModPerl::Registry routine.
If the log entry shows up in error_log, you're on your way...
Good Luck,
cmac
P.S. Google doesn't index some sites well.
Look at http://perl.apache.org/docs/2.0/
particularly its API link.
On Jan 25, 2010, at 5:49 PM, Tosh Cooey wrote:
Sorry, I couldn't figure out what at(1) meant (or maybe ap(1) which
you say below) is that an abbreviation for something?
And Perrin saying cleanup handler is right up there with
prostate exam in my list of things to get into, both scare me!
Of course at some point a man needs to do both...
So... If this magic: $r-push_handlers(PerlCleanupHandler =
\cleanup); is available in ModPerl::Registry context then I will
attempt to force all my forks into early retirement and work the
problem out that way.
Unfortunately Google doesn't return an easy answer, anybody know
this before I spend all day tomorrow in my struggle?
Thank-you all again,
Tosh
macke...@animalhead.com wrote:
You made no comment on the links I sent you earlier today.
They had lots of good advice. Particularly the first one
suggested not forking the Apache process, but using an
ap(1) call to start a process to do the additional processing.
OK, the ap(1) alternative was a bit light on details.
How about the alternative offered by Perrin Hawkins in the
same thread, of using a cleanup handler to do the follow-up
processing rather than a forked process.
From p. 107 of mod_per2 User's Guide:
$r-push_handlers(PerlCleanupHandler = \cleanup);
print $in-redirect... # to redirect the browser
Now cleanup (which receives $r as its operand) can do
whatever slow stuff you need to, can probably use DBI
without all the pain you have below, and can access the
request to find out what to do.
In some past context you may have learned how to get hold of
a $r to use in these calls, and hopefully you're no longer
scared of $r. But there does remain the question of whether
a ModPerl::Registry module can do such calls.
Hopefully someone who knows can chime in on this.
If not, for me it would be worth the editing of getting the
module out from under ModPerl::Registry and into the native
mode of SetHandler modperl.
Best of luck,
cmac
On Jan 25, 2010, at 1:54 PM, Tosh Cooey wrote:
Ok, then maybe I need to supply some code here to try and get
clarification:
mailfile.pl
###
use strict;
...
use POSIX;
#gather needed modules and objects
my $fileOBJ = new MyOBJS::FILE($in-param('id'));
my $clientOBJ = new ...
my $userOBJ = new ...
# All OBJjects have a {DBH} property which is their DB handle
# I hear I have to disconnect these first, do I have to
disconnect ALL?
$fileOBJ-{DBH}-disconnect;
$SIG{CHLD} = 'IGNORE';
my $pid;
if ($pid = fork) {
warn Pid = $pid;
} elsif (defined $pid) {
close(STDOUT);
close(STDIN);
close(STDERR);
# chdir to /, stops the process from preventing an unmount
chdir '/' or die Can't chdir to /: $!;
# dump our STDIN and STDOUT handles
open STDIN, '/dev/null' or die Can't read /dev/null: $!;
open STDOUT, '/dev/null' or die Can't write to /dev/null: $!;
# redirect for logging
open STDERR, '/tmp/stderr' or die Can't write to /tmp/
stderr: $!;
# Prevent locking to apache process
setsid or die Can't start a new session: $!;
# Create file download link email
my $mailSTR = ...
# Send the mail possibly to many people
foreach my $person (@people) {
open(MAIL, '|' . cfg('sendmail_location') . ' -t');
print MAIL $mailSTR;
close(MAIL);
}
# Need to recreate the DBI connection on the $fileOBJ I hear
$fileOBJ = new MyOBJS::FILE($in-param('id'));
# Do some SQL to update the $fileOBJ status based on mailout
$fileOBJ-sql
# create LOGGING Objects to log stuff
my $logOBJ = new ...
$logOBJ-sql...
CORE::exit(0);
}
print $in-redirect... # For the parent to redirect the browser
# Done.
Is there a glaring mistake in the above?
The parent does no more DB stuff, it just sends a redirect.
This runs under ModPerl::Registry.
I'd like to get at least one thing working tonight, either the
forking or the DBI, I'll be happy!
Tosh
Perrin Harkins wrote:
On Mon, Jan 25, 2010 at 3:48 PM, Tosh Cooey t...@1200group.com
wrote:
Thanks Perrin, the forking, my child got a PID of 30033 and
then afterwards
when I checked the processes (ps) for 30033 I see:
Re: FreeBSD 7.2, mod_perl2 Apache2::Cookie (libapreq2)
The man you want at the FreeBSD lists is Philip M. Gollucci. He maintains libapreq2 in the ports collection, and was very helpful to me when I was trying to get the module to build, earlier this year. Good Luck and please report your results to this list when this is settled, cmac On Oct 30, 2009, at 9:44 AM, Sin wrote: When I build from ports the make config usually brings up a window that has options for both 32 bit and 64 bit. If I had to guess I'd say its the same Makefile and source code that builds the binary files. Or in your case a 64 bit bin file. But honestly I really don't know, we should steer this thread over to the FreeBSD mailling lists. Perhaps its just a matter of pointing this out to a maintainer or the governing group at the FreeBSD mailling list people. Maybe they know the next step? - Original Message - From: metacyc...@gmail.com To: Sin Cc: Foo JH ; Adam Prime ; Joe Niederberger ; mod_perl list Sent: Friday, October 30, 2009 12:36 PM Subject: Re: FreeBSD 7.2, mod_perl2 Apache2::Cookie (libapreq2) This may be completely unrelated, but I had similar headaches installing libapreq2 on a 64 bit machine that had both 32 and 64 bit libs installed. I had to uninstall the offending 32 bit libs (which I didn't need). Do they have a similar setup? Dimitri On Fri, Oct 30, 2009 at 12:28 PM, Sin sinis...@gmail.com wrote: pkg_add -r just goes to a package repository and gets a package version thats allready complied for your distribution. So you can't build your options. However this apreq2.12 issue is interesting. I was going to try this again. I went to build this port but make errored out with: === libapreq2-2.12_1 : Error from bsd.apache.mk. apache13 is installed (or APACHE_PORT is defined) and port requires 2.0+. *** Error code 1 But if you look at the port it says apache-1.3.41_1 is the build and run dependency. So it should of built the port (because I just happen to have apache 1.3 installed ). I'm wondering if all the times I tried to build Apache2.x with mod_perl2 the application couldn't interface with it because of a missing dependency. I'm not a BSD expert but it looks like this port needs updating. Here's the description in ports: Port: libapreq2-2.12_1 Path: /usr/ports/www/libapreq2 Info: Generic Apache2 Request Library Maint: s...@freebsd.org B-deps: apache-1.3.41_1 autoconf-2.62 autoconf-wrapper-20071109 expat-2.0.1 gettext-0.17_1 gmake-3.81_3 libiconv-1.13.1 libtool-2.2.6a_1 m4-1.4.13,1 perl-5.8.9_3 R-deps: apache-1.3.41_1 expat-2.0.1 perl-5.8.9_3 WWW:http://httpd.apache.org/apreq/ Port: p5-libapreq2-2.12_1 Path: /usr/ports/www/p5-libapreq2 Info: Generic Apache2 Request Library Maint: s...@freebsd.org B-deps: apache-1.3.41_1 autoconf-2.62 autoconf-wrapper-20071109 expat-2.0.1 gettext-0.17_1 gmake-3.81_3 libiconv-1.13.1 libtool-2.2.6a_1 m4-1.4.13,1 mod_perl2-2.0.4_2,3 p5-BSD- Resource-1.2903 p5-ExtUtils-XSBuilder-0.28_1 p5-Parse- RecDescent-1.962.2 p5-Tie-IxHash-1.21 p5-version-0.76 perl-5.8.9_3 R-deps: apache-1.3.41_1 expat-2.0.1 mod_perl2-2.0.4_2,3 p5-BSD- Resource-1.2903 p5-ExtUtils-XSBuilder-0.28_1 p5-Parse- RecDescent-1.962.2 p5-Tie-IxHash-1.21 p5-version-0.76 perl-5.8.9_3 WWW:http://httpd.apache.org/apreq/ - Original Message - From: Foo JH jhfoo...@extracktor.com To: Adam Prime adam.pr...@utoronto.ca Cc: Joe Niederberger jniederber...@comcast.net; mod_perl list modperl@perl.apache.org Sent: Friday, October 30, 2009 3:15 AM Subject: Re: FreeBSD 7.2, mod_perl2 Apache2::Cookie (libapreq2) Just a thought: is install ap22 + mp2 + libapreq2 via pkg_add -r an option? That's what I normally do these days. Adam Prime wrote: You guys might want to take a look at this thread on apreq-dev http://marc.info/?t=12420765987r=1w=2 Specifically the last couple of posts from pgollucci (who is a freebsd, and mod_perl committer). If you can't get apreq2.12 to work, try 2.08. Adam Joe Niederberger wrote: How do I find out what *all* the special options needed are? Thanks, Joe N. - Original Message - From: gl...@gallien.net To: mod_perl list modperl@perl.apache.org Sent: Thursday, October 29, 2009 12:14 PM Subject: Re: FreeBSD 7.2, mod_perl2 Apache2::Cookie (libapreq2) I've been using apache2/mod_perl2 on FreeBSD for years. Currently using 6.3 and 7.2. Installing from ports should work fine, but I prefer to install separate versions of apache2 and mod_perl2 from source. Haven't had a problem installing either of those in sometime. Installing libapreq2 on FreeBSD requires some special options, like passing --with-expat=/usr/local to configure and using gmake. -Glenn
Re: mod_perl2 per directory
On page http://perl.apache.org/docs/2.0/user/config/ config.html#mod_perl_Directives_Argument_Types_and_Allowed_Location you can see the following line in the table that defines the scope of the various directives: PerlInterpStart TAKE1 SRV SRV means server scope which to me means that it cannot be applied to a Location. On Sep 18, 2009, at 4:14 AM, Philip Blatter wrote: Hey there, i've a strange problem... According to the mod_perl 2.0 server configuration pages, the following should work: Location /__ias/jobserver SetHandler perl-script PerlInterpStart 1 PerlInterpMax 1 PerlInterpMaxRequests 1000 PerlResponseHandler MyModule /Location paradoxically starting my apache2 gives me the following error message: PerlInterpStart not allowed here What's wrong with my setup? Am i missing something? Regards, Philip
Re: RFC: Apache2::CloseKeepAlive
Thanks for your comments. They lead to lots to think about and try. I looked up sprite sheets and they seem to be graphics that include lots of little tiled figures used in video games. How does a browser know to split them up into separate images that pages can then call out individually? Ask Google ;) http://www.websiteoptimization.com/speed/tweak/css-sprites/ You should have said CSS sprites rather than sprite sheets. The techniques cited look much harder to use than CloseKeepAlive, plus the comments at the end of the article say they don't work on MSIE, or (per another comment) on MSIE 8. Looks like a technique that needs a new CPAN module to help mere mortals use it. Something to keep you out of mischief? Anyway I guess I'll muddle through with submitting CloseKeepAlive to CPAN. (This list has been quite helpful as to how to do that.) Obviously it's not for you, but maybe it can be useful to people less technologically advanced than you. Or as an example of how to use IPC::MMA. Thanks again, cmac
Re: RFC: Apache2::CloseKeepAlive
On Aug 26, 2009, at 7:39 AM, Adam Prime wrote: A non-code comment, but there seems to be some consternation about the same terms as perl itself license. See the link below: http://perlbuzz.com/2009/07/help-end-licensing-under-same-terms-as- perl-itself.html Inside many programmers, nascent lawyers lurk? The linked-to CPAN Licensing Guidelines http://www.perlfoundation.org/cpan_licensing_guidelines was easier to read and didn't cut off the right sides of code segments. I modified the end of http://www.animalhead.com/CloseKeepAlive.html accordingly. Thanks, cmac
Re: RFC: IPC::MMA
IMO, the perl threads sharing mechanism is flawed because (in my short experience with it) it does not allow certain interesting items (e.g. tied) to be shared. I hope that if Perl 6 is ever released it will have a better threading component. w/r/t figuring out itself, in looking at the code it looks like good old Ralf had quite a time just coping with the various Unices and Linuces. cmac www.animalhead.com On Feb 17, 2009, at 6:43 AM, Perrin Harkins wrote: On Tue, Feb 17, 2009 at 8:13 AM, André Warnier a...@ice-sa.com wrote: Now, how about extending this so it would work on both Unix/Linux *and* Windows platforms, figuring out itself where it is and doing whatever is needed to use the OS mechanisms available ? The underlying library does not support Windows: http://www.ossp.org/pkg/lib/mm/ It was built by Ralf Engelschall to allow sharing between processes on unix, for use in Apache HTTPD 1.x. Since Windows uses threads instead, it doesn't need this. You should be able to use the perl threads sharing mechanism to do the same thing. - Perrin
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
I started with 'worker', and it had similar performance to what I saw thereafter with 'event'. Actually slightly better, perhaps because 'event' gained no advantage from KeepAlive because each test process stayed locked to one connection. 'worker' and 'event' were experiments that I decided to terminate, in part because I was so annoyed with Perl ithreads after I had laboriously eliminated many global variables from my mod_perl scripts that didn't want to be shared. This activity was based on the Programming Perl description of threads 5.005. Oh well, now the code should be better set for the future. Best Regards, cmac www.animalhead.com On Jan 16, 2009, at 8:14 AM, Adam Prime wrote: macke...@animalhead.com wrote: Apache is forgiving/robust about specifying nonexistent phase handlers in httpd.conf and inserts thereto. I was using event, and had to let each thread open its own DBs based on an undefined global. Then I found that the identical traffic-test exerciser could get 66 files/second back in event and 1500/second in prefork. This is under FreeBSD 6.3. No doubt I had something wrong or tunable in event, but at that point I gave up and will wait for the simple MPM. The event MPM is also currently referred to as being 'experimental' by the httpd folks. have you tried worker instead? Adam
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
What I did to get worker and event working under FreeBSD 6.3,
was to eliminate the child_init handler, and at the start of
the response handler do something like
my $env;
sub handler { # this is the response handler
my ($r) = @_;
if (!$env) {
$env = DbEnv-new;
$env-open( $envdir, Db::DB_USE_ENVIRON | Db::DB_THREAD);
...
}
...
}
What this means is that each thread must open the db's for itself.
The amount of data stored for each open DB connection, times
THREADS_PER_CHILD times the number of Apache children at any
given point, makes for some memory. But
1) the separate connections help the DB package be thread-safe,
2) the first-used threads keep getting re-used in preference to
threads not yet used.
3) if you consider each thread as more or less equivalent to a
child process in prefork, your total memory requirement is less.
Guten abend (in meiner Zeitzone),
cmac
www.animalhead.com
On Jan 16, 2009, at 11:34 AM, Michael Ludwig wrote:
Mark Hedges schrieb:
*** glibc detected *** free(): invalid pointer: 0x084e6a14 ***
If this didn't work in a response handler, I'd guess there's
something else wrong. That's the kind of error you get when
you use things under threads that are not thread-safe in
some way, I remember that sort of thing trying to use
List::MoreUtils::natatime() under threads. (But the rest of
that library works.) It's a sad state of affairs that a
large chunk of CPAN doesn't play nice with threads, and
there's usually not any way of knowing which library is
causing the problem. It might not even be the oracle
thingy, maybe something else you're loading.
It may have been XML::LibXML, XML::LibXSLT, or - most probably - my
lack of understanding of the worker MPM and threaded Perl.
I made some progress and got it working using package-level
my-variables to cache the Berkeley handles.
Still, concurrent access quite reliably manages to produce SEGVs.
This may have something to do with with not finalizing the handles
properly.
So I think I have to use PerlChildInitHandler and
PerlChildExitHandler.
I spent some time wondering why the variables I initialized in the
PerlChildInitHandler Eumel::A::child_init were undefined when
accessed from the PerlResponseHandler Eumel::B::handler, regardless
of whether they were declared with our or with my and then returned
by a function.
I thought that moving the child_init method into the same package
as the handler and having it initialize package level my-variables
or our-variables might do the trick. But it doesn't.
Looks like the variables that the PerlChildInitHandler sees are
different beasts than those the PerlResponseHandler sees, even
though they are textually identical.
Turns out under the threaded worker MPM things behave differently:
| Global variables are only global to the interpreter in which
| they are created. Other interpreters from other threads
| can't access that variable. Though it's possible to make
| existing variables shared between several threads running in
| the same process by using the function
| threads::shared::share().
http://perl.apache.org/docs/2.0/user/coding/
coding.html#Shared_Variables
Should have RTFM before.
I haven't used threads to this date, nor threads::shared. Looks like
I'd have to share the Berkeley handles in order for it to *maybe*
work.
Now here's a funny one.
use threads;
use threads::shared;
our $env;
sub child_init {
share($env);
$env = DbEnv-new;
$env-open( $envdir, Db::DB_USE_ENVIRON | Db::DB_THREAD);
print STDERR child_init: Umgebung geoeffnet\n;
...
}
I never see the log message - the PerlChildInitHandler thing is
hanging. Which bizarrely doesn't stop the PerlResponseHandler, by
the way.
If I uncomment use threads, the threads::shared stuff conveniently
turns into no-ops, and I get back the old behaviour, which is that
the PerlChildInitHandler and the PerlResponseHandler do not see the
same variables in spite of their being textually identical.
I'm at my wit's end. Any clues?
Michael Ludwig
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
Apache is forgiving/robust about specifying nonexistent phase handlers in httpd.conf and inserts thereto. I was using event, and had to let each thread open its own DBs based on an undefined global. Then I found that the identical traffic-test exerciser could get 66 files/second back in event and 1500/second in prefork. This is under FreeBSD 6.3. No doubt I had something wrong or tunable in event, but at that point I gave up and will wait for the simple MPM. Best Regards, cmac www.animalhead.com On Jan 15, 2009, at 7:49 AM, Michael Ludwig wrote: Mark Hedges schrieb: Probably what you're thinking of is a PerlChildInitHandler so that each mod_perl child process does your connection for you when the child process first forks. Yes, that's what I thought. But then I noticed that the PerlChildInitHandler I set up is ignored. Apache configuration snippet: PerlRequire /home/milu/www/eumel/startup.pl PerlChildInitHandler Eumel::Gurke::bla; PerlChildExitHandler Eumel::Gurke2::bla; These two classes do not even exist, but Apache starts fine. When I replace those dummy entries with existing classes, I can see they're ignored because I don't see the log output I should see. Is is possible these are ignored for the worker MPM? | In the prefork MPM this phase is useful for initializing | any data structures which should be private to each process. http://perl.apache.org/docs/2.0/user/handlers/ server.html#C_PerlChildInitHandler_ It is unclear to me whether this applies to the prefork MPM exclusively. Does anyone know? Michael Ludwig
ThreadsPerChild vs THR in top using Event MPM
This is not a mod_perl problem, but I received no response to it on the httpd-users mailing list. I send it in the hope that there are many smart people subscribed to this list who may be able to answer it. If you know of another forum or person that might answer it, please tell me where. Our server is running Apache 2.2.11 with the Event MPM under FreeBSD 6.3, on a site that's largely mod_perl2 driven. ThreadsPerChild is specified as 25 in an include file of httpd.conf. The documentation of the Worker MPM (which is said to have the same configuration characteristics as Event) states that each child process creates ThreadsPerChild threads when it starts, and never changes the number of threads. But the THR column of the 'top' utility shows 11 threads per process. Is 'top' just wrong, or is something constraining how many threads are being created? Might my Internet Hosting Provider have built FreeBSD with a 'max threads per process' parameter of 11? How can I get more data? The only way I know to ask Apache about its threads configuration is the Apache2::MPM-query facility. Using this facility for all of the values noted in the Apache2::Const documentation yields: MPMQ_MAX_DAEMON_USED(1) = -1 MPMQ_IS_THREADED(2) = 1 MPMQ_IS_FORKED(3) = 2 MPMQ_HARD_LIMIT_DAEMONS(4) = 16 MPMQ_HARD_LIMIT_THREADS(5) = 64 MPMQ_MAX_THREADS(6) = 25 MPMQ_MIN_SPARE_DAEMONS(7) = 0 MPMQ_MIN_SPARE_THREADS(8) = 25 MPMQ_MAX_SPARE_DAEMONS(9) = 0 MPMQ_MAX_SPARE_THREADS(10) = 150 MPMQ_MAX_REQUESTS_DAEMON(11) = 0 MPMQ_MAX_DAEMONS(12) = 10 MPMQ_MAX_THREADS has the same value as ThreadsPerChild, but its name suggests that this number is a maximum. Does this imply that this number may not be created? Help will be much appreciated, cmac www.animalhead.com
Re: Info about mp2 and threaded MPMs
One of the fun aspects of this is that the dark-blue-going- on-purple book (which I have) describes how global variables are sometimes shared between threads if they're not declared shared, while your linked page says only data that is explicitly requested to be shared will be shared between threads. I think both are correct within their frame of reference, the latter page is not talking about mod_perl2 nor threads created by Apache2. I used to like new subjects to learn about... cmac On Dec 29, 2008, at 2:37 PM, André Warnier wrote: cr...@animalhead.com wrote: [...] My own not very reliable 2 cent : The mod_perl 2 User's Guide (Pub: Onyx Neon, authors: Stas Beckman and Jim Brandt) has apparently only part of 2 pages (365-366) on the subject. Also, the Suse Enterprise Linux 10.x system that I recently installed for a customer has a worker (threaded) Apache2 + mod_perl2 installed by default. None of the above constitute firm recommendations, but in the absence of forecasts of doom, I would tend to see these as encouraging signs. Apart of the usual warnings about the underlying thread-safety of underlying C libraries, there are apparently a couple of potentially nasty side-effects, such as if you would be using chdir () in some of your modules (because a chdir in one thread affects all the others). There is a reference in the book to this, which I have not checked myself : http://www.perl.com/lpt/2002/06/11/threads.html
Re: [mp2] undefined symbol in make test with threaded Apache 2.2.11
On Dec 24 00:57, Philip M. Gollucci pgollu...@p6m7g8.com wrote: Subject: Re: [mp2] undefined symbol in make test with threaded Apache 2.2.11 cr...@animalhead.com wrote: Neither mod_perl 2.0.4 nor the current build modperl-2.0_20081223052020 If you're going to do that 'current build', I'd use revision numbers of SVN instead of the data. The page on the mod_perl site about what to do before submitting a problem report suggested trying the current build from the location I used, as a data point for the overall report. So I complied with those instructions. If some other source is better, perhaps the web page in question wants editing. PL_markstack_ptr This isn't perl 5.8.8 vs perl 5.8.9 related, if you diff the trees, only 1 line is changed - *PL_markstack_ptr = (p) - PL_stack_base;\ + *PL_markstack_ptr = (I32)((p) - PL_stack_base);\ PL_markstack_ptr is in the error message that occurred. Perhaps it's no longer exported, or global'ed, or whatever the right term is nowadays? MP_APXS= /usr/local/apache2/bin/apxs *** /usr/local/apache2/bin/httpd -V You did not install www/apache22 port, but rather compiled by hand ? It has been years since I compiled or assembled anything by hand, but I do go back in the software field to 1963, when such things were sometimes still done. All of several set of instructions I have read, as to how to build mod-perl, have recommended the apxs method. *** /usr/local/bin/perl -V Summary of my perl5 (revision 5 version 8 subversion 9) configuration: So this isn't in the ports tree yet, so you also compiled perl by hand I have never had anything to do with the ports facility, and have built all of our site's software from source for years. The build sequences used for Aapche 2.2.11, perl 5.8.9, and mod-perl 2.0.4 have all been used on previous occasions with full success. The big change is building 2.2.11 for the event MPM. @INC: /usr/local/lib/perl5/site_perl/5.8.9 /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl/5.8.7 /usr/local/lib/perl5/vendor_perl/5.8.9 /usr/local/lib/perl5/vendor_perl/5.8.7 Looks like your perl tree has some history, I'd try with a clean tree .. my money is here. The Perl5.8.9 in question has worked very well with the previous prefork Apache 2.2.10 and a mod-perl built against it. A clean tree approach is complicated by the fact the tree has essential modules in it from my internet hosting provider (IHP), mostly in the vendor-perl branch but perhaps in other branches as well. *** Packages of interest status: mod_perl : 1.29 mod_perl2 : 2.03, 2.05 Speaking of clean trees, that can't be good. you have multiple versions of mod_perl2 installed which almost certainly means that you'll get a mix match between .pm and .so files. I'd clean that up My IHP provides a large number of .so's in the include directory, and I just let the mod-perl 2.0.4 that I built exist next to the 2.0.3 from my IHP's original setup. This has never caused any problems in the past. (httpd.conf calls out the 2.0.4.so.) But on your advice I will rename the 2.0.3 with an extra .hide after the .so, and retry the thing as soon as I get back to my development machine next Monday. FWIW I'm startled by the 2.05 but I assume that's because the last thing built was the _20081223052020 on the recommendations of the mod_perl web site. FWIW, you really shouldn't install your own perl into /usr/local/bin/perl on FreeBSD you should put yours somewhere else. There is great value to me in having a single perl that is used for all applications, rather than a my perl and their perl and the perl used by that other gang over there. For example, commonality of use of Berkeley DBs by perl programs from various sources discourages having multiple Perl binaries in use. (Older perl's are still around for reference but nothing uses them on our site.)
mutual exclusion Q
My Q today may be affected by Apache and/or Perl, so it seems appropriate for this mailing list. I've written a mod-perl module that needs mutual exclusion among Apache child processes using the prefork MPM, so I decided to use a file as a semaphore using Perl's 'flock'. 1. The mutual exclusion doesn't work (in the sense that multiple processes are allowed to execute mutually-excluded code simultaneously) if the post-config phase (before forking) just sysopens the file for write with permissions 0666, and the child processes flock the resulting filehandle ($outer_lock_fh) when they need to. 2. It works if the post-config phase does the same sysopen (to ensure the file exists) and then immediately closes the file, and each child process opens the file for input in its child-init phase. 3. Mutual exclusion does not work (in the same way as in 1) if the post-config phase sysopens for write, closes, and then opens for reading in the same way that the child-init phases do in case 2. This suggests that the forking operation (with demotion to a lower priority user) prevents flock from working properly on the inherited filehandle. Yet my Apache child processes can inherit other open filehandles and tied hashes from the original process, and do I/O via them just fine, without having to re-open them. Can anyone this difference in use of these filehandles? If I could understood why 1-3 act as they do, this would probably help me in the future. Thanks, cmac www.animalhead.com
Outgoing content-length apparently 0
Hope this is the address to which to send a problem/question!
Our Verio Virtual Private Server is FreeBSD-based with Apache 1.3.33.
I've installed mod-perl 1.29 and mod_gzip 1.3.19.2a, and revised our former
CGI scripts to run as perl modules under the Apache API.
They are operating fine, except for one residual problem.
When a script builds a response, mod_gzip squawks in the error_log like
this:
... [error] mod_gzip: EMPTY FILE [/tmp/_16777_106_7.wrk] in sendfile2
... [error] mod_gzip: Make sure all named directories exist and have ...
The corresponding entry in access_log looks like this:
... POST /contact HTTP/1.1 200 0or
... GET /x/whoami.cgi HTTP/1.1 200 0
The zero outgoing length in the log entry agrees with mod_gzip's complaint,
but in reality the correct body is going out to the browser (I was the
browser user for both of the access_log cases above). The former case
(contact) runs under mod_perl, while the latter runs under mod_cgi.
Today I tried an approach to solving this, wherein I build up the whole body
in a scalar $body, and then do a
send_response('text/html');
where send_response is:
sub send_response {
$r-header_out(Content-Length, length $body);
$r-send_http_header(shift);
if (!$r-header_only) {$r-print($body)}
undef $body;
}
The problem still persists. So I searched our logs. The first 200 0 log
entry for a script was seen before I installed mod_gzip, and after I
followed the advice of the Practical mod-perl book, and moved all of the
scripts from /usr/local/apache/cgi-bin to a new directory
/usr/local/apache/perl. Before that the scripts had been running fairly
correctly under mod_perl and Apache::Registry in .../cgi-bin, without
showing any 200 0 log entries for scripts.
None of the following changes in httpd.conf seem to affect the problem:
PerlSendHeaders On/Off
PerlSetupEnv On/Off
Option +ExecCGI in the Location block of a script
Finally I tried making a Location /cgi-bin/contact.cgi block in an attempt
to get as close to the conditions under which the last script output with a
non-zero length was seen. No help!
Scripts that do a redirect using the same send_response routine as above,
show log entries like 302 771, although $body contained just a few bytes.
So my question is: what do I need to do, to make Apache hand off the output
from the script to mod_gzip properly, and (far less important) show the
proper length in the log?
Thanks to anyone who can help,
Craig MacKenna
Los Gatos, CA
408-353-5037
