Re: ApacheCon 2009 in Oakland
On Mon, Oct 19, 2009 at 12:15 AM, Fred Moyer wrote: > Greetings, > > Is anyone here attending ApacheCon in Oakland this year? I am > organizing a mod_perl social. I'll be at the conference at least one > day hacking mod_perl. This lurker will be there.
DBI Connectons accumulate under Mod_perl
Hello! I have a very weird situation. I use MYSQL. Apache 2.2 with mod_perl2 O use Modperl::RegistryBB to run the script, but the script itself is written very nicely for mod_perl. That is no globals at all. Everything is in sub handler. I do not use Apache::DBI, just plain "use DBI;" So, what it does (basic idea) sub handler { db->connect(); .. does all the thing... db->disconnect(); print $output; } however, whe i do in mysql show processlist; I see many connection hanging in sleep state and they grow and grow slowly! Any idea why this happens? Artem
Custom INC per-directory
Hi there, list, I've been trying for what feels like months now to find some way to have two directories specify the same module name to PerlInitHandler but to use two different modules. For example, I'd like to be able to have: File 1: /usr/foo/.htaccess: PerlInitHandler MyAuth::Authnz.pm # We want this to load /modules/foo/MyAuth/Authnz.pm File 2: /usr/bar/.htaccess: PerlInitHandler MyAuth::Authnz.pm # We want this to load /modules/bar/MyAuth/Authnz.pm Impetus: version control. Imagine 'foo' as the production code and 'bar' as the testing code. I've come across quite a few ways we can fake it... but none of them elegant. Can anybody think of a way to achieve directory-based @INC? -- Devin
Re: Custom INC per-directory
> For example, I'd like to be able to have: > > File 1: /usr/foo/.htaccess: > PerlInitHandler MyAuth::Authnz.pm > # We want this to load /modules/foo/MyAuth/Authnz.pm > > > File 2: /usr/bar/.htaccess: > PerlInitHandler MyAuth::Authnz.pm > # We want this to load /modules/bar/MyAuth/Authnz.pm Think what would happen if you did this in a Perl script: BEGIN { @INC = '/modules/foo' } use My::Module; BEGIN { @INC, '/modules/bar' } use My::Module; The second use wouldn't do anything because Perl sees that it has already loaded My::Module. So the only way to do what you want is to run two separate servers (or just use plain CGI - no mod_perl). You're going to need to use different module names. But, if MyAuth::Authnz contains a lot of functionality which you need in both modules, then why not subclass it and put the foo/bar specific stuff into a module which inherits from it? clint
Re: Custom INC per-directory
Well, I was thinking I could get away with something like this... foo's .htaccess: use ModPerl::Util; ModPerl::Util::unload_package(MyAuth::Authnz); unshift(@INC, '/modules/foo'); push @PerlInitHandler, 'MyAuth::Authnz'; bar's .htaccess: use ModPerl::Util; ModPerl::Util::unload_package(MyAuth::Authnz); unshift(@INC, '/modules/bar'); push @PerlInitHandler, 'MyAuth::Authnz'; Though even that has some weirdness to it and doesn't quite work right. I wonder if ModPerl::PackageRegistry in CPAN ( http://search.cpan.org/~crakrjack/ModPerl- PackageRegistry-0.02/lib/ModPerl/PackageRegistry.pm ) could help. It looks to be what I want, but seems a bit strange. I think ultimately, I may have to code some kind of dispatch module that is called from both the production and testing trees and then that module dynamically dispatches the call to the proper module based on the chain of command (as in, who called it). -- Devin On Tue, 2009-10-20 at 19:38 +0200, Clinton Gormley wrote: > > For example, I'd like to be able to have: > > > > File 1: /usr/foo/.htaccess: > > PerlInitHandler MyAuth::Authnz.pm > > # We want this to load /modules/foo/MyAuth/Authnz.pm > > > > > > File 2: /usr/bar/.htaccess: > > PerlInitHandler MyAuth::Authnz.pm > > # We want this to load /modules/bar/MyAuth/Authnz.pm > > Think what would happen if you did this in a Perl script: > > BEGIN { @INC = '/modules/foo' } > use My::Module; > BEGIN { @INC, '/modules/bar' } > use My::Module; > > The second use wouldn't do anything because Perl sees that it has > already loaded My::Module. > > So the only way to do what you want is to run two separate servers (or > just use plain CGI - no mod_perl). > > You're going to need to use different module names. But, if > MyAuth::Authnz contains a lot of functionality which you need in both > modules, then why not subclass it and put the foo/bar specific stuff > into a module which inherits from it? > > clint > -- Cheers, Devin Teske -> CONTACT INFORMATION <- Field Engineer FIS - Vicor Business Unit 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> END TRANSMISSION <-
Re: Custom INC per-directory
On 10/20/2009 01:06 PM, Devin Teske wrote: I've been trying for what feels like months now to find some way to have two directories specify the same module name to PerlInitHandler but to use two different modules. This isn't a limitation of mod_perl but a limitation of Perl. You can't have 2 different modules with the same name. This comes up from time to time on this list and the best advice usually comes down to this: Run multiple apaches (on the same machine if you want). Then put a proxy in front to redirect to the right one based on the host or path or whatever. This is just slightly more complicated to set up, but is the most flexible and doesn't require as many resources as it initially appears. -- Michael Peters Plus Three, LP
Re: GnuPG module producing empty files
Laurent MARTIN wrote: > Has anyone on the list some experience with mod_perl and an alternative > to GnuPG module (which is "simply" a wrapper around pgp binary). I use GnuPG::Interface, and it works fine. Regards, Michael Schout
Re: Custom INC per-directory
Yes, but that's a horrible solutions (thinking "utter nightmare" when it comes to system administration, et. al.). I've currently got it working with this (which seems to be MUCH more elegant, only requires a single Apache server, no virtual-hosts, is directory-based, and requires no special configs in httpd.conf -- just .htaccess files -- given of course that MP2 is loaded): my $env = $0; $env =~ s:^(/usr/env\d+)/.*:$1:; push @PerlInitHandler, "MyAuth::Dispatch"; push @PerlSetEnv, [ LIBDIR => "$env/lib" ]; I feel that's a much better solution. Here's Dispatch: package MyAuth::Dispatch; use strict; use warnings; sub handler() { my ( $r ) = @_; my $libdir = $ENV{LIBDIR} || ""; unshift @INC, $libdir if $libdir; require MyAuth::Authnz; return &MyAuth::Authnz::handle($r); } 1; Now doesn't that work a heck of a lot better? -- Devin On Tue, 2009-10-20 at 14:32 -0400, Michael Peters wrote: > On 10/20/2009 01:06 PM, Devin Teske wrote: > > > I've been trying for what feels like months now to find some way to have > > two directories specify the same module name to PerlInitHandler but to > > use two different modules. > > This isn't a limitation of mod_perl but a limitation of Perl. You can't > have 2 different modules with the same name. This comes up from time to > time on this list and the best advice usually comes down to this: > > Run multiple apaches (on the same machine if you want). Then put a proxy > in front to redirect to the right one based on the host or path or > whatever. This is just slightly more complicated to set up, but is the > most flexible and doesn't require as many resources as it initially appears. > -- Cheers, Devin Teske -> CONTACT INFORMATION <- Field Engineer FIS - Vicor Business Unit 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> END TRANSMISSION <-
Re: Custom INC per-directory
Ok, my last attempt, which involved using blocks, didn't work either (and I know why). Here's the final solution that is working 100% of the time. So... the problem is that 'require' and 'use' want to conditionally not re- import the library based on the module name (which doesn't change)... ...the solution is to not use 'use' or 'require' and instead beat down on the INC hash yourself, and perform a manual 'do' statement. FILE: /foo/htdocs/.htaccess: == allow from all PerlInitHandler MyAuth::Dispatch PerlSetEnv LIBDIR /foo/lib FILE: /bar/htdocs/.htaccess: == allow from all PerlInitHandler MyAuth::Dispatch PerlSetEnv LIBDIR /bar/lib FILE: /usr/local/apache2/MyAuth/Dispatch.pm: == package MyAuth::Dispatch; use strict; use warnings; sub handler() { my ( $req_conn ) = @_; my $LIBDIR = $ENV{LIBDIR} || ""; my $LIBNAME = 'MyAuth/Allow.pm'; $INC{$LIBNAME} = "$LIBDIR/$LIBNAME"; do "$LIBDIR/$LIBNAME"; return &MyAuth::Allow::handler($req_conn); } 1; FILE: /foo/lib/MyAuth/Allow.pm: == package MyAuth::Allow; use strict; use warnings; use Apache2::Const -compile => qw(OK); sub handler() { my ( $req_conn ) = @_; warn 'foo-version of allow called'; return Apache2::Const::OK; } 1; FILE: /bar/lib/MyAuth/Allow.pm: == package MyAuth::Allow; use strict; use warnings; use Apache2::Const -compile => qw(OK); sub handler() { my ( $req_conn ) = @_; warn 'bar-version of allow called'; return Apache2::Const::OK; } 1; This allows me to use a different library depending on what directory the user is browsing (coded in the .htaccess file of the directory tree). -- Devin On Tue, 2009-10-20 at 12:07 -0700, Devin Teske wrote: > Yes, but that's a horrible solutions (thinking "utter nightmare" when it > comes to system administration, et. al.). > > > I've currently got it working with this (which seems to be MUCH more > elegant, only requires a single Apache server, no virtual-hosts, is > directory-based, and requires no special configs in httpd.conf -- > just .htaccess files -- given of course that MP2 is loaded): > > > my $env = $0; > $env =~ s:^(/usr/env\d+)/.*:$1:; > push @PerlInitHandler, "MyAuth::Dispatch"; > push @PerlSetEnv, [ LIBDIR => "$env/lib" ]; > > > > I feel that's a much better solution. Here's Dispatch: > > package MyAuth::Dispatch; > use strict; > use warnings; > sub handler() > { > my ( $r ) = @_; > my $libdir = $ENV{LIBDIR} || ""; > unshift @INC, $libdir if $libdir; > require MyAuth::Authnz; > return &MyAuth::Authnz::handle($r); > } > 1; > > > Now doesn't that work a heck of a lot better? > -- > Devin > > > > > > > > On Tue, 2009-10-20 at 14:32 -0400, Michael Peters wrote: > > On 10/20/2009 01:06 PM, Devin Teske wrote: > > > > > I've been trying for what feels like months now to find some way to have > > > two directories specify the same module name to PerlInitHandler but to > > > use two different modules. > > > > This isn't a limitation of mod_perl but a limitation of Perl. You can't > > have 2 different modules with the same name. This comes up from time to > > time on this list and the best advice usually comes down to this: > > > > Run multiple apaches (on the same machine if you want). Then put a proxy > > in front to redirect to the right one based on the host or path or > > whatever. This is just slightly more complicated to set up, but is the > > most flexible and doesn't require as many resources as it initially appears. > > -- Cheers, Devin Teske -> CONTACT INFORMATION <- Field Engineer FIS - Vicor Business Unit 626-573-6040 Office 510-735-5650 Mobile devin.te...@metavante.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> END TRANSMISSION <-
Re: Custom INC per-directory
On Tue, Oct 20, 2009 at 5:16 PM, Devin Teske wrote: > the problem is that 'require' and 'use' want to conditionally not re- > import the library based on the module name (which doesn't change)... > > ...the solution is to not use 'use' or 'require' and instead beat down > on the INC hash yourself, and perform a manual 'do' statement. Performance will suffer, but it will work for most code. It will fail on the same kind of things that Apache::Reload fails on. BTW, there are many threads on this in the list archives, with other possible solutions. - Perrin
Re: Custom INC per-directory
On 10/20/2009 05:24 PM, Perrin Harkins wrote: Performance will suffer, but it will work for most code. Right. Your Perl interpreter will be persistent, but none of your code will be cached and thus every module you use will be reloaded on every request. It also means you can't do pretty standard things like preloading modules on start up. And if you're going to have your own dispatch class, doing your own module loading, why is having separate namespaces a big deal? > It will fail > on the same kind of things that Apache::Reload fails on. This usually affects modules that dynamically add methods to classes (like most ORMs, Moose, etc). -- Michael Peters Plus Three, LP
Re: Custom INC per-directory
Wouldn't using the Parent option ( http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_ ) work better for what you're trying to do? -- Alan
Re: Custom INC per-directory
Alan Young wrote: Wouldn't using the Parent option ( http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_ ) work better for what you're trying to do? Parents requires vhosts, which he said he doesn't want to use. Adam
Re: Custom INC per-directory
I don't know the specifics of your project so it's quite possible that I'm missing something, but this all seems like an incredibly bad idea. Sure you can knock some cringe inducing code together and get it to technically work, but the very fact that you need to resort to these sort of unorthodox methods should be a hint that you might have something wrong in your systems or software design. Again, I lack a full perspective here and maybe you have good reasons, but this sounds like a problem screaming to be solved in conception, not in implementation. Adam Prime wrote: Alan Young wrote: Wouldn't using the Parent option ( http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_ ) work better for what you're trying to do? Parents requires vhosts, which he said he doesn't want to use. Adam
Re: Custom INC per-directory
Hey All, I'll try to answer multiple people's questions/suggestions in one e-mail. (and I notice that there are 2 more that I haven't read et from Adam and Brad). Use of 'Parent' implies that different virtual hosts. Imagine thusly... You want to run what are essentially "different websites" but Not accessible under different sub-domains (i.e. virtual hosting) but rather under different pathnames (URIs). Though more aptly, each base URI directory (Alias'd to a real directory on disk) is a sandbox for a different version of the same website. Therefore a central CVS repository feeds separate (and inequal) branches of the CVS repos. Each sandbox runs a different release/tag and receives updates automatically (this is also answering some other questions from the list). So the CVS can therefore move independently of the sandboxes and the sandboxes move at the pace of the tags. Achieved is the ability to have a "production" environment and a "testing" environment available under say, URI /foo and URI /bar. There are no virtual hosts in the schema. Though, that's not to say we couldn't perhaps explore that. We'd like to get away from that though and let the sandbox be a fully-owned entity of itself, blissfully unaware of anything outside its universe (including Apache, and vice versa). Apache shouldn't have to know about how the environment is configured, and the environment should need to know how Apache is configured. A solution that exists entirely in .htaccess files is actually preferred. Therefore, one environment could be using MP2 based authentication, another using Basic Auth, another using Digest auth, another using a different version of MP2-based auth, ad nauseum. Make sense? I knew that there would be performance limitations to manually performing a do everytime. Though it perhaps may be that if I add a few if's and checks here and there, it could be "pretty good", though not up to par with MP2's caching. What I experience under MPM prefork is that once the child has cached a copy of the module, it's not reloaded unless say, unload_package in ModPerl::Util is used, or Apache2::Reload is used (with extreme prejudice). I actually think that balancing %INC manually within the child process on an as-needed basis based on the access of the different environments is pretty efficient. It's somewhat of an even balance between caching, no-caching and re-caching. (we check $INC{$modname} and if it's for a different environment, we modify the value to be the current environment and thus MP2 should be caching that file upon 'do' assumedly -- based on what I've read so far in the online 'tomes'). Also, the different environments are not accessed evenly. There's usually one environment that receives 99% of the traffic throughout the day. So the caching should remain pretty static. Generating the htaccess files are no problem. -- Devin Teske VICOR, A Metavante Company Systems Programmer II / Field Engineer - Wells Fargo (LAX) dte...@vicor.com 510-735-5650 Mobile 510-621-2038 Office This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. - Original Message - From: "Alan Young" To: "Devin Teske" Cc: "Michael Peters" ; Sent: Tuesday, October 20, 2009 4:05 PM Subject: Re: Custom INC per-directory Wouldn't using the Parent option ( http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_ ) work better for what you're trying to do? -- Alan
Re: Custom INC per-directory
Hehe, it does seem kind of looney, doesn't it? In a broad scope (see last e-mail which explains the benefits we gain), it is solving a real problem. For example, which might be efficient enough... # ... my $MOD = SomeMod; sub handler() { # ... if ( $INC{$MOD} ne $ENV{'LIBDIR'}/$MOD ) { delete $INC{$MOD}; } require $MOD; # ... } # ... Might as well just make a new sub to replace require()... sub librequire() { my ($filename) = @_; if (exists $INC{$filename}) { return 1 if ( $INC{$filename} =~ m#^$ENV{'LIBDIR'}/# ); } my ($realfilename,$result); $realfilename = "$ENV{'LIBDIR'}/$filename"; if (-f $realfilename) { $INC{$filename} = $realfilename; $result = do $realfilename; } if ($@) { $INC{$filename} = undef; die $@; } elsif (!$result) { delete $INC{$filename}; die "$filename did not return true value"; } else { return $result; } } That would allow me to simply say "librequire Some::Mod" rather than "require Some::Mod" and have the same effect, allowing us to defeat the cache when necessary, always getting the right module, whether it's provided by cache or not. -- Devin Teske - Original Message - From: "Brad Van Sickle" To: "Adam Prime" Cc: "Alan Young" ; "Devin Teske" ; "Michael Peters" ; Sent: Tuesday, October 20, 2009 4:22 PM Subject: Re: Custom INC per-directory I don't know the specifics of your project so it's quite possible that I'm missing something, but this all seems like an incredibly bad idea. Sure you can knock some cringe inducing code together and get it to technically work, but the very fact that you need to resort to these sort of unorthodox methods should be a hint that you might have something wrong in your systems or software design. Again, I lack a full perspective here and maybe you have good reasons, but this sounds like a problem screaming to be solved in conception, not in implementation. Adam Prime wrote: Alan Young wrote: Wouldn't using the Parent option ( http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_ ) work better for what you're trying to do? Parents requires vhosts, which he said he doesn't want to use. Adam
Re: Custom INC per-directory
Ended up just knocking out the PerlSetEnv LIBDIR in favor of giving librequire a new first variable of $r and basing the lib path off of $r->filename() Why $r->filename() you say? Because $r->uri() is before any Alias translations with mod_alias. -- Devin Teske - Original Message - From: "Devin Teske" To: "Brad Van Sickle" ; "Adam Prime" Cc: "Alan Young" ; "Michael Peters" ; Sent: Tuesday, October 20, 2009 4:51 PM Subject: Re: Custom INC per-directory Hehe, it does seem kind of looney, doesn't it? In a broad scope (see last e-mail which explains the benefits we gain), it is solving a real problem. For example, which might be efficient enough... # ... my $MOD = SomeMod; sub handler() { # ... if ( $INC{$MOD} ne $ENV{'LIBDIR'}/$MOD ) { delete $INC{$MOD}; } require $MOD; # ... } # ... Might as well just make a new sub to replace require()... sub librequire() { my ($filename) = @_; if (exists $INC{$filename}) { return 1 if ( $INC{$filename} =~ m#^$ENV{'LIBDIR'}/# ); } my ($realfilename,$result); $realfilename = "$ENV{'LIBDIR'}/$filename"; if (-f $realfilename) { $INC{$filename} = $realfilename; $result = do $realfilename; } if ($@) { $INC{$filename} = undef; die $@; } elsif (!$result) { delete $INC{$filename}; die "$filename did not return true value"; } else { return $result; } } That would allow me to simply say "librequire Some::Mod" rather than "require Some::Mod" and have the same effect, allowing us to defeat the cache when necessary, always getting the right module, whether it's provided by cache or not. -- Devin Teske - Original Message - From: "Brad Van Sickle" To: "Adam Prime" Cc: "Alan Young" ; "Devin Teske" ; "Michael Peters" ; Sent: Tuesday, October 20, 2009 4:22 PM Subject: Re: Custom INC per-directory I don't know the specifics of your project so it's quite possible that I'm missing something, but this all seems like an incredibly bad idea. Sure you can knock some cringe inducing code together and get it to technically work, but the very fact that you need to resort to these sort of unorthodox methods should be a hint that you might have something wrong in your systems or software design. Again, I lack a full perspective here and maybe you have good reasons, but this sounds like a problem screaming to be solved in conception, not in implementation. Adam Prime wrote: Alan Young wrote: Wouldn't using the Parent option ( http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_ ) work better for what you're trying to do? Parents requires vhosts, which he said he doesn't want to use. Adam