Re: dynamic vs. mostly static data
Matt Sergeant wrote: On Thu, 2 Nov 2000, Neil Conway wrote: I'm writing a web app in mod_perl, using a PostgreSQL database backend and HTML::Template. In looking for ways to optimize performance, I noticed that although my code is doing several (say, 4-5) database queries per handler/webpage, a large part of the data (~2 queries) is mostly static (it will change perhaps once per week, or once per month). It's obviously inefficient to run these queries on the database for every single request. How can I 'cache' this data so that all Apache children can access it quickly? Is there a way to automatically update this cache periodically (say every 10 minutes)? Also, this solution should work on any reasonably modern UNIX system (Win32 is not important for now). Along with all the other fine suggestions here, you might like to check out the Memoize module on CPAN. I don't think it yet offers a way to invalidate the cache, but I know that is planned. What about IPC::SharedCache ? This should allow you to cache stuff in IPC and updateit from outside apache via a crond job ? just my 2 euro's worth. Greg -- Matt/ /||** Director and CTO ** //||** AxKit.com Ltd ** ** XML Application Serving ** // ||** http://axkit.org ** ** XSLT, XPathScript, XSP ** // \\| // ** Personal Web Site: http://sergeant.org/ ** \\// //\\ // \\
Re: Apache_1.3.14 mod_layout 1.24_01 core dump
Hi there, On Fri, 3 Nov 2000, George Sanderson wrote: First, off, let me clairfy, I am a Newbie in almost every way. . . We've all been there sometime. Some of us never left. I have the following configuration Apache 1.3.14, mod_perl-1.24_01 (static), Perl-5.6.0, Linux 2.2.14. If I load load Apache::Icon, and mod_layout 2.8 (DSO), Apache will core 1. Make sure that the same compiler built your Perl as built mod_perl. 2. Try compiling all the modules into Apache statically. 3. Try Perl 5.005_03. 4. You might want to get 2.2.16 - nothing to do with mod_perl, or your problem, but there's a security bug in Linux 2.2 less than 2.2.16 which allows local users to get root access. I have a similar problem when I replace mod_layout with mod_auth_mda (DSO), however I do not have a problem if i do not load mod_perl with the other modules You said you'd got mod_perl statically compiled. You mean you have another copy of Apache, right? No error_log output is generated. Yeah, core dumps will do that. 73, Ged.
RH7.0+Apache_1.3.14+mod_perl-1.24_01 -- bad combination ?
When I'm compiling mod_perl on RH6.2 everything works just fine, but when I try the same steps on RH7.0, it refuses to run apache server, saying that I don't have a lockfile on the local disk when I do. (compiling stand alone apache on RH7.0 works but when mod_perl compiles apache, it doesn't work) I tried downloading perl5.6.0 and compiling it and mod_perl with the same compiler .. it still didn't work .. did anybody have the same problem ? is there a solution ? or should I stick to RH6.2 for now .. Dennis
Re: POST results in HTTP/1.0 (null) ??
On Sat, 4 Nov 2000, G.W. Haywood wrote: On Fri, 3 Nov 2000, Paul J. Lucas wrote: Why is the content merely "HTTP/1.0 (null)"? What happened to the other 6900 bytes or so? Maybe you need to generate them... I did: if you read my original post, I wrote: Orig If I write a mini server in Perl that just dumps the contents Orig of the HTTP post, the posted data is correct. I'm doing a POST with curl: the data curl generates is fine as is in evidence by the above mini server. do you think you could give us a bit more to go on? I'd be happy to: what else would you like to know? - Paul
ANNOUNCE: Apache::SSI 2.14
The URL http://mathforum.com/~ken/modules/archive/Apache-SSI-2.14.tar.gz has entered CPAN as file: $CPAN/authors/id/K/KW/KWILLIAMS/Apache-SSI-2.14.tar.gz size: 23758 bytes md5: 805ef1fb308a014dad5c923cfa676c26 Revision history for Perl extension Apache::SSI. 2.14 Fri Nov 3 16:21:45 EST 2000 We now use Apache::test for the 'make test' part of installation. Made changes throughout the distribution for this. Now correctly sets the DOCUMENT_URI environment variable. [[EMAIL PROTECTED] (Jacqui Caren)] Nested if...endif sections now work correctly. Previously they were not allowed. [[EMAIL PROTECTED] (David Burry)] If the structure of if...endif XSSI sections is malformed, we now return an error instead of dying. This seems better. Note that mod_include will just continue on without any errors, giving really strange results that are hard to debug. The error() method will not attempt to write to the error logs in standalone mode anymore. This was causing a fatal error in standalone mode. Added a test to the test suite that makes sure simple subclassing of Apache::SSI works as expected. Added a couple of tests (not exercised yet) for nested if...elsif XSSI structures. Corresponding code is mostly written, but needs some more scrutiny testing. ------ Ken Williams Last Bastion of Euclidity [EMAIL PROTECTED]The Math Forum
Re: how to do this??
On Thu, 2 Nov 2000, Michael wrote:
#handler
package MyHandler;
use Module qw(sub1 sub2);
use Apache;
use vars qw(@ISA)
@ISA = qw (Module Apache);
calling sub1 from apache
using THIS handler named "MyHandler" results in Module::sub1
identifing the "caller" as Apache.
Okay, I'll bite... Why do you want to change the caller? I mean,
Apache really is the caller if this is being invoked directly in
response to a request.
You had to ask heh... heh... code re-use of course.
I want to be able to use a custom set of variables (multiple
personalities). i.e. create a module to do a common job. Apache
directs the return to a handler for a particular user via a
PerlSetVar in the httpd.conf script. Rather than duplicate code, for
each user a module is used, but the variables belong to the user. One
could write a lot of code to pass the variables into the module for
EACH subroutine called from the handler, but the AUTOLOAD trick and
passing the caller name make for a very compact piece of user code.
Basically all that is there is an init routine which is only used if
the user calls his handler from user space and the AUTOLOAD piece
which is used by both the user and Apache.
The module is 600 lines of perl ~ 16kb containing 20 subroutines,
each with differing variable requirements.
The whole user handler consists of the code below which allows the
Module free access to all of the variables in the users handler
space by virtue of knowing the "caller" name.
package user32::Access;
use Common::Access;
use vars qw($LogSerial %var @ISA);
@ISA = qw(Common::Access);
my $acc = 'Common::Access';
# a bunch of variables here
use constant BLAH1 =
{VAR1 ='some stuff',
VAR2= 'more stuff'
};
use constant var = \%var;
#
# package reference, called only from user space
#
sub pkg {
my ($proto) = @_;
my $class = ref($proto) || $proto;
my $self = {};
bless ($self, $class);
return $self;
}
#
# autoload extensions from 'ACCESS package'
#
sub AUTOLOAD {
no strict;
my @sub = split('::', $AUTOLOAD);
my $sub = $sub[$#sub];
return undef unless $subroutine;
return $acc-{$sub}(__PACKAGE__,@_); # execute remote sub
}
1;
[EMAIL PROTECTED]
input from external process
Hi Is it possible to have an existing mod_perl application read from an external process? The case in point is procmail piping mails to the mod_perl application. With high volumes of email, we would prefer not to have to launch the perl interpreter each time to perform the tasks specified by the incoming email. Nasty way of effecting this communication is to have each mail message saved to a file and do a readdir every time interval. Is it possible to have the mod_perl process listen on a socket (or something) that procmail may write to? -- Eric Smith Fruitcom.com Amsterdam Wire phone : +31 20 528 7340 Mobile: +31 6 241 259 16 www.fruitcom.com
Dealing with spiders
This is slightly OT, but any solution I use will be mod_perl, of course. I'm wondering how people deal with spiders. I don't mind being spidered as long as it's a well behaved spider and follows robots.txt. And at this point I'm not concerned with the load spiders put on the server (and I know there are modules for dealing with load issues). But it's amazing how many are just lame in that they take perfectly good HREF tags and mess them up in the request. For example, every day I see many requests from Novell's BorderManager where they forgot to convert HTML entities in HREFs before making the request. Here's another example: 64.3.57.99 - "-" [04/Nov/2000:04:36:22 -0800] "GET /../../../ HTTP/1.0" 400 265 "-" "Microsoft Internet Explorer/4.40.426 (Windows 95)" 5740 In the last day that IP has requested about 10,000 documents. Over half were 404 requests where some 404s were non-converted entities from HREFs, but most were just for documents that do not and have never existed on this site. Almost 1000 request were 400s (Bad Request like the example above). And I'd guess that's not really the correct user agent, either In general, what I'm interested in stopping are the thousands of requests for documents that just don't exist on the site. And to simply block the lame ones, since they are, well, lame. Anyway, what do you do with spiders like this, if anything? Is it even an issue that you deal with? Do you use any automated methods to detect spiders, and perhaps block the lame ones? I wouldn't want to track every IP, but seems like I could do well just looking at IPs that have a high proportion of 404s to 200 and 304s and have been requesting over a long period of time, or very frequently. The reason I'm asking is that I was asked about all the 404s in the web usage reports. I know I could post-process the logs before running the web reports, but it would be much more fun to use mod_perl to catch and block them on the fly. BTW -- I have blocked spiders on the fly before -- I used to have a decoy in robots.txt that, if followed, would add that IP to the blocked list. It was interesting to see one spider get caught by that trick because it took thousands and thousands of 403 errors before that spider got a clue that it was blocked on every request. Thanks, Bill Moseley mailto:[EMAIL PROTECTED]
ANNOUNCE: Apache::Filter
The URL http://forum.swarthmore.edu/~ken/modules/archive/Apache-Filter-1.013.tar.gz has entered CPAN as file: $CPAN/authors/id/K/KW/KWILLIAMS/Apache-Filter-1.013.tar.gz size: 25569 bytes md5: e022ef5a8c526e04d0f25f9efe7de8ba I decided that Apache::Filter had to change its interface or be forever stuck against the roadblocks it's been stuck against for a while. So there's a big implementation change in this version, and a small interface change. Version: 1.013 Date: 2000/11/05 04:37:10 Cleaned up new implementation begun in 1.012. Revised documentation accordingly. Added overloaded method for $r-print(). Version: 1.012 Date: 2000/09/07 05:17:27 (not released publically) Major reorganization of code in Apache::Filter. Now Apache::Filter is a subclass of Apache, which allows much smoother overriding of the functionality that needs to be changed for filtering. In particular, send_http_headers() and send_fd() behave well now. Added new filter_register() method, which must always be called. filter_input() isn't necessary anymore when you don't actually need the input filehandle it returns. Thanks much to Geoffrey Young, who helped discuss ideas for the code changes and helped prod me along when I dragged my feet. =) ------ Ken Williams Last Bastion of Euclidity [EMAIL PROTECTED]The Math Forum
ANNOUNCE: Apache::SSI 2.15
The URL http://forum.swarthmore.edu/~ken/modules/archive/Apache-SSI-2.15.tar.gz has entered CPAN as file: $CPAN/authors/id/K/KW/KWILLIAMS/Apache-SSI-2.15.tar.gz size: 23853 bytes md5: 3bbf7470d9324c13d4105e534f189310 Revision history for Perl extension Apache::SSI. 2.15 Sun Nov 5 00:24:52 EST 2000 Updated code to work with Apache::Filter 1.013 and greater. If you're using Apache::Filter 1.012 or lower, you must use 2.14 or lower of Apache::SSI. If you don't use Apache::Filter, you don't need to care. ------ Ken Williams Last Bastion of Euclidity [EMAIL PROTECTED]The Math Forum
ANNOUNCE: Apache::Compress 1.003
The URL http://forum.swarthmore.edu/~ken/modules/archive/Apache-Compress-1.003.tar.gz has entered CPAN as file: $CPAN/authors/id/K/KW/KWILLIAMS/Apache-Compress-1.003.tar.gz size: 11028 bytes md5: 9d1b571995961223bdb5e005f42e1c76 Version: 1.003 Date: 2000/11/05 05:36:46 Updated code to work with Apache::Filter 1.013 and greater. If you're using Apache::Filter 1.012 or lower, you must use 2.14 or lower of Apache::Compress. If you don't use Apache::Filter, you don't need to care. Added Compress::Zlib to the PREREQ_PM list in Makefile.PL. ------ Ken Williams Last Bastion of Euclidity [EMAIL PROTECTED]The Math Forum
