Determining when a cached item is out of date
I'm moving into the XML space and one of the things I see is that XML processing is very expensive, so AxKit, PageKit, et al make extensive use of caching. I'm keeping all of my data in a MySQL DB with about 40 tables. I'm pretty clear about how to turn that MySQL data into XML and turn the XML into HTML, WML, or what have you. But I haven't been able to wrap my skull around knowing when the data in Mysql is fresher than what is in the cache without doing a major portion of the work needed to generate that web page to begin with. Do AxKit and PageKit pay such close attention to caching because XML processing is so deadly slow that one doesn't have a hope of reasonable response times on a fast but lightly loaded server otherwise? Or is it because even a fast server would quickly be on its knees under anything more than a light load? With a MVC type architecture, would it make sense to have the Model objects maintain the XML related to the content I want to serve as static files so that a simple stat of the appropriate XML file tells me if my cached HTML document is out of date? One more thing. Perrin Harkins' eToys case study casually mentions a a means of removing files from the mod_proxy cache directory so that mod_proxy had to go back to the application servers to get an up to date copy. I haven't seen anything in the mod_proxy docs that says this is possible. Does something like that exist outside of eToys? I don't know, maybe my Prussian Perfection gene has taken over again and wants a bigger win than I need to get ... -- Christopher L. Everett Chief Technology Officer The Medical Banner Exchange Physicians Employment on the Internet
Re: Determining when a cached item is out of date
Perrin Harkins wrote: Christopher L. Everett wrote: But I haven't been able to wrap my skull around knowing when the data in Mysql is fresher than what is in the cache without doing a major portion of the work needed to generate that web page to begin with. There are three ways to handle cache synchronization: 1) Time to Live (TTL). This approach just keeps the data cached for a certain amount of time and ignores possible updates. This is the most popular because it is easy to implement and gives good performance. Cache::Cache and friends work this way. I'm cursed by my installed base. Our users go into our site to make sure their changes are up correctly. I don't think a 15 second TTL would do us any good :) 2) Polling. This involves checking the freshness of the data before serving it from cache. This is only feasible if you have a way to check freshness that is faster than re-generating the data. This is difficult in most situations. 3) Invalidation. This approach involves removing cache entried whenever you update something that would make them out of date. This is only feasible if you have total control over the update mechanism and can calculate all the dependencies quickly. I see where one could combine polling and invalidation, for instance by having empty files representing a page that get touched when the data for them go out of date. But again, there is the issue of mapping changed data onto dependent pages. I guess one way to do that is to track which database rows appear in which pages in the database. Since typically I do several database operations to generate a page, adding one more delete or insert operation whanever a new page is generated won't kill me. Could get nasty in a big hurry if I'm not careful though. Perhaps a cache manager object/class that handles cache mappings invalidation would be handy. Or maybe do that as part the PageKit base Model class. One more thing. Perrin Harkins' eToys case study casually mentions a a means of removing files from the mod_proxy cache directory so that mod_proxy had to go back to the application servers to get an up to date copy. I haven't seen anything in the mod_proxy docs that says this is possible. Does something like that exist outside of eToys? Not in mod_proxy. We added it ourselves. I don't have the code for that anymore, but it's not hard to do if you have a competent C hacker handy. Maybe mod_accel has this feature. Well, I like to think I'm language independent, heh. But reinventing the wheel isn't cheap. I'll root around some more. -- Christopher L. Everett Chief Technology Officer The Medical Banner Exchange Physicians Employment on the Internet
Re: OSCON ideas - more talk ideas
If its not too late to weigh in with ideas: 1) A large chunk how to on doing advanced types of XML processing with perl, I'm really interested in the idea that I can serve my content to PDA's (which is the up-and-coming killer platform for my market) and in AppML (Application Markup Language) for which I can find only dribs and draps of information (a description of a toy application, an X Schema, and not much more). 2) Tutorial on persistent objects stored in MySQL, with a view to encoding business rules. 3) Implementing a toy web application using Apache::PageKit -- Christopher L. Everett Chief Technology Officer The Medical Banner Exchange Physicians Employment on the Internet
mod_proxy problems
I was updating my front end to 1.3.24 (I know, should have checked for problems in the newsgroups/mailing lists). While I was at it I updated debian. Now, the apache child processes on my front end server are segfaulting whenever a URL defined by ProxyPass/ReverseProxyPass statements gets accessed. Assuming that whatever I messed up with mod_proxy won't get fixed for a long while (this is way over my head), and because I have no real way of stepping back to my previous config, what would be another way of bringing my mod_perl server back into the game with a front end server? Christopher L. Everett
Weird IE cookie behaviour
Hello, I've gotten Apache::AuthCookie to run against Netscape and Mozilla browsers. However, when I try to get it to work with IE with prompting enabled for accepting cookies, I never get prompted to accept a cookie, so it appears that the browser is refusing to acknowlegde that the cookie was received. I've looked high and low for resources on this. So far I've rewritten my AuthCookie subclass to use CGI::Cookie, and added a ton more debug statements, with no effect or light on the problem. I _must_ get this working with IE. Does anyone have a clue stick for me? --Christopher
SSL Certificate Distingushed Name size in bytes
Hello, This is slightly off topic, but I can't find a firm specification anywhere on the Internet that says how big a X.500 Distinguished Name could be. I've set up MySQL to have fields cert_subject_dn and cert_issuer_dn as char(255) not null. But I'm wondering if these things could get longer? Does anyone have anything useful to tell me? --Christopher
SSL and thin/fat server setups.
Hello all, I've been running apache+mod_perl servers with apache+mod_ssl front-ends, and been quite happy with this type of setup for quite some time. Now I need to use SSL certificates for authenticating users of an online database. It seems like there's no way to get the SSL information that the front-end sees to the back-end server because the SSL protocol underlies the HTTP protocol (outside of writing a custom apache module, and passing back the cert info in headers) and there's no such thing as an SSL proxy module that I've been able to find. Right now, I'm considering setting up a very lightweight apache+mod_perl+ssl+mod_proxy frontend with just a single perl auth/authz handler installed, and have that decrypt, authenticate, authorize, and proxy all SSL requests back to the fat server. Then I revert the apache+mod_ssl front end to a vanilla apache server and have it handle all plain HTTP requests. Before I do this, I'd just like to know if anyone has any other ideas on how to do this. --Christopher
SSL and thin/fat server setups.
Hello all, I've been running apache+mod_perl servers with apache+mod_ssl front-ends, and been quite happy with this type of setup for quite some time. Now I need to use SSL certificates for authenticating users of an online database. It seems like there's no way to get the SSL information that the front-end sees to the back-end server because the SSL protocol underlies the HTTP protocol (outside of writing a custom apache module, and passing back the cert info in headers) and there's no such thing as an SSL proxy module that I've been able to find. Right now, I'm considering setting up a very lightweight apache+mod_perl+ssl+mod_proxy frontend with just a single perl auth/authz handler installed, and have that decrypt, authenticate, authorize, and proxy all SSL requests back to the fat server. Then I revert the apache+mod_ssl front end to a vanilla apache server and have it handle all plain HTTP requests. Before I do this, I'd just like to know if anyone has any other ideas on how to do this. --Christopher
Apache::Session problems, film at 11:00 ...
All:
I'm getting very odd behavior out of Apache::Session, with
serious problems using both the MySQL and File variants.
Yes, I know I've come here with this problem before. Sigh.
I even fixed it, although it was one of those things where I
didn't quite know why it started working. Anyway, it stopped
working about a week ago, and, as usual, I have no clue. Hence
this plea for help:
With Apache::Session::File, this code creates a new session id
with every request. The lock file for each session remains in
the lock directory. I ran a 'chmod -R 777 dirname' on both
the session store and lock directories.
With Apache::Session::MySQL, this code behaves more normally:
it reuses the session id, the way [I believe] it should, except
$session{state} never seems to make it into the database. I
say that because I look at the contents of the sessions table
between transactions, and it looks like this:
mysql select * from sessions;
+--+---+
| id |
a_session |
+--+---+
| 4def39f4e8144aede90532951232c040
| |
+--+---+
1 row in set (0.00 sec)
I did make sure that the right privileges existed for the
database user accessing the sessions table.
I tried uninstalling Apache::Session ('rm -rf
/usr/local/lib/perl5/site_perl/5.6.0/Apache/Session*'),
and reinstalled it using CPAN, on the theory that I may have
diddled it while checking out its code. But that didn't help.
Here's the (relevant) code, with short, annotated, log extract
following:
##
## Physemp::Search
##
package Physemp::Search;
use strict;
use Apache;
use Apache::Request;
use Apache::Constants qw( :common );
use CGI::Cookie;
use Apache::Session::MySQL;
use DBI;
use Data::Dumper;
my (%states, %_CACHE);
sub handler ($$) {
my ($class, $q) = @_;
my $self = $class-new(my $r = Apache::Request-new($q));
my $html = '';
$self-get_session($r);
my $coderef =
$self-{make}-{$self-frame}-{$self-page}-{$self-command}
|| \unimplemented;
$html = $self-$coderef($r);
$r-content_type('text/html');
$self-put_or_del_session($r);
$r-send_http_header;
print $html;
return OK;
}
sub get_session {
my ($self, $r) = @_;
my %session;
my $cookie_str = $r-header_in('Cookie');
my %cookies = $cookie_str eq '' ? ( ) :
CGI::Cookie-parse($cookie_str);
if (exists $cookies{SessionID}) {
(my $session_id = $cookies{SessionID}-value) =~ s/([0-9a-f]+)/$1/;
eval {
tie %session, $self-{tieclass}, $session_id, $self-{tieattrs};
};
if ($@) {
$r-log_error($@);
$r-log_error(get_session: No session data found.);
$self-{state} = { };
$self-{session_id} = '';
} else {
$r-log_error(get_session: Session data found.);
$r-log_error(get_session: \$session{state} is \n, Dumper
$session{state});
$session{state} = { account = {} } unless exists $session{state};
$self-{session_id} = $session{_session_id};
$self-{state} = $session{state};
}
undef %session;
} else {
$r-log_error(get_session: No Session ID cookie.);
$self-{state} = { };
$self-{session_id} = '';
}
$r-log_error(get_session: Session ID is '$self-{session_id}'.);
$r-log_error(get_session: State is \n, Dumper $self-{state});
}
sub put_or_del_session {
my ($self, $r) = @_;
my (%session, $cookie);
if ($self-command eq 'make' or $self-page eq 'action') {
eval {
tie %session,
$self-{tieclass},
($self-{session_id} eq '' ? undef : $self-{session_id}),
$self-{tieattrs};
};
if ($@) {
$r-log_error(put_or_del_session: $@);
eval { tie %session, $self-{tieclass}, undef, $self-{tieattrs};
};
if ($@) {
$r-log_error(put_or_del_session: $@);
return; # WTH, we can't do any good here
}
}
if ($self-command eq 'logout') {
$r-log_error(put_or_del_session: deleting session.);
$cookie = CGI::Cookie-new( -name= 'SessionID',
-path= $self-{uri},
-domain = '.physemp.com',
-expires = '-10m',
-value = '' );
tied(%session)-delete;
} else {
$r-log_error(put_or_del_session: updating session.);
$session{state} = $self-{state};
$session{changes}++;
$r-log_error(put_or_del_session: Session ID is
'$session{_session_id}'.);
$r-log_error(put_or_del_session: State is \n, Dumper
$session{state});
$cookie = CGI::Cookie-new( -name= 'SessionID',
Re: Optimizing memory use of modperl servlets
Why not store all that static information using Cache::File? --Christopher Everett
mirroring data across a server cluster
This is important when clustering for redundancy purposes, I'm trying to address 2 issues: A. Avoiding a single point of failure associated with a having a central repository for the data, such as a NFS share or a single database server. B. Avoiding the overhead from using heavyweight tools like database replication. So I've been thinking about how to pull that off, and I think I've figured out how, as long as I don't need every machine to have exactly the same version of the data structure at all times. What it comes down to is implementing 2 classes: one implements a daemon running on each server in the cluster, responsible for handling requests to update the data across the network and one a class usable inside mod_perl to handle local updates and inform other servers of updates. I believe I wouldn't be the only person finding something like this terrifically useful. Furthermore, I see that Cache::Cache could be the underlying basis for those classes. Most of the deep network programming is already there in Net::Daemon. What say y'all to something like Cache::Clustered::Server and Cache::Clustered::Client::* ? --Christopher Everett
Re: Apache::Session problems
Cees Hek wrote:
On Mon, 26 Mar 2001, Christopher L. Everett,,, wrote:
Apache::Session::MySQL won't save session state.
Apache::Session::File returns the following error:
Insecure dependency in open while running with -T switch at
/usr/local/lib/perl5/site_perl/5.6.0/Apache/Session/Lock/File.pm
line 40.
Well, line 40 of Apache/Session/Lock/File.pm contains the following bit of
code:
open($fh,"+".$LockDirectory."/Apache-Session-".$session-{data}-{_session_id}.".lock")
|| die $!;D
So perl is telling you that one of the variables being used in the open
command is Tainted (you are running perl in Taint mode with the -T
switch turned on). I'm guessing it is probably
$session-{data}-{_session_id}, which is really just the $session_id
variable that you pulled out of a Cookie in your code below (and
cookies are automatically tainted since it comes from the user). You will
have to untaint the $session_id variable before you pass it to
Apache::Session, and this error message should go away. See the perl
manpages on how to untaint variables...
Aargh! struck by the blindingly obvious again. I have got to stop
posting in the early morning ... I also figured out the next day
why Apache::Session::MySQL didn't work right, when I investigated
the nature of tied variables a little more closely. undef'ing the
variable at the end of get_session and re-tying %session at the
beginning of put_or_del_session, plus shuffling some code around
in get_seesion pretty well solved that problem. Seemed to me you
can't do something like:
tie %session, 'Apache::Session::MySQL', undef, \%attrs;
$self-{session} = %session;
ant then later on do
%session = $self-{session}
--Christopher
Apache::Session problems
Apache::Session::MySQL won't save session state.
Apache::Session::File returns the following error:
Insecure dependency in open while running with -T switch at
/usr/local/lib/perl5/site_perl/5.6.0/Apache/Session/Lock/File.pm
line 40.
here's the code in question:
sub put_or_del_session {
my ($self, $r, %session) = @_;
if ($self-command eq 'logout') {
tied{%session}-delete;
my $cookie = Apache::Cookie-new( $r,
-name= 'SessionID',
-path= $self-{uri},
-domain = $self-{config}-{TicketServerName},
-expires = '-10m',
-value = '' );
$cookie-bake;
} elsif (($self-page eq 'frame' $self-command eq 'make') or
$self-page eq 'action') {
$session{state} = $self-{state};
$session{timestamp} = time;
}
$r-log_error("put_or_del_session: session_id is $self-{session_id}");
$r-log_error("put_or_del_session: state is " . Dumper $session{state});
undef %session;
}
sub get_session {
my ($self, $r) = @_;
my %session;
my $cookie_str = $r-header_in('Cookie');
my %cookies = $cookie_str eq '' ? ( ) :
Apache::Cookie-parse($cookie_str);
if (exists $cookies{SessionID}) {
my $session_id = $cookies{SessionID}-value;
#tie %session, 'Apache::Session::MySQL', $session_id,
#{
# DataSource = $self-{config}-{Session_DB},
# UserName = $self-{config}-{Search_DB_User},
# Password = $self-{config}-{Search_DB_Password},
# LockDataSource = $self-{config}-{Session_DB},
# LockUserName = $self-{config}-{Search_DB_User},
# LockPassword = $self-{config}-{Search_DB_Password},
#};
tie %session, 'Apache::Session::File', $session_id,
{
Directory = '/tmp/apache/session',
LockDirectory = '/tmp/apache/session/lock'
};
} else {
#tie %session, 'Apache::Session::MySQL', undef,
#{
# DataSource = $self-{config}-{Session_DB},
# UserName = $self-{config}-{Search_DB_User},
# Password = $self-{config}-{Search_DB_Password},
# LockDataSource = $self-{config}-{Session_DB},
# LockUserName = $self-{config}-{Search_DB_User},
# LockPassword = $self-{config}-{Search_DB_Password},
#};
tie %session, 'Apache::Session::File', undef,
{
Directory = '/tmp/apache/session',
LockDirectory = '/tmp/apache/session/lock'
};
$session{state} = {
account = {},
command = '',
step= '',
order = {}
};
my $cookie = Apache::Cookie-new( $r,
-name= 'SessionID',
-path= $self-{uri},
-domain = 'www.physemp.com',
-value = $session{_session_id} );
$cookie-bake;
}
$self-{state} = $session{state};
$self-{session_id} = $session{_session_id};
$r-log_error("get_session: session_id is $self-{session_id}");
$r-log_error('get_session: $session{state} is ' . Dumper
$session{state});
$r-log_error('get_session: $self-{state} is ' . Dumper $self-{state});
return %session;
}
Re: mod_perl segfault
Doug MacEachern wrote: snipped ... it would probably be worth your while to try 5.6.1-trial1. Where does one download that? I have those in my logs, one more bug to kill. Someday, I'll go 24 hours without adding to my error logs, but that day won't happen too soon ... --Christopher Everett
Re: Help: Can't use string (Exchange::Account::My) as a HASH ref while strict refs in use trying to use instance variables in my handler
Anyway, what you should do is create a constructor:
sub new {
my $class = shift;
my $self {@_};
bless $self, $class;
return $self;
}
You mean like this code segment that I included in my original post
just below the handler code :)
sub init {
my $invocant = shift;
my $class = ref ($invocant) || $invocant;
my $self = {};
bless ($self, $class);
$self-{config}= $self-init_config;
$self-{dispatch} = $self-init_dispatch_table;
$self-{templates} = $self-init_templates;
$self-{_child_started_up} = 0;
return $self;
}
... straight out of "Programming Perl" ...
Then rewrite the above snippet of your code to:
sub handler ($$) {
my ($class, $q) = @_;
my $r = Apache::Request-new($q);
my $self = $class-new(request=$r);
Hunh?!?
Wait a second. I have a startup.pl, and inside that I have the lines:
use Exchange::Account::My;
my $account_interface = Exchange::Account::My-init;
Won't that do what I need it to do? When the root process forks off
children, won't a complete copy of $account_interface go with it, with
everything all set and ready to go?
$self-child_init unless $self-{_child_started_up};
# The rest of the code...
Then you should be good to go (instance variables and all!). Hope that
helps,
Chris
Except for calling the contructor with every call of the handler,
I think I've done everything right. Isn't the part of idea behind
mod_perl handlers that one _doesn't_ have to call the contructor
_every_ time the handler gets called? Otherwise invites massive
overhead.
Obviously, what I'm doing doesn't work. But could someone show me
how to call the constructor just once in in a childs lifetime?
Please?
--Christopher
Repost with typos corrected--Instance variable inheritance: is it just me or am I doing something dumb?]
Sorry about that. It was 3 AM after 12 hours beating my brains out.
I guess I didn't have any left.
Check these modules:
package simian;
use fields qw (name);
use strict; #very important!
use Apache;
use Apache::Request;
use Apache::Constants qw(:common);
sub new {
my $type = shift;
my simian $self = fields::new(ref $type || $type);
$self-{name} = 'Jane';
return $self;
}
sub name {
my $self = shift;
$self-{name} = shift if @_;
return $self-{name}; # error here, I mean
}
sub handler ($$) {
my ($self, $q);
my $r = Apache::Request-new($q);
# send headers here
print $self-name;
return OK;
}
package macaque;
use base "simian";
sub new {
my macaque $self = fields::new("macaque");
return $self;
}
return 1;
then put this line in your startup.pl or a Perl section:
my $tree_climber = macaque-new;
Then set up the mod perl server to use this class for a handler,
Location /macaque
# all the other stuff
PerlHandler macaque
/Location
and try to access it on the web browser. You will get an error like
the following:
Can't use string ("Exchange::MyAccount3") as a HASH ref while "strict
refs" in use at /usr/local/lib/perl5/site_perl/MyApp/Framework3.pm line
245.
This happens using "use fields" and "use base" or using @ISA
inheritance.
What the heck am I doing wrong?
--Christopher
Instance variable inheritance: is it just me or am I doing something dumb?
Check these modules:
package simian;
use fields qw (name);
use strict; #very important!
use Apache;
use Apache::Request;
use Apache::Constants qw(:common);
sub new {
my $type = shift;
my class1 $self = fields::new(ref $type || $type);
$self-{name} = 'Jane';
return $self-{name}; # error here
}
sub name {
my $self = shift;
$self-{name} = shift if @_;
return $self-{name};
}
sub handler ($$) {
my ($self, $q);
my $r = Apache::Request-new($q);
# send headers here
print $self-name;
return OK;
}
package macaque;
use base "simian";
sub new {
my macaque $self = fields::new("macaque");
return $self;
}
return 1;
then put this line in your startup.pl or a Perl section:
my $tree_climber = macaque-new;
Then set up the mod perl server to use this class for a handler,
and try to access it on the web browser:
You will get an error like the following:
Can't use string ("Exchange::MyAccount3") as a HASH ref while "strict
refs" in use at /usr/local/lib/perl5/site_perl/MyApp/Framework3.pm line
245.
What the heck am I doing wrong?
--Christopher
dir_config at startup: I know what doesn't work, so what does?
All:
I want to specify things like MySQL login info, names of tables
containing
user login information. I'd like to do it by putting it all into a
class
variable at server startup, using my startup.pl:
my $Account_Interface = $Exchange::MyAccount-init;
Then inside Exchange::MyAccount::init,
we have code like this:
$config{DBI_DSN} = Apache-server-dir_config('DBI_DSN');
and then of course we have the corresponding
PerlSetVar DBI_DSN "DBI:mysql:exchange_db"
What I'm running into is that dir_config always returns undef, no matter
what:
- PerlSetVars inside or outside the Location tag
- dir_config called at server startup or child startup
I've used dir_config before, but only at the PerlHandler level.
So what I'd like to know is: is there any way of picking up
configuration
info from the httpd-perl.conf at server startup? Picking all this stuff
up with the first request isn't always the most desirable way of doing
it,
especially if processing the directives takes any appreciable amount of
time. Putting config info into other files just spreads out what should
be in a single place.
Thanks,
Christopher Everett
Apache::AuthTicket
All: I really hate to bother you yet again, but I'm still unable to make Apache::AuthTicket work for me. I realize you're all busy, and you're dealing with deadlines just as I am, but I thought I'd take one last shot at asking for help. I tried another tack, using Apache::TicketAccess from the eagle book. I was able to make that work, after I figured out how to proxy cookies in and out through the lightweight front end Apache. Unfortunately, Apache::TicketAccess does not suit my purposes. However, I was able to prove to my satisfaction that my server handles cookies just fine. However, when I adopted those same techniques for Apache::AuthTicket, all that happens when I submit my login credentials was that Apache::AuthTicket sent my browser a blank cookie. Frankly, I'm lost and I need all the help I can get. I've laced both AuthTicket.pm and AuthCookie.pm with my own $r-log_error calls, not to mention turned on all the debugging already built into AuthCookie and AuthTicket. Everything seems to behave just as I would expect it to. I'm using Apache 1.3.14, mod_perl 1.24_01 I'm willing to send conf files, error_logs or anything else necessary. Christopher Everett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: sending receiving cookies through the light front end
Vivek Khera wrote: "CLE" == Christopher L Everett [EMAIL PROTECTED] writes: CLE 2) If 1 above is "yes", since the cookie was intended for a CLE.baz.org server, won't the perl.foo.org Apache drop the CLEcookie in the bit-bucket? The client (IE, Netscape) won't send a cookie for .baz.org to the perl.foo.org host; also, it probably won't let you to set such a cookie unless you are a *.baz.org host. This is the way I have it now: 1) set up IP aliases for the loopback adapter like so: ifconfig lo:0 192.168.0.1 netmask 255.255.255.0 ifconfig lo:1 192.168.0.2 netmask 255.255.255.0 ifconfig lo:2 192.168.0.3 netmask 255.255.255.0 2) added entries to the hosts file like this 192.168.0.1 perl.foo.org 192.168.0.2 perl.bar.org 192.168.0.3 perl.baz.org 3) modified the virtual hosts in the httpd-lite.conf to look like this VirtualHost 123.123.123.123 ServerName www.foo.org ProxyPass/nit/ httpd://perl.foo.org/nit/ ProxyPassReverse /nit/ httpd://perl.foo.org/nit/ # and whatever else I need /VirtualHost 4) modified the httpd-perl.conf to have virtual hosts like this VirtualHost 192.168.0.1 ServerName perl.foo.org ProxyPass/nit/ httpd://perl.foo.org/nit/ ProxyPassReverse /nit/ httpd://perl.foo.org/nit/ # and whatever else I need /VirtualHost Now, mod_perl and the front end are both in the same domain, so the cookie should go through, right? But it doesn't. --Christopher - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: sending receiving cookies through the light front end
"Christopher L. Everett" wrote: This is the way I have it now: 1) set up IP aliases for the loopback adapter like so: ifconfig lo:0 192.168.0.1 netmask 255.255.255.0 ifconfig lo:1 192.168.0.2 netmask 255.255.255.0 ifconfig lo:2 192.168.0.3 netmask 255.255.255.0 2) added entries to the hosts file like this 192.168.0.1 perl.foo.org 192.168.0.2 perl.bar.org 192.168.0.3 perl.baz.org 3) modified the virtual hosts in the httpd-lite.conf to look like this VirtualHost 123.123.123.123 ServerName www.foo.org ProxyPass/nit/ httpd://perl.foo.org/nit/ ProxyPassReverse /nit/ httpd://perl.foo.org/nit/ # and whatever else I need /VirtualHost 4) modified the httpd-perl.conf to have virtual hosts like this VirtualHost 192.168.0.1 ServerName perl.foo.org ProxyPass/nit/ httpd://perl.foo.org/nit/ ProxyPassReverse /nit/ httpd://perl.foo.org/nit/ # and whatever else I need /VirtualHost Now, mod_perl and the front end are both in the same domain, so the cookie should go through, right? But it doesn't. Ahhh, but it does work. Changed Apache::Cookie-fetch($r) to Apache::Cookie-parse and all was well. What a relief, five days of agony vindicated (kind of). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Still need help w/ Apache::AuthTicket
Hello: I'm still trying to get Apache::AuthTicket working. As of right now, what's happening is that the cookie with the ticket doesn't get sent. This is true for MSIE/Win98, Netscape/Linux, and Lynx running directly on the server itself, bypassing the front end. What would keep the cookie from being sent? --Christopher Christopher L. Everett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Need help with Apache::AuthTicket
In the docs for Apache::AuthTicket it says you need to specify PerlSetVar FooTicketLoginHandler /foologin PerlSetVar FooLoginScript/foologinform and then you need to add Location /foologin AuthType Apache::AuthTicket AuthName Foo SetHandler perl-script PerlHandler Apache::AuthTicket-login /Location Location /foologinform AuthType Apache::AuthTicket AuthName Foo SetHandler perl-script PerlHandler Apache::AuthTicket-login_screen /Location Since I am using a front-end proxy, I have this in the httpd-lite.conf: ProxyRequestsOn ProxyPass/secure/ httpd://localhost:8080/secure/ ProxyPassReverse /secure/ httpd://localhost:8080/secure/ and I do this in the httpd-perl.conf: PerlSetVar PhysempTicketLoginHandler /secure/login PerlSetVar PhysempLoginScript/secure/loginform and then I added Location /secure/login AuthType Apache::AuthTicket AuthName Physemp SetHandler perl-script PerlHandler Apache::AuthTicket-login /Location Location /secure/loginform AuthType Apache::AuthTicket AuthName Physemp SetHandler perl-script PerlHandler Apache::AuthTicket-login_screen /Location so that /secure/login gets proxied back with everything else. My problem is that when I access the /secure, I get the login screen OK, but when I login, I keep getting hte login screen and 403 Access Forbidden errors in the httpd-perl log, but even with $Apache::AuthTicket::DEBUG = 1 and including PerlSetVar AuthCookieDebug 3 in the conf file, I have no clue why the login fails over and over. Does anyone have a clue for me? Thanks ... --Christopher - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_perl on specific scripts
David Hodgkinson wrote: Paonia Ezrine [EMAIL PROTECTED] writes: I have a number of scripts in places other then /perl that I want to use mod_perl for. However, I can't turn it on for all scripts in a specific directory or even a certain extension. Is there any way to do this or am I going to need to do a redirect of some sort (anyone have one)? I trust you've set up a thin apache at the front? Then it's easy to pass only the scripts you want back to the mod_perl server. Did I just condemn you to learning mod_rewrite? Ooops :-) Assuming your script is written as a mod_perl handler, and assuming the aforesaid thin Apache in front, try mod_proxy on the front end and Location on the back end. What I do in my httpd-static.conf (cribbed wholesale from the Guide, names changed to protect the innocent): VirtualHost www.xxx.yyy.zzz:ppp ServerName www.foo.com ProxyRequests on ProxyPass/cgi-bin/ http://localhost:8080/bar/baz/ ProxyPassReverse /cgi-bin/ http://localhost:8080/bar/baz/ RewriteRule ^proxy:.* - [F] # keep others from using your proxy ProxyReceiveBufferSize 65536 # buffer more data thru the proxy # put whatever else you need in here too, like SSL configs, root dirs, # and whatnot /VirtualHost and in the httpd-perl.conf: Location /boo/baz/noot.cgi SetHandler perl-script PerlHandler Knights::who::say::Nit # likely more stuff goes in here too, see the Guide for details /Location If your scripts are of type Apache::Registry, instead do Alias /bar/baz/ /your/special/path/to/your/scripts/ Location /bar/baz/ SetHandler perl-script PerlHandler +Apache::Registry Options ExecCGI PerlSendHeader On /Location META Would something like Alias /bar/baz/foo.cgi /your/special/path/to/foo.cgi work? Because then this will work: Location /bar/baz/foo.cgi SetHandler perl-script PerlHandler +Apache::Registry Options ExecCGI PerlSendHeader On /Location and we can mix PerlHandlers and Apache::Registry scripts in the same (virtual) direactory. That would be a win. /META At least this handles the directory problem. The file extension problem really does require mod_rewrite, I believe. The beauty of this is that what the client sees really has nothing to do with where the scripts are on your server. On my box, I used mod_proxy to virtualize the /cgi-bin/ directory. When a request comes in for http://www.foo.com/cgi-bin/noot.cgi, the request gets passed back, the proper PerlHandler gets called, and life goes on. I admit that I've deliberately avoided learning mod_rewrite; I took one look at it and said "I'm doing something else!". So I set up all my sites to avoid using it. God help me if I ever really _have_ to use it, though :) --Christopher Christopher L. Everett [EMAIL PROTECTED]
Re: how to really bang on a script?
Perrin Harkins wrote: On Sat, 28 Oct 2000, Matthew Byng-Maddick wrote: On Sat, 28 Oct 2000, Matt Sergeant wrote: exactly the same thing (changing server logs into a benchmark tool) at ApacheCon, only I can't for the life of me remember who it was. Theo, during the mod_backhand talk, or at lunch just before, I can't remember. It was during the talk. The tool is called Daquiri, and he said it was available in the mod_backhand CVS tree. I have also found httperf and http_load pretty useful for this stuff, although they don't support logfile playback. Hey, I went by http://www.mod_backhand.org/, and found no mention of anything called Daquiri, nor any description of how to access the mod_backhand CVS tree, not even an email address to ask for information. I downloaded the mod_backhand source, and I found nothing there either. Does anyone have a clue about where to start getting my hands on this neat and nifty tool? --Christopher
Re: how to really bang on a script?
Adi wrote: "Christopher L. Everett" wrote: Adi wrote: martin langhoff wrote: Chris, i'd bet my head a few months ago someone announced an apache::bench module, that would take a log and run it as a benchmarking secuence of HTTP requests. just get to the list archives and start searching with benchmarks and logs. CPAN is your friend, also. It was HTTPD::Bench::ApacheBench. It is a Perl API to ab. It doesn't take a log per se, it simply sends sequences of HTTP requests and benchmarks the results. I'm sure you could very easily write a script to parse a log and then make a benchmarking run out of it. Yes, I considered ab and I did find HTTPD::Bench::ApacheBench, while excellently done and copiously documented, isn't quite what I need: 1) I want to spoof the IP addresses of the browsers (I just realized that since I'm using mod_proxy_add_forward anyway, I can make the requester script like a proxy; the rest is cookbook). I can't find provision for that in the interface for HTTPD::Bench::ApacheBench. Yeah, that would require adding arbitrary HTTP headers to each benchmark request... that's currently on our to-do list (I think). 2) Record the query parameters as well as the response's MD5 checksum directly in a database table on the fly. This is all possible with ApacheBench. It wouldn't be stored in real-time (i.e. as the request is sent) but you set up the benchmarking runs, so you know all the query parameters (which are all either in the URI or the postdata for each request), and ApacheBench returns all the response data, which you can then pass thru an MD5 hash and store in the database. 3) The interface is more suited to setting up, then executing a batch run programmatically, rather than replaying a log. Right.. it was designed to be generally useful. To replay a log you simply need to set up a batch that exactly duplicates your log. As I understand ApacheBench, to set up a really large (1M) run with say 200 or 300 URI's with ApacheBench distributed randomly across 400K unique IP addresses, you just about would end up doing 1 run per access log entry, which turns it into a really massive data structure, at which point a file starts looking like a better place to put all that. Or, I could try forking off ApacheBench objects one by one, putting runs into the next object to fork while the most recently forked object beats on the server. Kind of a roundabout way of trading off complexity against RAM. [snippage] to me in that the last line in the execute method, "return $self-ab;" is the only mention of the class method "ab" in the entire file. Obviously I That's because "ab" is the XS function that sends the HTTP requests and builds up a hash with all the response data and times. All the looping is done in C for speed. Take a look at ApacheBench.xs. (especially if you feel like adding the arbitrary HTTP request header functionality, hint hint :) What would be nice from the self-documenting code point of view for a newby like me would be a clue _in_the_code_itself_ that ApacheBench.xs was the place to look for the definition of ab. That's how I would expect a programming language to behave :), but that's something more for the Perl 6 list. have _much, much_ more to learn ... :) No, actually you pointed out some good feature additions that we should think about making to ApacheBench. Thanks. You're welcome. -Adi --Christopher Christopher L. Everett [EMAIL PROTECTED]
Re: how to really bang on a script?
Adi wrote: martin langhoff wrote: Chris, i'd bet my head a few months ago someone announced an apache::bench module, that would take a log and run it as a benchmarking secuence of HTTP requests. just get to the list archives and start searching with benchmarks and logs. CPAN is your friend, also. It was HTTPD::Bench::ApacheBench. It is a Perl API to ab. It doesn't take a log per se, it simply sends sequences of HTTP requests and benchmarks the results. I'm sure you could very easily write a script to parse a log and then make a benchmarking run out of it. Yes, I considered ab and I did find HTTPD::Bench::ApacheBench, while excellently done and copiously documented, isn't quite what I need: 1) I want to spoof the IP addresses of the browsers (I just realized that since I'm using mod_proxy_add_forward anyway, I can make the requester script behave as a proxy; the rest is cookbook). I can't find provision for that in the interface for HTTPD::Bench::ApacheBench. 2) Record the query parameters as well as the response's MD5 checksum directly in a database table on the fly. 3) The interface is more suited to setting up, then executing a batch run programmatically, rather than replaying a log. Having examined the ApacheBench.pm source, I don't see how I can make it do what I want by subclassing it. Also the code is a little bit mystifying to me in that the last line in the execute method, "return $self-ab;" is the only mention of the class method "ab" in the entire file. Obviously I have _much, much_ more to learn ... :) --Christopher Christopher L. Everett [EMAIL PROTECTED]
Re: how to really bang on a script?
"G.W. Haywood" wrote: Hi there, On Fri, 27 Oct 2000, Christopher L. Everett wrote: snipped helpful advice I need to prove to myself and my marketing guy that my script has certain statistical properties, not the least of which is the question of whether my activity logs match what actually happened. You've been spending too much time with your marketing guy. "Certain statistical properties" is gobbledygook. What properties? Activity logs don't match statistically. Either they match or they don't. If they don't then either the logging is turned off or it isn't working. OK, I confess: I've written (probably yet another) mod_perl banner exchange. I need to know that when we serve 100K banners to 40K different IP addresses a day, and we are selling 30K banners/day, the 500 sites that hosted our banner ads are getting the 50K banners that we promised them, and the 30K banners that we sold, we really did serve. Also, I want to know that the banners my logs say the script sent are really the ones people saw on their browsers. That's what I meant by "certain statistical properties". So, I apologize for not describing my problem clearly in the first place. And again, my questions are: How would I go about proving to myself that my script does what I designed it to do? Has anyone else dealt with a similar problem, and how did they go about doing it? If I solve it for myself, would anyone else find the solution useful, and how would I make it more useful to them? Usually, I would test by running through the script a few times with some variations, but we are so freaked out by our experience with the 2 other banner exchange scripts we tried, we find a lot of value in being certain. Thanks again for your kind help. --Christopher Everett
