problem porting to threaded mode
Trying to shift our largely mod_perl2 web site to an Apache2 threaded
MPM and perl ithreads.
The following works under the non-threaded prefork MPM:
use DB_File;
my @dbs; # array of hash references
my @dbModTime; # mod times of db files
my @dbfns; # array of database pathnames
# executed before fork into child processes
sub post_config {
my $db;
my $s = $_[3];
# tie the DBs and get their mod times
for ($db = 0; $db < @dbfn; $db++) {
$dbs[$db] = {};
tie %{$dbs[$db]}, "DB_File", $dbfn[$db], O_RDONLY
or die ((caller 0)[3]. " can't tie " . $dbfn[$db] . ": $!");
$dbModTime[$db] = (CORE::stat($dbfn[$db]))[9]
or die ((caller 0)[3]. " can't stat " . $dbfn[$db] . ": $!");
} }
The routines that use the databases re-stat the DB files and untie and
re-tie a DB
that has changed. Each child process must do this for itself.
In the threaded environment, any thread within a process may discover
that such
an untie and re-tie is necessary, but such an operation should be
effective
for the other threads in the process as well. This means that @dbs and
@dbModTime should be shared among the threads:
use threads;
use threads::shared;
my @dbs :shared; # array of hash references
my @dbModTime :shared; # mod times of db files
Making only the changes above makes perl complain "Invalid value for
shared scalar"
about the '$dbs[$db] = {};' line. This error message can be fixed as
follows:
for ($db = 0; $db < @dbfn; $db++) {
$dbs[$db] = shared_clone({});
tie %{$dbs[$db]}, "DB_File", $dbfn[$db], O_RDONLY
or die ((caller 0)[3]. " can't tie " . $dbfn[$db] . ": $!");
$dbModTime[$db] = (CORE::stat($dbfn[$db]))[9]
or die ((caller 0)[3]. " can't stat " . $dbfn[$db] . ": $!");
$s->log->notice ($dbfn[$db]." has " .scalar(keys(%{$dbs[$db]}))."
entries");
}
Unfortunately, when this is done the server starts, but the DBs look
empty and
the log notices for each DB show "0 entries".
Removing the ':shared' tag for @dbs and the 'shared_clone()' wrapper
for '{}'
causes the log notices to show the proper number of entries for each DB,
but blows up the Apache configuration process (before the 'resuming
normal
operations' message) with
httpd in free(): error: chunk is already free
in error_log and the following on the terminal:
Abort trap (core dumped)
Error invoking apachectl start command
I guess not having databases is better. I've tried using @dbs as an
array of
references to named, shared hashes: also no database content. The
'worker'
and 'event' MPMs work identically w/r/t this problem.
Suggestions of things to try will be very welcome.
Happy New Year, cmac
Re: preserving request body across redirects
> > Oh no, the framework doesn't depend on using a session > > at all. Just this particular auth module depends on > > plugging in the session beforehand. > > So I have to have a persistant backstoring session in > order to use the auth module? Well, now, but as you recommended, maybe not necessary. > I'd think 'implementation subclass must implement a method > that returns validity of username' would be more flexible > and concise. You could use apache environment variables > and other request state information to pass extra data to > that function based on what or or > its in, as a forinstance. Well, it would get $r. Yes, I thought it would be a good idea to rework it so getting the username is its own method, that way you could subclass and get it from LDAP or whatever. Actually in that case, it's not clear that even a DBI handle is necessary for the top-level implementation, although if you use the connector handler to get one in your subclass you could do joins or whatever you need to do. Then I can also drop the directives for the table name and field names. Thanks for the advice! There may be something wrong with your mail program, or with alpine's way of reading your mail. It's hard to sort out where I wrote and where you wrote because everything has only one level of '> ' quotes. Mark
Re: preserving request body across redirects
Mark Hedges wrote:
- a lot of times people use references to
other structures when they should subclass... these
references function only to re-map arguments to other
modules, which is ridiculous.
Careful on the should. It can seem extra and possibly confusing but
isn't always. Delegation is a valid pattern that is cleaner than
inheriting at times, particularly when you're mixing in a few different
modules at the same time to do something. If you're merely extending an
existing, then yes, inheritance is good. Multi-directional
multi-inheritance can get really messy... (as I dug myself out of in
development recently myself) if you haven't read, at least scan...
http://www.perldesignpatterns.com/?MixIns
http://www.perldesignpatterns.com/?DelegationConcept
http://www.perldesignpatterns.com/?CompositePattern
And forgive me if I'm too talky on the subject. :)
On Tue, 30 Dec 2008, David Ihnen wrote:
Yes. But timtowdti on how that information is
distributed. In my opinion any *framework* must not
depend on the *application* having established a
persistent backstore of shared session data, so that it
can persist put/posts. You're *significantly*
constraining the parameters of the implementations
utilizing the framework by requiring this, which I
consider to be exactly what frameworks shouldn't do. We
may disagree. :)
Oh no, the framework doesn't depend on using a session at
all. Just this particular auth module depends on plugging
in the session beforehand.
So I have to have a persistant backstoring session in order to use the
auth module?
It requires server storage to know what usernames are
available in this situation *yes*. But that is merely at
authentication time - not during the lifetime of the
session. An auth service is not a session service.
It does not require server session storage inherently to
associate a session cookie with a user - the cookie has
plenty of storage to do that itself.
Hrmm. I'll think about that... maybe the only thing
necessary is that a database handle is available from
Apache2::Controller::DBI::Connector.
I'd think 'implementation subclass must implement a method that returns
validity of username' would be more flexible and concise. You could use
apache environment variables and other request state information to pass
extra data to that function based on what or or
its in, as a forinstance.
That way you wouldn't attempt to corral the user into using a particular
method of acquiring user data. I mean, think of the directive
proliferation that impends from trying to configure all the DBI stuff.
Sure, perhaps your connector subclass defines the connection string and
all that hooey (hopefully THAT stuff isn't in the directive
proliferation...) - but how do you query in this database once you have
the connection? Do you define table, column, where conditions, joins?
An arbitrary SQL string (language in language, ack!) What if I did
things way differently... and had an external service providing the data
that doesn't even require a query, as such, but of course I could layer
through DBI if I wanted/had to (SELECT openid_identifier FROM USERS
WHERE username = ? translating into a json-rpc call to my user store
service {id:1, method:'get_users', parameters:[?]) that really does do
the same job as a query, conceptually, but does not utilize the
assumption that we have a database backed store available at all?
Example code could show how a dbi connection could be used and
concisely, but is that part of the requirements of utilizing the framework?
David
Re: preserving request body across redirects
On Wed, 31 Dec 2008, Foo JH wrote: > Mark Hedges wrote: > > http://search.cpan.org/~markle/Apache2-Controller-1.000.001/ > Very interesting. I have a controller which functions in a > slightly different way, but it's good to see alternative > approaches. I think yours makes life easier. Thanks Foo! I've seen the idea implemented a number of ways, open source and not. Part of my point was proof of concept that a lot of the abstractions were useless hoops to jump through -- a lot of times people use references to other structures when they should subclass... these references function only to re-map arguments to other modules, which is ridiculous. On Tue, 30 Dec 2008, David Ihnen wrote: > Yes. But timtowdti on how that information is > distributed. In my opinion any *framework* must not > depend on the *application* having established a > persistent backstore of shared session data, so that it > can persist put/posts. You're *significantly* > constraining the parameters of the implementations > utilizing the framework by requiring this, which I > consider to be exactly what frameworks shouldn't do. We > may disagree. :) Oh no, the framework doesn't depend on using a session at all. Just this particular auth module depends on plugging in the session beforehand. > It requires server storage to know what usernames are > available in this situation *yes*. But that is merely at > authentication time - not during the lifetime of the > session. An auth service is not a session service. > > It does not require server session storage inherently to > associate a session cookie with a user - the cookie has > plenty of storage to do that itself. Hrmm. I'll think about that... maybe the only thing necessary is that a database handle is available from Apache2::Controller::DBI::Connector. > Your session times out when you say it times out. And > changes states when you say it changes states. You can > have two time periods - one being re-auth request time > period, another being true expiration time period. The > purpose of a session timeout is to stop large time delta > recycling of session data. The purpose of re-auth time > period is to nudge your flow into getting a new > authentication token without the interruption of actual > logout. Neither of these is an infinite state, and would > not be replicated by infinite timeout. That could work. Sort of want to avoid a proliferation of directives... but I guess that's a potential control that would be less complicated and wouldn't require a persistent session store on the server. And can set a flag in notes that this happened for the app to deal with it if it wants to, as you said. Data::UUID's are a good idea for the auth tracking cookie. For the backend-session plugin right now I'm just using the _session_id from Apache::Session as the cookie value.
Re: HTTP Response Headers fixup
André Warnier wrote: One of my questions is thus : assuming I keep adding this custom header "Test-Header" via mod_headers, will it be visible by the HTTPHeadersFixup subclass I create and configure ? Its an output connection filter, so I can't imagine it wouldn't work as you expect. if yes, I could make a simple test in my subclass to detect if this request is indeed one that needs to be modified. If the filter configuration in the is insufficient you mean? You certainly can inspect and react. Or is there another/better/simpler way to resolve my problem ? http://perl.apache.org/docs/2.0/user/handlers/filters.html#C_PerlOutputFilterHandler_ You can also use output filter handler stage... just look for the header in the brigade and insert your own header instead. Though that sounds more complicated than the ready-made header fixup handler. :) David
HTTP Response Headers fixup
Hi. I have a peculiar problem, and need a peculiar solution. In an Apache 2.2.3, mod_jk, Tomcat 5.5.x, mod_perl2, perl 5.8.8, Linux environment, some links are being re-directed by Apache via mod_jk, to a back-end Tomcat server, and a webapp running in that server. For reasons that are off-topic and obscure, that webapp generates html pages that are really encoded in iso-8859-2, but with a Content-Type response header that says they are iso-8859-1. It's like that, and I cannot change this at the webapp or Tomcat level, not even with a servlet filter. So I need to modify the Content-Type response header, after-the-fact (meaning when the response from Tomcat has been channeled back to Apache via mod_jk). My first inspiration was to do it using mod_headers. But this does not work for some reason. I have been told on either the Apache or Tomcat forum, that it was because Apache or mod_jk does something internally, to prevent the response headers generated by Tomcat, to be modifiable by Apache later on. I do not really know if that is true, but it matches the symptoms that I am seeing : a custom HTTP header that I *add* to the response after mod_jk shows up properly at browser level, but it conpletely ignores my attempts at deleting/editing/replacing the Content-Type header for the same response. Then I found Apache2::Filter::HTTPHeadersFixup, which seems to be almost exactly what I am looking for. Except that I have a doubt : HTTPHeadersFixup is a connection filter, I guess by necessity for that kind of thing. Which means that it affects all requests that go through that VHost. However, only a small fraction of these requests (the ones that are re-directed to that rogue Tomcat application) should have their response Content-Type modified, and not all requests. So my question is, rather generally, what would be the best way to implement what I need, either with HTTPHeadersFixup or without it ? Here is a snippet of the configuration I have at the moment, including what I tried with mod_headers (but which does not work) : (lines with ### prefix are my comments to explain what it does/should do) Start : ### set the is-jk variable only for requests that will be re-directed to the rogue Tomcat application SetEnvIf REQUEST_URI "/servlet\.[^\.]+$" is-jk ### if the above var is set, then try modifying the mod_jk response header after-the-fact, at VHost level Header always set Content-Type: "text/html; charset=ISO-8859-2" env=is-jk ### just to check the above, also add a custom header to the response, under the same conditions Header always set Test-Header: "VHost level" env=is-jk ### the following Location is where the mod_jk redirect really happens (the setHandler plays the same role as a JkMount directive) SetHandler jakarta-servlet ### try the same as before, but within the specific Location scope Header always set Content-Type: "text/html; charset=ISO-8859-2" Header always set Test-Header: "Location level" :End The result of the above, is that the custom header "Test-Header" gets indeed added to the appropriate responses, at the "Location" level (the Location directive replaces the one generated by the generic one). But, the Content-Type header obstinately stays at what the rogue webapp set it, which is "Content-Type: text/html; charset=ISO-8859-1" (instead of -2 at the end). One of my questions is thus : assuming I keep adding this custom header "Test-Header" via mod_headers, will it be visible by the HTTPHeadersFixup subclass I create and configure ? if yes, I could make a simple test in my subclass to detect if this request is indeed one that needs to be modified. Or is there another/better/simpler way to resolve my problem ? And, best wishes to all for tomorrow and on.
