problems with %Location inside perl-sections
I have two problems related to setting Location directives from within
perl sections that I don't understand (I've looked at the mod_perl guide
and the archive but a RTFM is certainly welcome ;)
First problem: Location works on url space only while $Location
requires existing directories (full httpd.conf atatched):
DocumentRoot /tmp
Location /perl-status
SetHandler perl-script
PerlHandler Apache::Status
/Location
Perl
$Location{'/test'} = {
SetHandler = 'perl-script',
PerlHandler = "Apache::Status",
};
/Perl
The first location works fine. The second one doesn't UNTIL I mkdir
/tmp/test (there is no /tmp/perl-status), after which I get the
perl-status page on both urls.
The second problem (which I can't reproduce with a small example) is that,
when using a lot of entries in %Location, they suddenly start to behave
like LocationMatch, i.e.:
$Location{'/admin'} = { ... };
matches /test/admin, /something/else/admin/jump, /cgi-bin/printenv/admin
and everything else that contains /admin IFF the url does not directly
point to something hat can be displayed (e.g. the printenv is a cgi script
but it doesn't get called).
If I change this to:
$Location{'^/admin'} = { ... };
it suddenly starts to work as expected, indeed as if LocationMatch was
used. However, this is not deterministic. Sometimes the above simply
matches nothing ;)
Does this ring a bell for somebody? I ahd the same problem with
apache-1.3.14 and mod_perl-1.24 I upgraded to apache-1.3.17 and modperl
from cvs but the symptoms didn't change so I guess it might not be a bug
but rather some misunderstanding on my side :(
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
ServerType standalone
PerlFreshRestart off
HostnameLookups off
User nobody
Group nobody
ServerName doom.laendle
ServerRoot /tmp
DocumentRoot /tmp
Port 81
PerlModule Apache::Status
Location /perl-status
SetHandler perl-script
PerlHandler Apache::Status
/Location
Perl
$Location{'/test'} = {
SetHandler = 'perl-script',
PerlHandler = "Apache::Status",
};
/Perl
Re: problems with %Location inside perl-sections
On Mon, Feb 12, 2001 at 08:48:57AM +0800, Stas Bekman [EMAIL PROTECTED] wrote:
Looks like Apache doing stat() calls problem. Try to run the request under
strace(1) or truss(1). See:
http://perl.apache.org/guide/performance.html#Reducing_the_Number_of_stat_Ca
this is with perl-status:
[pid 14461] stat("/tmp/perl-status", 0xb3cc) = -1 ENOENT (No such file or
directory)
[pid 14461] stat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=3505, ...}) = 0
[pid 14461] open("/.htaccess", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 14461] open("/tmp/.htaccess", O_RDONLY) = -1 ENOENT (No such file or directory)
and this is the same as with /test when there is no /tmp/test. With an
existing /tmp/test, I get:
[pid 14460] stat("/tmp/test", {st_mode=S_IFDIR|0755, st_size=35, ...}) = 0
[pid 14460] open("/.htaccess", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 14460] open("/tmp/.htaccess", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 14460] open("/tmp/test/.htaccess", O_RDONLY) = -1 ENOENT (No such file or
directory)
$Location{'/admin'} = { ... };
$Location{'^/admin'} = { ... };
Again, what strace tells you?
apache is stat()'ing the path as long as it can, i.e. until
.../cgi-bin/printenv oder . (there is no /admin directory in my
DocumentRoot).
You will see everything that Apache does
while looking at the output.
I couldn't try with PerlTransHandler yet (since I seem to have left this
out when compiling mod_perl), but my question is: does this also fix the
problems I encounter? I am not concerned about speed here, but rather
about correctness, namely that I need ^/admin which shouldn't match at all
in a Location directive.
Thanks a lot for your reply!
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: mod_perl guide corrections: in uris
On Mon, Feb 12, 2001 at 03:13:55AM +0100, Robin Berjon [EMAIL PROTECTED] wrote: I don't think so. The browser would be right to treat reg; as an entity, not reg. But why? It's not HTML in the first place, so expecting from clients to interpret it in one way or another is not sensible. If it had proper heuristics for dealing with poor HTML, it'd detect that there is no ; in sight for the next n chars, or that reg= isn't While one might rgue that clients should apply heuristics, given the large amount of borken html out there, one has to remember that the source for this broken html WAS sloppy html parsers with heuristics. If netscape and mosaic had flagged syntax errors nobody would expect browsers to implement heuristics today :( However, I agree that people should try their best to write proper html. If Especially since you can only choose between theoretically incorerct and in practise sometimes not working AND theoretically correct and working alway sin practise I think the choise should be clear ;) somehow despise it. Think twice: take out the html, what's left of your site ? XHTML clearly forbids such wrong constructs (won't even parse if you get it wrong) and that's cool. It's like use strict for HTML. Which is exactly my point ;) Implying this is a browser bug will only make more people insist on outputting "correct" code. After all, the clients must be fixed ;- -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
bug in mod_perl-1.24 request-args request-content
Apache.pm documents two methods "args" and "content" that should return
argument = value pairs (when called appropriately). In fact, args is
implemented as:
return map { Apache::unescape_url_info($_) } split /[=;]/, $string, -1;
However, this might return an odd number of values, for example for this url:
httpurl?arg1arg2=val2
I get (arg1 = "arg2", val2), which is not as documented.
This url format is often used for boolean arguments (CGI.pm creates an
arg1 = "", while I would prefer arg1 = undef in this case).
So either the documentation or the implementation is in error. I would
prefer if the implementation were in error ;)
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing question (low priority ;)
On Thu, Jun 01, 2000 at 11:59:53AM -0700, Doug MacEachern [EMAIL PROTECTED] wrote: will not reload a module if it's already in %INC. but that's doesn't mean a Perl environment cannot un-cache that entry so it will be reloaded. consider the code below, pretend that loop is a long-lifetime server, Doing that, however, breaks code that uses documented and recommended techniques (see perltoot for the best example). My problem is not that it is possible to actively break code within normal perl, this is easy in any language. Module::Reload checks the mtime of Foo.pm (and all loaded modules) on disk, if it has changed since the last time it was require'd, it will be reloaded. this is perfectly valid Perl, the language supports this feature. You miss a very large point: Modules are not just files containing perl code. One of the features of modules is that, how often you might use them, they are only loaded once (unlike "do EXPR", for example). Sure, you can load modules in different ways, even in ways that break them, and the code still runs under perl (that is, without syntax error), but this has not much to do with perl's concept of modules anymore. i'm quite certain your code be changed to adapt to such an environment. It's easy, I just have to kick my ass each time I want to use a lexical for data abstraction and use a package variable instead, with only the exception that I have to be very careful that I never re-use the same name. This is quite difficult for code that resides in many files (package variables are, of course, global to the whole package not just the file or the block), but not unsolvable. You see, my first problem was misunderstanding (or not understanding) what apache does during the "module initialization phase", and how "reload" is defined (namely by not deleting the package). I assumed that a "fresh restart" would imply a fresh perl interpreter, or at least a fresh package, not just loading the code over the old one, preserving it. While I am not sure why PerlFreshRestart is implemented in this misleading manner, I can definitely live with that, as it is an option that can be turned off, which is much more than one can expect (namely nothing at all!), and, given that this is not considered a bug but a limitation with mod_perl (spec. that single option), can be worked around consistently (as it is relatively minor). The biggest problem with PerlFreshRestart is that it causes bugs which are very difficult to find and that might only be found after a lot of debugging, as the problem might be in a third-party module you have never seen the source code of. The more important aspects of mod_perl of course work fine, and this is what counts most. So consider that totally solved to my satisfaction. (Indirectly) calling my code broken (ah, you probably see where the problem is, now ;), however, will require some very good arguments from your side, which you didn't give so far. Especially I'd like to hear why perltoot (and others) actively encourage these "broken" programming techniques that enable data hiding and definitely work with "perl modules": Surely I can load a .pm file using other methods, like "do", or by nuking the %INC entry, but changing the behaviour of "use" so drastically changes the notion of "modules" as well. In my opinion, using a different way of loading modules than the documented one, a way that breaks modules that work fine under the documented mechanism, does not magically turn recommended code practises into broken code practises. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing question (low priority ;)
On Fri, May 26, 2000 at 10:33:15AM -0400, Geoffrey Young [EMAIL PROTECTED] wrote: mod_perl sometimes requires special perl coding guidelines, due in part to the way mod_perl works with and within apache. I know this and I have no problems with that (as I made very clear in my last mail). But when mod_perl requires special programming techniques this does not mean that code not using that techniques is "broken anyway", as dougm said, at least not in perl. It might be "broken code", but then the language is not perl (where the problematic technique is documented to work in perltoot). For example, you shouldn't do things under Registry that are otherwise ok Exactly what I said. These things are otherwise ok, but mod_perl requires special treatment. mod_perl. that doesn't make mod_perl a dialect of perl, However, you can't claim that mod_perl _is_ perl and at the same time claim that perfectly valid perl code is broken. Either it's broken code in the mod_perl dialect or it's valid code not working due to environment constraints. actually, it makes perfect sense, if you think of it in apache/mod_perl I can understand why it happens, if you mean that. Wether it makes perfect sense is, unfortunately, not so clear to me. and, there's no reason to get real argumentative or defensive about any of this, Marc - we're all friends here, in modest persuit of the same things :) As I explained I was getting defensive simply because _not_ all are friends here. I am still defensive because calling somebody else's code broken should not be done so lightly. What happened, if you go back and read the thread, is that I had a question and had a very hard time explaining what I actually meant (because my initial mails were simply not verbose enough on the problem). Many responses (especially the private ones) were very aggressive, so "we are all friends" is a myth. Since this is my first contact with modperl, however, this just means that I wasn't able to adjust myself to the tone on (and around) the modperl mailinglist.. Next time I will simply ignore most of the responses. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing (short reproducible example)
On Thu, May 25, 2000 at 12:09:09PM -0700, Doug MacEachern [EMAIL PROTECTED] wrote: You can only configure Apache from Perl sections, but you can load all your modules, shared data, etc. from a file pulled in with PerlRequire. actually you can, if a module defines variables in the Apache::ReadConfig:: namespace, they are fed to the apache config gears just as Perl sections are. Erhm, I am doing this right now (the functions I call just implant Location directives etc.. into the caller's package). Now it would be interesting to know wether that works only when called from perl sections or also form perlrequire? (Hmm, I'll just try it out and assume it's supported if it works ;) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing question (low priority ;)
On Thu, May 25, 2000 at 11:58:38AM -0700, Doug MacEachern [EMAIL PROTECTED] wrote:
You must be kidding here!!! Using lexicals on package level is broken??? If
it is broken, then _why_ is it recommended programming practise in perl (see
perltoot for example)?
i'm not kidding. i don't understand whay you mean by 'using lexicals on
package level is broken'.
here is my module, in very terse pseudocode:
=
package mymodule;
my some"globals";
sub functiousing someglobals {
}
now load other code (in some kind of xml) and compile it into it's
package, UNLESS the package already exists.
==
And this obviously breaks for the newly compiled code. The caching,
unfortunately, is very important. Surely I could make an exception when
running under mod_perl with PerlFreshRestart, but that code runs nicely in
other environments, since the apache bits have been abstracted away. Read
below.
and, like i said, i realize that certain modules make assumptions that
they will only be loaded once in the same process and that re-parsing them
can uncover subtle problems. but i still consider such assumptions to be
broken.
Sorry, but reparsing is never done by perl itself (it explicitly says so),
so mod_perl clearly is not acting like perl.
So, in in the mod_perl dialect of perl, modules that work fine in perl
magically break on restarts. This is because mod_perl "restarts" by
fooling the perl interpreter to think it hasn't seen _some_ parts of the
program(s) and tries to reload all on top of it.
What's confusing here is the term "FreshRestart", which doesn't do
anything like a "fresh restart".
The reason this is important to me is not because I couldn't make an
exception to my code for apache - it's the real world, so hacks are
necessary in many cases anyway - but because what's broken is mod_perls
idea of "fresh restarting" by cheating the perl interpreter rather than my
code.
Now that I know what's going on it's not a big deal, but if you call
my code broken you should have rather good arguments on the basis of
the perl language and protgramming, not because mod_perl does some very
undocumented things differently to perl itself.
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing (short reproducible example)
On Tue, May 23, 2000 at 04:07:40PM -0700, Perrin Harkins [EMAIL PROTECTED] wrote:
This is a combination of closures and PerlFreshRestart biting
(still no closures)
My example might be misleading, since I used x before it was defined (to make
the example short). Typical examples look like this:
my $global1;
sub func1_using_global1 {
}
sub func2_using_global2_and_func1 {
}
etc... Users of the module outside the module itself can get incosistent
views about which fucntion accesses which global.
Most people do a "PerlRequire startup.pl" and then put all that stuff in
startup.pl, which runs before the fork. It will not be run twice because
PerlRequire will see it in %INC, unless you use PerlFreshRestart.
I was under the impression that you cannot configure Apache from a
PerlRequire. If that is not the case (and somehow works) I'd really like
to get away from perlsections.
Your sub x is a closure. That's why it returns the previous value of
$x. When it gets re-defined, it should start looking at the value of the
new $x.
You cannot redefine closures as well.
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing (short reproducible example)
On Wed, May 24, 2000 at 12:52:37AM +0300, Stas Bekman [EMAIL PROTECTED] wrote: You can control what's being reloaded and what's not: http://perl.apache.org/guide/config.html#Apache_Restarts_Twice_On_Start Not really. This would require intimate knowledge of some module's implementation (which files it requires and which it doesn't). My hackish example did something similar, indeed, but it had to be embedded into the module source, which is somewhat inconvinient. BTW, what is the connection between that part of config.html and PerlFreshRestart? Can it happen that modules get reparsed even when PerlFreshRestart is off? Should I abstain from trying to restart apache when using any modules requiring full perl semantics? And is SIGUSR1 safe? (Sorry, so many questions.. :) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing question (low priority ;)
Huh? Why is "do" a bad thing Do is bad because it is called every time, even if you've already executed You are confused about the two different forms of do. The do BLOCK form I used has nothing to do with the do EXPR form you seem to be confused about. perldoc -f do explains the differences quite in detail. flag to keep from compiling again and checking $@ yourself, so you're getting around these problems, but the file form of do is generally a red flag. This is just as saying "division is a bad thing in general, because it let's you try to divide by zero". Of course you could abuse do, but you should read about the different forms of do in the perl documentation where it explains what do is for, and what it isn't for. (let's direct followup discussion about perl to private mail, thanks) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing (short reproducible example)
On Tue, 23 May 2000, Perrin Harkins wrote: Your sub x is a closure. That's why it returns the previous value of No. In perl, a closure is *defined* as "anonymous subroutine" (see the documentation). If you define it different, you are right, but you are not talking about perl then. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing question (low priority ;)
On Tue, May 23, 2000 at 08:22:59PM -0700, Doug MacEachern [EMAIL PROTECTED] wrote: If this were true, it would be very bad. If there is no technical need to do this "half-reloading" then it should definitely be turned off. it is off by default, you turned it on with 'PerlFreshRestart On' Which had undocumented side effects, breaking valid perl. (No problem, if everywhere were documented... :) It breaks perfectly valid perl code, creating some kind of it's not breaking code, it's exposing broken code. if a module can't then the module is broken. You must be kidding here!!! Using lexicals on package level is broken??? If it is broken, then _why_ is it recommended programming practise in perl (see perltoot for example)? Sorry, you can say that this and that programming technique is invalid under PerlFreshRestart, which is entirely ok, and just means that mod_perl != perl (which it is in other details as well). Calling recommended perl programming techniques "broken", however, is something entirely different :-| Especially since this results in _very_ subtle problems, in where, for example, splitting a function into two functions might render a program not working for reasons well outside the understanding of many perl programmers. In any case, mod_perl is fine to do this. It would have bitten me much later if it didn't do this double-parsing at loading time. This, however, is an incompatibility from mod_perl to standard perl, not "broken". What is broken is the notion of "deleting from INC" is equivalent to "unloading a module". -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing (short reproducible example)
On Wed, May 24, 2000 at 11:30:40AM -0700, Perrin Harkins [EMAIL PROTECTED] wrote: example did something similar, indeed, but it had to be embedded into the module source, which is somewhat inconvinient. If you don't have PerlFreshRestart turned on (it is not recommended on production servers), only the stuff in your config file (Perl sections and PerlModule/PerlRequire) gets called twice. You can block things out like this: Actually, and to my surprise, PerlModule/PerlRequire is not done twice, at leats not with respect to reparsing. They simply don't reparse the modules, so mod_perl works *perfectly* without PerlFreshRestart, and without special measures. SUGUSR1. Personally, I always do cold stop and start on my production servers, in order to maximize shared memory, but this is difficult when you don't have a load-balanced cluster at your disposal... *) It's also more disruptive ;) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing question (low priority ;)
On Tue, May 23, 2000 at 12:56:28AM -0500, Autarch [EMAIL PROTECTED] wrote: On Tue, 23 May 2000, Marc Lehmann wrote: stable (mod_perl really is very unstable for large applications). Apart Wow, I wish you'd warned me before I did several large applications using mod_perl. Fortunately, they haven't experienced any mod_perl related Well, add XML::Parser, some large and twisted perl modules (like XML::XSLT), do a lot of things dynamically (recompiling etc..) and apache segfaults quite a lot, at random times, and even with some weird effects (adding a comment makes httpd segfault, removing it and it works again). Fortunately, once you found a source form where httpd starts up, mod_perl seems to run fine for a long time. I, too, used mod_perl for large projects _before_ and they ran stable (== Apache::Registry). But the picture changes quite a bit when I started to write my own mod_perl handler and to use the Apache::API extensively. But this is very off-topic, at leats to my real concern (double parsing). OTOH, the instability could very well be related to this as well. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing question (low priority ;)
On Tue, May 23, 2000 at 12:27:58PM +0800, Gunther Birznieks [EMAIL PROTECTED]
wrote:
replace my $global with use vars qw($global); and your problem should
disappear.
If you had read my mail you would have known that I do not search for a
workaround. While in this simple example it is possible to create a package
variable instead of the lexical, it is ugly (I mean, the variable is now
visible everywhere). In other circumstances it is very hard to usea package
variable instead, e.g.:
my $v;
sub xxx { $v }
my $v;
sub yyy { $v }
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing question (low priority ;)
On Mon, May 22, 2000 at 11:24:10PM -0700, Perrin Harkins [EMAIL PROTECTED] wrote: business about being parsed twice only applies to Apache's config file and Perl sections in it, not to your modules. A little followup: Looking at the mod_perl source, I see that INC is tinkered with in a lot of places. Removing modules from %INC would result in exactly the kind of behaviour I described. Unfortunately I don't know under which circumstances this happens, so I am still egaer to find out why and when this happens, and how to shield my application against that. I also forgot (uh-oh) to describe my system configuration (sorry, it was 5am): Currently, I use perl5.6 + mod_perl-1.23, but I see the behaviour since at least perl5.005_03 + mod_perl-1.21. The system is linux + apache 1.3.12 (problem seen since apache-1.3.9). -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing question (low priority ;)
On Tue, May 23, 2000 at 09:26:13AM +0100, Matt Sergeant [EMAIL PROTECTED] wrote: Hmm... AxKit does all this, and is very stable for me, and I've only had a couple of reports of segfaults, none of which went unsolved as far as I know... OK. To be fair, I am not 100% sure wether it's an mod_perl problem. The same application works fine in standalone (e.g. SpeedyCGI) mode. It also seems to be related to high perl parsing activity (the application parses at least 60k of perl source per second ;) So, as to not clobber this dicussion with any claims of mod_perl stability, let's consider mod_perl rock-solid, since the I can handle the stability problems (segfaults do not occur on requests, but either after them or while startup, so users aren't affected) What I cannot handle is the reparsing of my modules :( -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
global variables and reparsing (short reproducible example)
On Tue, May 23, 2000 at 07:15:46AM -0500, Ken Williams [EMAIL PROTECTED]
wrote:
my $global = 5;
sub set_global {
$global = shift;
}
othermodule::set_global 7;
=
Then, to my surprise, _sometimes_ the $global will be set, and sometimes not.
You know, if I didn't know any better, I'd say that you're simply setting
$global (not the best name) in one httpd child, but of course it's not getting
No, it's completely deterministic ("sometimes" == depending on the when
the function was compiled). The technical aspect is clear (to me at
least, but I am bad at explaining ;): the file is sourced twice, and the
function is compiled twice (and since each "my" generates another instance
of a lexical, there are two variables).
The problem is that some code that was compiled "in between" (i.e. after
the first compilation and before the second) gets "bound" to the first
version of the function (think of it as a kind of callback). This means
that some modifications are not seen by my module.
This is indeed related to config-file-parsing.
As a related note, I wondered why there isn't a mod_perl callback that is
clled _before_ forking, but after configuration parsing. This would allow
a lot of data sharing between the httpd servers. My module requires you to
call "configured PApp" at the end of the configuration section so that it
can pull in most of the code and big data structures before it forks (so
the data gets shared). AFAIK there is no way to make this automatic.
Then, when the file is parsed again, _another_ instance of $global gets
created, and another instance of set_global is compiled. Now there are two
versions of set_global, two versions of $global, and which one gets called
very much depends on random factors.
I don't think I believe that.
You won't believe how long it took until _I_ believed that!!
Nothing gets parsed twice except http configuration files. Is this file
used inside perl sections of config files, maybe?
"Used"? I definitely call it from within my perl sections, but it's not
embedded there (see the httpd.conf excerpt I posted). What happens is
that, somehow, %INC get's reset so perl thinks it hasn't loaded the module
yet, and just loads it again.
Ok, I tried a little bit and hopefully here is a small example that is
reproducible. Install this module as "X.pm" somewhere where mod_perl will
find it:
=
package X;
my $x;
$g;
my $o;
BEGIN {
$d = defined X::x;
$o = x if $d;
$x++; $g++;
}
open FILE, "/tmp/log";
print FILE "hello pid=$$ x=$x g=$g d=$d o=$o\n";
close FILE;
sub x { $x }
=
And use this httpd.conf:
=
ServerType standalone
PerlFreshRestart On
HostnameLookups off
Port 83
ServerRoot /tmp
ErrorLog error
PerlModule X
=
And then start httpd (I had to create conf/mime.types and a logs directory
to make apache run). Then look at the file /tmp/log:
cerebro:/tmp# rm /tmp/log; httpd -f /tmp/httpd.conf; sleep 1; cat /tmp/log
hello pid=32011 x=1 g=1 d= o=
hello pid=32011 x=1 g=2 d=1 o=1
The module is indeed compiled twice. Please note that, on the first time,
X::x is not yet defined (it wasn't compiled yet) but on the second time,
it is ($d==1). Also note that the global "$g" isn't changed (it survives,
as expected), while "$x" (a lexical) is allocated anew. However, a call
to the previous version of X::x returns the old value, i.e. o=1 (X::x is
still bound to the old copy since it wasn't recompiled yet, even if "$x"
is undefined at this moment).
I assumed it _had_ something to do with parsing twice, but, in fact, it
happens without using a perl section. My mod_perl is configured like this:
=
PERL_SECTIONS=1
PERL_STACKED_HANDLERS=1
PERL_DIRECTIVE_HANDLERS=1
PERL_RESTART=1
PERL_AUTHEN=1
PERL_AUTHZ=1
PERL_TYPE=1
DO_HTTPD=1
=
Am I the only one who thinks that
{
my $x = ...;
sub the_only_function_using_x {
}
}
is cleaner then using a global variable? Heck, it's even recommended by
the perl documentation!
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing question (low priority ;)
On Tue, May 23, 2000 at 10:08:46AM -0700, Gustavo Duarte [EMAIL PROTECTED] wrote:
I'm not sure this makes sense for your case, but it might help, so...
It probably makes a lot of sense. Thanks!
"When the server [apache] is restarted, the configuration and module
initialization phases are called again. To ensure such restarts will be
uneventful, Apache actually runs these two phases twice during server startup
just to check that all modules can survive a restart."
If this were true, it would be very bad. If there is no technical
need to do this "half-reloading" then it should definitely be turned
off. It breaks perfectly valid perl code, creating some kind of
"mod_perl-language-subset". At least _I_ lost days of debugging to find
out about these subtle differences to standard perl. (And this also means
that some CPAN modules would need to be patched to run with mod_perl).
Or in other words: mod_perl should either do it cleanly or not at all. Or
have a very good reason to break code ;)
Does the book say what "module initialization phase" means, and how one
can work around that? (Apart from patching _every_ module with a hack like
the one I posted)?
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing (short reproducible example)
On Tue, May 23, 2000 at 11:53:04AM -0700, Doug MacEachern [EMAIL PROTECTED] wrote: a lot of data sharing between the httpd servers. My module requires you to call "configured PApp" at the end of the configuration section so that it can pull in most of the code and big data structures before it forks (so the data gets shared). AFAIK there is no way to make this automatic. what's wrong with having 'PerlModule foo' at the bottom of your config Nothing. It's just that users of my module constantly forget that ;) file? apache 1.3 has no 'post-config' hook, 2.0 does though. Another non-issue, then ;) PerlFreshRestart On PerlModule X yes, because apache does module-init/config-read twice at startup, since you have 'PerlFreshRestart On', your module is re-compiled on this second BINGO, exactly. Thanks a lot for clarifying!! Here is a proposed extension for perldoc mod_perl / RESTARTING: PerlFreshRestart On Please note that setting this option results in parsing your modules twice at server startup, and reparsing at every restart. Old code and variables are not cleared and might interfere with your module. Avoiding lexical variables at global scope and not referencing subs before declaration/definition is therefore recommended. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
global variables and reparsing question (low priority ;)
While I understand how my problem happens, it just caught me again
(letting me debug another two hours), and so I wanted to ask why this
peculiar behaviour is really necessary.
But straight to the point. If I have a module like this:
=
package othermodule;
my $global = 5;
sub set_global {
$global = shift;
}
=
And use this from another module:
=
othermodule::set_global 7;
=
Then, to my surprise, _sometimes_ the $global will be set, and sometimes not.
The reason for this seems to be that othermodule is parsed twice: Once
during configuration parsing and once later. The first time it is parsed,
$global is created and bound to ste_global. Other modules that call the
function get a reference to it cimpiled in.
Then, when the file is parsed again, _another_ instance of $global gets
created, and another instance of set_global is compiled. Now there are two
versions of set_global, two versions of $global, and which one gets called
very much depends on random factors.
To my pity, _some_ global (non-lexical) variables also get cleared, but
not in all packages.
Now, since this is really hard to debug (the order of function declaration
within the same file and wether the funciton is exported or not, and
wether there are other callers outside the file etc...
To remedy this, many of my modules (the bad point is that even modules I have
no control over suffer from this problem) start with something like this:
=
if (!defined $PApp::Apache::compiled) { eval do { local $/; DATA };
die $@ if $@ } 1;
__DATA__
#line 6 "(PApp::Apache source)"
=
This mostly remedies the situation. The only "side effects" that I found
so far is that my httpds reuqire much _less_ memory and run _much_ more
stable (mod_perl really is very unstable for large applications). Apart
from expected semantics of global lexical variables.
So my very own problem is solvable (either by using package variables
or that ugly but working trick above). Third-party modules (e.g. from
CPAN) however, still suffer from this problem, and due to my dumbness I
fall into that trap again and again.
And so my question is: why does this behaviour exist, and why is it
necessary (the documents I saw so far only told me that this "has
something to do with apache's configuration file parsing", which doesn't
explain much, especially as it does seem unnecessary).
Thanks a lot for any feedback ;)
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
