Re: global variables and reparsing (short reproducible example)
On 24 May 2000, Randal L. Schwartz wrote:
"tayers" == [EMAIL PROTECTED] writes:
tayers Given the above more complete descriptions I would say the usage in
tayers Lperlsub is confusing.
I agree with that, and with your general observation. From my hanging
out on P5P, a subroutine is only a closure when it sees lexical
variables and must therefore create a distinct clone of the
environment on the way out.
And both named subroutines and anonymous subroutines can be closures.
Here's how to tell if a subroutine is a closure or not:
for (1..5) {
push @a, sub { "hi there" };
}
for (1..5) {
{
my $b;
push @b, sub { $b."hi there" };
}
}
print "anon normal: @a\n";
print "anon closure: @b\n";
which generates
anon normal: CODE(0x80ce9d4) CODE(0x80ce9d4) CODE(0x80ce9d4) CODE(0x80ce9d4)
CODE(0x80ce9d4)
anon closure: CODE(0x80c83ac) CODE(0x80d2440) CODE(0x80d24a0) CODE(0x80d2500)
CODE(0x80d2560)
Note how each coderef from the non-closure is identical, but the
closure form must generate distinct coderefs to point at the distinct
instances of the closure.
This is the underpinnings of that famous "won't stay shared" message.
A "my" variable in a named subroutine context is generating distinct
coderefs.
So, in summary, "closure" vs "non-closure", and "named subroutine" vs
"anonymous subroutine" are orthogonal concepts. Pick one from each
column. :) And it's too bad that we have people mixing the terms up,
because cargo-cult meme virus items spread fast in the Perl extended
community.
Well, it's been about 6 months ago when Randal wrote this, but only now
I've finally got the time to reply to this. That's why the whole post is
quoted.
Apparently, the above explanation is not so complete and clear and while
"closure" vs "non-closure", and "named subroutine" vs "anonymous
subroutine" are indeed orthogonal concepts, only the "closure"+"anonymous
subroutine" appears to work, and "closure"+"named subroutine" is something
that should never happen, unless it's a desired effect and then the
programmer knows what's he doing.
Here is the part that Randal didn't post:
And now the same with named subroutines:
for (1..5) {
sub a { "hi there" };
push @a, \a;
}
for (1..5) {
{
my $b;
sub b { $b."hi there" };
push @b, \b;
}
}
print "normal:\n", join "\t\n",@a,"\n";
print "closure:\n",join "\t\n",@b,"\n";
which generates:
anon normal:
CODE(0x80568c0)
CODE(0x80568c0)
CODE(0x80568c0)
CODE(0x80568c0)
CODE(0x80568c0)
anon closure:
CODE(0x8056998)
CODE(0x8056998)
CODE(0x8056998)
CODE(0x8056998)
CODE(0x8056998)
We can see that both versions has generated the same code reference. For
the subroutine Ia it's easy, since it doesn't include any lexical
variables defined outside it in the same lexical scope.
As for the subroutine Ib, it's indeed a closure sub, but Perl won't
recompile it since it's a named subroutine (see the perlsub manpage). It's
something that we don't want to happen in our code.
This is the underpinnings of that famous I"won't stay shared" message.
A Imy variable in a named subroutine context is generating identical
code references and therefore it ignores any future changes to the lexical
variables outside of it.
BTW, for those who aren't yet familiar with the famous I"won't stay
shared" message and the prove that "closure"+"anonymous subroutine" work,
please read:
http://perl.apache.org/guide/perl.html#my_Scoped_Variable_in_Nested_S
Thanks!
_
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:[EMAIL PROTECTED] http://apachetoday.com http://jazzvalley.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: global variables and reparsing (short reproducible example)
On Fri, 26 May 2000, Marc Lehmann wrote: On Thu, May 25, 2000 at 12:09:09PM -0700, Doug MacEachern [EMAIL PROTECTED] wrote: You can only configure Apache from Perl sections, but you can load all your modules, shared data, etc. from a file pulled in with PerlRequire. actually you can, if a module defines variables in the Apache::ReadConfig:: namespace, they are fed to the apache config gears just as Perl sections are. Erhm, I am doing this right now (the functions I call just implant Location directives etc.. into the caller's package). Now it would be interesting to know wether that works only when called from perl sections or also form perlrequire? (Hmm, I'll just try it out and assume it's supported if it works ;) yes, it's supported with PerlRequire and PerlModule.
Re: global variables and reparsing (short reproducible example)
On Wed, 24 May 2000, Perrin Harkins wrote: On Wed, 24 May 2000, Marc Lehmann wrote: I was under the impression that you cannot configure Apache from a PerlRequire. If that is not the case (and somehow works) I'd really like to get away from perlsections. You can only configure Apache from Perl sections, but you can load all your modules, shared data, etc. from a file pulled in with PerlRequire. actually you can, if a module defines variables in the Apache::ReadConfig:: namespace, they are fed to the apache config gears just as Perl sections are.
Re: global variables and reparsing (short reproducible example)
On Thu, May 25, 2000 at 12:09:09PM -0700, Doug MacEachern [EMAIL PROTECTED] wrote: You can only configure Apache from Perl sections, but you can load all your modules, shared data, etc. from a file pulled in with PerlRequire. actually you can, if a module defines variables in the Apache::ReadConfig:: namespace, they are fed to the apache config gears just as Perl sections are. Erhm, I am doing this right now (the functions I call just implant Location directives etc.. into the caller's package). Now it would be interesting to know wether that works only when called from perl sections or also form perlrequire? (Hmm, I'll just try it out and assume it's supported if it works ;) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing (short reproducible example)
On Tue, May 23, 2000 at 04:07:40PM -0700, Perrin Harkins [EMAIL PROTECTED] wrote:
This is a combination of closures and PerlFreshRestart biting
(still no closures)
My example might be misleading, since I used x before it was defined (to make
the example short). Typical examples look like this:
my $global1;
sub func1_using_global1 {
}
sub func2_using_global2_and_func1 {
}
etc... Users of the module outside the module itself can get incosistent
views about which fucntion accesses which global.
Most people do a "PerlRequire startup.pl" and then put all that stuff in
startup.pl, which runs before the fork. It will not be run twice because
PerlRequire will see it in %INC, unless you use PerlFreshRestart.
I was under the impression that you cannot configure Apache from a
PerlRequire. If that is not the case (and somehow works) I'd really like
to get away from perlsections.
Your sub x is a closure. That's why it returns the previous value of
$x. When it gets re-defined, it should start looking at the value of the
new $x.
You cannot redefine closures as well.
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing (short reproducible example)
On Wed, May 24, 2000 at 12:52:37AM +0300, Stas Bekman [EMAIL PROTECTED] wrote: You can control what's being reloaded and what's not: http://perl.apache.org/guide/config.html#Apache_Restarts_Twice_On_Start Not really. This would require intimate knowledge of some module's implementation (which files it requires and which it doesn't). My hackish example did something similar, indeed, but it had to be embedded into the module source, which is somewhat inconvinient. BTW, what is the connection between that part of config.html and PerlFreshRestart? Can it happen that modules get reparsed even when PerlFreshRestart is off? Should I abstain from trying to restart apache when using any modules requiring full perl semantics? And is SIGUSR1 safe? (Sorry, so many questions.. :) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing (short reproducible example)
On Tue, 23 May 2000, Perrin Harkins wrote: Your sub x is a closure. That's why it returns the previous value of No. In perl, a closure is *defined* as "anonymous subroutine" (see the documentation). If you define it different, you are right, but you are not talking about perl then. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing (short reproducible example)
"tayers" == [EMAIL PROTECTED] writes:
tayers Given the above more complete descriptions I would say the usage in
tayers Lperlsub is confusing.
I agree with that, and with your general observation. From my hanging
out on P5P, a subroutine is only a closure when it sees lexical
variables and must therefore create a distinct clone of the
environment on the way out.
And both named subroutines and anonymous subroutines can be closures.
Here's how to tell if a subroutine is a closure or not:
for (1..5) {
push @a, sub { "hi there" };
}
for (1..5) {
{
my $b;
push @b, sub { $b."hi there" };
}
}
print "anon normal: @a\n";
print "anon closure: @b\n";
which generates
anon normal: CODE(0x80ce9d4) CODE(0x80ce9d4) CODE(0x80ce9d4) CODE(0x80ce9d4)
CODE(0x80ce9d4)
anon closure: CODE(0x80c83ac) CODE(0x80d2440) CODE(0x80d24a0) CODE(0x80d2500)
CODE(0x80d2560)
Note how each coderef from the non-closure is identical, but the
closure form must generate distinct coderefs to point at the distinct
instances of the closure.
This is the underpinnings of that famous "won't stay shared" message.
A "my" variable in a named subroutine context is generating distinct
coderefs.
So, in summary, "closure" vs "non-closure", and "named subroutine" vs
"anonymous subroutine" are orthogonal concepts. Pick one from each
column. :) And it's too bad that we have people mixing the terms up,
because cargo-cult meme virus items spread fast in the Perl extended
community.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
[EMAIL PROTECTED] URL:http://www.stonehenge.com/merlyn/
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Re: global variables and reparsing (short reproducible example)
On Wed, 24 May 2000, Marc Lehmann wrote:
On Wed, May 24, 2000 at 12:52:37AM +0300, Stas Bekman [EMAIL PROTECTED] wrote:
You can control what's being reloaded and what's not:
http://perl.apache.org/guide/config.html#Apache_Restarts_Twice_On_Start
Not really. This would require intimate knowledge of some module's
implementation (which files it requires and which it doesn't). My hackish
example did something similar, indeed, but it had to be embedded into the
module source, which is somewhat inconvinient.
If you don't have PerlFreshRestart turned on (it is not recommended on
production servers), only the stuff in your config file (Perl sections
and PerlModule/PerlRequire) gets called twice. You can block things out
like this:
Perl
unless ($Apache::Server::ReStarting) {
...
}
/Perl
BTW, what is the connection between that part of config.html and
PerlFreshRestart? Can it happen that modules get reparsed even when
PerlFreshRestart is off?
Only the stuff in your config file should get reparsed.
Should I abstain from trying to restart apache when using any modules
requiring full perl semantics? And is SIGUSR1 safe?
People have reported some growth in memory consumption when using
SUGUSR1. Personally, I always do cold stop and start on my production
servers, in order to maximize shared memory, but this is difficult when
you don't have a load-balanced cluster at your disposal...
- Perrin
Re: global variables and reparsing (short reproducible example)
On Wed, 24 May 2000, Marc Lehmann wrote:
I was under the impression that you cannot configure Apache from a
PerlRequire. If that is not the case (and somehow works) I'd really like
to get away from perlsections.
You can only configure Apache from Perl sections, but you can load all
your modules, shared data, etc. from a file pulled in with PerlRequire.
Your sub x is a closure. That's why it returns the previous value of
$x. When it gets re-defined, it should start looking at the value of the
new $x.
You cannot redefine closures as well.
Closures are a confusing aspect of perl, and I could easily be mistaken
about them, but Randal and Doug seem to agree that this is a closure:
my $x;
sub show_x {
print $x;
}
The easiest definition I've found is the one that Damian Conway has in his
OO Perl book:
"In Perl, a closure is just a subroutine that refers to one or more
lexical variables declared outside the subroutine itself."
When this code gets parsed, the $x goes out of scope at the end of the
block, but show_x keeps a copy of it.
I think that this is the source of the behavior you saw where it looked
like there were multiple versions of subs and variables being created.
Anyway, I was just trying to help you solve your problem. There's no need
to get defensive.
- Perrin
Re: global variables and reparsing (short reproducible example)
On Wed, May 24, 2000 at 11:30:40AM -0700, Perrin Harkins [EMAIL PROTECTED] wrote: example did something similar, indeed, but it had to be embedded into the module source, which is somewhat inconvinient. If you don't have PerlFreshRestart turned on (it is not recommended on production servers), only the stuff in your config file (Perl sections and PerlModule/PerlRequire) gets called twice. You can block things out like this: Actually, and to my surprise, PerlModule/PerlRequire is not done twice, at leats not with respect to reparsing. They simply don't reparse the modules, so mod_perl works *perfectly* without PerlFreshRestart, and without special measures. SUGUSR1. Personally, I always do cold stop and start on my production servers, in order to maximize shared memory, but this is difficult when you don't have a load-balanced cluster at your disposal... *) It's also more disruptive ;) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
global variables and reparsing (short reproducible example)
On Tue, May 23, 2000 at 07:15:46AM -0500, Ken Williams [EMAIL PROTECTED]
wrote:
my $global = 5;
sub set_global {
$global = shift;
}
othermodule::set_global 7;
=
Then, to my surprise, _sometimes_ the $global will be set, and sometimes not.
You know, if I didn't know any better, I'd say that you're simply setting
$global (not the best name) in one httpd child, but of course it's not getting
No, it's completely deterministic ("sometimes" == depending on the when
the function was compiled). The technical aspect is clear (to me at
least, but I am bad at explaining ;): the file is sourced twice, and the
function is compiled twice (and since each "my" generates another instance
of a lexical, there are two variables).
The problem is that some code that was compiled "in between" (i.e. after
the first compilation and before the second) gets "bound" to the first
version of the function (think of it as a kind of callback). This means
that some modifications are not seen by my module.
This is indeed related to config-file-parsing.
As a related note, I wondered why there isn't a mod_perl callback that is
clled _before_ forking, but after configuration parsing. This would allow
a lot of data sharing between the httpd servers. My module requires you to
call "configured PApp" at the end of the configuration section so that it
can pull in most of the code and big data structures before it forks (so
the data gets shared). AFAIK there is no way to make this automatic.
Then, when the file is parsed again, _another_ instance of $global gets
created, and another instance of set_global is compiled. Now there are two
versions of set_global, two versions of $global, and which one gets called
very much depends on random factors.
I don't think I believe that.
You won't believe how long it took until _I_ believed that!!
Nothing gets parsed twice except http configuration files. Is this file
used inside perl sections of config files, maybe?
"Used"? I definitely call it from within my perl sections, but it's not
embedded there (see the httpd.conf excerpt I posted). What happens is
that, somehow, %INC get's reset so perl thinks it hasn't loaded the module
yet, and just loads it again.
Ok, I tried a little bit and hopefully here is a small example that is
reproducible. Install this module as "X.pm" somewhere where mod_perl will
find it:
=
package X;
my $x;
$g;
my $o;
BEGIN {
$d = defined X::x;
$o = x if $d;
$x++; $g++;
}
open FILE, "/tmp/log";
print FILE "hello pid=$$ x=$x g=$g d=$d o=$o\n";
close FILE;
sub x { $x }
=
And use this httpd.conf:
=
ServerType standalone
PerlFreshRestart On
HostnameLookups off
Port 83
ServerRoot /tmp
ErrorLog error
PerlModule X
=
And then start httpd (I had to create conf/mime.types and a logs directory
to make apache run). Then look at the file /tmp/log:
cerebro:/tmp# rm /tmp/log; httpd -f /tmp/httpd.conf; sleep 1; cat /tmp/log
hello pid=32011 x=1 g=1 d= o=
hello pid=32011 x=1 g=2 d=1 o=1
The module is indeed compiled twice. Please note that, on the first time,
X::x is not yet defined (it wasn't compiled yet) but on the second time,
it is ($d==1). Also note that the global "$g" isn't changed (it survives,
as expected), while "$x" (a lexical) is allocated anew. However, a call
to the previous version of X::x returns the old value, i.e. o=1 (X::x is
still bound to the old copy since it wasn't recompiled yet, even if "$x"
is undefined at this moment).
I assumed it _had_ something to do with parsing twice, but, in fact, it
happens without using a perl section. My mod_perl is configured like this:
=
PERL_SECTIONS=1
PERL_STACKED_HANDLERS=1
PERL_DIRECTIVE_HANDLERS=1
PERL_RESTART=1
PERL_AUTHEN=1
PERL_AUTHZ=1
PERL_TYPE=1
DO_HTTPD=1
=
Am I the only one who thinks that
{
my $x = ...;
sub the_only_function_using_x {
}
}
is cleaner then using a global variable? Heck, it's even recommended by
the perl documentation!
--
-==- |
==-- _ |
---==---(_)__ __ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e|
-=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
Re: global variables and reparsing (short reproducible example)
On Tue, 23 May 2000, Marc Lehmann wrote: As a related note, I wondered why there isn't a mod_perl callback that is clled _before_ forking, but after configuration parsing. This would allow a lot of data sharing between the httpd servers. My module requires you to call "configured PApp" at the end of the configuration section so that it can pull in most of the code and big data structures before it forks (so the data gets shared). AFAIK there is no way to make this automatic. what's wrong with having 'PerlModule foo' at the bottom of your config file? apache 1.3 has no 'post-config' hook, 2.0 does though. Then, when the file is parsed again, _another_ instance of $global gets PerlFreshRestart On PerlModule X The module is indeed compiled twice. yes, because apache does module-init/config-read twice at startup, since you have 'PerlFreshRestart On', your module is re-compiled on this second pass.
Re: global variables and reparsing (short reproducible example)
On Tue, May 23, 2000 at 11:53:04AM -0700, Doug MacEachern [EMAIL PROTECTED] wrote: a lot of data sharing between the httpd servers. My module requires you to call "configured PApp" at the end of the configuration section so that it can pull in most of the code and big data structures before it forks (so the data gets shared). AFAIK there is no way to make this automatic. what's wrong with having 'PerlModule foo' at the bottom of your config Nothing. It's just that users of my module constantly forget that ;) file? apache 1.3 has no 'post-config' hook, 2.0 does though. Another non-issue, then ;) PerlFreshRestart On PerlModule X yes, because apache does module-init/config-read twice at startup, since you have 'PerlFreshRestart On', your module is re-compiled on this second BINGO, exactly. Thanks a lot for clarifying!! Here is a proposed extension for perldoc mod_perl / RESTARTING: PerlFreshRestart On Please note that setting this option results in parsing your modules twice at server startup, and reparsing at every restart. Old code and variables are not cleared and might interfere with your module. Avoiding lexical variables at global scope and not referencing subs before declaration/definition is therefore recommended. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | |
Re: global variables and reparsing (short reproducible example)
On Tue, 23 May 2000, Perrin Harkins wrote: Your sub x is a closure. That's why it returns the previous value of $x. When it gets re-defined, it should start looking at the value of the new $x. nevermind what i said in the other reply about not being a closure. you're right, it is by definition, i just assumed you were refering to the Apache::Registry trap.
