Re: memory leaking with closures
Hi,
With this simple test script:
print Content-type: text/html\n\n;
my $var = 'x' x 50;
our $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; }; };
print Done\n;
$var does not get freed, and the process grows each request. Has anyone
seen this sort of behavior and have any ideas/workarounds (besides don't
yes. this is the documented behavior for closures. the anonymous sub
must have it's own copy of $var which happens to be 500k plus perl
overhead on each invocation.
I think the problem is that $var doesn't get freed even when $var and
$sub and $sub2 have all gone out of scope. That's a perl bug for which I
can fathom no workaround.
If any of those variables are still in scope, then of course $var can't
get freed.
Yes, upon further investigation, this perl script:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; } };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
will grow forever as $var never seems to be freed even when everything
should go out of scope, definately a perl bug, not a mod_perl one. =)
Oddly, if you just do
my $sub = sub { $var; };
it does not grow, definately something strange going on. Happens on
perl 5.004_04, 5.005_03 and 5.6.1.
Cheers,
Alex
Re: memory leaking with closures
On Fri, 7 Sep 2001, Alex Krohn wrote:
Hi,
With this simple test script:
print Content-type: text/html\n\n;
my $var = 'x' x 50;
our $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; }; };
print Done\n;
$var does not get freed, and the process grows each request. Has anyone
seen this sort of behavior and have any ideas/workarounds (besides don't
yes. this is the documented behavior for closures. the anonymous sub
must have it's own copy of $var which happens to be 500k plus perl
overhead on each invocation.
I think the problem is that $var doesn't get freed even when $var and
$sub and $sub2 have all gone out of scope. That's a perl bug for which I
can fathom no workaround.
If any of those variables are still in scope, then of course $var can't
get freed.
Yes, upon further investigation, this perl script:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; } };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
will grow forever as $var never seems to be freed even when everything
should go out of scope, definately a perl bug, not a mod_perl one. =)
Look like memory leaking to me. The reference count is done incorrectly,
which leads to memory leaking:
use GTop;
use Devel::Peek;
my $gtop = GTop-new;
for (1..3) {
my $a = 'x' x 50;
{
sub { $a; };
}
printf RERCNT %s: \$a=%d\n,
GTop::size_string($gtop-proc_mem($$)-rss),
Devel::Peek::SvREFCNT($a);
}
prints:
RERCNT 3.5M: $a=1
RERCNT 3.5M: $a=1
RERCNT 3.5M: $a=1
all fine!
use GTop;
use Devel::Peek;
my $gtop = GTop-new;
for (1..3) {
my $a = 'x' x 50;
{
sub { my $sub2 = sub { $a; }; };
}
printf RERCNT %s: \$a=%d\n,
GTop::size_string($gtop-proc_mem($$)-rss),
Devel::Peek::SvREFCNT($a);
}
prints:
RERCNT 3.5M: $a=4
RERCNT 4.0M: $a=3
RERCNT 4.5M: $a=3
1. the reference counting is wrong
2. memory leaks (because of 1)
This is a hack to fix the leaking (well, almost):
use GTop;
use Devel::Peek;
my $gtop = GTop-new;
for (1..3) {
my $a = 'x' x 50;
{
sub { my $sub2 = sub { $a; }; };
Devel::Peek::SvREFCNT_dec($a);
Devel::Peek::SvREFCNT_dec($a);
}
printf RERCNT %s: \$a=%d\n,
GTop::size_string($gtop-proc_mem($$)-rss),
Devel::Peek::SvREFCNT($a);
}
prints:
RERCNT 3.5M: $a=2
RERCNT 4.0M: $a=1
RERCNT 4.0M: $a=1
Oddly, if you just do
my $sub = sub { $var; };
it does not grow, definately something strange going on. Happens on
perl 5.004_04, 5.005_03 and 5.6.1.
You mean this works for you?:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { $var; };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
that's because your $sub goes out of scope. the other one seems like a
bug.
If I didn't miss something, it seems that we need to run this through p5p.
_
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:[EMAIL PROTECTED] http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: memory leaking with closures
Oddly, if you just do
my $sub = sub { $var; };
it does not grow, definately something strange going on. Happens on
perl 5.004_04, 5.005_03 and 5.6.1.
You mean this works for you?:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { $var; };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
that's because your $sub goes out of scope.
The leaking behavior only happens when you use nested anonymous subs. It
should always be possible to avoid it, although your code may look uglier.
the other one seems like a
bug.
If I didn't miss something, it seems that we need to run this through p5p.
It's been known about for a while. I'm not sure what the status of getting
a fix is. I assume that it must be difficult to fix or it would have been
changed a long time ago.
- Perrin
RE: memory leaking with closures
On Fri, Sep 07, 2001 at 12:27:58AM -0700, Alex Krohn wrote:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; } };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
will grow forever
it does not grow, definately something strange going on. Happens on
perl 5.004_04, 5.005_03 and 5.6.1.
confirmed in bleadperl (patch 11936). CCing p5p.
atleast one sub { sub{} } leak was fixed recently, but not this one.
- Barrie
Seems like we are not properly freeing the prototype CV which is cloned.
Arthur
SV: memory leaking with closures
On Fri, 07 Sep 2001 18:06:17 +0200, Arthur Bergman wrote:
On Fri, Sep 07, 2001 at 12:27:58AM -0700, Alex Krohn wrote:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; } };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
will grow forever
it does not grow, definately something strange going on. Happens on
perl 5.004_04, 5.005_03 and 5.6.1.
confirmed in bleadperl (patch 11936). CCing p5p.
atleast one sub { sub{} } leak was fixed recently, but not this one.
- Barrie
Seems like we are not properly freeing the prototype CV which is cloned.
Not likely, since there are always a fixed number of closure prototypes
when there is no eval to create new ones.
Silly me.
It is more likely that the reference loop between the inner and outer
CVs is preventing the freeing of either of them. I'm not in fact sure
that the reference loop *can* be eliminated trivially, given these two
CVs can have different lifetimes. Perhaps the right solution is to
move to using weakrefs for CvOUTSIDE(), I dunno.
Sounds like an idea anyway. I will give it a try. Only problem I see it will be messed
up after perl_clone.
Arthur
Re: memory leaking with closures
On Fri, Sep 07, 2001 at 12:27:58AM -0700, Alex Krohn wrote:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; } };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
will grow forever
it does not grow, definately something strange going on. Happens on
perl 5.004_04, 5.005_03 and 5.6.1.
confirmed in bleadperl (patch 11936). CCing p5p.
atleast one sub { sub{} } leak was fixed recently, but not this one.
- Barrie
Re: memory leaking with closures
On Fri, 07 Sep 2001 18:06:17 +0200, Arthur Bergman wrote:
On Fri, Sep 07, 2001 at 12:27:58AM -0700, Alex Krohn wrote:
while (1) {
{
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; } };
}
# $var and $sub should be gone, but memory is never freed
sleep 1; # Don't crash things =)
}
will grow forever
it does not grow, definately something strange going on. Happens on
perl 5.004_04, 5.005_03 and 5.6.1.
confirmed in bleadperl (patch 11936). CCing p5p.
atleast one sub { sub{} } leak was fixed recently, but not this one.
- Barrie
Seems like we are not properly freeing the prototype CV which is cloned.
Not likely, since there are always a fixed number of closure prototypes
when there is no eval to create new ones.
It is more likely that the reference loop between the inner and outer
CVs is preventing the freeing of either of them. I'm not in fact sure
that the reference loop *can* be eliminated trivially, given these two
CVs can have different lifetimes. Perhaps the right solution is to
move to using weakrefs for CvOUTSIDE(), I dunno.
Sarathy
[EMAIL PROTECTED]
memory leaking with closures
Hi,
With this simple test script:
print Content-type: text/html\n\n;
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; }; };
print Done\n;
$var does not get freed, and the process grows each request. Has anyone
seen this sort of behavior and have any ideas/workarounds (besides don't
do that).
Cheers,
Alex
Re: memory leaking with closures
With this simple test script:
print Content-type: text/html\n\n;
my $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; }; };
print Done\n;
$var does not get freed, and the process grows each request. Has anyone
seen this sort of behavior and have any ideas/workarounds (besides don't
do that).
I've commented on this before, as have others, in the context of using
Error.pm. There is no workaround for this except don't do that.
- Perrin
Re: memory leaking with closures
Alex Krohn wrote:
Hi,
With this simple test script:
print Content-type: text/html\n\n;
my $var = 'x' x 50;
our $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; }; };
print Done\n;
$var does not get freed, and the process grows each request. Has anyone
seen this sort of behavior and have any ideas/workarounds (besides don't
yes. this is the documented behavior for closures. the anonymous sub must
have it's own copy of $var which happens to be 500k plus perl overhead on each
invocation.
do that).
Cheers,
Alex
--
___cliff [EMAIL PROTECTED]http://www.genwax.com/
Re: memory leaking with closures
___cliff rayman___ [EMAIL PROTECTED] wrote:
Alex Krohn wrote:
Hi,
With this simple test script:
print Content-type: text/html\n\n;
my $var = 'x' x 50;
our $var = 'x' x 50;
my $sub = sub { my $sub2 = sub { $var; }; };
print Done\n;
$var does not get freed, and the process grows each request. Has anyone
seen this sort of behavior and have any ideas/workarounds (besides don't
yes. this is the documented behavior for closures. the anonymous sub
must have it's own copy of $var which happens to be 500k plus perl
overhead on each invocation.
I think the problem is that $var doesn't get freed even when $var and
$sub and $sub2 have all gone out of scope. That's a perl bug for which I
can fathom no workaround.
If any of those variables are still in scope, then of course $var can't
get freed.
-Ken
