Re: memory leaking with closures

[Stas Bekman]

On Fri, 7 Sep 2001, Alex Krohn wrote:

> Hi,
>
> > >> With this simple test script:
> > >>
> > >> print "Content-type: text/html\n\n";
> > >> my $var = 'x' x 500000;
> > >
> > > our $var = 'x' x 500000;
> > >
> > >>
> > >> my $sub = sub { my $sub2 = sub { $var; }; };
> > >> print "Done\n";
> > >>
> > >> $var does not get freed, and the process grows each request. Has anyone
> > >> seen this sort of behavior and have any ideas/workarounds (besides don't
> > >
> > > yes.  this is the documented behavior for closures.  the anonymous sub
> > > must have it's own copy of $var which happens to be 500k plus perl
> > > overhead on each invocation.
> >
> > I think the problem is that $var doesn't get freed even when $var and
> > $sub and $sub2 have all gone out of scope.  That's a perl bug for which I
> > can fathom no workaround.
> >
> > If any of those variables are still in scope, then of course $var can't
> > get freed.
>
> Yes, upon further investigation, this perl script:
>
> while (1) {
>       {
>               my $var = 'x' x 500000;
>               my $sub = sub { my $sub2 = sub { $var; } };
>       }
>       # $var and $sub should be gone, but memory is never freed
>       sleep 1; # Don't crash things =)
> }
>
> will grow forever as $var never seems to be freed even when everything
> should go out of scope, definately a perl bug, not a mod_perl one. =)

Look like memory leaking to me. The reference count is done incorrectly,
which leads to memory leaking:

use GTop;
use Devel::Peek;
my $gtop = GTop->new;

for (1..3) {
    my $a = 'x' x 500000;
    {
        sub { $a; };
    }
    printf "RERCNT %s: \$a=%d\n",
        GTop::size_string($gtop->proc_mem($$)->rss),
        Devel::Peek::SvREFCNT($a);
}

prints:

RERCNT  3.5M: $a=1
RERCNT  3.5M: $a=1
RERCNT  3.5M: $a=1

all fine!

use GTop;
use Devel::Peek;
my $gtop = GTop->new;

for (1..3) {
    my $a = 'x' x 500000;
    {
        sub { my $sub2 = sub { $a; }; };
    }
    printf "RERCNT %s: \$a=%d\n",
        GTop::size_string($gtop->proc_mem($$)->rss),
        Devel::Peek::SvREFCNT($a);
}

prints:

RERCNT  3.5M: $a=4
RERCNT  4.0M: $a=3
RERCNT  4.5M: $a=3

1. the reference counting is wrong
2. memory leaks (because of 1)

This is a hack to fix the leaking (well, almost):

use GTop;
use Devel::Peek;
my $gtop = GTop->new;

for (1..3) {
    my $a = 'x' x 500000;
    {
        sub { my $sub2 = sub { $a; }; };
        Devel::Peek::SvREFCNT_dec($a);
        Devel::Peek::SvREFCNT_dec($a);
    }
    printf "RERCNT %s: \$a=%d\n",
        GTop::size_string($gtop->proc_mem($$)->rss),
        Devel::Peek::SvREFCNT($a);
}

prints:

RERCNT  3.5M: $a=2
RERCNT  4.0M: $a=1
RERCNT  4.0M: $a=1


> Oddly, if you just do
>
>       my $sub = sub { $var; };
>
> it does not grow, definately something strange going on. Happens on
> perl 5.004_04, 5.005_03 and 5.6.1.

You mean this works for you?:

while (1) {
       {
               my $var = 'x' x 500000;
               my $sub = sub { $var; };
       }
       # $var and $sub should be gone, but memory is never freed
       sleep 1; # Don't crash things =)
}

that's because your $sub goes out of scope. the other one seems like a
bug.

If I didn't miss something, it seems that we need to run this through p5p.

_____________________________________________________________________
Stas Bekman              JAm_pH     --   Just Another mod_perl Hacker
http://stason.org/       mod_perl Guide  http://perl.apache.org/guide
mailto:[EMAIL PROTECTED]   http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/