Re: leaks with Apache::Request?
Joe Schaefer [EMAIL PROTECTED] writes:
[...]
Somehow the assignment operator MUST be involved in the leak here.
(You only get a leak when the *same* reference (*SV) is on both sides
of the assignment).
Could someone with modperl 1.2x built using a perl 5.8 release candidate
please test this out:
Perl
sub Apache::Request::DESTROY{warn DEAD: $_[0]\n}
sub Apache::DESTROY{warn Dead: $_[0]\n}
use Devel::Peek;
use Apache:Request;
package Apache::test;
sub handler {
my $r = shift;
my $apr = Apache::Request-new($r);
Dump($apr); # OK
$r = $apr; # XXX: what's going on here???
Dump($apr); # fscked
$r-send_header;
$r-print('apr test');
return 200;
}
1;
/Perl
Location /test
SetHandler perl-script
PerlHandler Apache::test
/Location
My error log for a request to /test is below. The above
assignment on the line marked XXX is modifying $apr, which
it shouldn't.
I don't have a 5.8-RC handy to test this on, but if somebody
else sees the same problem on 5.8, I'll file a bug report to p5p.
Thanks alot.
--
SV = RV(0x814b154) at 0x82309b8
REFCNT = 1
FLAGS = (ROK)
RV = 0x82472f4
SV = PVMG(0x8244f58) at 0x82472f4
REFCNT = 1
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 136639644
NV = 0
PV = 0
MAGIC = 0x8244f80
MG_VIRTUAL = 0
MG_TYPE = '~'
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x822dec4
SV = RV(0x814b14c) at 0x822dec4
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x82472dc
SV = PVMG(0x8244f30) at 0x82472dc
REFCNT = 2
FLAGS = (OBJECT,IOK,pIOK)
IV = 136628428
NV = 0
PV = 0
STASH = 0x81420e4 Apache
MG_LEN = -1
MG_PTR = 0x824c8cc - please notify IZ
STASH = 0x8224a18 Apache::Request
SV = RV(0x814b154) at 0x82309b8
REFCNT = 1
FLAGS = (ROK)
RV = 0x82472f4
SV = PVMG(0x8244f58) at 0x82472f4
REFCNT = 2
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 136639644
NV = 0
PV = 0
MAGIC = 0x8244f80
MG_VIRTUAL = 0
MG_TYPE = '~'
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x822dec4
SV = RV(0x814b14c) at 0x822dec4
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x82472f4
SV = PVMG(0x8244f58) at 0x82472f4
REFCNT = 2
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 136639644
NV = 0
PV = 0
MAGIC = 0x8244f80
MG_VIRTUAL = 0
MG_TYPE = '~'
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x822dec4
SV = RV(0x814b14c) at 0x822dec4
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x82472f4
MG_LEN = -1
MG_PTR = 0x824c8cc - please notify IZ
STASH = 0x8224a18 Apache::Request
MG_LEN = -1
MG_PTR = 0x824c8cc - please notify IZ
STASH = 0x8224a18 Apache::Request
Dead: Apache=SCALAR(0x82472dc)
--
--
Joe Schaefer
Re: leaks with Apache::Request?
Joe Schaefer wrote: Somehow the assignment operator MUST be involved in the leak here. (You only get a leak when the *same* reference (*SV) is on both sides of the assignment). Could someone with modperl 1.2x built using a perl 5.8 release candidate please test this out: I got the same behavior (Apache/1.3.26 (Unix) mod_perl/1.27 perl@17440) with a modified version of your code (it doesn't work as is.) HTH.
Re: leaks with Apache::Request?
Does anyone know what's causing the Apache::Request object to
leak here? See # XXX comment below:
Perl
package Apache::test;
sub Apache::Request::DESTROY{warn DEAD: $_[0]\n}
sub Apache::DESTROY{warn Dead: $_[0]\n}
use Devel::Peek;
use Apache::Request;
sub handler {
my $r = shift;
my $apr = Apache::Request-new($r);
Dump($apr); # OK
$r = $apr; # XXX: what's going on here???
Dump($apr); # fscked
$r-send_http_header;
$r-print('apr test');
return 200;
}
1;
/Perl
Location /test
SetHandler perl-script
PerlHandler Apache::test
/Location
-- server error log for request to /test --
[Wed Jul 10 07:38:32 2002] [notice] Apache/1.3.27-dev (Unix)
mod_perl/1.27_01-dev Perl/v5.8.0 configured
[...]
SV = RV(0x87aca6c) at 0x89902b8
REFCNT = 1
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
SV = PVMG(0x8703130) at 0x8626da4
REFCNT = 1
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 148456972
NV = 0
PV = 0
MAGIC = 0x8d7b2c8
MG_VIRTUAL = 0
MG_TYPE = PERL_MAGIC_ext(~)
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x89975dc
SV = RV(0x87aca64) at 0x89975dc
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626c78
SV = PVMG(0x8703110) at 0x8626c78
REFCNT = 2
FLAGS = (OBJECT,IOK,pIOK)
IV = 148451868
NV = 0
PV = 0
STASH = 0x8298510 Apache
MG_LEN = -1
MG_PTR = 0x8d9321c - please notify IZ
STASH = 0x835f8ec Apache::Request
SV = RV(0x87aca6c) at 0x89902b8
REFCNT = 1
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
SV = PVMG(0x8703130) at 0x8626da4
REFCNT = 2
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 148456972
NV = 0
PV = 0
MAGIC = 0x8d7b2c8
MG_VIRTUAL = 0
MG_TYPE = PERL_MAGIC_ext(~)
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x89975dc
SV = RV(0x87aca64) at 0x89975dc
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
SV = PVMG(0x8703130) at 0x8626da4
REFCNT = 2
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 148456972
NV = 0
PV = 0
MAGIC = 0x8d7b2c8
MG_VIRTUAL = 0
MG_TYPE = PERL_MAGIC_ext(~)
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x89975dc
SV = RV(0x87aca64) at 0x89975dc
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
MG_LEN = -1
MG_PTR = 0x8d9321c - please notify IZ
STASH = 0x835f8ec Apache::Request
MG_LEN = -1
MG_PTR = 0x8d9321c - please notify IZ
STASH = 0x835f8ec Apache::Request
Dead: Apache=SCALAR(0x8626c78)
Re: leaks with Apache::Request?
[resent to modperl list; earlier copy mistakenly cc'd to p5p]
Does anyone know what's causing the Apache::Request object to
leak here? See # XXX comment below:
Perl
package Apache::test;
sub Apache::Request::DESTROY{warn DEAD: $_[0]\n}
sub Apache::DESTROY{warn Dead: $_[0]\n}
use Devel::Peek;
use Apache::Request;
sub handler {
my $r = shift;
my $apr = Apache::Request-new($r);
Dump($apr); # OK
$r = $apr; # XXX: what's going on here???
Dump($apr); # fscked
$r-send_http_header;
$r-print('apr test');
return 200;
}
1;
/Perl
Location /test
SetHandler perl-script
PerlHandler Apache::test
/Location
-- server error log for request to /test --
[Wed Jul 10 07:38:32 2002] [notice] Apache/1.3.27-dev (Unix)
mod_perl/1.27_01-dev Perl/v5.8.0 configured
[...]
SV = RV(0x87aca6c) at 0x89902b8
REFCNT = 1
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
SV = PVMG(0x8703130) at 0x8626da4
REFCNT = 1
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 148456972
NV = 0
PV = 0
MAGIC = 0x8d7b2c8
MG_VIRTUAL = 0
MG_TYPE = PERL_MAGIC_ext(~)
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x89975dc
SV = RV(0x87aca64) at 0x89975dc
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626c78
SV = PVMG(0x8703110) at 0x8626c78
REFCNT = 2
FLAGS = (OBJECT,IOK,pIOK)
IV = 148451868
NV = 0
PV = 0
STASH = 0x8298510 Apache
MG_LEN = -1
MG_PTR = 0x8d9321c - please notify IZ
STASH = 0x835f8ec Apache::Request
SV = RV(0x87aca6c) at 0x89902b8
REFCNT = 1
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
SV = PVMG(0x8703130) at 0x8626da4
REFCNT = 2
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 148456972
NV = 0
PV = 0
MAGIC = 0x8d7b2c8
MG_VIRTUAL = 0
MG_TYPE = PERL_MAGIC_ext(~)
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x89975dc
SV = RV(0x87aca64) at 0x89975dc
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
SV = PVMG(0x8703130) at 0x8626da4
REFCNT = 2
FLAGS = (OBJECT,RMG,IOK,pIOK)
IV = 148456972
NV = 0
PV = 0
MAGIC = 0x8d7b2c8
MG_VIRTUAL = 0
MG_TYPE = PERL_MAGIC_ext(~)
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0x89975dc
SV = RV(0x87aca64) at 0x89975dc
REFCNT = 2
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x8626da4
MG_LEN = -1
MG_PTR = 0x8d9321c - please notify IZ
STASH = 0x835f8ec Apache::Request
MG_LEN = -1
MG_PTR = 0x8d9321c - please notify IZ
STASH = 0x835f8ec Apache::Request
Dead: Apache=SCALAR(0x8626c78)
Re: leaks with Apache::Request?
On 8 Jul 2002, Joe Schaefer wrote: Write that like this, and I think your leak will disappear: my $r = Apache::Request-new( shift ); AFAICT, Apache::Request::new is NOT leaking here, since the REFCNT of its returned object IS 1. There might be some magic-related bug in perl that causes the assignment to bump $r's refcount to 2. This MIGHT be circumventable with some better code in Request.xs, but I really don't know how to fix it. Until some perl guru enlightens us, as a personal rule I try hard to avoid expressions like $foo = make_something_out_of($foo); I realize that this isn't always possible, but it often/usually is. Such advice would serve you well in this case; you could even get away with this my $r = shift; my $apr = Apache::Request-new($r); That's not going to leak, either. At least I hope not :-) I ended up using something like this and the leak went away. It seems to me that this might actually be a Perl bug. If I do this: my $x = shift; $x = make_something_from($x); then it seems like the original $x should go out of scope when it is assigned to, so its refcount should stay at 1. -dave /*== www.urth.org we await the New Sun ==*/
Re: leaks with Apache::Request?
Dave Rolsky [EMAIL PROTECTED] writes: On 8 Jul 2002, Joe Schaefer wrote: [...] my $r = shift; my $apr = Apache::Request-new($r); That's not going to leak, either. At least I hope not :-) I ended up using something like this and the leak went away. It seems to me that this might actually be a Perl bug. I doubt it's solely perl's fault. The problem is that you have to be an internals wizard to write safe XS; otherwise you just keep your fingers crossed. :-) Hopefully some generous soul will post a patch to Request.xs that prevents this problem from ever arising. If I do this: my $x = shift; $x = make_something_from($x); then it seems like the original $x should go out of scope when it is assigned to, so its refcount should stay at 1. ^^ Right, it should stay at 1. But all bets are off when $x is has magic and make_something_from() is an XS sub :-). -- Joe Schaefer
Re: leaks with Apache::Request?
* Joe Schaefer [EMAIL PROTECTED] [2002-07-09 12:47]: Dave Rolsky [EMAIL PROTECTED] writes: On 8 Jul 2002, Joe Schaefer wrote: If I do this: my $x = shift; $x = make_something_from($x); then it seems like the original $x should go out of scope when it is assigned to, so its refcount should stay at 1. ^^ Right, it should stay at 1. But all bets are off when $x is has magic and make_something_from() is an XS sub :-). But the leak is not with $x, it's with what $_[0] is; $x is just a reference to that, and the reassignment in the second line should reassign $x, and decrement the ref count for what $x is pointing to at that point. So, it all depends on what make_something_from() does with the $x's referent. (darren) -- Great minds discuss ideas. Average minds discuss events. Small minds discuss people. -- Admiral Hyman G. Rickover
Re: leaks with Apache::Request?
darren chamberlain [EMAIL PROTECTED] writes: * Joe Schaefer [EMAIL PROTECTED] [2002-07-09 12:47]: Dave Rolsky [EMAIL PROTECTED] writes: On 8 Jul 2002, Joe Schaefer wrote: If I do this: my $x = shift; $x = make_something_from($x); then it seems like the original $x should go out of scope when it is assigned to, so its refcount should stay at 1. ^^ Right, it should stay at 1. But all bets are off when $x is has magic and make_something_from() is an XS sub :-). But the leak is not with $x, it's with what $_[0] is; $x is just a reference to that, and the reassignment in the second line should reassign $x, and decrement the ref count for what $x is pointing to at that point. So, it all depends on what make_something_from() does with the $x's referent. I don't think it's as simple as that- THIS LEAKS: my $r = shift; $r = Apache::Request-new($r); my $r = shift; $r = Apache::Request-new($_) for $r; DOES NOT LEAK: my $apr = Apache::Request-new(shift); my $r = shift; my $apr = $r; # $r and $apr have the same referent now right? $apr = Apache::Request-new($r); my $r = shift; my $apr = Apache::Request-new($r); my $r = shift; $r = Apache::Request-new($_) for map $_, $r; Somehow the assignment operator MUST be involved in the leak here. (You only get a leak when the *same* reference (*SV) is on both sides of the assignment). -- Joe Schaefer
Re: leaks with Apache::Request?
Dave,
Perhaps this is why from the eagle book,
During the child exit phase, mod_perl invokes the Perl API function
perl_destruct( ) to run the contents of END blocks and to invoke the
DESTROY method for any global objects that have not gone out of scope
already.
So I think i'm right in saying that, When I run this, the DESTROY
method is not called until the server shuts down. is perfectly normal
behaviour. I don't know how you are actually testing your memory
usage, but I might suggest that if you are sending larger amounts of
data than previously, just once per 5000 requests then this memory is
gonna be consumed by apache forever (at least until httpd is killed).
Richard
Dave Rolsky wrote:
It looks like there may be a memory leak with Apache::Request. I'm using
version 1.0 with Perl 5.6.1, mod_perl 1.26, and Apache 1.3.26. mod_perl
is statically compiled into Apache.
Here's some code that I think demonstrates the leak:
package My::APRTest;
use strict;
use Apache::Request;
sub Apache::Request::DESTROY{warn DEAD: $_[0]\n}
sub handler
{
my $r = shift;
$r = Apache::Request-new($r);
$r-send_http_header;
$r-print('apr test');
return 200;
}
1;
When I run this, the DESTROY method is not called until the server shuts
down.
Watching memory using with top (against a server running as httpd -X) I
can see that memory usage is growing a little less the 500K every 5000
requests.
This isn't catastrophic but fixing it would be a good thing.
-dave
/*==
www.urth.org
we await the New Sun
==*/
Re: leaks with Apache::Request?
On Mon, 8 Jul 2002, Richard Clarke wrote: During the child exit phase, mod_perl invokes the Perl API function perl_destruct( ) to run the contents of END blocks and to invoke the DESTROY method for any global objects that have not gone out of scope already. Notice where it says ... for any global objects that have not gone out of scope already. In the code I posted, $r goes out of scope at the end of the handler subroutine. So I think i'm right in saying that, When I run this, the DESTROY method is not called until the server shuts down. is perfectly normal behaviour. I don't know how you are actually testing your memory Well, creating an Apache::DESTROY method works and shows the Apache object going out of scope on a per-request basis. usage, but I might suggest that if you are sending larger amounts of data than previously, just once per 5000 requests then this memory is gonna be consumed by apache forever (at least until httpd is killed). I tested with ab sending the same request thousands of times in a row. -dave /*== www.urth.org we await the New Sun ==*/
Re: leaks with Apache::Request?
Dave Rolsky [EMAIL PROTECTED] writes:
[...]
Here's some code that I think demonstrates the leak:
package My::APRTest;
use strict;
use Apache::Request;
sub Apache::Request::DESTROY{warn DEAD: $_[0]\n}
sub handler
{
my $r = shift;
$r = Apache::Request-new($r);
^
Write that like this, and I think your leak will
disappear:
my $r = Apache::Request-new( shift );
AFAICT, Apache::Request::new is NOT leaking here, since the
REFCNT of its returned object IS 1. There might be some
magic-related bug in perl that causes the assignment to bump
$r's refcount to 2. This MIGHT be circumventable with some better
code in Request.xs, but I really don't know how to fix it.
Until some perl guru enlightens us, as a personal rule I
try hard to avoid expressions like
$foo = make_something_out_of($foo);
I realize that this isn't always possible, but it often/usually
is. Such advice would serve you well in this case; you could
even get away with this
my $r = shift;
my $apr = Apache::Request-new($r);
That's not going to leak, either. At least I hope not :-)
HTH
--
Joe Schaefer
