Re: mod_perl memory
On Mar 17, 2010, at 11:27 AM, Torsten Förtsch wrote:
> On Wednesday 17 March 2010 12:15:15 Torsten Förtsch wrote:
>> On Tuesday 16 March 2010 21:09:33 Pavel Georgiev wrote:
>>> for () {
>>>$request->print("--$this->{boundary}\n");
>>>$request->print("Content-type: text/html; charset=utf-8;\n\n");
>>>$request->print("$data\n\n");
>>>$request->rflush;
>>> }
>>>
>>> And the result is endless memory growth in the apache process. Is that
>>> what you had in mind?
>>>
>>
>> I can confirm this. I have tried this little handler:
>>
>> sub {
>> my $r=shift;
>>
>> until( -e "/tmp/stop" ) {
>>$r->print(("x"x70)."\n");
>>$r->rflush;
>> }
>>
>> return Apache2::Const::OK;
>> }
>>
>> The httpd process grows slowly but unlimited. Without the rflush() it
>> grows slower but still does.
>>
> Here is a bit more stuff on the bug. It is the pool that grows.
>
> To show it I use a handler that prints an empty document. I think an empty
> file shipped by the default handler will do as well.
>
> Then I add the following filter to the request:
>
> $r->add_output_filter(sub {
> my ($f, $bb)=...@_;
>
> unless( $f->ctx ) {
>$f->r->headers_out->unset('Content-Length');
>$f->ctx(1);
> }
>
> my $eos=0;
> while( my $b=$bb->first ) {
>$eos++ if( $b->is_eos );
>$b->delete;
> }
> return 0 unless $eos;
>
> my $ba=$f->c->bucket_alloc;
> until( -e '/tmp/stop' ) {
>my $bb2=APR::Brigade->new($f->c->pool, $ba);
>$bb2->insert_tail(APR::Bucket->new($ba, ("x"x70)."\n"));
>$bb2->insert_tail(APR::Bucket::flush_create $ba);
>$f->next->pass_brigade($bb2);
> }
>
> my $bb2=APR::Brigade->new($f->c->pool, $ba);
> $bb2->insert_tail(APR::Bucket::eos_create $ba);
> $f->next->pass_brigade($bb2);
>
> return 0;
> });
>
> The filter drops the empty document and emulates our infinite output. With
> this filter the httpd process still grows. Now I add a subpool to the loop:
>
> [...]
> until( -e '/tmp/stop' ) {
>my $pool=$f->c->pool->new; # create a subpool
>my $bb2=APR::Brigade->new($pool, $ba); # use the subpool
>$bb2->insert_tail(APR::Bucket->new($ba, ("x"x70)."\n"));
>$bb2->insert_tail(APR::Bucket::flush_create $ba);
>$f->next->pass_brigade($bb2);
>$pool->destroy; # and destroy it
> }
> [...]
>
> Now it does not grow.
>
> Torsten Förtsch
>
> --
> Need professional modperl support? Hire me! (http://foertsch.name)
>
> Like fantasy? http://kabatinte.net
How would that logic (adding subpools and using them) be applied to my
simplified example:
for (;;) {
$request->print("--$this->{boundary}\n");
$request->print("Content-type: text/html; charset=utf-8;\n\n");
$request->print("$data\n\n");
$request->rflush;
}
Do I need to add an output filter?
Re: mod_perl memory
On Wednesday 17 March 2010 12:15:15 Torsten Förtsch wrote:
> On Tuesday 16 March 2010 21:09:33 Pavel Georgiev wrote:
> > for () {
> > $request->print("--$this->{boundary}\n");
> > $request->print("Content-type: text/html; charset=utf-8;\n\n");
> > $request->print("$data\n\n");
> > $request->rflush;
> > }
> >
> > And the result is endless memory growth in the apache process. Is that
> > what you had in mind?
> >
>
> I can confirm this. I have tried this little handler:
>
> sub {
> my $r=shift;
>
> until( -e "/tmp/stop" ) {
> $r->print(("x"x70)."\n");
> $r->rflush;
> }
>
> return Apache2::Const::OK;
> }
>
> The httpd process grows slowly but unlimited. Without the rflush() it
> grows slower but still does.
>
Here is a bit more stuff on the bug. It is the pool that grows.
To show it I use a handler that prints an empty document. I think an empty
file shipped by the default handler will do as well.
Then I add the following filter to the request:
$r->add_output_filter(sub {
my ($f, $bb)=...@_;
unless( $f->ctx ) {
$f->r->headers_out->unset('Content-Length');
$f->ctx(1);
}
my $eos=0;
while( my $b=$bb->first ) {
$eos++ if( $b->is_eos );
$b->delete;
}
return 0 unless $eos;
my $ba=$f->c->bucket_alloc;
until( -e '/tmp/stop' ) {
my $bb2=APR::Brigade->new($f->c->pool, $ba);
$bb2->insert_tail(APR::Bucket->new($ba, ("x"x70)."\n"));
$bb2->insert_tail(APR::Bucket::flush_create $ba);
$f->next->pass_brigade($bb2);
}
my $bb2=APR::Brigade->new($f->c->pool, $ba);
$bb2->insert_tail(APR::Bucket::eos_create $ba);
$f->next->pass_brigade($bb2);
return 0;
});
The filter drops the empty document and emulates our infinite output. With
this filter the httpd process still grows. Now I add a subpool to the loop:
[...]
until( -e '/tmp/stop' ) {
my $pool=$f->c->pool->new; # create a subpool
my $bb2=APR::Brigade->new($pool, $ba); # use the subpool
$bb2->insert_tail(APR::Bucket->new($ba, ("x"x70)."\n"));
$bb2->insert_tail(APR::Bucket::flush_create $ba);
$f->next->pass_brigade($bb2);
$pool->destroy; # and destroy it
}
[...]
Now it does not grow.
Torsten Förtsch
--
Need professional modperl support? Hire me! (http://foertsch.name)
Like fantasy? http://kabatinte.net
Re: mod_perl memory
2010/3/17 Torsten Förtsch : > The httpd process grows slowly but unlimited. Without the rflush() it grows > slower but still does. > > With the rflush() its size increased by 100MB for an output of 220MB. > Without it it grew by 10MB for an output of 2.3GB. > > I'd say it's a bug. I agree. This should not cause ongoing process growth. The data must be accumulating somewhere. - Perrin
Re: mod_perl memory
On 03/17/2010 05:15 AM, Torsten Förtsch wrote:
until( -e "/tmp/stop" ) {
$r->print(("x"x70)."\n");
$r->rflush;
}
Just for the record:
With mp1 there isn't any mem leak with or without rflush.
(After 10 mins: output 109GB. Fedora's 12 stock perl 5.10.0, apache
1.3.42, mod_perl 1.31)
Maybe is mp2's filters related.
Regards.
Salvador Ortiz.
Re: mod_perl memory
On Tuesday 16 March 2010 21:09:33 Pavel Georgiev wrote:
> for () {
> $request->print("--$this->{boundary}\n");
> $request->print("Content-type: text/html; charset=utf-8;\n\n");
> $request->print("$data\n\n");
> $request->rflush;
> }
>
> And the result is endless memory growth in the apache process. Is that what
> you had in mind?
>
I can confirm this. I have tried this little handler:
sub {
my $r=shift;
until( -e "/tmp/stop" ) {
$r->print(("x"x70)."\n");
$r->rflush;
}
return Apache2::Const::OK;
}
The httpd process grows slowly but unlimited. Without the rflush() it grows
slower but still does.
With the rflush() its size increased by 100MB for an output of 220MB.
Without it it grew by 10MB for an output of 2.3GB.
I'd say it's a bug.
Torsten Förtsch
--
Need professional modperl support? Hire me! (http://foertsch.name)
Like fantasy? http://kabatinte.net
