On Thu, 21 Nov 2002, Stas Bekman wrote:
> Esteban Fernandez Stafford wrote:
> >
> > Hello,
> >
> > I am currently developing a modperl filter that uses the streaming
> > approach. I started off with the example in
> >
> >
>http://perl.apache.org/docs/2.0/user/handlers/filters.html#Stream_oriented_Output_Filter
> >
> > sub handler {
> > my $filter = shift;
> >
> > my $left_over = '';
> > while ($filter->read(my $buffer, BUFF_LEN)) {
> > $buffer = $left_over . $buffer;
> > $left_over = '';
> > while ($buffer =~ /([^\r\n]*)([\r\n]*)/g) {
> > $left_over = $1, last unless $2;
> > $filter->print(scalar(reverse $1), $2);
> > }
> > }
> > $filter->print(scalar reverse $left_over) if length $left_over;
> >
> > Apache::OK;
> > }
> >
> > This seems to work OK when the file is small (smaller than 8192). When
> > the file is larger, then there is a line that gets cut. This is
> > because the handler gets called more than once for big requests. Is
> > there a nice way to overcome this problem? I was thinking using
> > filter context to store variable $left_over, but then I dont know how
> > to detect the "real" end of the stream.
>
> The problem is that I've written a patch that makes the $filter->print()
> outside the while() loop work, but it was never committed. So the doc
> is out of sync with the core code. For now please apply this patch:
> http://marc.theaimsgroup.com/?l=apache-modperl-dev&m=102828686110352&w=2
>
> I'll see that it gets into the core asap.
>
Thanks Stas,
the patch you sent me solved some of my trouble. The bad news is that I
still have problems when I filter large files. I have written an example
to expose this behaviour. It consists of modified versions of the beloved
SendAlphaNum.pm and FilterReverse1.pm.
----------------------------------------------------
package MyApache::SendAlphaNum;
use strict;
use warnings;
use Apache::RequestRec ();
use Apache::RequestIO ();
use Apache::Const -compile => qw(OK);
sub handler {
my $r = shift;
$r->content_type('text/plain');
foreach(1..300)
{
$r->print(1..9, "0", 'a'..'z', "\n");
}
Apache::OK;
}
1;
----------------------------------------------------
package MyApache::FilterReverse1;
use strict;
use warnings;
use base qw(Apache::Filter);
use Apache::Const -compile => qw(OK);
use constant BUFF_LEN => 1024;
my $count = 0;
sub handler
{
my $filter = shift;
$filter->print("\nI'm Starting: $count bytes so far...\n");
my $left_over = '';
while ($filter->read(my $buffer, BUFF_LEN)) {
$count+=length($buffer);
$buffer = $left_over . $buffer;
$left_over = '';
while ($buffer =~ /([^\r\n]*)([\r\n]*)/g) {
$left_over = $1, last unless $2;
$filter->print(scalar(reverse $1), $2);
}
}
$filter->print(scalar reverse $left_over) if length $left_over;
Apache::OK;
}
1;
----------------------------------------------------
PerlModule MyApache::FilterReverse1
PerlModule MyApache::SendAlphaNum
<Location /reverse1>
SetHandler modperl
PerlResponseHandler MyApache::SendAlphaNum
PerlOutputFilterHandler MyApache::FilterReverse1
</Location>
----------------------------------------------------
As an output I get the following:
$ wget -q -O - http://fangorn:3000/reverse1
I'm Starting: 0 bytes so far...
zyxwvutsrqponmlkjihgfedcba0987654321
zyxwvutsrqponmlkjihgfedcba0987654321
# A lot of these...
zyxwvutsrqponmlkjihgfedcba0987654321
zyxwvutsrqponmlkjihgfedcba0987654321
edcba0987654321
I'm Starting: 8192 bytes so far...
zyxwvutsrqponmlkjihgf
zyxwvutsrqponmlkjihgfedcba0987654321
zyxwvutsrqponmlkjihgfedcba0987654321
# ... and some more...
zyxwvutsrqponmlkjihgfedcba0987654321
zyxwvutsrqponmlkjihgfedcba0987654321
As you can see the filter is called twice and therefore there is one line
that gets broken in two. I could always store $left_over in the context
of the filter and prepend it to what I read when the filter starts again.
But the problem I find is that I have no way to know when the "real" eos
is reached in order to flush $left_over to the output. I thought there may
be some sort of $filter->eos() call or something.
E s t e b a n!
:wq
