Re: High CPU utilization on RHEL5.6/CentOS5.6

[Christopher Stanton]

Which benchmark module would work best with mod_perl, Perl 5.8 and Perl 5.12?

On Wed, Jul 27, 2011 at 7:22 AM, Dave Hodgkinson  wrote:
>
> On 26 Jul 2011, at 18:58, Christopher Stanton wrote:
>
>> I will see if I can get Perl 5.12 and an associated mod_perl up and
>> running on the problem machine and see if the CPU utilization goes
>> down.
>
> Is benchmarking out of the question?
>
>

Re: High CPU utilization on RHEL5.6/CentOS5.6

[Christopher Stanton]

I will see if I can get Perl 5.12 and an associated mod_perl up and
running on the problem machine and see if the CPU utilization goes
down.

Long long ago in a install far far away I did not run into this issue
on FC4 which has very similar packages other than apache (2.0 rather
than 2.2):
Requires: httpd >= 2.0.54-0; perl >= 5.8.6-0; perl-Digest-SHA1 >=
2.10-0; mod_perl >= 2.0.1-0;

On Tue, Jul 26, 2011 at 12:43 PM, Perrin Harkins  wrote:
> On Mon, Jul 25, 2011 at 6:43 PM, Christopher Stanton
>  wrote:
>> I am seeing high cpu utilization under RHEL5.6 which I don't see on
>> FC14. We are talking sub 10% on FC14 vs 80% on EL5. This is on
>> different hardware, but not that different.
>
> Well, you've changed hardware and lots of software versions, so there
> are many things it could be.  The only one that seems unlikely is your
> code, since that didn't change.
>
> I would suspect the major downgrade in Perl as the culprit.  You went
> from 5.12 to 5.8.  To test this, I suggest you install 5.12 on your
> new system and compile mod_perl against it.  If you don't want to
> touch the system mod_perl and apache you can install your own in a
> different location.  I use perlbrew to try out new Perl versions
> locally.
>
> - Perrin
>

Re: High CPU utilization on RHEL5.6/CentOS5.6

[Christopher Stanton]

Before I start playing mix-and-match with kernels, I would prefer some
feedback on mod_perl and the script's code.

It doesn't sleep in the main loop, but it should block on
my @ready = $select->can_read(1);
and not block on
if ( $sock->read(my $data, POSIX::BUFSIZ*10) ){

since the socket is set with Blocking => 0.

A JPEG (in a 25fps stream) arrives every 40,000 microseconds.

Christopher


On Mon, Jul 25, 2011 at 6:43 PM, Fred Moyer  wrote:
> On Mon, Jul 25, 2011 at 4:02 PM, Christopher Stanton
>  wrote:
>> Neither install is running within VMWare.
>
> Might be worthwhile trying a different rate though, or going tickless.
>  Perl 5.8.8 isn't that much (if at all) slower than 5.12.  Looks like
> the FC14 kernel is tickless:
>
> http://docs.fedoraproject.org/en-US/Fedora/14/pdf/Power_Management_Guide/Fedora-14-Power_Management_Guide-en-US.pdf
>
>
>> On Mon, Jul 25, 2011 at 5:55 PM, Fred Moyer  wrote:
>>> This may not be related, but when I was working with Centos 5 in a
>>> Vmware environment, I ran into an issue of high cpu since the default
>>> clock rate with Linux 2.6 is 1000Hz.  I changed to a 100Hz clock rate
>>> and got much lower cpu usage.
>>>
>>> I'm not sure if that is your issue, perhaps FC14 was using less than a
>>> 1000Hz clock rate.  You might try a tickless kernel, or dig further
>>> into what clock rate your kernel is using.
>>>
>>> I don't know where the authoritative source for this information is,
>>> but here's what I found with a search:
>>>
>>> http://yate.null.ro/pmwiki/index.php?n=Main.YateAndVMWare
>>>
>>> On Mon, Jul 25, 2011 at 3:43 PM, Christopher Stanton
>>>  wrote:
>>>> I have a MJPEG streaming system which uses mod_perl in the web
>>>> interface to supply the final stream to the client.
>>>>
>>>> I am seeing high cpu utilization under RHEL5.6 which I don't see on
>>>> FC14. We are talking sub 10% on FC14 vs 80% on EL5. This is on
>>>> different hardware, but not that different. And on EL5 the httpd
>>>> session is way higher (cpu wise) than the supplying proxy. On FC14 on
>>>> my test rig I see (for a 25fps stream) 15-18% CPU for the proxy which
>>>> can handle multiplexing etc and 5.5-8% for the mod_perl httpd session.
>>>>
>>>> I think this is an issue with that platform, but I haven't seen
>>>> any/many reports. Does anyone have a sec to look at the source? I
>>>> don't think I have implemented this is a horrible inefficient way but
>>>> who knows. It is true, I could have byte markers and then know the
>>>> size of the JPEG I am expecting rather than just splitting on the
>>>> boundary field, but in tests (and production) it seemed to work ok.
>>>>
>>>> http://svn.codaxus.com/flexTPS/2.x.x/trunk/portal/2.4.x/site/perl/nph-mjpeg_stream.pl
>>>>
>>>> Any comments or critiques would be appreciated,
>>>> Christopher
>>>>
>>>> --
>>>>
>>>> On FC14:
>>>> Requires: httpd >= 2.2.17-1
>>>> Requires: mod_perl >= 2.0.4-11
>>>> Requires: mod_ssl >= 1:2.2.17-1
>>>> Requires: perl >= 5.12.3-141
>>>> Requires: perl-CGI >= 3.51-1
>>>> Requires: perl-Digest-SHA1 >= 2.12-4
>>>> Requires: perl-XML-Filter-BufferText >= 1.01-9
>>>> Requires: perl-XML-Simple >= 2.18-7
>>>> Requires: perl-XML-Validator-Schema >= 1.10-5
>>>> Requires: perl-Tree-DAG_Node >= 1.06-8
>>>> Requires: perl-LDAP >= 0.40-2
>>>> Requires: perl-Crypt-SSLeay >= 0.58-1
>>>> Requires: ffmpeg >= 0.6-4
>>>>
>>>> On EL5:
>>>> Requires: httpd >= 2.2.3-45
>>>> Requires: mod_perl >= 2.0.4-6
>>>> Requires: mod_ssl >= 2.2.3-45
>>>> Requires: perl >= 5.8.8-32
>>>> Requires: perl-Digest-SHA1 >= 2.11-1
>>>> Requires: perl-XML-Filter-BufferText >= 1.01
>>>> Requires: perl-XML-Simple >= 2.14-4
>>>> Requires: perl-XML-Validator-Schema >= 1.10-1
>>>> Requires: perl-Tree-DAG_Node >= 1.06
>>>> Requires: perl-LDAP >= 0.33-3
>>>> Requires: perl-Crypt-SSLeay >= 0.51-11
>>>> Requires: ffmpeg >= 0.6.1-1
>>>>
>>>
>>
>

Re: High CPU utilization on RHEL5.6/CentOS5.6

[Christopher Stanton]

Neither install is running within VMWare.

On Mon, Jul 25, 2011 at 5:55 PM, Fred Moyer  wrote:
> This may not be related, but when I was working with Centos 5 in a
> Vmware environment, I ran into an issue of high cpu since the default
> clock rate with Linux 2.6 is 1000Hz.  I changed to a 100Hz clock rate
> and got much lower cpu usage.
>
> I'm not sure if that is your issue, perhaps FC14 was using less than a
> 1000Hz clock rate.  You might try a tickless kernel, or dig further
> into what clock rate your kernel is using.
>
> I don't know where the authoritative source for this information is,
> but here's what I found with a search:
>
> http://yate.null.ro/pmwiki/index.php?n=Main.YateAndVMWare
>
> On Mon, Jul 25, 2011 at 3:43 PM, Christopher Stanton
>  wrote:
>> I have a MJPEG streaming system which uses mod_perl in the web
>> interface to supply the final stream to the client.
>>
>> I am seeing high cpu utilization under RHEL5.6 which I don't see on
>> FC14. We are talking sub 10% on FC14 vs 80% on EL5. This is on
>> different hardware, but not that different. And on EL5 the httpd
>> session is way higher (cpu wise) than the supplying proxy. On FC14 on
>> my test rig I see (for a 25fps stream) 15-18% CPU for the proxy which
>> can handle multiplexing etc and 5.5-8% for the mod_perl httpd session.
>>
>> I think this is an issue with that platform, but I haven't seen
>> any/many reports. Does anyone have a sec to look at the source? I
>> don't think I have implemented this is a horrible inefficient way but
>> who knows. It is true, I could have byte markers and then know the
>> size of the JPEG I am expecting rather than just splitting on the
>> boundary field, but in tests (and production) it seemed to work ok.
>>
>> http://svn.codaxus.com/flexTPS/2.x.x/trunk/portal/2.4.x/site/perl/nph-mjpeg_stream.pl
>>
>> Any comments or critiques would be appreciated,
>> Christopher
>>
>> --
>>
>> On FC14:
>> Requires: httpd >= 2.2.17-1
>> Requires: mod_perl >= 2.0.4-11
>> Requires: mod_ssl >= 1:2.2.17-1
>> Requires: perl >= 5.12.3-141
>> Requires: perl-CGI >= 3.51-1
>> Requires: perl-Digest-SHA1 >= 2.12-4
>> Requires: perl-XML-Filter-BufferText >= 1.01-9
>> Requires: perl-XML-Simple >= 2.18-7
>> Requires: perl-XML-Validator-Schema >= 1.10-5
>> Requires: perl-Tree-DAG_Node >= 1.06-8
>> Requires: perl-LDAP >= 0.40-2
>> Requires: perl-Crypt-SSLeay >= 0.58-1
>> Requires: ffmpeg >= 0.6-4
>>
>> On EL5:
>> Requires: httpd >= 2.2.3-45
>> Requires: mod_perl >= 2.0.4-6
>> Requires: mod_ssl >= 2.2.3-45
>> Requires: perl >= 5.8.8-32
>> Requires: perl-Digest-SHA1 >= 2.11-1
>> Requires: perl-XML-Filter-BufferText >= 1.01
>> Requires: perl-XML-Simple >= 2.14-4
>> Requires: perl-XML-Validator-Schema >= 1.10-1
>> Requires: perl-Tree-DAG_Node >= 1.06
>> Requires: perl-LDAP >= 0.33-3
>> Requires: perl-Crypt-SSLeay >= 0.51-11
>> Requires: ffmpeg >= 0.6.1-1
>>
>

High CPU utilization on RHEL5.6/CentOS5.6

[Christopher Stanton]

I have a MJPEG streaming system which uses mod_perl in the web
interface to supply the final stream to the client.

I am seeing high cpu utilization under RHEL5.6 which I don't see on
FC14. We are talking sub 10% on FC14 vs 80% on EL5. This is on
different hardware, but not that different. And on EL5 the httpd
session is way higher (cpu wise) than the supplying proxy. On FC14 on
my test rig I see (for a 25fps stream) 15-18% CPU for the proxy which
can handle multiplexing etc and 5.5-8% for the mod_perl httpd session.

I think this is an issue with that platform, but I haven't seen
any/many reports. Does anyone have a sec to look at the source? I
don't think I have implemented this is a horrible inefficient way but
who knows. It is true, I could have byte markers and then know the
size of the JPEG I am expecting rather than just splitting on the
boundary field, but in tests (and production) it seemed to work ok.

http://svn.codaxus.com/flexTPS/2.x.x/trunk/portal/2.4.x/site/perl/nph-mjpeg_stream.pl

Any comments or critiques would be appreciated,
Christopher

--

On FC14:
Requires: httpd >= 2.2.17-1
Requires: mod_perl >= 2.0.4-11
Requires: mod_ssl >= 1:2.2.17-1
Requires: perl >= 5.12.3-141
Requires: perl-CGI >= 3.51-1
Requires: perl-Digest-SHA1 >= 2.12-4
Requires: perl-XML-Filter-BufferText >= 1.01-9
Requires: perl-XML-Simple >= 2.18-7
Requires: perl-XML-Validator-Schema >= 1.10-5
Requires: perl-Tree-DAG_Node >= 1.06-8
Requires: perl-LDAP >= 0.40-2
Requires: perl-Crypt-SSLeay >= 0.58-1
Requires: ffmpeg >= 0.6-4

On EL5:
Requires: httpd >= 2.2.3-45
Requires: mod_perl >= 2.0.4-6
Requires: mod_ssl >= 2.2.3-45
Requires: perl >= 5.8.8-32
Requires: perl-Digest-SHA1 >= 2.11-1
Requires: perl-XML-Filter-BufferText >= 1.01
Requires: perl-XML-Simple >= 2.14-4
Requires: perl-XML-Validator-Schema >= 1.10-1
Requires: perl-Tree-DAG_Node >= 1.06
Requires: perl-LDAP >= 0.33-3
Requires: perl-Crypt-SSLeay >= 0.51-11
Requires: ffmpeg >= 0.6.1-1

allow_methods not setting list when request if of type OPTIONS

[Christopher Stanton]

Hey Everyone,

I am trying to set the allowed bitmask in a custom request handler
when I receive the OPTIONS method (and when I receive a method request
for a method I do not support). This includes possibly not listing
support for GET or POST.

I am using:
Fedora Core 6
Apache 2.2.4-2.1
mod_perl 2.0.2-6.2

In short I am using the allow_methods function to try to set the
returned list of allowed methods both when the request type is OPTIONS
and when it is an unsupported type.
This works when it is an unsupported type but Apache is ignoring it
with it is of type OPTIONS. To see a basic example handler and output
just skip all the additional explanations and head to the end of this
e-mail.

http://perl.apache.org/docs/2.0/api/Apache2/Access.html#C_allow_methods_

$r->allow_methods($reset, $list) only seems to respect the reset
flag if the return code from the handler is either
Apache2::Const::HTTP_METHOD_NOT_ALLOWED or
Apache2::Const::HTTP_NOT_IMPLEMENTED which is not in the
documentation.

The OPTIONS request is supposed to be URI/resource specific (unless
the URI is '*') but Apache seems to be treating it as '*' plus
whatever other additional options I set. OPTIONS is supposed to return
the set of methods available for the specific URI.

So can anyone shed some light on why Apache is ignoring the reset flag
when I am returning Apache2::Const::DECLINED?

And why is $r->allowed() also listed in the docs but does not work
either as advertised in the documentation?
(http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_allowed_)

And which way is the proper/standard way of:
1) setting the "Allow" header field when responding to an OPTIONS
request on a URI
2) returing from handling an OPTIONS request. is it Apache2::Const::DECLINED?
2) setting the "Allow" header field when responding with either
HTTP_METHOD_NOT_ALLOWED or HTTP_NOT_IMPLEMENTED return codes

I have included a basic example handler and and returned output below.
The Allow hearder field for both OPTION and GET requests should be the
same, but as you can see, they are not.

thanks,
Christopher

--

sub handler {
 my $r = shift;
 my $status_code = Apache2::Const::SERVER_ERROR;

 if ($r->method eq "OPTIONS"){
   $r->allow_methods(1, qw(PUT DELETE));
   $status_code = Apache2::Const::DECLINED;
 } else {
   $r->allow_methods(1, qw(PUT DELETE));
   $status_code = Apache2::Const::HTTP_METHOD_NOT_ALLOWED;
 }
 return $status_code
}

--

#curl -D header.txt -X OPTIONS http://localhost:9080/somedir
#cat header.txt
HTTP/1.1 200 OK
Date: Tue, 09 Oct 2007 22:10:58 GMT
Server: Apache/2.2.4 (Fedora)
Allow: GET,HEAD,POST,OPTIONS,DELETE,PUT,TRACE
Content-Length: 0
Connection: close
Content-Type: text/plain; charset=UTF-8

--

#curl -D header.txt -X GET http://localhost:9080/somedir
#cat header.txt
HTTP/1.1 405 Method Not Allowed
Date: Tue, 09 Oct 2007 22:14:26 GMT
Server: Apache/2.2.4 (Fedora)
Allow: DELETE,PUT,TRACE
Content-Length: 308
Connection: close
Content-Type: text/html; charset=iso-8859-1

Re: mod_perl Apache2 RequestRec allowed problem

[Christopher Stanton]

$r->allowed_methods($reset, $list) only seems to respect the reset
flag if the return code from the handler is either
Apache2::Const::HTTP_METHOD_NOT_ALLOWED or
Apache2::Const::HTTP_NOT_IMPLEMENTED which is not in the
documentation.

The OPTIONS request is supposed to be URI/resource specific (unless
the URI is '*') but Apache seems to be treating it as '*' plus
whatever other additional options I set. OPTIONS is supposed to return
the set of methods available for the specific URI.

So can anyone shed some light on why Apache is ignoring the reset flag
when I am returning Apache2::Const::DECLINED?

And why $r->allowed() does not work as advertised in the documentation?

And which way is the proper/standard way of:
1) setting the "Allow" header field when responding to an OPTIONS
request on a URI
2) returing from handling an OPTIONS request. is it Apache2::Const::DECLINED?
2) setting the "Allow" header field when responding with either
HTTP_METHOD_NOT_ALLOWED or HTTP_NOT_IMPLEMENTED return codes

I have included a basic example handler and and returned output below.
The Allow hearder field for both OPTION and GET requests should be the
same, but as you can see, they are not.

I am using:
Fedora Core 6
Apache 2.2.4-2.1
mod_perl 2.0.2-6.2

thanks,
Christopher

--

sub handler {
  my $r = shift;
  my $status_code = Apache2::Const::SERVER_ERROR;

  if ($r->method eq "OPTIONS"){
$r->allow_methods(1, qw(PUT DELETE));
$status_code = Apache2::Const::DECLINED;
  } else {
$r->allow_methods(1, qw(PUT DELETE));
$status_code = Apache2::Const::HTTP_METHOD_NOT_ALLOWED;
  }
  return $status_code
}

--

#curl -D header.txt -X OPTIONS http://localhost:9080/somedir
#cat header.txt
HTTP/1.1 200 OK
Date: Tue, 09 Oct 2007 22:10:58 GMT
Server: Apache/2.2.4 (Fedora)
Allow: GET,HEAD,POST,OPTIONS,DELETE,PUT,TRACE
Content-Length: 0
Connection: close
Content-Type: text/plain; charset=UTF-8

--

#curl -D header.txt -X GET http://localhost:9080/somedir
#cat header.txt
HTTP/1.1 405 Method Not Allowed
Date: Tue, 09 Oct 2007 22:14:26 GMT
Server: Apache/2.2.4 (Fedora)
Allow: DELETE,PUT,TRACE
Content-Length: 308
Connection: close
Content-Type: text/html; charset=iso-8859-1

Re: mod_perl Apache2 RequestRec allowed problem

[Christopher Stanton]

Thanks for the reply Geoffrey.

I have updated the code to use $r->allow_methods rather than
$r->allowed per the documentation at:
http://perl.apache.org/docs/2.0/api/Apache2/Access.html#C_allow_methods_

Now additional methods are being included in the response, but the
reset boolean does not seem to completely reset the list of Allow
results.

$r->allow_methods(1, qw(PUT));
return Apache2::Const::DECLINED;
results in
Allow: GET,HEAD,POST,OPTIONS,PUT,TRACE
being returned to the client rather than
Allow: OPTIONS,PUT,TRACE

$r->allow_methods(1, qw(HEAD GET));
return Apache2::Const::DECLINED;
results in
Allow: GET,HEAD,POST,OPTIONS,TRACE
being returned to the client rather than maybe
Allow: HEAD,GET,OPTIONS,TRACE

So, this does not seem to be working as I would expect either.

And additional source of confusion is the fact that the documentation
(http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_allowed_)
about the $r->allowed() method states:
"This bitvector is used to construct the "Allow:" header required for
OPTIONS requests, and Apache2::Const::HTTP_METHOD_NOT_ALLOWED (405)
and Apache2::Const::HTTP_NOT_IMPLEMENTED (501) status codes."

Are the docs just wrong?

thanks,
Christopher

On 10/9/07, Geoffrey Young <[EMAIL PROTECTED]> wrote:
>
>
> Christopher Stanton wrote:
> > I am trying to set the allowed bitmask in a custom request handler
> > when I receive the OPTIONS method (and when I receive a method request
> > for a method I do not support).
> >
> > I have followed the example in
> > http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_allowed_
> > but Apache always returns "Allow: GET,HEAD,POST,OPTIONS,TRACE" in the
> > header irrespective of what options I set.
> >
> > For example, lets say I only accept PUT to try to restrict the
> > available options to OPTIONS, TRACE, and PUT.
> >
> > I have tried both
> > $r->allowed($r->allowed | (1<< Apache2::Const::M_PUT));
> > and
> > $r->allowed(1< > with a return of Apache2::Const::DECLINED in either case in my handler.
> >
> > Neither ends up modifying the supported options sent to the client.
> >
> > In tha handler if I print the return value of $r->allowed:
> > print $r->allowed(1< > print $r->allowed() . "\n";
> > return Apache2::Const::DECLINED;
> >
> > I get:
> > 1
> > 2
> >
> > So the record is being updated, but for some reason Apache is not
> > basing its response to the client off of the record.
> >
> > Can anyone provide any insight into why I am unable to modify the list
> > of request methods my handler is capable of servicing?
>
> the Allow header appears to be built not from r->allowed but
> r->allowed_methods->method_mask.  so, try this:
>
> http://perl.apache.org/docs/2.0/api/Apache2/Access.html#C_allow_methods_
>
> HTH
>
> --Geoff
>

mod_perl Apache2 RequestRec allowed problem

[Christopher Stanton]

I am trying to set the allowed bitmask in a custom request handler
when I receive the OPTIONS method (and when I receive a method request
for a method I do not support).

I have followed the example in
http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_allowed_
but Apache always returns "Allow: GET,HEAD,POST,OPTIONS,TRACE" in the
header irrespective of what options I set.

For example, lets say I only accept PUT to try to restrict the
available options to OPTIONS, TRACE, and PUT.

I have tried both
$r->allowed($r->allowed | (1<< Apache2::Const::M_PUT));
and
$r->allowed(1allowed(1

[mp2] $r->print produces errors when Content-Disposition contains "built" filename

[Christopher Stanton]

Fedora Core 4
httpd-2.0.54-10
mod_perl-2.0.0-0.rc5.3
nph mod_perl script

If I user $r->print to print the response to the client, it returns that
it sent more bytes than length($write_buffer) and the JPEG, which is 
part of the response, is bad if the response header contains a 
Content-Disposition in which the filename field is set from appended 
hash values.


Code Example 1:
--
my $snapshot_filename = "$stream->{'feed-id'}-$stream->{'id'}.jpg";
print STDERR "snap=#$snapshot_filename#\n";
$write_buffer = "$ENV{'SERVER_PROTOCOL'} 200 OK\r\n" .
   "Pragma: no-cache\r\n" .
   "Cache-Control: must-revalidate, no-cache, no-store\r\n" .
   "Content-Type: image/jpeg\r\n";
if ( defined($attachment) && $attachment eq "true" ){
  $write_buffer .= "Content-Disposition: attachment; 
filename=\"$snapshot_filename\"\r\n";

}
$write_buffer .= "Content-Length: " . length($jpeg) . "\r\n";
$write_buffer .= "\r\n" . $jpeg . "\r\n\r\n";
$sent_bytes = $r->print($write_buffer);
--

Code Example 2:
--
my $snapshot_filename = "$stream->{'feed-id'}-$stream->{'id'}.jpg";
print STDERR "snap=#$snapshot_filename#\n";
$write_buffer = "$ENV{'SERVER_PROTOCOL'} 200 OK\r\n" .
   "Pragma: no-cache\r\n" .
   "Cache-Control: must-revalidate, no-cache, no-store\r\n" .
   "Content-Type: image/jpeg\r\n";
if ( defined($attachment) && $attachment eq "true" ){
  $write_buffer .= "Content-Disposition: attachment; 
filename=\"$snapshot_filename\"\r\n";

}
$write_buffer .= "Content-Length: " . length($jpeg) . "\r\n";
$write_buffer .= "\r\n";
$sent_bytes = $r->print($write_buffer);
$sent_bytes += $r->print($jpeg);
$sent_bytes += $r->print("\r\n\r\n");
--

Code Example 3:
--
my $snapshot_filename = "$stream->{'feed-id'}-$stream->{'id'}.jpg";
print STDERR "snap1=#$snapshot_filename#\n";
$snapshot_filename = "feed_1-stream_2.jpg";
print STDERR "snap2=#$snapshot_filename#\n";
$write_buffer = "$ENV{'SERVER_PROTOCOL'} 200 OK\r\n" .
   "Pragma: no-cache\r\n" .
   "Cache-Control: must-revalidate, no-cache, no-store\r\n" .
   "Content-Type: image/jpeg\r\n";
if ( defined($attachment) && $attachment eq "true" ){
  $write_buffer .= "Content-Disposition: attachment; 
filename=\"$snapshot_filename\"\r\n";

}
$write_buffer .= "Content-Length: " . length($jpeg) . "\r\n";
$write_buffer .= "\r\n" . $jpeg . "\r\n\r\n";
$sent_bytes = $r->print($write_buffer);
--

$attachment="true" for all tests.

In Code Example 1, snap=#feed_1-stream_2.jpg# and $r-print reports that 
it has sent more bytes than length($write_buffer) (bytes_sent=10951 vs 
length($write_buffer)=7648) and the attachment on the client's end is 
corrupted.


If I change "$sent_bytes = $r->print($write_buffer);" to "print 
$write_buffer;" in Code Example 1, the attachment is a valid JPEG on the 
client's end.


In Code Example 2, snap=#feed_1-stream_2.jpg# and I use $r->print to 
send the message, but I send each "section" as a separate $r->print 
call. In this case, $r-print prints the correct number of bytes and the 
attachment is a valid JPEG on the client's end.


In Code Example 3, snap1=#feed_1-stream_2.jpg# and 
snap2=#feed_1-stream_2.jpg#, I print the unified message just like I did 
in Example 1 and the reported bytes sent is correct and the JPEG is 
valid on the client's end. If I test snap1 and snap2 for string 
equality, they match.


So, Code Example 1 and 3 should be exactly the the save except the in 
Code Example 3 I do not build the string from values in a hash. But, the 
two strings match, so why does Code Example 3 work where Code Example 1 
failes? Why does Code Example 1 work if replace "$bytes_sent = 
$r->print($write_buffer)" with "print $write_buffer". Why does Code 
Example 2 work even though it should be equivalent to Code Example 1?


And how can $r->print legitimately return bytes_sent larger than the 
length of the buffer?


Has anyone see this behavior before? I am not sure where to go from here...

thanks for any help,
Christopher

Re: Strange $r-print($buffer)/Content-Disposition problem

[Christopher Stanton]

In the second example I have already appended "\r\n" to the write buffer.

So it really is:
$write_buffer .= "\r\n";
$r->print($write_buffer);
$r->print($jpeg);
$r->print("\r\n\r\n");

I try to write the same data to the client. In one case I write it as 
one big buffer, in the other case I write the header, then the JPEG, 
then the footer.


I can use print and $r->print to write the unified buffer if the 
Content-Disposition tag is not added or if it does not have the "; 
filename=.*" substring as part of the value


Works:
""
"Content-Disposition: attachment"

$r->print failes on the unified buffer if it contains the full 
Content-Disposition value:

"Content-Disposition: attachment; filename=\"$snapshot_filename\"\r\n"

The standalone print works and I can get $r->print to work if I split 
the printing into the three chunks I mentioned.


$r->print on the unified buffer returns that it sent more bytes than 
were even in the buffer (Example: bytes sent=10882 and 
length($write_buffer)=7647).


So, the two big questions are:
1) why is $r->print returning more bytes sent than are in the buffer
2) why does print work but $r->print does not on this case

sorry about the confusion,
Christopher

Larry Leszczynski wrote:

Hi Christopher -

I don't know if it's a typo but this:


$write_buffer .= "\r\n" . $jpeg . "\r\n\r\n";
$r->print($write_buffer);



is not the same as this:


$r->print($write_buffer);
$r->print($jpeg);
$r->print("\r\n\r\n");



since the latter does not insert "\r\n" between $write_buffer and $jpeg...


Larry

Strange $r-print($buffer)/Content-Disposition problem

[Christopher Stanton]

Fedora Core 4
httpd-2.0.54-10
mod_perl-2.0.0-0.rc5.3

I have an nph mod_perl script which will send a JPEG to a client either
inline or as an attachment (if a param is passed). If it is an
attachment, I set the Content-Disposition to:
"Content-Disposition: attachment; filename=\"$snapshot_filename\"\r\n"
otherwise, I do not include this tag in the header.

I am creating the buffer as I go (appending header tags). I append the
separator and the JPEG:
$write_buffer .= "\r\n" . $jpeg . "\r\n\r\n";

And then send the buffer to the client:
$r->print($write_buffer);

If the Content-Disposition line is not present, $r->print returns the
number of bytes sent to the client which equals length($write_buffer).
7628/7628 (bytes_sent/buffer_length)

If the Content-Disposition line is present, $r->print returns the number
of bytes sent to the client which is larger than length($write_buffer).
10882/7647 (bytes_sent/buffer_length)

(the buffer length is different because the image sent was different)

In addition to this, the JPEG is valid in the first case but is
"corrupt" in the second. Saving the attachment through Firefox or using
curl results in a file which is not a valid JPEG.

If I break the attachment send into 3 sends:
$r->print($write_buffer);
$r->print($jpeg);
$r->print("\r\n\r\n");

Each returns bytes sent equal to the length of the buffer and everything
is valid on the client's end.

"print $write_buffer" works in both the inline and append cases without
needing to separate the transfer into parts.

Is this a bug in mod_perl or am I missing something??

thanks,
Christopher