[correction] Benchmarking Apache::Registry and Perl Content Handler

[Stas Bekman]

Well, following the guardian Gunther suggestions I've tried to rerun these
tests as well, preloading the script to make the benchmark fair. This has
improved the results significantly, shortening the gaps. 

But read the both sets of tests and try to explain the phenomena that you
will see. You will find it at the end, under the [ReaderMETA] tag. Thanks!




=head1 Benchmarking Apache::Registry and Perl Content Handler

=head2 Light (Empty) Code

First lets see the overhead that Apache::Regsitry adds. In order to do
that we will use an almost empty scripts, that only send a basic
header and one word as a content.

The I<registry.pl> script running under C<Apache::Registry>:

  benchmarks/registry.pl
  ----------------------
  use strict;
  print "Content-type: text/plain\r\n\r\n";
  print "Hello";

The Perl Content handler:

  Benchmark/Handler.pm
  --------------------
  package Benchmark::Handler;
  use Apache::Constants qw(:common);
  
  sub handler{
    $r = shift;
    $r->send_http_header('text/html');
    $r->print("Hello");
    return OK;
  }
  1;

with settings:

  PerlModule Benchmark::Handler
  <Location /benchmark_handler>
    SetHandler perl-script
    PerlHandler Benchmark::Handler
  </Location>

so we get C<Benchmark::Handler> preloaded.

We will use the C<Apache::RegistryLoader> to preload the script as
well, so the benchmark will be fair and only the processing time will
be measured. In the I<startup.pl> we add:

  use Apache::RegistryLoader ();
  Apache::RegistryLoader->new->handler(
              "/perl/benchmarks/registry.pl",
   "/home/httpd/perl/benchmarks/registry.pl");

And we if we check the I<Compiled Registry Scripts" section with help
of L<Apache::Status|debug/Apache_Status_Embedded_Inter> (
http://localhost/perl-status?rgysubs ), where we see the listing of the
already compiled scripts :

  Apache::ROOT::perl::benchmarks::registry_2epl

So now we can proceed with the benchmark:

  % ab -n 1000 -c 10 http://localhost/perl/benchmarks/registry.pl
  
  Time taken for tests:   16.148 seconds
  Requests per second:    61.93
  Connnection Times (ms)
                min   avg   max
  Connect:        0     2   202
  Processing:    81   158    60
  Total:         81   160   262

  % ab -n 1000 -c 10 http://localhost/benchmark_handler
  
  Time taken for tests:   5.097 seconds
  Requests per second:    196.19
  Connnection Times (ms)
                min   avg   max
  Connect:        0     0     3
  Processing:    40    50   237
  Total:         40    50   240

So we can see that the average added overhead is about:

  160 - 50 = 110 milli-seconds

per script.

=head2 Heavy Code

Of course this overhead is insignificant when the code itself is
significantly heavier and slower. Let's leave the above code examples
umodified but add some CPU intensive processing operation (it can be
also an IO operation or a database query.)

  my $x = 100;
  my $y = log ($x ** 100)  for (0..10000);

  % ab -n 1000 -c 10 http://localhost/perl/benchmarks/registry.pl
  
  Time taken for tests:   82.614 seconds
  Requests per second:    12.10
  Connnection Times (ms)
                min   avg   max
  Connect:        0     3   670
  Processing:   187   819  1211
  Total:        187   822  1881

  % ab -n 1000 -c 10  http://localhost/benchmark_handler
  
  Time taken for tests:   15.000 seconds
  Requests per second:    66.67
  Connnection Times (ms)
                min   avg   max
  Connect:        0     2   112
  Processing:    22   147   770
  Total:         22   149   882

The processing time delta has grown to 673 milli-seconds (822-149).

[ReaderMETA]: Anyone knows to explain this phenomena? It's not clear
to me why adding the same CPU intesive code to the two handlers under
test, enlarges the delta of the average processing time between the
two handlers. I'd expect to see the same delta (of 110 msec) in this
case, but that's not what's happenning. Any ideas?

Notice that the hardware used in this test is not important since
what's important is the delta (because we are interested in the
comparison and not the absolute values).

The SW that was used: Apache/1.3.10-dev, (Unix) mod_perl/1.21_01-dev,
Perl5.005_03 for i386-linux.

The relevant server configuration:

  MinSpareServers 10
  MaxSpareServers 10
  StartServers 10
  MaxClients 20
  MaxRequestsPerChild 10000



______________________________________________________________________
Stas Bekman             | JAm_pH    --    Just Another mod_perl Hacker
http://stason.org/      | mod_perl Guide  http://perl.apache.org/guide 
mailto:[EMAIL PROTECTED]  | http://perl.org    http://stason.org/TULARC/
http://singlesheaven.com| http://perlmonth.com http://sourcegarden.org
----------------------------------------------------------------------