On Fri, 8 Jun 2001, will trillich wrote:
> On Thu, Jun 07, 2001 at 06:48:38AM +0200, Gerald Richter wrote:
> > > regarding the tools that dovetail into the mod_perl paradigm,
> > > who's got a comparison over relative performance (and other
> > > strengths/weaknesses) of various templating methods?
> >
> > There are various discussions on the mod_perl list about this topic in the
> > past (so take a look at the archives). Also there was an start to write such
> > a comparsion, but I am not aware that anybody has really finished it. The
> > only benchmarks I know are from Joshua. I append his mail below.
> >
> > NOTE: While the hello.xxx benchmarks only prints "Hello world", so they only
> > measures the startup overhead of the toolkit, the h2000.xxx tests tends a
> > little bit more towards a real application.
>
> this is interesting information -- perhaps misleading to use the
> microscopic "hello world" but still it gives a starting point:
>
> here i sorted by hits-per-second
>
> > Test Name Test File Hits/sec Bytes/Hit
> > ------------ ---------- ---------- ----------
> > HTML static hello.html 1158.4 311 bytes
> > mod_include SSI hello.shtm 996.6 198 bytes
> > mod_caucho JSP hello.jsp 860.6 230 bytes
> > mod_perl handler hello.benc 852.6 196 bytes
> > mod_php PHP hello.php 734.8 225 bytes
> > Apache::Registry v2.01 CGI Raw hello_raw. 706.4 52 bytes
> > Apache::Dispatch v0.08 handler hello/worl 656.1 196 bytes
> > HTML::Template v2.0 hello.htmp 567.2 198 bytes
> > Apache::SSI v2.16 hello.shtm 559.4 199 bytes
> > Template v2.00 Toolkit hello.tt 522.1 198 bytes
> > Apache::Registry v2.01 CGI.pm hello.reg 458.5 216 bytes
> > HTML::Embperl v2.0a18 hello.epl 458.2 219 bytes
> > Apache::ASP v2.07 hello.asp 390.6 241 bytes
> > Apache::ePerl hello.eper 344.8 217 bytes
> > HTML::Mason v0.895 hello.mas 365.3 197 bytes
>
> i bet CGI would be 200.0 or so... ?
>
> and here's the "handler actualy does some work" set:
>
> > mod_caucho JSP 2000 h2000.jsp 328.9 28964 byte
> > mod_php PHP 2000 h2000.php 261.8 28865 byte
> > HTML::Embperl v2.0a18 2000 h2000.epl 247.3 28809 byte
> > Apache::ASP v2.07 2000 h2000.asp 228.0 28997 byte
> > HTML::Mason v0.895 2000 h2000.mas 222.9 28798 byte
> > Template v2.00 Toolkit 2000 h2000.tt 55.6 28888 byte
>
> wow. template toolkil took a big hit, there. (no mod_perl on
> this list? hmm!)
This benchmark can be very non-representive. If you don't know how to
optimize each and every "thing" under test, you end up with unfair
benchmark and come to potentially wrong conclusions. Take TT, add compiled
template caching on the disk and shared TT object and I bet TT won't be at
the bottom.
In any case always remember that it's extremely hard to run a fair
benchmark. I'd say it's almost impossible. The only fair benchmarking can
be done if you know all the in's and out's of the 'things' under test and
provide many benchmark tests each exploring a single property and not just
'one for all' benchmark.
Of course it's a good thing to have benchmarks, but they all should be
taken with a grain of salt.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:[EMAIL PROTECTED] http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
