I don't normally like responding to the vendor comparison messages because it doesn't
really seem to be appropriate to the list and has the potential to turn into a
useless vendor war. The model 2 architecture stuff is much more interesting. But I
did try very hard to make those benchmarks fair, and have added some comments
defending them below.
Karl Avedal wrote:
> As a note, the Orion version tested was 0.7.6b. Orion has since then become very
> much faster (even if the benchmark shows it was already very fast and already beat
> Resin for their own DB-test using the same VM).
>
> Our own internal testing shows that the latest version of Orion is faster than all
> other JSP engines, including Resin. (Could this be why they're not including Orion
> in their latest test?)
:-) No, I haven't re-tested the servlet engines because of time constraints. If you
notice, the Resin version in the servlet engine comparison is also an old one.
Just for fun and because Orion is pretty easy to configure, I downloaded the latest
Orion and tested it against Resin:
1 client/no keepalive (in ops per second)
File Servlet JSP Loop Big
Resin/IBM 620 529 553 43 84
Orion/JDK1.2 192 190 428 5 49
Resin/JDK1.2 267 212 234 6 71
4 clients/ 4 keepalives
Resin/IBM 1588 1120 1132 45 95
Orion/JDK1.2 236 208 717 6 94
Resin/JDK1.2 372 273 311 6 94
So, indeed, Orion's Hello World JSP significantly beats Resin's. Great job from the
Orion team!
> But please, don't just look at the vendors' own tests. Do your own tests and see
> what you find. Don't just trust the vendors.
Agreed. Most importantly, you need to test your actual application. Just running
the same benchmark on your own system doesn't help. These benchmarks, like most, are
toy benchmarks. For example, these benchmarks don't address "model
2" architectures. (The reason it's missing from the Caucho benchmarks, of course, is
that a "model 2" benchmark doesn't make sense against Perl or mod_php.)
Also, after a certain point, the servlet engine performance really doesn't matter.
You'll be stuck with database overhead or your own code.
>From Vyacheslav Pedak:
> About Resin benchmarks, see here
>
http:[EMAIL PROTECTED]
The benchmark that's on www.caucho.com has incorporated their changes for months.
You know, for all the complaining that the Perl folks have done about the benchmarks,
it's clear from their results that they had to cripple Resin (by turning off socket
keepalive) to get those numbers.
http://www.chamas.com/bench/hello_bysystem.html shows results comparable to our
numbers, not the Perl numbers.
Internally, I'd duplicated the slash/1.0 front page with Java/JDBC to test
performance against Perl, because toy benchmarks are only useful if you've got
nothing else. Java beat Perl by about a factor of 10. That shows one reason why toy
benchmarks are misleading. The toy benchmarks don't compare Perl's functions calls
or field references to Java. The factor of 10 is not surprising. Once you start
doing anything more complicated than a "hello, world" test, Java with a decent JIT is
going to crush Perl. So actually, the Caucho benchmarks are biased in favor of
Perl. They should be thanking us for making Perl look decent.
-- Scott
===========================================================================
To unsubscribe: mailto [EMAIL PROTECTED] with body: "signoff JSP-INTEREST".
Some relevant FAQs on JSP/Servlets can be found at:
http://java.sun.com/products/jsp/faq.html
http://www.esperanto.org.nz/jsp/jspfaq.html
http://www.jguru.com/jguru/faq/faqpage.jsp?name=JSP
http://www.jguru.com/jguru/faq/faqpage.jsp?name=Servlets
