Sean Chittenden wrote:
> <http://www.caucho.com/articles/benchmark.html>
>
> Supposedly, according to its benchmarks, it's faster than mod_perl...
> impressive to say the least. Any chance someone has any experience
> with this or would like to benchmark this technology? External validation
> would be pretty useful.
It is not faster than mod_perl. We tested it and found that the Resin
author used code that didn't take good advantage of mod_perl, i.e.
Apache::Registry scripts rather than PerlHandler modules. We sent our
results and code to Caucho, and received a cordial reply, but they have
not updated their page. The following is an excerpt from the message one
of our engineers sent to them:
> I've recently been testing Resin 1.1b4's performance compared with
> mod_perl. The "benchmark.jar" from your web site included perl scripts
> that really didn't take full advantage of the capabilities that mod_perl
> offers. So, I wrote three perl modules that attempt to be similar to
> examples included in the Resin 1.1b4 distribution and better reflect
> what mod_perl can do.
>
> Attached is a gzipped tar file with these perl modules, and you're free
> to do whatever you want with them.
>
> Overall, I've found Resin to be a very good product, yet ran into a few
> problems during testing. These may be caused by the IBM 1.1.8 JDK rather
> than the Resin package itself. For example when running Resin as a
> stand-alone httpd server, then using "httperf" to benchmark it, the
> entire JVM would freeze up when the --close-with-reset option was given
> to httperf.
>
> The JVM used in testing is IBM 1.1.8, on Red Hat Linux 6.0, 2x500MHz
> Pentium III, 512MB RAM, Oracle 8.0.5 client libraries (JDBC). Apache
> 1.3.9 with DSO support was used for "Resin+Apache", with 60 max clients.
> A recent version of mod_perl and Apache (without DSO) was used for the
> "mod_perl+Apache" tests, also with 60 max clients. Finally, "Resin
> httpd" is Resin running its own web server, with max threads set to 60.
>
> These tests used httperf with persistent connections, making either
> 1000 or 100 requests per connection, depending on the test. The results
> on "hello" are astounding:
>
> Test: "hello" -- displays "Hello, World"
> Size: 450 bytes
> httperf: 40 concurrent connections, 1000 requests/connection
> Results:
> mod_perl+Apache: 1600 req/sec
> Resin+Apache: 500 req/sec
> Resin httpd: 4600 req/sec
>
> Test: "bighello" -- lots of HTML + "Hello World"
> Size: 23888 bytes
> httperf: 40 concurrent connections, 100 requests/connection
> Results:
> mod_perl+Apache: 480 req/sec
> Resin+Apache: 300 req/sec
> Resin httpd: 280 req/sec
>
> Test: "database" -- query: "select NAME from DIVISION", returns 11
> rows
> Size: 460 bytes
> httperf: 40 concurrent connections, 100 requests/connection
> Results:
> mod_perl+Apache: 570 req/sec
> Resin+Apache: 300 req/sec
> Resin httpd: 450 req/sec
>
> So, although "Resin httpd" performs very well with many small transfers
> ("hello"), the performance relative to mod_perl appears to degrade with
> larger transfers ("bighello"). The database performance improved nicely
> from where it was under Resin+Apache, but mod_perl still has the edge
> there.
>
> I also tested the "file.jsp", which I renamed "fortune" in the perl
> tests. With http_load, the results again show mod_perl ahead:
>
> Resin:
> 1584 fetches, 10 max parallel, 489610 bytes, in 10 seconds
> 309.097 mean bytes/connection
> 158.4 fetches/sec, 48961 bytes/sec
>
> mod_perl:
> 6190 fetches, 10 max parallel, 1.98814e+06 bytes, in 10 seconds
> 321.186 mean bytes/connection
> 619 fetches/sec, 198814 bytes/sec
>
> I'm not trying to belittle Resin in any way -- in fact I'm impressed by
> both its design and performance. In learning about Resin during these
> tests, I found that JSP is in many ways easier to use than mod_perl. The
> smart caching that Resin does (with compiling .java -> .class, etc.) is
> effective, and the XML configuration is a joy to deal with compared to
> Apache's httpd.conf.
So, it's nice, but not as fast as mod_perl on Linux. If anyone wants to
try this on another OS, I can supply a tar of the files we used for
testing.
- Perrin
