Perrin Harkins wrote:
>
> On Fri, 28 Jan 2000, Joshua Chamas wrote:
> > I have updated the Hello World Web Application Benchmarks,
> > now at http://www.chamas.com/bench/
>
> The end result of all this is that you have benchmark numbers which, while
> sort of entertaining, should not be used to make any kind of decision. I
> would hate to think that someone used these numbers as the basis for
> making decisions and didn't benchmark the options himself. I almost fell
> into that trap when I saw the low scores for certain things and wrote them
> off without doing my own testing. (Yes, I know a good developer should
> never believe other people's benchmarks.)
>
> Although there is a disclaimer on the page, I wish it said more. In some
> ways, it's worse to have these misleading benchmarks than no benchmarks at
> all. I shudder to think what a naive manager might do with these.
>
> Okay, I'm done complaining for now. Sorry Joshua, I know you put effort
> into this, and I do appreciate it.
>
I hear you Perrin. Its better to have something than nothing
though. There is no way that people are going to benchmark
10+ different environments themselves, so this merely offers
a quick fix to get people going with their own comparisons.
Do you have any idea how much time it takes to do these?
Run a few on different operating systems, the time involved
is humbling to say the least.
In order to improve the benchmarks, like the Resin & Velocigen
ones that you cited where we have a very small sample, we simply
need more numbers from more people. That these benchmarks are
open to anyone submitting, means we all can scrutinize those
benchmarks submitted by those with agendas and submit our own.
If you can find any way to make these benchmarks better than
simply more data, please feel free to suggest improvements.
Also, any disclaimer modifications might be good if you feel
there can be more work done there.
-- Joshua
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
