Hi again Roy.
Roy Wilson wrote:
>
> Jason,
>
> A matter of small importance, and of philosophy, but ...
>
> <snip>
>
> > The great thing about this is that
> > it gets some people jazzed about the project without making any
> > real, solid claims.
>
> I think I'm ok on this.
>
> > And, about the sentence that says "performs well under heavy
> > load, even compared to the native Apache web server".. This
> > works the same way, since we're not saying which web server,
> > nor running on which operating system, nor under what conditions.
> > We're not even saying a major version number. So why is it in
> > here? Because, I heard through the grapevine that someone ran
> > some meaningful (but not necessarily bullet-proof) performance
> > tests and had this finding: that the version of Catalina that
> > was tested at the time performed about as well on most tests
> > as the version of their native Apache web server did on the same
> > tests.
>
> But what if there happens to also be in the gv a story about someone who
> has run some meaningful performance tests and found the contrary? To mix
> metaphors, the can-o-worms that you don't want to open gets opened
> anyway, does it not?
So if someone else wants to dispute this claim, they'll first run
into the fact that we're not making any exact claims in the FAB,
only relative statements. Then, if they want to try to make some
kind of exact claim that Catalina's web server doesn't perform well
in comparison to some version (that they choose) of the Apache web
server, then they'll first need to run some meaningful benchmarks
and publicly post the results. Most people know that benchmarks
are often misleading, and are often swayed in favor of whoever runs
the benchmark. So, that would only cause more talk about how the
benchmark should have been tuned, or which tests really should have
been run, etc.. And, I think that's all good because the result
is that more people are thinking and talking about Catalina. :)
We learn more about how to tune it, how to test it, etc, all in
comparison to the native Apache web server.
The end result of it all is that it is our word against theirs,
and people will believe what they want to believe and what they
can prove on their own machine. That's about it.
> > I personally don't want to spend time validating that,
> > nor would I want to be the one to try to defend the results
> > (it would likely turn into another "Java's not as fast as C"
> > arguement).
>
> I agree with this, sort of.
>
> > But, I believe it's possible, and I also believe
> > that Catalina's web server performs quite nicely.
>
> I am too am inclined to think that Catalina's webserver will perform well
> enough to satisfy someone somewhere, but aren't you claiming more?
This is one thing I find awkward about marketing-speak: it sure
seems like it makes wild, solid claims, but in reality it doesn't.
> > Everyone should keep in mind that the purpose of our FAB is not
> > to prove any performance numbers or to help engineers to
> > understand Catalina's deep implementation details. It's to make
> > some not-so-technically-saavy readers want to help the project,
> > promote the project, hire people to get it running and use it,
> > etc. We were primarily thinking of people like:
>
> > - CEOs, CTOs, and other management personnel
> > - Venture Capitalists
> > - Marketing staff
> > - [insert random Joe "Non-Technical" Businessman here]
>
> I agree with the language used for the target audience, but see below.
>
> > Reallly, people who don't know much about why their company
> > should be interested in this thing called Catalina who aren't
> > technically saavy enough to understand other docs that they
> > can find about it (like the User docs, or other technical docs
> > like the Classloader explanation doc or even the status doc..)
> > How else can we each get permission to work full-time on cool
> > projects like Catalina? The better Catalina looks to both the
> > technical and non-technical population at large, the more support
> > the project will get overall, I think.
>
> I think that the marketing-speak can't be broadcast until there is
> technical data to support it in some fashion. Otherwise, perhaps, some
> CEO/CTO/etc., asks a question that ultimately gets answered by a
> technically savvy person such as yourself who (in the absence of
> technical performance docs) might then be forced to reply "there's no
> !#%?! evidence!"
So then the Boss's next reply would be "Well then benchmark it and
have the results ready by friday.", and that means they'll download
it, install it, figure out some way of benchmarking it, and compare
with Apache. If they get bad numbers, then they'll probably think
that we know something they don't. And, we just might.
Also, having no "solid" numbers to back up a claim that's actually
a relative statement won't necessarily mean that we're just plain
wrong.
Besides, I think those who benchmark will find that it performs
quite comparably to the native Apache web server of their choice.
Notice, though, that the FAB makes no mention that currently the
Catalina web server doesn't have a plethora of configuration
features like the native Apache web server has. Of course the
Catalina web server should have less features at this point, it's
a much younger project. So, even benchmarking doesn't give you
the full picture. You have to decide what kind of a web server
you need for any given project. Performance isn't always everything
(but it sure is nice! :).
> I don't see how a market-first/measure-later approach can sustain growth
> in Tomcat support. But, then, maybe you guys are simply getting the
> market-speak in place :-).
Yes, for us it was mostly a matter of having our ducks in a row.
We wanted to be prepared for non-techies asking us the question
"So what's this Catalina thing you mentioned, and why should I
care?". That was our real motive for writing this kind of text.
--
Jason Brittain
Software Engineer, Olliance Inc. http://www.Olliance.com
Current Maintainer, Locomotive Project http://www.Locomotive.org
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
