Re: production quality?

[Martin Algesten]

Hello again,

The quote helps pointing out one important thing "It's all a matter of expectations". That's just another way of twisting my point, what we're dealing with here is expectation management.

Apache Tomcat has reached a state where it is deployed widely around the world in live environments. The "Apache" name is for most people synonymous with quality, and on top of that we got Sun Java's endorsment of Tomcat by virtue of J2EE. This means that wether we like it or not "Apache Tomcat" does raise whole lot of expectation. These expectations are a form of trust, which is a nice thing and that we need to responsibly care for, rather than brushing them under the carpet with a "it's your problem to test your software before going to production".

Making sure known bugs are addressed before releasing a new version and perhaps being more explicit about what issues a release might have, is only acting responsibly towards the trust people have in what we do.

Martin

Shapira, Yoav wrote:

Hi,
Every now and then a message along those lines comes up. I've seen it
in all the open sources projects I've contributed to, and (what does
that say? I have too much free time? ;)) these are many.

I will quote a response, not written by me, that I think is a perfect
way to look at the process. I hope Will won't mind being quoted ;)

From: "Milt Epstein" <[EMAIL PROTECTED]>
Sent: Friday, August 23, 2002 10:11 AM
I mean, I understand (and sympathize with what you're saying), but I
also understand the position of those who want to wait until there's
"release"-quality code available. I mean, what will the typical
manager say after the production system breaks down and you tell them
that you went with "beta"-quality code instead of "release"-quality
code.

It's all a matter of expectations. I find that most people want the

system

to work and aren't really caught up on release numbers and labels.

Now, through probably painful experience, people have lower

expectations of

1.0 code, "Beta" code, etc. They pretty much expect it to fail in some,
perhaps critical, way. They may use the label as an indicator whether

the

code is worth testing.

But the real problem is not necessarily whether its Beta or Release
quality, it is the testing facilities available to the consumers.

If you had a deployed application and a solid, thorough feature and

load

test suite of your application, then it shouldn't matter what the back

end

code is labeled. If your test is complete and robust enough and thus
confirms that your application will behave as designed, then Release or
Beta are irrelevant.

The dark side of this is that folks will say "Well, gee I don't have to
'test' release code, it's already tested!".

To wit everybody falls over on the floor in hysterics.

The acceptance process for the software should be the same whether Beta

or

Release. No matter how thorough the testing by the folks developing the
application, no doubt they have NOT tested their software on YOUR
application with YOUR load conditions on YOUR hardware.

And it should go without saying that this applies to any major
infrastructure component in your application, not just Tomcat. Don't

just >blindly upgrade your system to Solaris 9, Oracle 9i and new
Firewall >software and then throw the application live because it's all
"release >quality" software.

There are disclaimers in those licenses for a reason. Caveat Emptor and

all

that.

Regards,

Will Hartung
([EMAIL PROTECTED])

Yoav Shapira
Millennium ChemInformatics

-----Original Message-----
From: Martin Algesten [mailto:puckman@;taglab.com]
Sent: Wednesday, October 30, 2002 6:19 AM
To: [EMAIL PROTECTED]
Subject: production quality?

Hi all,

Just some thoughts.

I've been using the 3.3.1 release for quite some time in a
mod_jk/apache/linux kind of setup and all was fine. Though a couple of
weeks ago I felt a need to start looking at new versions of all my
API's/products in order to make sure I stay on top of things and don't
end up with unsupported versions.

What do we mean with production quality?
According to the Tomcat project home page, 4.1.12 is a production
quality release, however using it in real life makes me question the
usefulness of such status. I've been monitoring this list and also

tried

to contribute by discussing/submitting patches for the bugs I've
encountered. I don't have an issue with how long it takes to resolve
these issues, after all we are all doing this for fun (more or less ;)
). However I do think we have a responsibility in what signals we're
sending regarding how useful a release really is. The current 4.1.12
release have some quite nasty issues that in some production setups
makes it more or less useless. In my opinion the most nasty issues are
those that directly breaks internet standards and the core API (10373,
13846, 13040).

What about quality control?
Well, the bugzilla do allow for a Severity, Priority classification.
Perhaps we should start classifying bugs more actively using these
switches. This could be combined with a policy that we don't make a
release before certain levels of issues are all resolved? I also get a
feeling that we as developers are somewhat in the wrong position to
label things as "production quality". The watchdog tests are all very
well, but can never replace real life scenarios. Perhaps we need a new
label such as release candidate? Perhaps we should be more in the

spirit

of other open source projects, (OpenSSL and Mozilla comes to mind),
where the release cycle seems to more involve end users and have a more
cautious labelling of releases.

This is not intended as criticism of anyone. After all Tomcat is a
fantastic project with fantastic people contributing. Good work all!

Martin


--
To unsubscribe, e-mail:   <mailto:tomcat-dev-
[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:tomcat-dev-
[EMAIL PROTECTED]>

------------------------------------------------------------------------

This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you.


------------------------------------------------------------------------

--
To unsubscribe, e-mail: <mailto:tomcat-dev-unsubscribe@;jakarta.apache.org>
For additional commands, e-mail: <mailto:tomcat-dev-help@;jakarta.apache.org>

--
To unsubscribe, e-mail:   <mailto:tomcat-dev-unsubscribe@;jakarta.apache.org>
For additional commands, e-mail: <mailto:tomcat-dev-help@;jakarta.apache.org>