Hi Everyone,
I would like to thank Nancy for her nice explanation of the beta process we
use at Henter-Joyce. This is one of the most common processes in the
software industry and HJ is, after all, a software company.
Like most software companies, we do a multi-tiered testing process. The
first people to see our software is our internal test department. These
people find the largest number of bugs in the software and we fix them
before anyone outside HJ gets the program.
When we feel that the product is not likely to crash too often and that most
of the new features are present in the product, we send it out to our
private beta group. membership in this very hard working group of about
forty people from all over the world is restricted to those we invite to
join. The private team is required to test the product, verify each others'
bug reports and download hundreds of megabytes of software during a product
cycle. In the 3.31 process, this gang has installed nearly every build from
13 to 35. They have reported and verified hundreds of bugs which have been,
for the most part, fixed well before the first public releases.
The selection criteria for invitation to join our private team is pretty
simple. Individuals who download the public releases and send in a lot of
very specific bug reports are often asked to join the group. People who are
on our private team and do not do the work required are removed after each
product release cycle. The 3.31 process has resulted in four new
invitations - all to people on this users list.
After the private group has banged on the software for a month or so, we do
a public beta. If you read the text on the web page which sits directly
above the ftp link one needs to choose to download our beta software, we
state with extreme clarity, that this is pre-release software and that
problems are likely. Our users then have the option to risk experiencing
some inconvenience and, in exchange, they get to start using the new
features before the final release of the product.
People who download our public beta software are not required to fill out a
form so we do not know who among our users has decided to run the new
version of the product. There is no requirement to send in bug reports and
if you don't like the new release, you can run the uninstall program and it
will go away.
There are two schools of thought on this matter: one group wants to see
everything we do nearly as soon as we code it and the other never wants to
run anything but final release software. We walk the fine line between the
two groups, the first gets angry when the discover that we've been "sitting
on new technology" for a few months and the second gets angry when we put
out product that is not entirely debugged. I think we do a nice job of this
by labeling the pre-release product as such and wrapping it with warnings.
Finally, if you count up every single keystroke in every application we
officially support in JFW, then add all of the default behaviors, a
scripting language which offers nearly infinite possibilities, the millions
of web sites with which we need to be compatible and all of the myriad JFW
configuration possibilities, plus all of the possible hardware variations,
the result is an astronomical test matrix. Without actually doing the math,
I would estimate that the total number of permutations runs well into the
billions of potential situations. Without expanding the test bed to as many
users as possible, we couldn't even hit the most common setups without years
invested in testing each separate build of JFW. Needless to say, this would
raise the price of JFW into the hundreds of thousands of dollars just to
cover the cost of a test department.
We're always looking for ways to improve and we take all suggestions, both
for product features and software engineering procedures, very seriously.
Keep the ideas coming and we'll do what we can to please as many people as
possible. JFW is nothing without its users and we respect their opinions
very highly.
Thanks,
cdh
Chris Hofstader
Development Manager
Henter-Joyce, Inc.
(727) 803-8000
[EMAIL PROTECTED]
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of Nancy Feldman
Sent: Sunday, October 10, 1999 12:47 AM
To: [EMAIL PROTECTED]
Subject: Re: Release Candidate 2 is now posted on the hj web page
No! I much prefer HJ's current practice for beta testing. This practice is
a commonly used one among many companies, allowing any of their registered
users who wish to do so to participate in the comment/testing phase. This
way we do have input, any of us who choose to. We who test their products
are fully aware that we may be installing an unstable program; we don't go
into this unaware.
As an instructor at the Colorado Center for the Blind, I know that my
students enjoy playing with the new modifications, to see what works and
what does not. We sent in a few messages for this last release, and were
pleased to receive replies. It's nice to know that our suggestions count
for something.
If you are uncomfortable installing software on your computer which may have
flaws, don't do it! Wait until the official release, where you can be
certain that most of the kinks have been ironed out.
Nancy Feldman, Technology Instructor
Colorado Center for the Blind
> even more than usual! Is the a JFW version 3.3, a version 3.31, a version
> 3.31.31 and now, a version 3.31.35 Release Candidate 2???
>
> What in the hell is going on? Why is HJ putting up program versions full
> of problems and bugs for us to find for them. Doesn't HJ have competent
> enough people to find their programming problems an bugs? If we are going
> to do their work for them, then, it seems as if some of us should be hired
> as paid beta testers.
>
> Do not misunderstand. I am very reluctant to criticize HJ because I am
> deeply appreciative of their work and am a genuine admirer of Ted Henter
> and what he has accomplished. However, releasing a program, even as a
> "release candidate" prematurely, with the objective of letting us find the
> problems and bugs, is a highly questionable practice. I suggest that they
> establish a "representative" group of typical users as paid beta testers,
> send them the release candidates, and then, after the problems and bugs
are
> worked out, HJ can put it up for the rest of us to download and/or they
can
> ship them out to us as officially released upgrades or new versions. I
> realize that even then we will probably find a few small problems, but the
> current approach is only creating chaos, giving them a poor reputation,
and
> confusing many users.
>
> Ron Milliman
>
>
>
>
> -
> Visit the jfw ml web page: http://jfw.cjb.net
>
-
Visit the jfw ml web page: http://jfw.cjb.net
-
Visit the jfw ml web page: http://jfw.cjb.net
