The SCons build system has had a testing infrastructure for many years,
unfortunately with only a minimal number of actual test cases.
As I have seen, the CMake system has one too, suffering the same
problem.
It's just a matter of interested parties contributing test cases.
Covering all the existing code is one challenge. But it would be most
helpful
if anyone adding features also provided test cases to demonstrate what
they are
supposed to do, and even more importantly what they are NOT supposed to
do.
Those could be just small(!) datasets with a description, and not
necessarily
already integrated into a test suite. Someone who is familiar with one
of the
suites can then integrate them. In the ideal case, no new feature wold
go into
a "final release" without a comprehensive set of test cases covering it.
Besides that, given the "organic" development and limited resources, I'm
not
sure if changing the release cycle would make much of a difference in
when bugs
are found. There's only a small number of people checking out the HEAD
version,
and those only try it on a very limited data set.
The majority will only download after a new final release has been
announced,
no matter wether the release candidate has been around for a week or a
month.
And only then will the new release be confronted with the "interesting"
data
that actually triggers most bugs.
The important part here is to then create test cases specifically
testing for
those bugs, to make sure they won't resurface later (regression
testing).
QA is a dirty job, but someone's gotta do it...
-schorsch
Am 2016-03-14 02:20, schrieb Gregory J. Ward:
We definitely have a more organic, non-standard release process for
Radiance. I am open to suggestions for alternatives. One of our
ongoing issues is the lack of a test suite to verify a release
candidate and identify regressions. We haven't had the resources to
create such a test suite, though Rob took a shot at it at some point.
-Greg
From: Rob Guglielmetti <rob.guglielme...@gmail.com>
Subject: Re: [Radiance-dev] Release numbering
Date: March 12, 2016 6:44:54 PM PST
Actually, the release candidate is the ‘b’ release in Greg’s parlance,
which usually exists for a very brief time. If you look at the GitHub
releases, we did make a 4.2.b.0, followed a week later by the official
4.2 release. It’s my understanding the HEAD is considered alpha code
until right before an official release. Radiance major versions are
released when they are released, generally following a brief period
where it’s considered ‘beta’, and then the version goes back to ‘a’ in
the HEAD and stays there until shortly before the next major release,
again to ‘b’, and then, new version number. Twas ever thus.
At NREL we hang an additional identifier after the ‘a’ on each of our
“releases” (which are really just snapshots, or tags), just to keep
them in order. It’s not true semantic versioning, but it’s a way for
us to keep track of which tag we’re using with which OpenStudio
releases and whatnot. It can be confusing because our v5.0.a.5
predates the official release v5.0, which itself predates the latest
set of packages we made: v5.0.a.8.
On Mar 12, 2016, at 7:24 PM, Randolph M. Fritz <rmfri...@gmail.com>
wrote:
It occurs to me that what we are calling a "release," the rest of the
world has taken to calling a "release candidate" -- the version that
is almost complete, but still has a couple of annoying bugs hiding
that users identify as soon as they put it into service. Perhaps this
would be a convention we could adopt.
--
Randolph
_______________________________________________
Radiance-dev mailing list
Radiance-dev@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-dev
--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/
_______________________________________________
Radiance-dev mailing list
Radiance-dev@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-dev
