Re: [Rd] Alpha and Beta testing of R versions
On Nov 4, 2005, at 6:58 AM, Prof Brian Ripley wrote:
> Martin's point is generally very valid, but in the case of the
> 2.2.0 release remarkably few of the bugs found since release were
> new in 2.2.0.
> One thing we have learnt is that none of the testers seem to look
> at HTML help (which accounts for 2 of the 4 2.2.0-only bugs I
> counted).
>
> What we need most is persistent help in testing each release,
> especially on unusual platforms. How do we `incentivize' that?
I suspect that in the particular case of OS X the problem was
probably visibility - it was the first time ever that nightly OS X
binaries were available during alpha/beta phase (afaict), but I'm not
sure how many people knew about it. I think posted about it on R-SIG-
Mac during some discussion, but maybe I should have announced it more
specifically somewhere. I'm, not even sure whether there was a link
from the main page on CRAN. I would think that OS X users are more
likely to rely on binaries, so the above is more relevant than on
other platforms.
>> - being listed as helpful person in R's 'THANKS' file
>> {but that may not entice those who are already listed},
>> or even in the NEWS of the new relase
>> or on the "Hall of fame of R beta testers"
The latter sounds good to me, although I'm not sure how many of our
users are striving for fame ;).
Cheers,
Simon
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Alpha and Beta testing of R versions
Martin's point is generally very valid, but in the case of the 2.2.0
release remarkably few of the bugs found since release were new in 2.2.0.
One thing we have learnt is that none of the testers seem to look at HTML
help (which accounts for 2 of the 4 2.2.0-only bugs I counted).
What we need most is persistent help in testing each release, especially
on unusual platforms. How do we `incentivize' that?
On Fri, 4 Nov 2005, Martin Maechler wrote:
> [Mainly for R-foundation members; but kept in public for general
> brainstorming...]
>
>> "Simon" == Simon Urbanek <[EMAIL PROTECTED]>
>> on Thu, 3 Nov 2005 12:16:25 -0500 writes:
>
> <>
>
>Simon> As Brian was saying, the error was fixed in R
>Simon> immediately after the release - strangely enough no
>Simon> one reported the error during the alpha and beta
>Simon> cycle although both the GUI and R binaries were
>Simon> available for download :(.
>
> Unfortunately, the phrase "strangely enough" could be replaced with
> ``as almost always''.
>
> Maybe we (the R-foundation) should give serious thoughts to
> offer prizes for valid bug reports during alpha and beta
> testing. These could include
> - Reduced fee for 'useR' and 'DSC' conferences
> - being listed as helpful person in R's 'THANKS' file
> {but that may not entice those who are already listed},
> or even in the NEWS of the new relase
> or on the "Hall of fame of R beta testers"
>
> In order to discourage an increased number of non-bug reports we
> may have to also open a "hall of shame" though...
>
> Martin
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Alpha and Beta testing of R versions
[...]
> Maybe we (the R-foundation) should give serious thoughts to
> offer prizes for valid bug reports during alpha and beta
> testing. These could include
> - Reduced fee for 'useR' and 'DSC' conferences
> - being listed as helpful person in R's 'THANKS' file
> {but that may not entice those who are already listed},
> or even in the NEWS of the new relase
> or on the "Hall of fame of R beta testers"
... formalized as a bug finding contest, for example?
-d
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Alpha and Beta testing of R versions
[Mainly for R-foundation members; but kept in public for general
brainstorming...]
> "Simon" == Simon Urbanek <[EMAIL PROTECTED]>
> on Thu, 3 Nov 2005 12:16:25 -0500 writes:
<>
Simon> As Brian was saying, the error was fixed in R
Simon> immediately after the release - strangely enough no
Simon> one reported the error during the alpha and beta
Simon> cycle although both the GUI and R binaries were
Simon> available for download :(.
Unfortunately, the phrase "strangely enough" could be replaced with
``as almost always''.
Maybe we (the R-foundation) should give serious thoughts to
offer prizes for valid bug reports during alpha and beta
testing. These could include
- Reduced fee for 'useR' and 'DSC' conferences
- being listed as helpful person in R's 'THANKS' file
{but that may not entice those who are already listed},
or even in the NEWS of the new relase
or on the "Hall of fame of R beta testers"
In order to discourage an increased number of non-bug reports we
may have to also open a "hall of shame" though...
Martin
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
