Hi,
I recently provoked some discussion over on the BioConductor developer
list about testing and R CMD check. In brief, the salient points that
arose from the discussion are:
[1] Because of the need for a nightly build of BioConductor, the tests
(in the ./tests directory) of a package that run routinely as part of "R
CMD check" must complete in five minutes.
[2] Nontrivial regression testing of complex algorithms on real data can
easily take longer than five minutes.
[3] Maintaining and improving code that works on various kinds of
"omics" data is greatly facilitated by the inclusion of nontrivial
regression tests.
Of course, points [1] and [3] are incompatible in the current setup.
Both could, however, be accommodated by changing the way "R CMD check"
runs test scripts. There are at least two ways this could be accomplished.
One possibility would be to define a separate directory as part of the
package structure to store the more complex, longer, "deep" tests. That
is, one might have
package/R
package/man
package/tests
package/deeptests [or some better name]
Then the default for "R CMD check" would be to run all the .R scripts in
tests but none of the .R scripts in deep tests. However, one would also
have to add an option, perhaps something like
R CMD check --run-deeptests [package]
to run the more elaborate test scripts.
The second possibility would be to keep all the tests in the same
(existing ./tests) directory but include a standard file in that
directory to indicate which scripts are more complex. This would be a
little trickier to design, since it would be nice to keep the existing
behavior for someone who ignores the new structure. One, could however,
allow for something like
----------------------------
## CONTROLTESTS
complex: test1.R, test3.R
simple: test2.R, test4.R
-----------------------------
The default (in the absence of A CONTROLTESTS file or for scripts in the
directory that are not listed in the file) is to assume "simple", which
would then get the existing behavior without changes to existing packages.
How hard would it be to implement one of these approaches for R CMD check?
-- Kevin
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
