On Tue, Sep 4, 2012 at 5:32 PM, Duncan Murdoch <murdoch.dun...@gmail.com> wrote: > On 04/09/2012 5:21 PM, John Fox wrote: >> >> Dear all, >> >> I'd like to second this fairly simple request. I currently enclosed some >> of >> the examples in the effects package in \donttest{} blocks to satisfy the >> CRAN timing requirements for examples. It would be nice to have something >> like a \donttestcran{} block that suppresses the tests when --as-cran is >> set >> (and on CRAN itself). >> >> I'm sure that I've missed many of the nuances in this discussion, but this >> seems like a simple solution to me. > > > That would work for examples, but not tests. Many packages have scripts > that are in the tests directory, not just test code in the .Rd files. What > I think you should use is my suggested HowMuchTimeLeftBeforeTimeout() > function, but if that's not writeable, then simply having a TIMELIMIT > environment variable or similar, so your code could see if the there's a > timeout pending.
I don't like this idea, because then I need to think about the order in which the tests are run. You are right that we could simulate this by having an environment variable, and if that variable is set, we could do the full tests. Like Dirk is mentioning, the (sensible, I might add) requirement that the tests should be "quick" suddenly makes the quick testing the default. What I - and I think a lot of other people - would like, is to do this in a standard way so it would be uniform across packages. I don't think it would be good if I start using R_LONG_TESTS and Dirk starts to use __NON_CRAN_TESTS or whatever. Then all of us need to know the package-specific variables when we test other people packages (and that happens occasionally, for example when you have a package that other packages depend upon). In this use case, standardization would be hugely beneficial in my opinion. In my reading of the discussion, this is really what all of us are saying. > > Duncan Murdoch > > >> >> Best, >> John >> >> > -----Original Message----- >> > From: r-devel-boun...@r-project.org [mailto:r-devel-bounces@r- >> > project.org] On Behalf Of Kasper Daniel Hansen >> > Sent: Tuesday, September 04, 2012 5:12 PM >> > To: Warnes, Gregory >> > Cc: Terry Therneau; r-devel@r-project.org >> > Subject: Re: [Rd] if(--as-cran)? >> > >> > On Tue, Sep 4, 2012 at 4:53 PM, Warnes, Gregory >> > <gregory.war...@novartis.com> wrote: >> > > >> > > On 9/4/12 3:58 PM, "Duncan Murdoch" <murdoch.dun...@gmail.com> wrote: >> > > >> > >>On 04/09/2012 3:44 PM, Terry Therneau wrote: >> > >>>ly in >> > >>> On 09/04/2012 01:57 PM, Duncan Murdoch wrote: >> > >>> > On 04/09/2012 2:36 PM, Warnes, Gregory wrote: >> > >>> >> On 9/4/12 8:38 AM, "Duncan Murdoch" <murdoch.dun...@gmail.com> >> > wrote: >> > >>> >> >> > >>> >> >> > >>> >> >On 04/09/2012 8:20 AM, Terry Therneau wrote: >> > >>> >> >> >> > >>> >> >> On 09/04/2012 05:00 AM, M >> > >>><mailto:r-devel-requ...@r-project.org>artin wrote: >> > >>> >> >> > The issue is not just about "CRAN" vs "off CRAN". >> > >>> >> >> > It is good to think about a more general scheme of >> > >>> >> >> > "light testing" vs "normal testing" vs "extensive testing", >> > >>> >> >> > e.g., for the situation where the package implements >> > >>> >> >> > (simulation/bootstrap/ ..) based inference, and the >> > developer >> > >>> >> >> > (but not only) should be able to run the extensive tests. >> > >>> >> >> > >> > >>> >> >> > Martin >> > >>> >> >> >> > >>> >> >> I agree with Martin. A mechanism to specify testing level >> > would >> > >>>be the >> > >>> >> >> best. Then CRAN can choose to set that variable to "3" say, >> > with >> > >>>level >> > >>> >> >> 1 for extensive and 2 for usual. >> > >>>>> >> >> > >>> >> > >>>[snip] >> > > >> > >>The testingLevel() function is supposed to be a way to know that a >> > >>certain level of testing is being done, to allow such tailoring. >> > >>However, I don't think it's practical. I think you can ask whether a >> > >>specific test is being run (my "D" %in% tests() example), but you >> > can't >> > >>reasonably convert the set of tests chosen by a tester into a single >> > >>number. >> > >> >> > >>What I think you and Greg are talking about is something different. >> > You >> > >>are asking that we set up more suites of tests, corresponding to >> > >>numerical levels. Currently we have two suites: the default, and the >> > >>--as-cran suite. But we also have completely customized suites, set >> > by >> > >>users who want to check specific things. They can do that the way you >> > >>do (by calling the tests explicitly), or by setting environment >> > >>variables (as described in the Tools chapter of the R Internals >> > manual). >> > > >> > > No! We're not asking for the r-core to create more test suites, or >> > even >> > > to do anything different based on the test intensity level. >> > > >> > > We're just asking for a standard way to control the intensity of the >> > tests >> > > *we* write to prevent us from duplicating this functionality in our >> > own >> > > packages, probably in incompatible ways. >> > >> > And given that CRAN recently put down timing requirements (and >> > Bioconductor has had them for a long time), it could be extremely >> > useful to have one system. It is not clear to me whether it needs >> > more than 2 levels ("slow" and "fast"), but I'll leave that up to >> > people who have thought longer about this. >> > >> > I could certainly use it in several packages to differentiate between >> > slow and quick tests. >> > >> > Kasper >> > >> > ______________________________________________ >> > R-devel@r-project.org mailing list >> > https://stat.ethz.ch/mailman/listinfo/r-devel >> > ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel