Re: [R-pkg-devel] Submission to CRAN when package needs personal data (API key)

[Duncan Murdoch]

On 06/09/2018 10:32 AM, Hadley Wickham wrote:

On Wed, Sep 5, 2018 at 3:03 PM Duncan Murdoch  wrote:


On 05/09/2018 2:20 PM, Henrik Bengtsson wrote:

I take a complementary approach; I condition on, my home-made,
R_TEST_ALL variable.  Effectively, I do:

if (as.logical(Sys.getenv("R_TEST_ALL", "FALSE"))) {
 ...
}

and set R_TEST_ALL=TRUE when I want to run that part of the test.  You
can also imagine refined versions of this, e.g. R_TEST_SETS=foo,bar
and test scripts with:

if ("foo" %in% strsplit(Sys.getenv("R_TEST_SETS"), split="[, ]+")[[1]]) {
 ...makes no assumption
}

That avoids making assumptions on where the tests are submitted/run,
may it be CRAN, Bioconductor, Travis CI, ...


This is the right way to do it.


I would like to gently push back on this assertion: if CRAN set an
environment variable we would have one single convention that all
packages could rely on.


When packages delete tests just for CRAN, the quality of the repository 
suffers.  Users should be able to check an install by running the tests 
that passed on CRAN and seeing them pass on their system as well.


 The current system relies on each package

author evolving their own solution. This makes life difficult when you
are running local reverse dependency checks: there is no way to
systematically assert that you want to run tests in a way as similar
as possible to CRAN.


Most packages don't need to evolve anything:  the CRAN tests are sufficient.


I know that the CRAN maintainers already have a very large workload,
and I hate to add to it, but setting CRAN=1 in a few profile files
doesn't seem excessively burdensome.


It would be easy to do that, but then CRAN wouldn't be testing the same 
things that users would test.  A user might see a test failure that 
didn't happen on CRAN, and suspect that there was something wrong with 
their install, when in fact it was an author trying to hide a deficiency 
in their package from CRAN.






This discussion has come up before.  If you want to submit to CRAN, you
should include tests that satisfy their requests.  If you want even more
tests, there are several ways to add them in addition to the CRAN tests.
   Henrik's is one, "R CMD check --test-dir=myCustomTests" is another.

Rainer's package is unusual, in that from his description it can't
really work unless the user obtains an API key.  There are other
packages like that, and those cases need manual handling by CRAN:  they
don't really run full tests by default.  But the vast majority of
packages should be able to live within the CRAN guidelines.


10 years ago, I would have definitely supported this statement. But I
am not sure it is still correct today, as there are now many packages
that require a connection to web API to work (or depend on a package
that uses an API). Additionally, CRAN only allows a limited amount of
compute time for each check, so there are often longer tests that you
want to run locally but not on CRAN. CRAN is a specialised testing
service and it does have different constraints to your local machine,
travis, and bioconductor.

A quick search of the CRAN mirror on github
(https://github.com/search?q=org%3Acran+skip_on_cran=Code)
reveals that there are ~2700 tests that use testthat::skip_on_cran().
This is obviously an underestimate of the total number of tests
skipped on CRAN, as many packages don't use testthat, or use an
alternative technique to avoid running code on CRAN.


That's not so obviously an underestimate, as packages that use that 
technique use it many times, not just once per package.  (A sample I 
looked at averaged 15 calls per package, but I don't know if that's 
unbiased.)


But in any case, the skip_on_cran() function implements a version of 
Henrik's approach.  The name of the function is misleading, it doesn't 
attempt to distinguish between CRAN and a regular user.


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] [FORGED] Re: Recommendations about adding options to a package in order to change default values of some functions on-the-fly

[Rolf Turner]

On 09/07/2018 04:15 AM, Alexandre Courtiol wrote:




you could:

1. directly write and then read elements in the (hidden) list .Options that
is present in the global environment:




Point of order Mr. Chairman (and I'm pretty sure it doesn't matter a 
damn) but (on my system at least) .Options is created in the *base* 
package, not in the global environment.


E.g.:

# Freshly started R session.
> find(".Options")
[1] "package:base"
> options(mung="gorp")
> find(".Options")
[1] "package:base"
> options("mung")
$mung
[1] "gorp"
> .Options$mung
[1] "gorp"

cheers,

Rolf Turner

--
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] questions about \dontrun{} etc.

[Ben Bolker]

  {Sorry this is long, just trying to be clear and thorough.}

  I'm struggling a bit with some CRAN recommendations; would appreciate
opinions/suggestions about how other people would handle this case.

  I'm working on a package (https://github.com/bbolker/broom.mixed) that
processes model fits from lots of other packages.  The time taken to fit
the original model varies a lot, from a few seconds to several minutes.

 My typical framework for the examples looks something like this:

if (require("lme4")) {
   \dontrun{
  lmm1 <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
   }
   load(system.file("extdata","lme4_example.rda",package="broom.mixed"))
   < stuff ...>
}

or sometimes the \dontrun{}  would be replaced by a commented-out code
line instead,

   ## lmm1 <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)

 The purpose of these lines is to show readers what model was fitted to
produce the stored model object that's going to be processed.

  CRAN responded:

===
Please replace \dontrun{} by \donttest{} or unwap [sic] the examples if
they can be executed in less than 5 sec per Rd-file.

Please do not comment out examples.
===

I see that the deprecation of (¿ too strong; "preference for avoiding"
?) \dontrun{} was previously discussed here:
https://stat.ethz.ch/pipermail/r-package-devel/2018q2/002811.html : Uwe
Ligges responded

> Sure, \dontrun{} markup is intended for cases where neither automated
chercks nor users can expect that the given code work out of the box
and, e.g., changes are needed to make it work, e.g. isertion of otehr
useranmes ... while \donttest{} is expected to work out of the box
(perhaps code that only work interactively or excluded in order to omit
ling running examples from the checks). Nothing new here.

   My desiderata:

1. make CRAN happy
2. consistent format across examples
3. avoid having users run the long examples, even by accident; I
personally would find it a little frustrating if a cut-and-paste to try
out the examples resulted in a 5-minute run time ...
4. readability
5. not letting examples get out of sync, e.g. because of downstream
package changes [I do have code for updating all of the fitted examples
at once, but it's not run automatically]

 * My preference would be for using \dontrun{} universally (consistent,
readable, mostly avoids accidental runs), but this goes against
(informal?) CRAN policy

 * universal \donttest{} doesn't seem to do what I want, since I *do*
want to avoid users having to run long examples

  * commenting (with something like "original model:" to make it clearer
that it's not a commented-out example) is even better for avoiding
accidental runs (harder to accidentally cut-and-paste, if doing that
rather than using example()), but less readable.

  * putting slow model fits in \dontrun{} and letting fast model fits be
executed does what CRAN asks and helps avoiding example-drift, but is
less consistent -- and I can't be sure that a model fit won't take much
longer on one of CRAN's test machines ...

  Thoughts?

  cheers
   Ben Bolker

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Recommendations about adding options to a package in order to change default values of some functions on-the-fly

[David Hugh-Jones]

Hi all,

A simple solution - if indeed you want to go down this route - is to use
options() and getOption(), ensuring all options are namespaced, e.g. by
prefixing them with the package name.

David

On Thu, 6 Sep 2018 at 17:15, Alexandre Courtiol <
alexandre.court...@gmail.com> wrote:

> Dear Samuel,
>
> Many may object (for good reasons) that adding options would clash with
> functional paradigm, but it is possible.
> I don't know about the least worse practice but as far as I would do it,
> you could:
>
> 1. directly write and then read elements in the (hidden) list .Options that
> is present in the global environment:
> options(test = "blabla") ## add element in .Options
> .Options$test ## read element (for use in your own function)
>
> 2. you could also create your own alternative to options(), as ex:
> https://github.com/courtiol/IsoriX/blob/master/IsoriX/R/options.R
> Option 2 has the benefits of not risking conflict with existing names in
> .Options and it would thus also not mess up the way other packages may
> work.
> Option 1 uses what is already in place.
>
> I hope this help,
> Best
>
> Alex
>
> On Thu, 6 Sep 2018 at 17:18, Samuel  wrote:
>
> > Hi,
> >
> > I would like to change the default value of some arguments of some
> > functions in a package of mine. I don't want to change explicitly the
> > calls in the many scripts that have been written. For example, I would
> > to change the delimiter in all write.mytable() without changing any
> > calls it and without changing the default value currently assigned to
> > that delimiter in the declaration of this function. What I would like is
> > to add a command at the beginning of the scripts that says "since now,
> > the delimiter is ;".
> >
> > I am thinking about setting global options that those functions will
> > read and use to overwrite the default values. Of course the
> > write.mytable function has to be modify to exploit these settings, but
> > none of the scripts. I think about something similar to the par() and
> > options() functions.
> >
> > I noticed the settings package, but I would like to know if there is a
> > simple and recommended solution without adding any dependency.
> >
> > Hope my question is clear.
> >
> > Thanks for any advice,
> > Samuel
> >
> > __
> > R-package-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >
>
>
> --
> Alexandre Courtiol
>
> http://sites.google.com/site/alexandrecourtiol/home
>
> *"Science is the belief in the ignorance of experts"*, R. Feynman
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Recommendations about adding options to a package in order to change default values of some functions on-the-fly

[Alexandre Courtiol]

Dear Samuel,

Many may object (for good reasons) that adding options would clash with
functional paradigm, but it is possible.
I don't know about the least worse practice but as far as I would do it,
you could:

1. directly write and then read elements in the (hidden) list .Options that
is present in the global environment:
options(test = "blabla") ## add element in .Options
.Options$test ## read element (for use in your own function)

2. you could also create your own alternative to options(), as ex:
https://github.com/courtiol/IsoriX/blob/master/IsoriX/R/options.R
Option 2 has the benefits of not risking conflict with existing names in
.Options and it would thus also not mess up the way other packages may work.
Option 1 uses what is already in place.

I hope this help,
Best

Alex

On Thu, 6 Sep 2018 at 17:18, Samuel  wrote:

> Hi,
>
> I would like to change the default value of some arguments of some
> functions in a package of mine. I don't want to change explicitly the
> calls in the many scripts that have been written. For example, I would
> to change the delimiter in all write.mytable() without changing any
> calls it and without changing the default value currently assigned to
> that delimiter in the declaration of this function. What I would like is
> to add a command at the beginning of the scripts that says "since now,
> the delimiter is ;".
>
> I am thinking about setting global options that those functions will
> read and use to overwrite the default values. Of course the
> write.mytable function has to be modify to exploit these settings, but
> none of the scripts. I think about something similar to the par() and
> options() functions.
>
> I noticed the settings package, but I would like to know if there is a
> simple and recommended solution without adding any dependency.
>
> Hope my question is clear.
>
> Thanks for any advice,
> Samuel
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>


-- 
Alexandre Courtiol

http://sites.google.com/site/alexandrecourtiol/home

*"Science is the belief in the ignorance of experts"*, R. Feynman

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] Recommendations about adding options to a package in order to change default values of some functions on-the-fly

[Samuel]

Hi,

I would like to change the default value of some arguments of some 
functions in a package of mine. I don't want to change explicitly the 
calls in the many scripts that have been written. For example, I would 
to change the delimiter in all write.mytable() without changing any 
calls it and without changing the default value currently assigned to 
that delimiter in the declaration of this function. What I would like is 
to add a command at the beginning of the scripts that says "since now, 
the delimiter is ;".


I am thinking about setting global options that those functions will 
read and use to overwrite the default values. Of course the 
write.mytable function has to be modify to exploit these settings, but 
none of the scripts. I think about something similar to the par() and 
options() functions.


I noticed the settings package, but I would like to know if there is a 
simple and recommended solution without adding any dependency.


Hope my question is clear.

Thanks for any advice,
Samuel

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Submission to CRAN when package needs personal data (API key)

[Hadley Wickham]

On Wed, Sep 5, 2018 at 3:03 PM Duncan Murdoch  wrote:
>
> On 05/09/2018 2:20 PM, Henrik Bengtsson wrote:
> > I take a complementary approach; I condition on, my home-made,
> > R_TEST_ALL variable.  Effectively, I do:
> >
> > if (as.logical(Sys.getenv("R_TEST_ALL", "FALSE"))) {
> > ...
> > }
> >
> > and set R_TEST_ALL=TRUE when I want to run that part of the test.  You
> > can also imagine refined versions of this, e.g. R_TEST_SETS=foo,bar
> > and test scripts with:
> >
> > if ("foo" %in% strsplit(Sys.getenv("R_TEST_SETS"), split="[, ]+")[[1]]) {
> > ...makes no assumption
> > }
> >
> > That avoids making assumptions on where the tests are submitted/run,
> > may it be CRAN, Bioconductor, Travis CI, ...
>
> This is the right way to do it.

I would like to gently push back on this assertion: if CRAN set an
environment variable we would have one single convention that all
packages could rely on. The current system relies on each package
author evolving their own solution. This makes life difficult when you
are running local reverse dependency checks: there is no way to
systematically assert that you want to run tests in a way as similar
as possible to CRAN.

I know that the CRAN maintainers already have a very large workload,
and I hate to add to it, but setting CRAN=1 in a few profile files
doesn't seem excessively burdensome.

> This discussion has come up before.  If you want to submit to CRAN, you
> should include tests that satisfy their requests.  If you want even more
> tests, there are several ways to add them in addition to the CRAN tests.
>   Henrik's is one, "R CMD check --test-dir=myCustomTests" is another.
>
> Rainer's package is unusual, in that from his description it can't
> really work unless the user obtains an API key.  There are other
> packages like that, and those cases need manual handling by CRAN:  they
> don't really run full tests by default.  But the vast majority of
> packages should be able to live within the CRAN guidelines.

10 years ago, I would have definitely supported this statement. But I
am not sure it is still correct today, as there are now many packages
that require a connection to web API to work (or depend on a package
that uses an API). Additionally, CRAN only allows a limited amount of
compute time for each check, so there are often longer tests that you
want to run locally but not on CRAN. CRAN is a specialised testing
service and it does have different constraints to your local machine,
travis, and bioconductor.

A quick search of the CRAN mirror on github
(https://github.com/search?q=org%3Acran+skip_on_cran=Code)
reveals that there are ~2700 tests that use testthat::skip_on_cran().
This is obviously an underestimate of the total number of tests
skipped on CRAN, as many packages don't use testthat, or use an
alternative technique to avoid running code on CRAN.

Hadley

-- 
http://hadley.nz

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] Vignette title problem

[Christopher John]

Dear CRAN

I have made a vignette and followed the instructions for adding a title and
name at the top of the vignette, the title and name appears locally when I
build it from RStudio. However, whenever it gets uploaded to CRAN, it is
not there in the online vignette. The rest of the vignette is fine, just
the title is missing.

Code (works locally to make header in RStudio):
https://github.com/cran/clusterlab/blob/master/vignettes/introduction.Rmd

Vignette (without header):
https://cran.r-project.org/web/packages/clusterlab/vignettes/introduction.html

Hopefully some one can help with this problem.

Best wishes,

Chris

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Submission to CRAN when package needs personal data (API key)

[Rainer M Krug]

> On 5 Sep 2018, at 20:20, Henrik Bengtsson  > wrote:
> 
> I take a complementary approach; I condition on, my home-made,
> R_TEST_ALL variable.  Effectively, I do:
> 
> if (as.logical(Sys.getenv("R_TEST_ALL", "FALSE"))) {
>   ...
> }
> 
> and set R_TEST_ALL=TRUE when I want to run that part of the test.  You
> can also imagine refined versions of this, e.g. R_TEST_SETS=foo,bar
> and test scripts with:
> 
> if ("foo" %in% strsplit(Sys.getenv("R_TEST_SETS"), split="[, ]+")[[1]]) {
>   ...makes no assumption
> }
> 
> That avoids making assumptions on where the tests are submitted/run,
> may it be CRAN, Bioconductor, Travis CI, …

Thanks Henrik - that sounds like a good idea. At the moment, I am checking for 
the name of my machine, which is obviously not to robust.

I will implement this before submission to CRAN.

Thanks,

Rainer

> 
> /Henrik
> 
> On Wed, Sep 5, 2018 at 11:08 AM Spencer Graves
>  > wrote:
>> 
>> 
>> 
>> On 2018-09-05 12:52, Gábor Csárdi wrote:
>>> On Wed, Sep 5, 2018 at 6:34 PM Spencer Graves
>>> >> > wrote:
 The fda package disables some tests on CRAN using:
 
 
 if(!fda::CRAN()){
  test you don't want run on CRAN
 }
>>> Seemingly this just checks for a couple of (5 by default) environment
>>> variables that match "^_R_":
>>> https://github.com/cran/fda/blob/0549e6a86aa144d8c2aade2d8704a269fb8a94ed/R/CRAN.R#L14
>>>  
>>> 
>>> 
>>> I am not sure if this is a good indication of the machine being a CRAN
>>> machine, I do set  some of these
>>> occasionally on my machine.  It would be great if CRAN machines could
>>> set a specific environment
>>> variable globally, e.g. ON_CRAN, or _R_ON_CRAN_, so people could skip
>>> these tests and examples
>>> easily.
>> 
>> 
>> Agreed.  Before I wrote that function, I asked this list for
>> suggestions.  The suggestion I got was, "Don't do it."  I felt that was
>> not acceptable so wrote what I did.  I think we need a better solution.
>> Spencer
>>> 
>>> Gabor
>>> 
 
I've used this in other contexts with tests that are too long to
 run on CRAN but that I want to run otherwise during "R CMD check".
 
 
Spencer Graves
 
 
 On 2018-09-05 12:13, Peter Meissner wrote:
> Hey,
> 
> Maybe webmockr (or something alike) can help:
> https://cran.r-project.org/web/packages/webmockr/index.html 
> 
> 
> Best, Peter
> 
> Rainer M Krug  schrieb am Mi., 5. Sep. 2018, 15:38:
> 
>> OK - thanks. I will than disable the checks for CRAN.
>> 
>> Thanks,
>> 
>> Rainer
>> 
>> 
>>> On 5 Sep 2018, at 15:37, Uwe Ligges 
>> wrote:
>>> You should not run on CRAN. Note that the checks are done almost daily
>> oin several platforms, hence the API key is used for about 10 check runs 
>> a
>> day.
>>> Best,
>>> Uwe Ligges
>>> 
>>> 
>>> 
>>> On 05.09.2018 09:28, Rainer Krug wrote:
 Hi
 I have a package at GitHub (https://github.com/rkrug/ROriginStamp)
>> which I am pre[paring for CRAN.
 It creates a trusted timestamp using the API fro OriginStamp (
>> https://originstamp.org/home) which requires an API key. Now this API
>> should not be made public, as to much traffic through one API key will 
>> lead
>> to it’s blocking.
 I have stored the key encrypted in the travis.yml, and the package
>> passes all tests.
 But if I send it to CRAN, it would fail the tests, as the api key is
>> not in the package itself.
 I could disable all tests for CRAN which need the API key, but I think
>> it would be better tu run the tests there as well (as an additional check
>> to travis).
 My question:
 Is there a way of storing the API key encrypted, so that only the CRAN
>> test servers can decrypt it, or is there another way can steal with this?
 Thanks,
 Rainer
 --
 Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
>> Biology, UCT), Dipl. Phys. (Germany)
 University of Zürich
 Cell:   +41 (0)78 630 66 57
 email:  rai...@krugs.de
 Skype:  RMkrug
 PGP: 0x0F52F982
 __
 R-package-devel@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-package-devel
>> --
>> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
>> Biology, UCT), Dipl. Phys. (Germany)
>> 
>> University of Zürich
>> 
>> Cell:   +41 (0)78 630 66 57
>> email:  rai...@krugs.de
>> Skype:  RMkrug
>> 
>> PGP: 0x0F52F982
>>