Re: [Rd] Test suite failures in R-devel_2017-02-25_r72256

[Peter Simons]

Hi Martin,

 > Is it necessary to also run the 'make check' part in that restricted
 > environment? Or could that ('checking") not get more priviledges?

in NixOS, there is no way to run any kind of build process in an
environment that has network access. On one hand, that is a security
consideration, but the even more compelling reason for that choice is
that NixOS goes to great lengths to guarantee deterministic builds, and
a build process that connects to servers on the Internet and uses data
they provides is as non-deterministic as it can be, unfortunately.

What we can do for the 3.4.0 release is to extend our build environment
for R to ensure that all required resources are downloaded and made
available locally before the build starts.

 > Note that you can only run  "make check" if you don't install
 > recommended packages, whereas more thorough testing would
 > include
 >  make check-devel
 > or even
 >  make check-all
 >
 > but these do have quite a bit more requirements including recommended
 > packages being present.

Thank you for pointing that out. We do offer our users the choice of
whether they want the recommended packages included or not (the default
choice being to not include them), so we should in fact parameterize our
build to run the extended test suite if the recommended packages are
built.

Another option would be to always build the recommended packages, but to
not install them if the user doesn't want them included. I'm not sure,
though, whether the R build system supports such a type of build easily?

Best regards,
Peter

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Test suite failures in R-devel_2017-02-25_r72256

[Prof Brian Ripley]

On 27/02/2017 09:30, Peter Simons wrote:

Hi,

I tried compiling the latest pre-release for R 3.3.3 for the NixOS Linux
distribution [1], but the build fails during the "make check" phase
because of the following 2 issues:


Hmm, R-devel (your subject line) is not a pre-release of R 3.3.3: it is 
'R Under development' for what is planned as R 3.4.0.  Pre-release 
tarballs for 3.3.3 are things like R-rc_2017-02-26_r72260.tar.gz (and 
were previously labelled R-beta).


Your first point only occurs in R-devel, and was something already 
reported and under investigation.  Note that test does not actually 
depend on network access: it depends on having an accessible CRAN 
mirror.  The latter could be local (and is on the CRAN check farm, for 
example).  So we have been working on a more sophisticated condition to 
run that test.


There are lots of possible check environments: it is very time-consuming 
to check them all and reasonable coverage is only attempted once a 
pre-release reaches 'alpha' status -- 3.4.0 has not done so but is 
planned to be in late March.


I have just re-checked on Linux, and R 3.3.3 RC passed its checks 
without Internet access (which is checked for where needed).


I also re-checked 3.3.3 RC without recommended packages and got the 
expected messages about incomplete testing but no failures except in 
'make check-recommended' (expected!).


Our posting guide does ask you to include the output of sessionInfo(): 
that would have avoided the version confusion (and might even have 
alerted you to it).




1) The "tools" test in "tests/Examples" requires network access, which
   it doesn't have in our build environment. Therefore, it fails as
   follows according to "tools-Ex.Rout.fail":

   | [...]
   | > set.seed(11)
   | > ## End(Don't show)
   | > pdb <- CRAN_package_db()
   | Warning in url(sprintf("%s/%s", cran, path), open = "rb") :
   |   URL 'http://CRAN.R-project.org/web/packages/packages.rds': status was 
'Couldn't resolve host name'
   | Error in url(sprintf("%s/%s", cran, path), open = "rb") :
   |   cannot open the connection to 
'http://CRAN.R-project.org/web/packages/packages.rds'
   | Calls: CRAN_package_db -> as.data.frame -> read_CRAN_object -> gzcon -> url
   | Execution halted

   I'm wondering whether it would be possible to extend the test suite
   with a configure-time flag that disable tests which depend on network
   access? My experience is that most modern Linux distributions run
   their builds in a restricted environment and therefore will run into
   trouble if the suite assumes that it can access the Internet.

2) When R is compiled with the --without-recommended-packages flag
   (which is our preferred configuration), the "base" test in
   "tests/Examples" fails, apparently because it depends on MASS. I'm
   citing from "base-Ex.Rout.fail":

   | >  ## The string "foo" and the symbol 'foo' can be used interchangably 
here:
   | >  stopifnot( identical(isNamespaceLoaded(  "foo"   ), FALSE),
   | + identical(isNamespaceLoaded(quote(foo)), FALSE),
   | + identical(isNamespaceLoaded(quote(stats)), statL))
   | >
   | > hasM <- isNamespaceLoaded("MASS") # (to restore if needed)
   | > Mns <- asNamespace("MASS") # loads it if not already
   | Error in loadNamespace(name) : there is no package called 'MASS'
   | Calls: asNamespace ... tryCatch -> tryCatchList -> tryCatchOne -> 

   | Execution halted

I hope this helps!

Best regards,
Peter



[1] http://nixos.org/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel




--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Emeritus Professor of Applied Statistics, University of Oxford

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Test suite failures in R-devel_2017-02-25_r72256

[Martin Maechler]

> Martin Maechler 
> on Mon, 27 Feb 2017 16:08:40 +0100 writes:

> Peter Simons 
> on Mon, 27 Feb 2017 10:30:39 +0100 writes:

>> Hi, I tried compiling the latest pre-release for R 3.3.3
>> for the NixOS Linux distribution [1], but the build fails
>> during the "make check" phase because of the following 2
>> issues:

>> 1) The "tools" test in "tests/Examples" requires network
>> access, which it doesn't have in our build
>> environment. 

> One may argue that the 'make check' (or even 'make check-all')
> tests could / should be allowed more resources than the pure
> build environment.

>> Therefore, it fails as follows according to
>> "tools-Ex.Rout.fail":

>> | [...]
>> | > set.seed(11)
>> | > ## End(Don't show)
>> | > pdb <- CRAN_package_db()
>> | Warning in url(sprintf("%s/%s", cran, path), open = "rb") :
>> |   URL 'http://CRAN.R-project.org/web/packages/packages.rds': status 
was 'Couldn't resolve host name'
>> | Error in url(sprintf("%s/%s", cran, path), open = "rb") :
>> |   cannot open the connection to 
'http://CRAN.R-project.org/web/packages/packages.rds'
>> | Calls: CRAN_package_db -> as.data.frame -> read_CRAN_object -> gzcon 
-> url
>> | Execution halted

>> I'm wondering whether it would be possible to extend the test suite
>> with a configure-time flag that disable tests which depend on network
>> access? My experience is that most modern Linux distributions run
>> their builds in a restricted environment and therefore will run into
>> trouble if the suite assumes that it can access the Internet.

> [see above] Is it necessary to also run the 'make check' part in
> that restricted environment?  Or could that ('checking") not get
> more priviledges?

> Note that you can only run  "make check" if you don't install
> recommended packages, whereas more thorough testing would
> include
>  make check-devel
> or even
>  make check-all
>
> but these do have quite a bit more requirements including
> recommended packages being present.

I have to correct myself:  The above paragraph may be misleading:

Much, if not all of
  make check-devel
and   make check-all
have worked well since R version 3.1.0  which had in its NEWS an entry

 • More of 'make check' will work if recommended packages are not
   installed: but recommended packages remain needed for thorough
   checking of an R build.

Further, the 'R-admin' manual (on 'make check-all' etc) contains

 |  Note that these checks are only run completely 
 |  if the recommended packages are installed.

so their presence is not required  but much recommended for
thorough testing.

Martin

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Test suite failures in R-devel_2017-02-25_r72256

[Martin Maechler]

> Peter Simons 
> on Mon, 27 Feb 2017 10:30:39 +0100 writes:

> Hi, I tried compiling the latest pre-release for R 3.3.3
> for the NixOS Linux distribution [1], but the build fails
> during the "make check" phase because of the following 2
> issues:

> 1) The "tools" test in "tests/Examples" requires network
> access, which it doesn't have in our build
> environment. 

One may argue that the 'make check' (or even 'make check-all')
tests could / should be allowed more resources than the pure
build environment.

> Therefore, it fails as follows according to
> "tools-Ex.Rout.fail":

> | [...]
> | > set.seed(11)
> | > ## End(Don't show)
> | > pdb <- CRAN_package_db()
> | Warning in url(sprintf("%s/%s", cran, path), open = "rb") :
> |   URL 'http://CRAN.R-project.org/web/packages/packages.rds': status was 
'Couldn't resolve host name'
> | Error in url(sprintf("%s/%s", cran, path), open = "rb") :
> |   cannot open the connection to 
'http://CRAN.R-project.org/web/packages/packages.rds'
> | Calls: CRAN_package_db -> as.data.frame -> read_CRAN_object -> gzcon -> 
url
> | Execution halted

> I'm wondering whether it would be possible to extend the test suite
> with a configure-time flag that disable tests which depend on network
> access? My experience is that most modern Linux distributions run
> their builds in a restricted environment and therefore will run into
> trouble if the suite assumes that it can access the Internet.

[see above] Is it necessary to also run the 'make check' part in
that restricted environment?  Or could that ('checking") not get
more priviledges?

Note that you can only run  "make check" if you don't install
recommended packages, whereas more thorough testing would
include
make check-devel
or even
make check-all

but these do have quite a bit more requirements including
recommended packages being present.


> 2) When R is compiled with the --without-recommended-packages flag
> (which is our preferred configuration), the "base" test in
> "tests/Examples" fails, apparently because it depends on MASS. I'm
> citing from "base-Ex.Rout.fail":

> | >  ## The string "foo" and the symbol 'foo' can be used interchangably 
here:
> | >  stopifnot( identical(isNamespaceLoaded(  "foo"   ), FALSE),
> | + identical(isNamespaceLoaded(quote(foo)), FALSE),
> | + identical(isNamespaceLoaded(quote(stats)), statL))
> | >
> | > hasM <- isNamespaceLoaded("MASS") # (to restore if needed)
> | > Mns <- asNamespace("MASS") # loads it if not already
> | Error in loadNamespace(name) : there is no package called 'MASS'
> | Calls: asNamespace ... tryCatch -> tryCatchList -> tryCatchOne -> 

> | Execution halted

Yes, that example should not have assumed a recommended package
to be available unconditionally.

I've changed it, thank you!


> I hope this helps!

> Best regards,
> Peter



> [1] http://nixos.org/

> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Test suite failures in R-devel_2017-02-25_r72256

[Peter Simons]

Hi,

I tried compiling the latest pre-release for R 3.3.3 for the NixOS Linux
distribution [1], but the build fails during the "make check" phase
because of the following 2 issues:

1) The "tools" test in "tests/Examples" requires network access, which
   it doesn't have in our build environment. Therefore, it fails as
   follows according to "tools-Ex.Rout.fail":

   | [...]
   | > set.seed(11)
   | > ## End(Don't show)
   | > pdb <- CRAN_package_db()
   | Warning in url(sprintf("%s/%s", cran, path), open = "rb") :
   |   URL 'http://CRAN.R-project.org/web/packages/packages.rds': status was 
'Couldn't resolve host name'
   | Error in url(sprintf("%s/%s", cran, path), open = "rb") :
   |   cannot open the connection to 
'http://CRAN.R-project.org/web/packages/packages.rds'
   | Calls: CRAN_package_db -> as.data.frame -> read_CRAN_object -> gzcon -> url
   | Execution halted

   I'm wondering whether it would be possible to extend the test suite
   with a configure-time flag that disable tests which depend on network
   access? My experience is that most modern Linux distributions run
   their builds in a restricted environment and therefore will run into
   trouble if the suite assumes that it can access the Internet.

2) When R is compiled with the --without-recommended-packages flag
   (which is our preferred configuration), the "base" test in
   "tests/Examples" fails, apparently because it depends on MASS. I'm
   citing from "base-Ex.Rout.fail":

   | >  ## The string "foo" and the symbol 'foo' can be used interchangably 
here:
   | >  stopifnot( identical(isNamespaceLoaded(  "foo"   ), FALSE),
   | + identical(isNamespaceLoaded(quote(foo)), FALSE),
   | + identical(isNamespaceLoaded(quote(stats)), statL))
   | >
   | > hasM <- isNamespaceLoaded("MASS") # (to restore if needed)
   | > Mns <- asNamespace("MASS") # loads it if not already
   | Error in loadNamespace(name) : there is no package called 'MASS'
   | Calls: asNamespace ... tryCatch -> tryCatchList -> tryCatchOne -> 

   | Execution halted

I hope this helps!

Best regards,
Peter



[1] http://nixos.org/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel