[Rd] Help finding source of warnings
I've been implementing a wrapper to the 2011 Fortran version of L-BFGS-B. In optim(), R uses a C translation of a Fortran version (the version number does not appear to be documented by the original authors). The authors of the original Fortran code have updated it and published the reasons in ACM TOMS due to inefficiencies and a bug. In running the checks on the resulting package (which is on R-forge under the optimizer project), I'm getting a number of warning messages of the type Warning in file.copy(file.path(.Library, pkg, DESCRIPTION), pd) : problem copying /usr/lib/R/library/mgcv/DESCRIPTION to /tmp/Rtmp0kkeHo/RLIBS_1214765d1c5f/mgcv/DESCRIPTION: No such file or directory which reference DESCRIPTIONs for a number of packages other than the one being checked -- here mgcv -- and which are not referenced in my package as far as I can determine. Possibly unrelated, when I run the code on a problem, it works for one run, then gives a NAMESPACE error and failure on the second try. Apart from this, checks and unit tests appear to work correctly. Does anyone have pointers where I might find some ideas on the origin of the issue(s)? I suspect the warning messages are not particularly indicative of the source of the warnings, but that I have some subtle glitch in the setup and call to the Fortran. I suspect this is not platform dependent, but I'm running Linux Mint 17.1 (ubuntu derivative), and R 3.1.2. Cheers, JN __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Help finding source of warnings
Prof J C Nash (U30A) writes: I've been implementing a wrapper to the 2011 Fortran version of L-BFGS-B. In optim(), R uses a C translation of a Fortran version (the version number does not appear to be documented by the original authors). The authors of the original Fortran code have updated it and published the reasons in ACM TOMS due to inefficiencies and a bug. In running the checks on the resulting package (which is on R-forge under the optimizer project), I'm getting a number of warning messages of the type Warning in file.copy(file.path(.Library, pkg, DESCRIPTION), pd) : problem copying /usr/lib/R/library/mgcv/DESCRIPTION to /tmp/Rtmp0kkeHo/RLIBS_1214765d1c5f/mgcv/DESCRIPTION: No such file or directory which reference DESCRIPTIONs for a number of packages other than the one being checked -- here mgcv -- and which are not referenced in my package as far as I can determine. Possibly unrelated, when I run the code on a problem, it works for one run, then gives a NAMESPACE error and failure on the second try. Apart from this, checks and unit tests appear to work correctly. Does anyone have pointers where I might find some ideas on the origin of the issue(s)? I suspect the warning messages are not particularly indicative of the source of the warnings, but that I have some subtle glitch in the setup and call to the Fortran. I suspect this is not platform dependent, but I'm running Linux Mint 17.1 (ubuntu derivative), and R 3.1.2. John: maybe you did not install the recommended packages? (On Debian, the corresponding package would be r-recommended.) Best -k Cheers, JN __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Help finding source of warnings
On 18 January 2015 at 09:03, Prof J C Nash (U30A) wrote: | I've been implementing a wrapper to the 2011 Fortran version of | L-BFGS-B. In optim(), R uses a C translation of a Fortran version (the | version number does not appear to be documented by the original | authors). The authors of the original Fortran code have updated it and | published the reasons in ACM TOMS due to inefficiencies and a bug. | | In running the checks on the resulting package (which is on R-forge | under the optimizer project), I'm getting a number of warning messages | of the type If you were so kind to share the __name of the subpackage__ you fail to test, one could attempt to help you. | Warning in file.copy(file.path(.Library, pkg, DESCRIPTION), pd) : |problem copying /usr/lib/R/library/mgcv/DESCRIPTION to | /tmp/Rtmp0kkeHo/RLIBS_1214765d1c5f/mgcv/DESCRIPTION: No such file or | directory | | which reference DESCRIPTIONs for a number of packages other than the one | being checked -- here mgcv -- and which are not referenced in my package | as far as I can determine. I've seen that when R tries to be too clever by half -- somehow .libPaths() ends up being partial. Following the high-level decision by Hornik, Leisch and Eddelbuettel made circa 2003 in a pub in Vienna, the Debian packages use three entries, and if you have recommended packages installed via apt, R may now pretend they don't exist. I could work on an alternate setup via Github and Travis but as you managed to make this non-reproducible I cannot actually try that ... Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Help finding source of warnings
Kurt pointed to the issue. Thanks. I did install r-recommended, but it seems something went wrong at some point. A reinstall got rid of the warnings. Thanks to Dirk for his offer of help. Now I'm still getting a namespace issue on the second run of an optimization problem. However, I think I need to do some more digging to narrow down where this issue is lurking. It may be some local matter, as with the r-recommended links failing. Best, JN On 15-01-18 09:27 AM, Kurt Hornik wrote: Prof J C Nash (U30A) writes: I've been implementing a wrapper to the 2011 Fortran version of L-BFGS-B. In optim(), R uses a C translation of a Fortran version (the version number does not appear to be documented by the original authors). The authors of the original Fortran code have updated it and published the reasons in ACM TOMS due to inefficiencies and a bug. In running the checks on the resulting package (which is on R-forge under the optimizer project), I'm getting a number of warning messages of the type Warning in file.copy(file.path(.Library, pkg, DESCRIPTION), pd) : problem copying /usr/lib/R/library/mgcv/DESCRIPTION to /tmp/Rtmp0kkeHo/RLIBS_1214765d1c5f/mgcv/DESCRIPTION: No such file or directory which reference DESCRIPTIONs for a number of packages other than the one being checked -- here mgcv -- and which are not referenced in my package as far as I can determine. Possibly unrelated, when I run the code on a problem, it works for one run, then gives a NAMESPACE error and failure on the second try. Apart from this, checks and unit tests appear to work correctly. Does anyone have pointers where I might find some ideas on the origin of the issue(s)? I suspect the warning messages are not particularly indicative of the source of the warnings, but that I have some subtle glitch in the setup and call to the Fortran. I suspect this is not platform dependent, but I'm running Linux Mint 17.1 (ubuntu derivative), and R 3.1.2. John: maybe you did not install the recommended packages? (On Debian, the corresponding package would be r-recommended.) Best -k Cheers, JN __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] [Resolved] (Was: Request for help with UBSAN and total absense of CRAN) response
I would like to express my deepest gratitude to the CRAN Maintainers for keeping up with their perfect record of never responding to questions from mere mortals like myself. The ensuing utter and complete silence allowed me to work with a much higher concentration level. More seriously, I owe a big thank you to Martyn Plummer and Jan van der Laan for replying here (and in off-list follow-ups), and to Jeff Horner who offered help via Twitter and off-list emails. Thanks to their help, I did get to the bottom of this, and it seems that part of (either mine only, or a more general) confusion was that I wanted the test to actually _abort_ on error. Which, as I found after too many failed tries, requires also setting the -fno-sanitize-recover option as well. Long story short, we now have a working appliance to test this via Docker. This is building on some of the examples I showed at the end of my useR! 2014 presentation, the work Carl and I have been doing in the Rocker.org repo for Docker containers for R, some experimentation, and an only-at-GitHub-yet new script for littler. I hope to blog about this in some more detail, but if you have Docker set-up, pull the rocker/r-devel-ubsan-clang container. Then: i) to replicate the reported error: docker run --rm -ti -v $(pwd):/mnt rocker/r-devel-ubsan-clang \ check.r --setwd /mnt --install-deps RcppAnnoy_0.0.5.tar.gz which runs the container with - post-run cleanup (--rm), - terminal and interactive mode (-ti), - the current directory mounted as /mnt in the container (-v ...), - invoking the check.r script (from littler as on GitHub, in the path in the container) with options to install package dependencies - the known-bad tarball ii) to see a fix not triggering it docker run --rm -ti -v $(pwd):/mnt rocker/r-devel-ubsan-clang \ check.r --setwd /mnt --install-deps RcppAnnoy_0.0.5.1.tar.gz as above but using an updated tarball not showing the error. This is generic: you can plug any of your R package tarballs in there and it will run the test. [ This assumes dependencies can be satisfied via install.packages(); need to add a hook for apt-get as needed etc pp ] The UBSAN config I settled on for now is close to what Martyn had sent me (in off-list follow-up): CC=clang-3.5 -fsanitize=undefined \ -fno-sanitize=float-divide-by-zero,vptr,function -fno-sanitize-recover CXX=clang++-3.5 -fsanitize=undefined \ -fno-sanitize=float-divide-by-zero,vptr,function -fno-sanitize-recover CXX1X=clang++-3.5 -fsanitize=undefined \ -fno-sanitize=float-divide-by-zero,vptr,function -fno-sanitize-recover We may want to turn on other options; Jeff Horner sent me his config which has a lot more goodies enabled. I'd welcome follow-up by anyone interested in working on this, either by (off-list) email or directly over at the GitHub repo for this container: https://github.com/rocker-org/r-devel-san-clang Thanks, Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
