Re: [R-pkg-devel] Unexpected multi-core CPU usage in package tests
On Wed, 28 Feb 2024 at 17:36, Ivan Krylov wrote: > > В Tue, 27 Feb 2024 11:14:19 + > Jon Clayden пишет: > > > My testing route is to install the packages within the > > 'rocker/r-devel' Docker container, which is Debian-based, then use > > 'time' to evaluate CPU usage. Note that, even though 'RNifti' does not > > use OpenMP, setting OMP_NUM_THREADS changes its CPU usage > > I think that's because rocker/r-devel uses parallel OpenBLAS: > > $ podman run --rm -it docker.io/rocker/r-devel \ > R -q -s -e 'sessionInfo()' | grep -A1 BLAS > BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 > LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.24.so; > LAPACK version 3.11.0 > > The incoming CRAN check machine either sets the BLAS parallellism to 1 > or uses a non-parallel BLAS. With rocker/r-devel, you can run R with > the environment variable OPENBLAS_NUM_THREADS set to 1. It's been > effective in the past to run R -d gdb and set a breakpoint on > pthread_create before launching the test. (In theory, it may be > required to set a breakpoint on every system call that may be used to > create threads, including various variations of clone(), subject to > variations between operating systems, but pthread_create has been > enough for me so far.) > > With OPENBLAS_NUM_THREADS=1, I'm only seeing OpenMP threads created by > the mmand package during tests for your package tractor.base, and the > latest commit (that temporary disables testing of mmand) doesn't hit > the breakpoint or raise any NOTEs at all. Very many thanks for testing this, Ivan. I did come across the breakpoint suggestion when searching on this, and tried it, but I ran into some problems with gdb within the container. Your experiments confirm that my yardstick was just misleading me; I will add support for explicitly setting the thread count within mmand, which should give me a longer-term solution. Thanks again, Jon __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Unexpected multi-core CPU usage in package tests
В Tue, 27 Feb 2024 11:14:19 + Jon Clayden пишет: > My testing route is to install the packages within the > 'rocker/r-devel' Docker container, which is Debian-based, then use > 'time' to evaluate CPU usage. Note that, even though 'RNifti' does not > use OpenMP, setting OMP_NUM_THREADS changes its CPU usage I think that's because rocker/r-devel uses parallel OpenBLAS: $ podman run --rm -it docker.io/rocker/r-devel \ R -q -s -e 'sessionInfo()' | grep -A1 BLAS BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.24.so; LAPACK version 3.11.0 The incoming CRAN check machine either sets the BLAS parallellism to 1 or uses a non-parallel BLAS. With rocker/r-devel, you can run R with the environment variable OPENBLAS_NUM_THREADS set to 1. It's been effective in the past to run R -d gdb and set a breakpoint on pthread_create before launching the test. (In theory, it may be required to set a breakpoint on every system call that may be used to create threads, including various variations of clone(), subject to variations between operating systems, but pthread_create has been enough for me so far.) With OPENBLAS_NUM_THREADS=1, I'm only seeing OpenMP threads created by the mmand package during tests for your package tractor.base, and the latest commit (that temporary disables testing of mmand) doesn't hit the breakpoint or raise any NOTEs at all. -- Best regards, Ivan __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Unable to access log operator in C
Thank you, Dirk.
However, I am an absolute clod.I just realized; I was passing in the
SEXP indicating whether or not to log the function as "log", so I
"overwrote" the symbol.
Excuse me while I go bang my head into the wall a few dozen times.
My apologies,
Avi
On Wed, Feb 28, 2024 at 7:14 PM Dirk Eddelbuettel wrote:
>
>
> On 28 February 2024 at 19:05, Avraham Adler wrote:
> | I am hoping the solution to this question is simple, but I have not
> | been able to find one. I am building a routine in C to be called from
> | R. I am including Rmath.h. However, when I have a call to "log", I get
> | the error "called object 'log' is not a function or a function
> | pointer. When I "trick" it by calling log1p(x - 1), which I *know* is
> | exported from Rmath.h, it works.
> |
> | More completely, my includes are:
> | #include
> | #include
> | #include
> | #include
> | #include // for NULL
> | #include
> |
> | The object being logged is a double, passed into C as an SEXP, call it
> | "a", which for now will always be a singleton. I initialize a pointer
> | double *pa = REAL(a). I eventually call log(pa[0]), which does not
> | compile and throws the error listed above. Switching the call to
> | log1p(pa[0] - 1.0) works and returns the proper answer.
> |
> | Even including math.h explicitly does not help, which makes sense as
> | it is included by Rmath.h.
>
> Can you show the actual line? Worst case rename your source file to end in
> .cpp, include and call std::log.
>
> > Rcpp::cppFunction("double mylog(double x) { return std::log(x); }")
> > mylog(exp(42))
> [1] 42
> >
>
> Dirk
>
> --
> dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Unable to access log operator in C
On 28 February 2024 at 19:05, Avraham Adler wrote:
| I am hoping the solution to this question is simple, but I have not
| been able to find one. I am building a routine in C to be called from
| R. I am including Rmath.h. However, when I have a call to "log", I get
| the error "called object 'log' is not a function or a function
| pointer. When I "trick" it by calling log1p(x - 1), which I *know* is
| exported from Rmath.h, it works.
|
| More completely, my includes are:
| #include
| #include
| #include
| #include
| #include // for NULL
| #include
|
| The object being logged is a double, passed into C as an SEXP, call it
| "a", which for now will always be a singleton. I initialize a pointer
| double *pa = REAL(a). I eventually call log(pa[0]), which does not
| compile and throws the error listed above. Switching the call to
| log1p(pa[0] - 1.0) works and returns the proper answer.
|
| Even including math.h explicitly does not help, which makes sense as
| it is included by Rmath.h.
Can you show the actual line? Worst case rename your source file to end in
.cpp, include and call std::log.
> Rcpp::cppFunction("double mylog(double x) { return std::log(x); }")
> mylog(exp(42))
[1] 42
>
Dirk
--
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Unable to access log operator in C
I am hoping the solution to this question is simple, but I have not been able to find one. I am building a routine in C to be called from R. I am including Rmath.h. However, when I have a call to "log", I get the error "called object 'log' is not a function or a function pointer. When I "trick" it by calling log1p(x - 1), which I *know* is exported from Rmath.h, it works. More completely, my includes are: #include #include #include #include #include // for NULL #include The object being logged is a double, passed into C as an SEXP, call it "a", which for now will always be a singleton. I initialize a pointer double *pa = REAL(a). I eventually call log(pa[0]), which does not compile and throws the error listed above. Switching the call to log1p(pa[0] - 1.0) works and returns the proper answer. Even including math.h explicitly does not help, which makes sense as it is included by Rmath.h. Thank you, Avi __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Unexpected multi-core CPU usage in package tests
Dear all,
I'm slightly going out of my mind trying to resolve a CRAN rejection
based on the note
Flavor: r-devel-linux-x86_64-debian-gcc
Check: tests, Result: NOTE
Running 'tinytest.R' [20s/5s]
Running R code in 'tinytest.R' had CPU time 4.4 times elapsed time
This applies to package 'tractor.base', which contains no compiled
code and is not parallelised, but has (first-party) package 'mmand' as
a dependency, which uses OpenMP where available.
Further investigation suggests that skipping the tests that use that
package doesn't resolve the issue, and indeed it seems to occur in
another dependency, 'RNifti', which doesn't use OpenMP, and has no
hard dependencies of its own except 'Rcpp'.
My testing route is to install the packages within the
'rocker/r-devel' Docker container, which is Debian-based, then use
'time' to evaluate CPU usage. Note that, even though 'RNifti' does not
use OpenMP, setting OMP_NUM_THREADS changes its CPU usage:
# /usr/bin/time Rscript -e "library(RNifti);
testthat::test_dir('tests/testthat')"
...
5.46user 4.88system 0:04.06elapsed 254%CPU (0avgtext+0avgdata
387244maxresident)k
0inputs+8032outputs (208major+97868minor)pagefaults 0swaps
# /usr/bin/time env OMP_NUM_THREADS=1 Rscript -e "library(RNifti);
testthat::test_dir('tests/testthat')"
...
3.54user 0.13system 0:03.67elapsed 100%CPU (0avgtext+0avgdata
382060maxresident)k
0inputs+8032outputs (222major+98893minor)pagefaults 0swaps
I see similar results for packages using 'tinytest' as well as
'testthat'. I am not requesting parallelised testing from either
framework.
So, my questions are
(i) is there something wrong with this evaluation that's leading me astray?
(ii) what might the source of this unexpected parallelism be?
(iii) how can I reliably meet CRAN's requirement not to use too many
CPU cores, other than setting OMP_NUM_THREADS before starting R, which
I can't control?
Thanks in advance.
All the best,
Jon
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
