Re: [R-pkg-devel] CRAN uses an old version of clang

2024-02-09 Thread Marcin Jurek
All this makes sense, thanks for your tips, everyone!

Marcin

On Fri, Feb 9, 2024 at 9:44 AM Dirk Eddelbuettel  wrote:

>
> On 9 February 2024 at 08:59, Marcin Jurek wrote:
> | I recently submitted an update to my package. It previous version relied
> on
> | Boost for Bessel and gamma functions but a colleague pointed out to me
> that
> | they are included in the standard library beginning with the C++17
> | standard.
>
> There is an often overlooked bit of 'fine print': _compiler support_ for a
> C++ standard is not the same as the _compiler shipping a complete library_
> for that same standard. This can be frustrating. See the release notes for
> gcc/g++ and clang/clang++, IIRC they usually have a separate entry for C++
> library support.
>
> In this case, can probably rely on LinkingTo: BH which has been helping
> with
> Boost headers for over a decade.
>
> Writing R Extensions is also generally careful in reminding us that such
> language standard support is always dependent on the compiler at hand. So
> package authors ought to check, just like R does via its extensive
> configure
> script when it builds.
>
> Dirk
>
> --
> dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


[R-pkg-devel] CRAN uses an old version of clang

2024-02-09 Thread Marcin Jurek
Dear community,

I recently submitted an update to my package. It previous version relied on
Boost for Bessel and gamma functions but a colleague pointed out to me that
they are included in the standard library beginning with the C++17
standard.

I don't have access to a Mac so I tested my package on Rhub and on my local
Linux and everything ran fine. However, it seems like CRAN is using an old
version of Clang (14.03 vs 16 being the newest one) and it complained about
these Bessel functions. I'm pasting the installation log below. I wonder if
this is something I could hope to explain in cran-comments and have my
package accepted as is?

I could also revert to using Boost although I only need it for these
special functions and things are much cleaner without it. In addition, one
of the main reasons for this update was related to some warnings Boost
started throwing.

Really appreciate the help!

* installing *source* package ‘GPvecchia’ ...
** package ‘GPvecchia’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 14.0.3 (clang-1403.0.22.14.1)’
using C++17
using SDK: ‘MacOSX11.3.sdk’
clang++ -arch x86_64 -std=gnu++17
-I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/Rcpp/include'
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/RcppArmadillo/include'
-I/opt/R/x86_64/include -fPIC  -falign-functions=64 -Wall -g -O2
-c Esqe.cpp -o Esqe.o
clang++ -arch x86_64 -std=gnu++17
-I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/Rcpp/include'
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/RcppArmadillo/include'
-I/opt/R/x86_64/include -fPIC  -falign-functions=64 -Wall -g -O2
-c Matern.cpp -o Matern.o
clang++ -arch x86_64 -std=gnu++17
-I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/Rcpp/include'
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/RcppArmadillo/include'
-I/opt/R/x86_64/include -fPIC  -falign-functions=64 -Wall -g -O2
-c MaxMin.cpp -o MaxMin.o
clang++ -arch x86_64 -std=gnu++17
-I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/Rcpp/include'
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/RcppArmadillo/include'
-I/opt/R/x86_64/include -fPIC  -falign-functions=64 -Wall -g -O2
-c RcppExports.cpp -o RcppExports.o
clang++ -arch x86_64 -std=gnu++17
-I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/Rcpp/include'
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/RcppArmadillo/include'
-I/opt/R/x86_64/include -fPIC  -falign-functions=64 -Wall -g -O2
-c U_NZentries.cpp -o U_NZentries.o
clang++ -arch x86_64 -std=gnu++17
-I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/Rcpp/include'
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/RcppArmadillo/include'
-I/opt/R/x86_64/include -fPIC  -falign-functions=64 -Wall -g -O2
-c dist.cpp -o dist.o
clang++ -arch x86_64 -std=gnu++17
-I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/Rcpp/include'
-I'/Volumes/Builds/packages/big-sur-x86_64/Rlib/4.3/RcppArmadillo/include'
-I/opt/R/x86_64/include -fPIC  -falign-functions=64 -Wall -g -O2
-c fastTree.cpp -o fastTree.o
Matern.cpp:80:68: error: no member named 'cyl_bessel_k' in namespace 'std'
covmat(j1,j2) = normcon*pow( scaledist, covparms(2)
)*std::cyl_bessel_k(covparms(2),scaledist);
//Rf_bessel_k(scaledist,covparms(2),1.0);
  ~^
1 error generated.
make: *** [Matern.o] Error 1
make: *** Waiting for unfinished jobs
ERROR: compilation failed for package ‘GPvecchia’
* removing 
‘/Volumes/Builds/packages/big-sur-x86_64/results/4.3/GPvecchia.Rcheck/GPvecchia’

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] failing CRAN checks due to problems with dependencies

2024-02-08 Thread Marcin Jurek
Ok, this makes sense! I saw that Rcpp was failing the checks too but I
wasn't sure if I should resubmit or wait. Thanks!

On Thu, Feb 8, 2024 at 1:17 PM Ivan Krylov  wrote:

> В Wed, 7 Feb 2024 08:40:44 -0600
> Marcin Jurek  пишет:
>
> > Packages required but not available: 'Rcpp', 'FNN',
> > 'RcppArmadillo' Packages suggested but not available for checking:
> > 'fields', 'rmarkdown', 'testthat', 'maptools'
>
> One of the machines running the incoming checks was having problems. If
> you followed the failing dependency chain by looking at the CRAN check
> results of the packages described as "not available", you could
> eventually find a package needing compilation (Rcpp or stringi or
> something else), look at the installation log and see Make trying to
> run commands that are completely wrong.
>
> It looked like the path to the compiler was empty:
>
> https://web.archive.org/web/20240208191430/https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-debian-clang/Rcpp-00install.html
>
> I think that the problems are solved now, so it should be safe to
> increment the version and submit it again.
>
> --
> Best regards,
> Ivan
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


[R-pkg-devel] failing CRAN checks due to problems with dependencies

2024-02-07 Thread Marcin Jurek
Dear community,

I've been trying to submit an update to my GPvecchia package recently. This
morning I was notified that my package didn't pass incoming checks. When I
looked at the log this was the problem:

* checking package dependencies ... ERROR
Packages required but not available: 'sparseinv', 'GpGp'* checking
package dependencies ... ERROR

Packages required but not available: 'sparseinv', 'GpGp'

Like my package, these have been on CRAN for a while now and still seem to
be. However they both report an error in their respective CRAN checks, but
it's a weird one.

This is taken from GpGp checks but it's similar for sparseinv:

Result: ERROR  Packages required but not available: 'Rcpp', 'FNN',
'RcppArmadillo' Packages suggested but not available for checking:
'fields', 'rmarkdown', 'testthat', 'maptools' See section ‘The DESCRIPTION
file’ in the ‘Writing R Extensions’

I'm not sure how to proceed. I know the maintainer of the GpGp package but
I'm not sure what to alert him to if Rcpp does not work.

I really appreciate your help.

Marcin

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Note: information on .o files is not available / Found '_exit', possibly from '_exit' (C)

2020-07-17 Thread Marcin Jurek
So maybe you tried that already but I noticed that sometimes when I do the
check on the package directory rather than on the tarball I get similar
errors. I don't know if that's the problem but it should be a quick thing
to try.

On Fri, Jul 17, 2020 at 5:05 AM Fabio Sigrist 
wrote:

> Dear all,
>
> I am trying to get an R package with C++ code on CRAN and I have one NOTE
> remaining, for which I can't find a solution:
>
> Note: information on .o files for x64 is not available
>   File
> 'd:/RCompile/CRANincoming/R-devel/lib/gpboost/libs/x64/lib_gpboost.dll':
> Found '_exit', possibly from '_exit' (C)
> Found 'abort', possibly from 'abort' (C), 'runtime' (Fortran)
> Found 'exit', possibly from 'exit' (C), 'stop' (Fortran)
> Found 'printf', possibly from 'printf' (C)
>
> As much as I search through my code, I can't find the place / headers where
> these calls / symbols originate. Also, I have no idea how to add
> information on .o files (apart from the shared library, there are no .o
> files). The .tar.gz file for the package can be found on
> https://github.com/fabsig/GPBoost/blob/master/gpboost_0.2.0.tar.gz. Note
> that the shared library is compiled using install.libs.R (this is a
> deliberate choice) and the flag "GPB_R_BUILD" is set when compiling for the
> R package (I have tried to put "#ifndef GPB_R_BUILD" around all headers
> that could cause the problems with exit / abort calls, but apparently I
> have not been able to find all).
>
> Any help is greatly appreciated.
>
> Best regards,
> Fabio Sigrist
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


[R-pkg-devel] debugging memory sanitation errors

2020-04-20 Thread Marcin Jurek
Hello so I keep getting my package turned back because of errors detected
by UBSAN. I am trying to catch it using r-hub's platform with UBSAN but
there are no errors thrown there. What would be a good way for me to debug
this.

Obviously I did R CMD check --as-cran on my own machine, and on Travis and
I can't detect a thing.

What surprises me is that this is a runtime error and I wonder why all the
systems I test it on give it a pass...

Thanks for suggestions!

Marcin


-- Forwarded message -
From: Uwe Ligges 
Date: Sat, Apr 18, 2020 at 2:32 AM
Subject: Re: CRAN submission GPvecchia 0.1.3
To: Marcin Jurek , CRAN <
cran-submissi...@r-project.org>


Thanks, we see:

checking with clang-UBSAN still has

/data/gannet/ripley/R/test-clang/RcppArmadillo/include/armadillo_bits/subview_meat.hpp:1277:23:

runtime error: reference binding to null pointer of type 'const unsigned
int'
 #0 0x7f8b9133b5ae in arma::subview::colptr(unsigned
int)
/data/gannet/ripley/R/test-clang/RcppArmadillo/include/armadillo_bits/subview_meat.hpp:1277:12
 #1 0x7f8b9133b5ae in arma::subview_col::subview_col(arma::Mat const&, unsigned int, unsigned
int, unsigned int)
/data/gannet/ripley/R/test-clang/RcppArmadillo/include/armadillo_bits/subview_meat.hpp:3133:25
 #2 0x7f8b913218d3 in arma::Col::head(unsigned int)
/data/gannet/ripley/R/test-clang/RcppArmadillo/include/armadillo_bits/Col_meat.hpp:782:10
 #3 0x7f8b913218d3 in clusterEqual(arma::Mat, int, int)
/data/gannet/ripley/R/packages/incoming/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/fastTree.cpp:53:35
 #4 0x7f8b9132722d in knotTree(arma::Mat,
std::__1::map,
std::__1::allocator >, arma::Col,
std::__1::less,
std::__1::allocator > >,
std::__1::allocator, std::__1::allocator > const,
arma::Col > > >)
/data/gannet/ripley/R/packages/incoming/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/fastTree.cpp:241:21
 #5 0x7f8b91328fae in generateNNarray(arma::Mat,
arma::Col, int, arma::Col, int)
/data/gannet/ripley/R/packages/incoming/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/fastTree.cpp:279:39
 #6 0x7f8b91267769 in _GPvecchia_generateNNarray
/data/gannet/ripley/R/packages/incoming/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/RcppExports.cpp:93:34

and others not from this package.

Please fix the issue reported above and resubmit.

Best,
Uwe Ligges


On 16.04.2020 00:37, CRAN submission wrote:
> [This was generated from CRAN.R-project.org/submit.html]
>
> The following package was uploaded to CRAN:
> ===
>
> Package Information:
> Package: GPvecchia
> Version: 0.1.3
> Title: Scalable Gaussian-Process Approximations
> Author(s): Matthias Katzfuss [aut], Marcin Jurek [aut, cre], Daniel Zilber
>[aut], Wenlong Gong [aut], Joe Guinness [ctb], Jingjie Zhang
>[ctb], Florian Schaefer [ctb]
> Maintainer: Marcin Jurek 
> Suggests: mvtnorm, knitr, rmarkdown, testthat
> Description: Fast scalable Gaussian process approximations, particularly
>well suited to spatial (aerial, remote-sensed) and
>environmental data, described in more detail in Katzfuss and
>Guinness (2017) . Package also contains a
>fast implementation of the incomplete Cholesky decomposition
>(IC0), based on Schaefer et al. (2019)  and
>MaxMin ordering proposed in Guinness (2018)
>.
> License: GPL (>=2)
> Imports: Rcpp (>= 0.12.16), methods, stats, sparseinv, fields,
>Matrix(>= 1.2.14), parallel, GpGp, FNN
> LinkingTo: Rcpp, RcppArmadillo, BH
>
>
> The maintainer confirms that he or she
> has read and agrees to the CRAN policies.
>
> Submitter's comment: ### Resubmission 5
> This is a fifth resubmission
>
> *
>fixed problems reported in
>https://cran.r-project.org/web/checks/check_results_GPvecchia.html
>
>
>
>
>
> ###
>Resubmission 4
> This is a fourth resubmission
>
> * fixed
>a problem with unmatched variable types encountered
>during the last submission:
>runtime error:
>reference binding to null pointer of type 'const
>unsigned
>
> other check results as in submission
>3
>
>
>
>
>
>
> ### Resubmission 3
> This is a third
>resubmission
>
> * fixed the problem with variable types
>(unsigned int) in src/U_NZentries.cpp
>
>
> ## Test
>environments
> * local ubuntu 18.04, R 3.4.4
> * ubuntu
>16.04 (Travis CI), R-devel
> * Windows Server 2008 R2
>SP1, R-devel, 32/64 bit (R-hub)
>
> There was 1 NOTE:
> *
>checking installed package size ... NOTE
>installed
>size is  7.0Mb
>sub-directories of 1Mb or more:
>
>libs   6.5Mb
>
> This is not unexpected in packages with
>compiled code as observed by Dirk
>Eddelbuettel
>
https://stackoverflow

[R-pkg-devel] potential memory leak using openMP

2019-11-15 Thread Marcin Jurek
Hello everyone, so my package was rejected from CRAN because of the
following error:

==29416== 640 bytes in 2 blocks are possibly lost in loss record 162 of
2,089
==29416==at 0x4837B65: calloc
(coregrind/m_replacemalloc/vg_replace_malloc.c:762)
==29416==by 0x40118B1: allocate_dtv
(/build/glibc-suXNNi/glibc-2.29/elf/../elf/dl-tls.c:286)
==29416==by 0x40121FD: _dl_allocate_tls
(/build/glibc-suXNNi/glibc-2.29/elf/../elf/dl-tls.c:532)
==29416==by 0x5684BDD: allocate_stack
(/build/glibc-suXNNi/glibc-2.29/nptl/allocatestack.c:621)
==29416==by 0x5684BDD: pthread_create@@GLIBC_2.2.5
(/build/glibc-suXNNi/glibc-2.29/nptl/pthread_create.c:669)
==29416==by 0x565DF65: ??? (in
/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
==29416==by 0x5654A3C: GOMP_parallel (in
/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
==29416==by 0x1DBACEBA: U_NZentries(int, int, arma::Mat const&,
arma::Mat const&, arma::Mat const&, arma::Col
const&, arma::Col const&, std::__cxx11::basic_string, std::allocator >,  arma::Col)
(/home/Hornik/tmp/CRAN/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/U_NZentries.cpp:225)
==29416==by 0x1DBA3B90: _GPvecchia_U_NZentries
(/home/Hornik/tmp/CRAN/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/RcppExports.cpp:78)

First, I don't know how to reproduce it because running
R CMD check  --use-valgrind
does not produce any errors on my machine.

Second, see below the relevant part of the U_NZentries function (with the
offending line marked with (***).

*** #pragma omp parallel for
shared(locs,revNNarray,revCondOnLatent,nuggets, nnp,m,Lentries,COV)
private(k,dist,onevec,covmat,nug,n0,inds,revCon_row,inds00)
 schedule(static)

  for (k = 0; k < nnp; k++) {
inds=revNNarray.row(k).t();
revCon_row=revCondOnLatent.row(k).t();
n0 = get_nonzero_count_general(inds); // for general case
inds00 = get_idx_vals_general(n0, inds);
nug=nuggets.elem(inds00) % (ones(n0)-revCon_row(span(m+1-n0,m))); //
vec is vec, cannot convert to mat
dist = calcPWD(locs.rows(inds00));
#pragma omp critical
{
  if( COV=="matern"){
covmat= MaternFun(dist,covparms) + diagmat(nug) ; // summation from
arma
  } else if(COV=="esqe") {
covmat= EsqeFun(dist,covparms) + diagmat(nug) ; // summation from
arma
  }
}
onevec.resize(n0);
onevec = zeros(n0);
onevec[n0-1] = 1;
arma::vec M=solve(chol(covmat,"upper"),onevec);
// save the entries to matrix
Lentries(k,span(0,n0-1)) = M.t();
  }

I have googled around and some people are saying that what valgrind reports
is an unavoidable consequence of using openMP (for example here:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=36298 and here:
https://stackoverflow.com/questions/6973489/valgrind-and-openmp-still-reachable-and-possibly-lost-is-that-bad
).

Three questions: 1) what should I do to reproduce the error? 2) is it worth
trying to do so? 3) why does it come up?

Thanks a lot everyone!

Marcin

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


[R-pkg-devel] debugging memory errors

2019-11-06 Thread Marcin Jurek
Hello, I'm trying to submit my package to CRAN and received the following
error message:

Error(s) in re-building vignettes:
--- re-building ‘GPvecchia_vignette.Rmd’ using rmarkdown
.../RcppArmadillo/include/armadillo_bits/subview_meat.hpp:1223:54:
runtime error: reference binding to null pointer of type 'const unsigned
int'
...//RcppArmadillo/include/armadillo_bits/access.hpp:26:100: runtime
error: reference binding to null pointer of type 'unsigned int'
=
==40666==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x615000445e48 at pc 0x7faebceeaa54 bp 0x7ffd4d87a980 sp 0x7ffd4d87a970
READ of size 8 at 0x615000445e48 thread T0
 #0 0x7faebceeaa53 in createUcpp(Rcpp::Vector<14,
Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>,
arma::Mat, arma::Col)
/data/gannet/ripley/R/packages/incoming/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/U_NZentries.cpp:366
 #1 0x7faebce84b2d in _GPvecchia_createUcpp
/data/gannet/ripley/R/packages/incoming/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/RcppExports.cpp:118

I have very little clue what to do, above all because I don't know how to
reproduce the error. Here is what I tried:
1. Using the r-devel-san container from rocker. When I'm building the
package using
> RD CMD build GPvecchia
it tells me:

Error: package or namespace load failed for ‘GPvecchia’ in dyn.load(file,
DLLpath = DLLpath, ...):
 unable to load shared object
'/tmp/RtmpRfO4Ad/Rinst96f4db205fe/00LOCK-GPvecchia/00new/GPvecchia/libs/GPvecchia.so':

/tmp/RtmpRfO4Ad/Rinst96f4db205fe/00LOCK-GPvecchia/00new/GPvecchia/libs/GPvecchia.so:
undefined symbol: __asan_option_detect_stack_use_after_return

Here is the package Makevars (following the specification from
https://www.stats.ox.ac.uk/pub/bdr/memtests/README.txt):
## optional
CXX_STD = CXX11

#PKG_CFLAGS = $(SHLIB_OPENMP_CFLAGS)
PKG_LIBS = $(SHLIB_OPENMP_CFLAGS) -lasan
PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) -fsanitize=address,undefined
-fno-omit-frame-pointer -fno-sanitize=vptr
PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)

MAIN_LDFLAGS=-fsanitize=address,undefined -pthread

F77 = gfortran -fsanitize=address
FC = gfortran -fsanitize=address
FCFLAGS = -g -O2 -mtune=native
FFLAGS = -g -O2 -mtune=native

2. I also tried using r-hub and their platform with R compiled with the
appropriate flags. However, their configuration does not support openMP.

Please help, thanks!

Marcin

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


[R-pkg-devel] please help understand an error in openMP statements

2019-09-12 Thread Marcin Jurek
Hello everyone, I'm submitting a package to CRAN which I tested locally, on
Travis CI, R-hub and win builder. It worked no problem in all these
environments. However, after submission, I keep getting the error described
here:


https://win-builder.r-project.org/incoming_pretest/GPvecchia_0.1.0_20190912_201702/Debian/00install.out


I'm not a pro when it comes to openMP but since all previous tests
completed successfully, I thought things were alright. Could you help me
understand what's wrong and how to fix it? Source code of the package can
be found at http://github.com/katzfuss-group/GPvecchia Thanks a lot!


Marcin

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel