Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
On Wed, Sep 23, 2020 at 07:34:50PM +0100, Shayan Doust wrote: > This [commit] now rectifies the build issue for r-cran-prophet. > > I can build r-cran-prophet successfully after re-building r-cran-rstan with > the > new patch. Strange, when I try to build r-cran-rstan with your patch I get: g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"." -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -DBOOST_NO_AUTO_PTR -D_REENTRANT -DSTAN_THREADS -I'/usr/lib/R/site-library/Rcpp/include' -I'/usr/lib/R/site-library/RcppEigen/include' -I'/usr/lib/R/site-library/BH/include' -I'/usr/lib/R/site-library/StanHeaders/include' -I'/usr/lib/R/site-library/RcppParallel/include'-fpic -g -O2 -fdebug-prefix-map=/build/r-base-OT058M/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c stan/lang/grammars/whitespace_grammar_inst.cpp -o stan/lang/grammars/whitespace_grammar_inst.o g++ -std=gnu++14 -shared -L/usr/lib/R/lib -Wl,-z,relro -o rstan.so Module.o chains.o init.o misc.o pointer-tools.o sparse_extractors.o stan_fit_base.o stan_fit_rccp.o stanc.o stan/lang/ast_def.o stan/lang/grammars/bare_type_grammar_inst.o stan/lang/grammars/block_var_decls_grammar_inst.o stan/lang/grammars/expression07_grammar_inst.o stan/lang/grammars/expression_grammar_inst.o stan/lang/grammars/functions_grammar_inst.o stan/lang/grammars/indexes_grammar_inst.o stan/lang/grammars/local_var_decls_grammar_inst.o stan/lang/grammars/program_grammar_inst.o stan/lang/grammars/semantic_actions_def.o stan/lang/grammars/statement_2_grammar_inst.o stan/lang/grammars/statement_grammar_inst.o stan/lang/grammars/term_grammar_inst.o stan/lang/grammars/whitespace_grammar_inst.o /usr/lib/x86_64-linux-gnu -ltbb -ltbbmalloc -L/usr/lib/R/lib -lR /usr/bin/ld: cannot find /usr/lib/x86_64-linux-gnu: file format not recognized collect2: error: ld returned 1 exit status make[1]: *** [/usr/share/R/share/make/shlib.mk:6: rstan.so] Error 1 It looks as if some variable is not resloved properly resolved in the chroot I substituted s/inst.o /usr/lib/x86_64-linux-gnu -ltbb/inst.o -L/usr/lib/x86_64-linux-gnu -ltbb/ (in the very end) and it worked. I guess the patch in R/plugin.R is responsible for this since this is where I can find the string -ltbb. Any idea how to fix this? Kind regards Andreas. > Within r-cran-prophet: > $ autopkgtest . -- null > > [TRUNCATED] > > /usr/bin/ld: cannot find -lStanHeaders > collect2: error: ld returned 1 exit status > make: *** [/usr/share/R/share/make/shlib.mk:6: sourceCpp_2.so] Error 1 > -- 2. Error: (unknown) (@test_stan_functions.R#9) > - > $ operator is invalid for atomic vectors > Backtrace: > 1. base::tryCatch(...) > 2. base:::tryCatchList(expr, classes, parentenv, handlers) > 3. base:::tryCatchOne(expr, names, parentenv, handlers[[1L]]) > 4. value[[3L]](cond) > 5. rstan::expose_stan_functions(...) > > == testthat results > === > [ OK: 160 | SKIPPED: 0 | WARNINGS: 3 | FAILED: 2 ] > 1. Error: (unknown) (@test_prophet.R#9) > 2. Error: (unknown) (@test_stan_functions.R#9) > > There exists some failed tests, however 160 pass! > > Kind regards, > Shayan Doust > > > [commit]: > https://salsa.debian.org/r-pkg-team/r-cran-rstan/-/commit/7351289242ba080c112b1c15784e57f154a79076 pub RSA 4096/19D02395 2019-09-04 Shayan Doust (Personal EMAIL) > sub RSA 4096/00484410 2020-06-26 > -- http://fam-tille.de
Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
Hello again Andreas, This [commit] now rectifies the build issue for r-cran-prophet. I can build r-cran-prophet successfully after re-building r-cran-rstan with the new patch. Within r-cran-prophet: $ autopkgtest . -- null [TRUNCATED] /usr/bin/ld: cannot find -lStanHeaders collect2: error: ld returned 1 exit status make: *** [/usr/share/R/share/make/shlib.mk:6: sourceCpp_2.so] Error 1 -- 2. Error: (unknown) (@test_stan_functions.R#9) - $ operator is invalid for atomic vectors Backtrace: 1. base::tryCatch(...) 2. base:::tryCatchList(expr, classes, parentenv, handlers) 3. base:::tryCatchOne(expr, names, parentenv, handlers[[1L]]) 4. value[[3L]](cond) 5. rstan::expose_stan_functions(...) == testthat results === [ OK: 160 | SKIPPED: 0 | WARNINGS: 3 | FAILED: 2 ] 1. Error: (unknown) (@test_prophet.R#9) 2. Error: (unknown) (@test_stan_functions.R#9) There exists some failed tests, however 160 pass! Kind regards, Shayan Doust [commit]: https://salsa.debian.org/r-pkg-team/r-cran-rstan/-/commit/7351289242ba080c112b1c15784e57f154a79076 0x6D7D441919D02395.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
Hello Andreas,
I see the error within r-cran-prophet's build.
It seems like this issue stems from r-cran-rstan (as the call stack shows).
Specifically:
$ grep -r "system.file(\"lib\""
R/plugin.R:RcppParallel_pkg_libs <- system.file("lib", .Platform$r_arch,
R/plugin.R:rstan_StanServices <- system.file("lib", .Platform$r_arch,
"libStanServices.a",
I believe the first line (regarding RcppParallel_pkg_libs) is of concern and the
cause as again, the library location assumption is wrong.
I will write a patch for this too, and I'll test anymore findings.
Kind regards,
Shayan Doust
0x6D7D441919D02395.asc
Description: application/pgp-keys
signature.asc
Description: OpenPGP digital signature
Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
Hi Shayan,
On Wed, Sep 23, 2020 at 01:27:52PM +0100, Shayan Doust wrote:
> The [commit] is now pushed for r-cran-rcppparallel, which fixes the "TBB
> library
> not found" error thrown for r-cran-rstanarm.
>
> Just a note that I did not make it through the entire build process. I gave
> the
> build 40 mins before giving up. Compiling r-cran-stanarm used almost all of
> my 4
> GB of RAM on my test server and it resorted to using most of my swap space,
> so I
> am unable to test for any other issues. I've left this bug open.
I confirmt stanarm is a beast. However, you can test with r-cran-prophet as
well and this says:
Error in system.file("lib", .Platform$r_arch, package = "RcppParallel", :
no file found
Calls: ... .pluginIncludes -> plugin -> rstanplugin -> system.file
Execution halted
My guess is that your patch is not complete yet.
Kind regards
Andreas.
--
http://fam-tille.de
Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
Hello Andreas, The [commit] is now pushed for r-cran-rcppparallel, which fixes the "TBB library not found" error thrown for r-cran-rstanarm. Just a note that I did not make it through the entire build process. I gave the build 40 mins before giving up. Compiling r-cran-stanarm used almost all of my 4 GB of RAM on my test server and it resorted to using most of my swap space, so I am unable to test for any other issues. I've left this bug open. Kind regards, Shayan Doust [commit]: https://salsa.debian.org/r-pkg-team/r-cran-rcppparallel/-/commit/e8fcee92d68a420ca1441479661d74add27f519e 0x6D7D441919D02395.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
Hi Shayan, On Tue, Sep 22, 2020 at 06:44:39PM +0100, Shayan Doust wrote: > I'll try to write a simple patch for this hook file. For a Debian package, > this > assumption for library location is plain wrong. That would be really great. My guess is that once this is solved also r-cran-prophet[1] will build. Kind regards Andreas. [1] https://salsa.debian.org/r-pkg-team/r-cran-prophet -- http://fam-tille.de
Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
Hello Andreas, Yes, this does seem like a local issue to r-cran-rcppparallel. tbb and variants are loaded when r-cran-rcppparallel is imported, as seen in [hooks.R]. The function of interest is 'tbbLibPath()', which seems to depict tbb library path. With some more investigation, you can see this function within [build.R]. You can see for all supported platforms apart from Windows and Sparc systems, it seems like it expects the library to be situated under lib/ (due to the first argument used within system.file()). I'll try to write a simple patch for this hook file. For a Debian package, this assumption for library location is plain wrong. Kind regards, Shayan Doust [hooks.R]: https://github.com/RcppCore/RcppParallel/blob/b216ba27dcbd3c523932bd918b6dd0b1b08d3566/R/hooks.R#L8 [build.R]: https://github.com/RcppCore/RcppParallel/blob/b216ba27dcbd3c523932bd918b6dd0b1b08d3566/R/build.R#L66 On Tue, 8 Sep 2020 20:32:53 +0200 Andreas Tille wrote: > Control: tags -1 help > > Hi, > > any help why TBB is not found? I think this issue is actually causes by > r-cran-rcppparallel since the code copy of libtbb was removed there - > but it seems to not provide the Debian packaged lib properly. > > Kind regards > > Andreas. > > -- > http://fam-tille.de > > 0x6D7D441919D02395.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
Bug#963392: [Help] Re: r-cran-rstanarm: FTBFS: error: (converted from warning) TBB library not found.
Control: tags -1 help Hi, any help why TBB is not found? I think this issue is actually causes by r-cran-rcppparallel since the code copy of libtbb was removed there - but it seems to not provide the Debian packaged lib properly. Kind regards Andreas. -- http://fam-tille.de
