Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Dirk Eddelbuettel
This is getting off topic but as you James saw fit to advertise his package (as he should, it is clearly helpful to some, himself included), here are my $0.02 of why it is not for me: On 14 May 2021 at 20:54, Balamuta, James Joseph wrote: | For code searches, consider using the {searcher} package

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Balamuta, James Joseph
For code searches, consider using the {searcher} package: https://github.com/r-assist/searcher In particular, the search_github() function handles the query formatting. As an example, try: searcher::search_github("clusterEvalQ ") This opens a web browser with: https://github.com/search?q=clu

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Dirk Eddelbuettel
On 14 May 2021 at 14:07, Michael Weylandt wrote: | The CRAN organization / mirror on GitHub (github.com/cran) is very | useful for this sort of thing. | | Searching for "library(" and "clusterEvalQ" | (https://github.com/search?q=org%3Acran+library%28+clusterEvalQ&type=code) | in that organizatio

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Naeem Khoshnevis
This is great, James. Thank you so much for sharing. Best regards, Naeem On Fri, May 14, 2021 at 3:09 PM Balamuta, James Joseph < balam...@illinois.edu> wrote: > Naeem, > > > > The best path for including compiled code in a package is to place it > within the `src/` directory instead of using `R

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Balamuta, James Joseph
Naeem, The best path for including compiled code in a package is to place it within the `src/` directory instead of using `Rcpp::cppFunction()` to compile. The reasons for this are stated succiently here: https://stackoverflow.com/a/6074391/1345455 From there, the C++ can easily be exported ac

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Naeem Khoshnevis
Thank you so much, Michael and Jeff. I really appreciate your help. These are invaluable suggestions and recommendations. Thanks, Dirk, for bringing this email list to my attention. Best regards, Naeem ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forg

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Michael Weylandt
You "just" run clusterEvalQ(library(PKG)) on each cluster. Like Jeff says, this is much easier for you as a package developer and, I'd argue, easier for your users as well, since they just have to make sure the package can be installed once, rather than having compilers ready and behaving for each

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Naeem Khoshnevis
Thank you so much, Jeff. The part that I do not understand is the "and make that package available in your worker environment" part. Could you please let me know how I can make the package available for each worker. On Fri, May 14, 2021 at 1:44 PM Jeff Newmiller wrote: > clusterExport works just

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Jeff Newmiller
clusterExport works just fine if you put your Rcpp code into your own package and make that package available in your worker environment. Given the need for compilation in possibly a variety of computing environments for parallel processing this is definitely recommended. On May 14, 2021 10:35:

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Naeem Khoshnevis
Hi Michael, Thank you so much for your response. That is correct. One method for exporting required variables/functions is using the clusterExport function, which does not work for Rcpp-based functions. Another option is using clusterEvalQ (as mentioned in the shared post); however, I am not sur

Re: [Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Michael Weylandt
Hi Naeem, My (very quick) guess is that this isn't an Rcpp problem per se, but a parLapply problem. You need to explicitly load your package on each worker so that functions from it are available. See, e.g., the brief discussion here: https://stackoverflow.com/questions/18357788/parallel-parlappl

[Rcpp-devel] Exporting rcpp-based function into parLapply workers in an R package

2021-05-14 Thread Naeem Khoshnevis
Dear Rcpp developers: Thanks for developing and maintaining the Rcpp package. I wrote a function in Rcpp. It is available throughout the package and works as expected; however, it is not available for praLapply workers. A temporary fix is just using Rcpp::cppFunction inside the function that parLa

Re: [Rcpp-devel] Posting question's guidelines

2021-05-14 Thread Dirk Eddelbuettel
On 14 May 2021 at 11:12, Naeem Khoshnevis wrote: | Hope this email finds you well. Can I post Rcpp related questions (as a | user) on this mailing list? Yes please -- that is *exactly* for it is for: doing 'Rcpp devel'opment by which we also mean using it in your packages and work. Dirk -- ht

[Rcpp-devel] Posting question's guidelines

2021-05-14 Thread Naeem Khoshnevis
Hi, Hope this email finds you well. Can I post Rcpp related questions (as a user) on this mailing list? Best regards, Naeem Khoshnevis ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailma