Re: [Rcpp-devel] Distribution of Rcpp codebase
My 2 cents... On 07/04/14 10:12, Romain François wrote: It would also mean many copies of the same code base. To which I’m thinking: so what. No, it will mean many copies of /many different and mostly outdated/ code bases. You can count on me to forget to git pull next time I update my package. What about something like the BH package that contains the boost headers? I'm using it in a project I'm working on, and just use a LinkingTo declaration with something in Makevars. Of course ideally it would be in a build-depends type of declaration so it isn't pulled during binary installs. Xavier ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Le 8 avr. 2014 à 10:12, Xavier Robin xav...@cbs.dtu.dk a écrit : My 2 cents... On 07/04/14 10:12, Romain François wrote: It would also mean many copies of the same code base. To which I’m thinking: so what. No, it will mean many copies of /many different and mostly outdated/ code bases. Still: so what ;) You can count on me to forget to git pull next time I update my package. And as long as your package keeps working, this is not so much of an issue. You are then in control of which version of the codebase is used. If you want a new version of the codebase, you can get it, and the challenge for me is to make it easy enough for you to get and deploy it into your package, if you don’t that’s fine. Now consider the alternative. You have no idea of which version of the codebase is used with your package. You have no idea if the version used at runtime is the same as the version that was used when the package was compiled. When a new Rcpp package is released, you definitely have to recompile yours, and need to ask your users to recompile your package against this version. I prefer when I’m in control. What about something like the BH package that contains the boost headers? I'm using it in a project I'm working on, and just use a LinkingTo declaration with something in Makevars. That works fine because BH is truly header only. Rcpp is not. The change in Rcpp 0.11.0 was a big step forward, but it is a job half done. Functions in the .h files of Rcpp constantly poke into the Rcpp.so. Any binary change in Rcpp creates hard problem to fix. Currently the usual advice is to ask to recompile, etc ... It is really easy to make such a binary change. Of course ideally it would be in a build-depends type of declaration so it isn't pulled during binary installs. Xavier ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Martyn Plummer plumm...@iarc.fr writes: And another 2 cents from me. A package is the basic unit of functionality in R. Whatever functionality you are providing, I think a package is the best way to deliver it. There is a well developed framework for versioning, dependency resolution, testing, and distribution. If you choose some other mechanism, then I suspect that any developer problems you eliminate will just return as system administration problems. I thought I could keep out of this... rcpp... is so much the core of many packages, that the mechanism of packages really shows its shortcomings, namely the versioning. A different aspect was discussed on the r-devel list some time ago concerning snapshots of CRAN to provide reproducibility. The part which makes rcpp outstanding from other packages, is that it is used mainly during compilation and not that much during running - that is if I understand the header only concept correctly. In contrast, as far as I know, all other packages provide run-time features as well as compile time - and this is what the packages were designed for - and I might say they work quite well. I can see Romain's point (oparticularly the more fine grained control of the developer), but I also understand that Dirk will keep rcpp as a package. My question is: would it be possible to combine these two, i.e. have one package, which contains different versions, and to use one command to switch between these? So one would have one package, which will switch between different versions? I don't know if this would be possible or a suitable solution? The other would obviously be to have different versions of rcpp on CRAN, similar to ggplot or roxygen. Cheers, Rainer Martyn On Tue, 2014-04-08 at 10:12 +0200, Xavier Robin wrote: My 2 cents... On 07/04/14 10:12, Romain François wrote: It would also mean many copies of the same code base. To which I’m thinking: so what. No, it will mean many copies of /many different and mostly outdated/ code bases. You can count on me to forget to git pull next time I update my package. What about something like the BH package that contains the boost headers? I'm using it in a project I'm working on, and just use a LinkingTo declaration with something in Makevars. Of course ideally it would be in a build-depends type of declaration so it isn't pulled during binary installs. Xavier ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D):+49 - (0)3 21 21 25 22 44 email: rai...@krugs.de Skype: RMkrug PGP: 0x0F52F982 pgp40yhkiHOAy.pgp Description: PGP signature ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Le 8 avr. 2014 à 10:51, Rainer M Krug rai...@krugs.de a écrit : Martyn Plummer plumm...@iarc.fr writes: And another 2 cents from me. A package is the basic unit of functionality in R. Whatever functionality you are providing, I think a package is the best way to deliver it. There is a well developed framework for versioning, dependency resolution, testing, and distribution. If you choose some other mechanism, then I suspect that any developer problems you eliminate will just return as system administration problems. I thought I could keep out of this... rcpp... is so much the core of many packages, that the mechanism of packages really shows its shortcomings, namely the versioning. A different aspect was discussed on the r-devel list some time ago concerning snapshots of CRAN to provide reproducibility. The part which makes rcpp outstanding from other packages, is that it is used mainly during compilation and not that much during running - that is if I understand the header only concept correctly. Well with the current Rcpp, you still need the runtime for any of the functions that are registered here: https://github.com/RcppCore/Rcpp/blob/master/src/Rcpp_init.cpp#L85 That’s what I meant earlier when I mentioned job half done. The intention of the change in 0.11.0 was to move away from runtime, but we did not go far enough. In contrast, as far as I know, all other packages provide run-time features as well as compile time - and this is what the packages were designed for - and I might say they work quite well. I agree. I can see Romain's point (oparticularly the more fine grained control of the developer), but I also understand that Dirk will keep rcpp as a package. Sure. My question is: would it be possible to combine these two, i.e. have one package, which contains different versions, and to use one command to switch between these? So one would have one package, which will switch between different versions? I see your point, but I would not want to do that. I can definitely foresee a whole new class of issues with this scenario. I don't know if this would be possible or a suitable solution? The other would obviously be to have different versions of rcpp on CRAN, similar to ggplot or oxygen. I can see this working. But there could be Rcpp_0_12_0, Rcpp_0_12_1, etc .. as different packages so that developers who want to update their package would need to change the version on which they depend. Not sure CRAN would be impressed if I start releasing a new package for each version. Cheers, Rainer Martyn On Tue, 2014-04-08 at 10:12 +0200, Xavier Robin wrote: My 2 cents... On 07/04/14 10:12, Romain François wrote: It would also mean many copies of the same code base. To which I’m thinking: so what. No, it will mean many copies of /many different and mostly outdated/ code bases. You can count on me to forget to git pull next time I update my package. What about something like the BH package that contains the boost headers? I'm using it in a project I'm working on, and just use a LinkingTo declaration with something in Makevars. Of course ideally it would be in a build-depends type of declaration so it isn't pulled during binary installs. Xavier ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D):+49 - (0)3 21 21 25 22 44 email: rai...@krugs.de Skype: RMkrug PGP: 0x0F52F982 ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Romain François rom...@r-enthusiasts.com writes: Le 8 avr. 2014 à 10:51, Rainer M Krug rai...@krugs.de a écrit : Martyn Plummer plumm...@iarc.fr writes: And another 2 cents from me. A package is the basic unit of functionality in R. Whatever functionality you are providing, I think a package is the best way to deliver it. There is a well developed framework for versioning, dependency resolution, testing, and distribution. If you choose some other mechanism, then I suspect that any developer problems you eliminate will just return as system administration problems. I thought I could keep out of this... rcpp... is so much the core of many packages, that the mechanism of packages really shows its shortcomings, namely the versioning. A different aspect was discussed on the r-devel list some time ago concerning snapshots of CRAN to provide reproducibility. The part which makes rcpp outstanding from other packages, is that it is used mainly during compilation and not that much during running - that is if I understand the header only concept correctly. Well with the current Rcpp, you still need the runtime for any of the functions that are registered here: https://github.com/RcppCore/Rcpp/blob/master/src/Rcpp_init.cpp#L85 That’s what I meant earlier when I mentioned job half done. The intention of the change in 0.11.0 was to move away from runtime, but we did not go far enough. I must admit, I am not to familiar with rcpp to comment on these functions, but would it be possible to split Rcpp into headers-only, i.e. only for compiling and developing, and Rcpp-runtime for runtime? Because I really think it might make sense to have additional function at runtime? In contrast, as far as I know, all other packages provide run-time features as well as compile time - and this is what the packages were designed for - and I might say they work quite well. I agree. I can see Romain's point (oparticularly the more fine grained control of the developer), but I also understand that Dirk will keep rcpp as a package. Sure. My question is: would it be possible to combine these two, i.e. have one package, which contains different versions, and to use one command to switch between these? So one would have one package, which will switch between different versions? I see your point, but I would not want to do that. I can definitely foresee a whole new class of issues with this scenario. I don't know if this would be possible or a suitable solution? The other would obviously be to have different versions of rcpp on CRAN, similar to ggplot or oxygen. I can see this working. But there could be Rcpp_0_12_0, Rcpp_0_12_1, etc .. as different packages so that developers who want to update their package would need to change the version on which they depend. Well - I would stick with the usual system, that would mean Rcpp_0_12, Rcpp_0_13, ... and no further fine grained. Not sure CRAN would be impressed if I start releasing a new package for each version. Agreed. Cheers, Rainer Martyn On Tue, 2014-04-08 at 10:12 +0200, Xavier Robin wrote: My 2 cents... On 07/04/14 10:12, Romain François wrote: It would also mean many copies of the same code base. To which I’m thinking: so what. No, it will mean many copies of /many different and mostly outdated/ code bases. You can count on me to forget to git pull next time I update my package. What about something like the BH package that contains the boost headers? I'm using it in a project I'm working on, and just use a LinkingTo declaration with something in Makevars. Of course ideally it would be in a build-depends type of declaration so it isn't pulled during binary installs. Xavier ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D):+49 - (0)3 21 21 25 22 44 email: rai...@krugs.de Skype: RMkrug PGP: 0x0F52F982 ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44
Re: [Rcpp-devel] Distribution of Rcpp codebase
Le 8 avr. 2014 à 11:47, Rainer M Krug rai...@krugs.de a écrit : Romain François rom...@r-enthusiasts.com writes: Le 8 avr. 2014 à 10:51, Rainer M Krug rai...@krugs.de a écrit : Martyn Plummer plumm...@iarc.fr writes: And another 2 cents from me. A package is the basic unit of functionality in R. Whatever functionality you are providing, I think a package is the best way to deliver it. There is a well developed framework for versioning, dependency resolution, testing, and distribution. If you choose some other mechanism, then I suspect that any developer problems you eliminate will just return as system administration problems. I thought I could keep out of this... rcpp... is so much the core of many packages, that the mechanism of packages really shows its shortcomings, namely the versioning. A different aspect was discussed on the r-devel list some time ago concerning snapshots of CRAN to provide reproducibility. The part which makes rcpp outstanding from other packages, is that it is used mainly during compilation and not that much during running - that is if I understand the header only concept correctly. Well with the current Rcpp, you still need the runtime for any of the functions that are registered here: https://github.com/RcppCore/Rcpp/blob/master/src/Rcpp_init.cpp#L85 That’s what I meant earlier when I mentioned job half done. The intention of the change in 0.11.0 was to move away from runtime, but we did not go far enough. I must admit, I am not to familiar with rcpp to comment on these functions, but would it be possible to split Rcpp into headers-only, i.e. only for compiling and developing, and Rcpp-runtime for runtime? Because I really think it might make sense to have additional function at runtime? They are really too tightly coupled for that. I’ve been through making Rcpp11 truly header only and it was really hard. It was worth every second of it, but still hard. Doing it for Rcpp is impossible, at least with Dirk’s maintainer requirements. In contrast, as far as I know, all other packages provide run-time features as well as compile time - and this is what the packages were designed for - and I might say they work quite well. I agree. I can see Romain's point (oparticularly the more fine grained control of the developer), but I also understand that Dirk will keep rcpp as a package. Sure. My question is: would it be possible to combine these two, i.e. have one package, which contains different versions, and to use one command to switch between these? So one would have one package, which will switch between different versions? I see your point, but I would not want to do that. I can definitely foresee a whole new class of issues with this scenario. I don't know if this would be possible or a suitable solution? The other would obviously be to have different versions of rcpp on CRAN, similar to ggplot or oxygen. I can see this working. But there could be Rcpp_0_12_0, Rcpp_0_12_1, etc .. as different packages so that developers who want to update their package would need to change the version on which they depend. Well - I would stick with the usual system, that would mean Rcpp_0_12, Rcpp_0_13, ... and no further fine grained. Same difference. I think this would be kind of confusing, but at least it would be allowed as « Rcpp » is not the same name as « Rcpp_0_12 ». As a side note, I plan to version Rcpp11 according to the version of R, so the first release of Rcpp11 will be version 3.1.0, ... Not sure CRAN would be impressed if I start releasing a new package for each version. Agreed. Cheers, Rainer Martyn On Tue, 2014-04-08 at 10:12 +0200, Xavier Robin wrote: My 2 cents... On 07/04/14 10:12, Romain François wrote: It would also mean many copies of the same code base. To which I’m thinking: so what. No, it will mean many copies of /many different and mostly outdated/ code bases. You can count on me to forget to git pull next time I update my package. What about something like the BH package that contains the boost headers? I'm using it in a project I'm working on, and just use a LinkingTo declaration with something in Makevars. Of course ideally it would be in a build-depends type of declaration so it isn't pulled during binary installs. Xavier ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33
Re: [Rcpp-devel] Distribution of Rcpp codebase
On Mon, Apr 7, 2014 at 4:12 AM, Romain François rom...@r-enthusiasts.comwrote: [...] However, in terms of wins: - package developers would know for sure which version of the codebase is used with their package. Once they have done testing, they don't have to be hostage of api breakage and things like please recompile your package, etc ... - developers of Rcpp* are less trapped by the compatibility issues, hands are set free to innovate. The only small downsides I see here is that (1) users potentially have to do more work to include Rcpp* in their packages (although you can just write an R function to include/update their Rcpp* versions); and that (2) source packages will be somewhat bigger. Btw. you are essentially simulating versioned package dependencies this way. :) The next thing to consider is that Rcpp is not just Rcpp, there are really nice extensions like RcppArmadillo, etc ... perhaps we could setup some tools (e.g. RcppJam) to combine several header only libraries into the end package, instead of what we do now, which is have some headers in Rcpp, some in RcppArmadillo, some in RcppGSL, ... with every risk of one being outdated or out of sync with the other. Exactly. IMHO this could work well and take the pressure of both Rcpp* developers and users. Gabor ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Le 7 avr. 2014 à 15:13, Gábor Csárdi csardi.ga...@gmail.com a écrit : On Mon, Apr 7, 2014 at 4:12 AM, Romain François rom...@r-enthusiasts.com wrote: [...] However, in terms of wins: - package developers would know for sure which version of the codebase is used with their package. Once they have done testing, they don’t have to be hostage of api breakage and things like « please recompile your package, etc … » - developers of Rcpp* are less trapped by the compatibility issues, hands are set free to innovate. The only small downsides I see here is that (1) users potentially have to do more work to include Rcpp* in their packages (although you can just write an R function to include/update their Rcpp* versions); and that (2) source packages will be somewhat bigger. The difference is that if they don’t want a new version, they can stick with the version they have in their package. So when they want newer Rcpp, they can have it, but they are not forced to do anything when the Rcpp* team wants to update the codebase. About 1). We just need to find a good way to propagate newer Rcpp* into a repo. Maybe through git subtree, git sub modules or something. We can come up with some tools. About 2). So What ;) Btw. you are essentially simulating versioned package dependencies this way. :) Yep. The next thing to consider is that Rcpp is not just Rcpp, there are really nice extensions like RcppArmadillo, etc ... perhaps we could setup some tools (e.g. RcppJam) to combine several header only libraries into the end package, instead of what we do now, which is have some headers in Rcpp, some in RcppArmadillo, some in RcppGSL, … with every risk of one being outdated or out of sync with the other. Exactly. IMHO this could work well and take the pressure of both Rcpp* developers and users. Gabor ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
On Mon, Apr 7, 2014 at 9:20 AM, Romain François rom...@r-enthusiasts.comwrote: [...] The only small downsides I see here is that (1) users potentially have to do more work to include Rcpp* in their packages (although you can just write an R function to include/update their Rcpp* versions); and that (2) source packages will be somewhat bigger. The difference is that if they don't want a new version, they can stick with the version they have in their package. So when they want newer Rcpp, they can have it, but they are not forced to do anything when the Rcpp* team wants to update the codebase. About 1). We just need to find a good way to propagate newer Rcpp* into a repo. Maybe through git subtree, git sub modules or something. We can come up with some tools. Oh, I see, you mean installing Rcpp* dependent packages from git(hub) directly. I imagine some people would want to put the headers into their tree, as ordinary files. Others might want to have it as submodules. For the latter to work, I imagine you would need to set up some special repo structure. Btw. I obviously don't have to tell you, but I want to point it out that this snapshotting is already possible today. I can just take the Rcpp* headers and put them in my package, and then I don't have to worry about future changes. This is assuming the package is header-only. With compiled code it is trickier, because of the name clashes. So I think, the question is whether to support this with some infrastructure, e.g. to update the embedded headers, have them as a git submodule, etc. About 2). So What ;) Agreed. CRAN-core might be fussy about it, though, if source packages get much bigger because of this. Or might be not. Gabor [...] ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Le 7 avr. 2014 à 19:20, Gábor Csárdi csardi.ga...@gmail.com a écrit : On Mon, Apr 7, 2014 at 9:20 AM, Romain François rom...@r-enthusiasts.com wrote: [...] The only small downsides I see here is that (1) users potentially have to do more work to include Rcpp* in their packages (although you can just write an R function to include/update their Rcpp* versions); and that (2) source packages will be somewhat bigger. The difference is that if they don’t want a new version, they can stick with the version they have in their package. So when they want newer Rcpp, they can have it, but they are not forced to do anything when the Rcpp* team wants to update the codebase. About 1). We just need to find a good way to propagate newer Rcpp* into a repo. Maybe through git subtree, git sub modules or something. We can come up with some tools. Oh, I see, you mean installing Rcpp* dependent packages from git(hub) directly. I imagine some people would want to put the headers into their tree, as ordinary files. Having the files into the client package is what I meant. This way, the developer of the package is in control of what version is used. The challenge is how to make it easy. Others might want to have it as submodules. For the latter to work, I imagine you would need to set up some special repo structure. I think git subtree gives something close to what I mean. But obviously someone could just copy the files manually, it would just be a matter of copying the inst/include directory. Btw. I obviously don't have to tell you, but I want to point it out that this snapshotting is already possible today. I can just take the Rcpp* headers and put them in my package, and then I don't have to worry about future changes. This is assuming the package is header-only. With compiled code it is trickier, because of the name clashes. Good luck doing that with current Rcpp. Many functions are compiled into the shared library shipped with the package. It is not linking as it used to do it, which is the big win of 0.11.0, but there is still a library. Many of these functions go through the namespace of the Rcpp package, etc … so you could snapshot Rcpp, but it would definitely not be easy at all. So I think, the question is whether to support this with some infrastructure, e.g. to update the embedded headers, have them as a git submodule, etc. About 2). So What ;) Agreed. CRAN-core might be fussy about it, though, if source packages get much bigger because of this. Or might be not. This is a drop of water in the ocean of CRAN packages. I think the benefit would outweigh this quite easily. In any case, not gonna happen, at least not with Rcpp. Romain ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Here's some thoughts, from the perspective of a package developer who might want to use Rcpp11. One option is to have the Rcpp11 distribution live as a git repository within the `inst/include` directory of a developer's package. A package developer could clone the repository (or have it track the inst/include subdirectory of Rcpp11 -- not sure if that's possible), and at their leisure pull the current version, pull a certain tag / release, and so on.). Maybe there's something that needs to be in .Rbuildignore to ensure the .git repository doesn't get added into the built package on R CMD build. Some R functions, e.g. useRcpp11(), could be written that, when run in the package directory, clones a repository in inst/include/Rcpp, and also updates pertinent licensing information (this package uses Rcpp11, which is licensing under so-and-so...). Essentially they would be nice wrappers to git. Maybe package these as part of an Rcpp11 CRAN / GitHub release. I'm not sure how well a mechanism like this would work for syncing across the other Rcpp libraries, e.g. RcppArmadillo, though. One should expect that packages depending on Rcpp11 will still want to be submitted to CRAN, and one potential problem (I know we could say 'not our problem', but we should be good citizens) is that, if R-Core decides to make an R API change that causes these packages to fail R CMD check, it will be more painful for them to accept maintenance releases from all these packages. Finally -- we might consider moving this discussion to a new mailing list, since I think Rcpp and Rcpp11 are divergent enough that we should consider Rcpp-devel for Rcpp-the-package-specific discussion, and a different mailing list for Rcpp11 development. Thoughts, Romain? On Mon, Apr 7, 2014 at 10:20 AM, Gábor Csárdi csardi.ga...@gmail.com wrote: On Mon, Apr 7, 2014 at 9:20 AM, Romain François rom...@r-enthusiasts.com wrote: [...] The only small downsides I see here is that (1) users potentially have to do more work to include Rcpp* in their packages (although you can just write an R function to include/update their Rcpp* versions); and that (2) source packages will be somewhat bigger. The difference is that if they don't want a new version, they can stick with the version they have in their package. So when they want newer Rcpp, they can have it, but they are not forced to do anything when the Rcpp* team wants to update the codebase. About 1). We just need to find a good way to propagate newer Rcpp* into a repo. Maybe through git subtree, git sub modules or something. We can come up with some tools. Oh, I see, you mean installing Rcpp* dependent packages from git(hub) directly. I imagine some people would want to put the headers into their tree, as ordinary files. Others might want to have it as submodules. For the latter to work, I imagine you would need to set up some special repo structure. Btw. I obviously don't have to tell you, but I want to point it out that this snapshotting is already possible today. I can just take the Rcpp* headers and put them in my package, and then I don't have to worry about future changes. This is assuming the package is header-only. With compiled code it is trickier, because of the name clashes. So I think, the question is whether to support this with some infrastructure, e.g. to update the embedded headers, have them as a git submodule, etc. About 2). So What ;) Agreed. CRAN-core might be fussy about it, though, if source packages get much bigger because of this. Or might be not. Gabor [...] ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
On Mon, Apr 7, 2014 at 2:19 PM, Romain François rom...@r-enthusiasts.comwrote: [...] Oh, I see, you mean installing Rcpp* dependent packages from git(hub) directly. I imagine some people would want to put the headers into their tree, as ordinary files. Having the files into the client package is what I meant. This way, the developer of the package is in control of what version is used. The challenge is how to make it easy. Sure, we are on the same page here. What I meant is that if the client package is on CRAN, then you'll have to include the Rcpp* header files in the package, obviously. Unless you create Rcpp* packages that contain multiple versions of the headers, in different folders. Personally, if I created a client package for Rcpp11 today, I would just copy over the header files, in a subfolder, and keep them unchanged, so that it is easy to update them with a delete/copy. If there is an RcppJam package that can update these files, perhaps add compilation flags, etc., that's also great. Btw. I obviously don't have to tell you, but I want to point it out that this snapshotting is already possible today. I can just take the Rcpp* headers and put them in my package, and then I don't have to worry about future changes. This is assuming the package is header-only. With compiled code it is trickier, because of the name clashes. Good luck doing that with current Rcpp. Many functions are compiled into the shared library shipped with the package. It is not linking as it used to do it, which is the big win of 0.11.0, but there is still a library. Many of these functions go through the namespace of the Rcpp package, etc ... so you could snapshot Rcpp, but it would definitely not be easy at all. I agree. With header-only packages it'll be easy. With compiled code it is trickier, but C++ is actually easier than C, because for the name clashes, you can just put the code in a new namespace. Then you'll need to redefine :: and .Call in your package, and even this might not be enough, especially for Rcpp. Gabor ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
Thanks to have interest in this discussion. Le 7 avr. 2014 à 20:30, Kevin Ushey kevinus...@gmail.com a écrit : Here's some thoughts, from the perspective of a package developer who might want to use Rcpp11. One option is to have the Rcpp11 distribution live as a git repository within the `inst/include` directory of a developer's package. A package developer could clone the repository (or have it track the inst/include subdirectory of Rcpp11 -- not sure if that's possible), and at their leisure pull the current version, pull a certain tag / release, and so on.). Maybe there's something that needs to be in .Rbuildignore to ensure the .git repository doesn't get added into the built package on R CMD build. Some R functions, e.g. useRcpp11(), could be written that, when run in the package directory, clones a repository in inst/include/Rcpp, and also updates pertinent licensing information (this package uses Rcpp11, which is licensing under so-and-so...). Essentially they would be nice wrappers to git. Maybe package these as part of an Rcpp11 CRAN / GitHub release. I think git subtree is the right tool. I'm not sure how well a mechanism like this would work for syncing across the other Rcpp libraries, e.g. RcppArmadillo, though. As long as these other are header only, that would be pretty easy. We’d have to diverge them as well as they carry various fastLm, etc ... which really don’t belong there (before the Rcpp police gets involves : I know they will stay). One should expect that packages depending on Rcpp11 will still want to be submitted to CRAN, and one potential problem (I know we could say 'not our problem', but we should be good citizens) is that, if R-Core decides to make an R API change that causes these packages to fail R CMD check, it will be more painful for them to accept maintenance releases from all these packages. There are guarantees on R api changes. I don’t see this as an issue. Finally -- we might consider moving this discussion to a new mailing list, since I think Rcpp and Rcpp11 are divergent enough that we should consider Rcpp-devel for Rcpp-the-package-specific discussion, and a different mailing list for Rcpp11 development. Thoughts, Romain? I though this mailing list was about discussing R and C++, but apparently I was wrong. Fair enough if this is just Rcpp related. Perhaps I’ll leave the mailing list as I’m not that interested in Rcpp anymore. Rcpp11 has its own github issues, this I think is enough for now, but sure maybe a new mailing list would be good. On Mon, Apr 7, 2014 at 10:20 AM, Gábor Csárdi csardi.ga...@gmail.com wrote: On Mon, Apr 7, 2014 at 9:20 AM, Romain François rom...@r-enthusiasts.com wrote: [...] The only small downsides I see here is that (1) users potentially have to do more work to include Rcpp* in their packages (although you can just write an R function to include/update their Rcpp* versions); and that (2) source packages will be somewhat bigger. The difference is that if they don't want a new version, they can stick with the version they have in their package. So when they want newer Rcpp, they can have it, but they are not forced to do anything when the Rcpp* team wants to update the codebase. About 1). We just need to find a good way to propagate newer Rcpp* into a repo. Maybe through git subtree, git sub modules or something. We can come up with some tools. Oh, I see, you mean installing Rcpp* dependent packages from git(hub) directly. I imagine some people would want to put the headers into their tree, as ordinary files. Others might want to have it as submodules. For the latter to work, I imagine you would need to set up some special repo structure. Btw. I obviously don't have to tell you, but I want to point it out that this snapshotting is already possible today. I can just take the Rcpp* headers and put them in my package, and then I don't have to worry about future changes. This is assuming the package is header-only. With compiled code it is trickier, because of the name clashes. So I think, the question is whether to support this with some infrastructure, e.g. to update the embedded headers, have them as a git submodule, etc. About 2). So What ;) Agreed. CRAN-core might be fussy about it, though, if source packages get much bigger because of this. Or might be not. Gabor [...] ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] Distribution of Rcpp codebase
On Mon, Apr 7, 2014 at 2:59 PM, Romain François rom...@r-enthusiasts.comwrote: [...] Some R functions, e.g. useRcpp11(), could be written that, when run in the package directory, clones a repository in inst/include/Rcpp, and also updates pertinent licensing information (this package uses Rcpp11, which is licensing under so-and-so...). Essentially they would be nice wrappers to git. Maybe package these as part of an Rcpp11 CRAN / GitHub release. I think git subtree is the right tool. Personally, I would not make git compulsory for developers of clients, and I would just add/modify R code in Rcpp11 to download/update the clones and include them in the client. I think it is simpler than setting up git subtree, etc. One should expect that packages depending on Rcpp11 will still want to be submitted to CRAN, and one potential problem (I know we could say 'not our problem', but we should be good citizens) is that, if R-Core decides to make an R API change that causes these packages to fail R CMD check, it will be more painful for them to accept maintenance releases from all these packages. They can just update to a newer embedded Rcpp11. You can keep two Rcpp11 branches alive if you want, so that users can upgrade to a bug-fix version without breaking the Rcpp11 API in their package. Gabor ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
