Re: [Rd] src/Makevars ignored ?
You're then asking CRAN to violate your "ideal contract" w/r/t compiler switching inside src/Makevars since CRAN needs to setup and produce standard, predictable, repeatable builds, including binary generation for two platforms (much to Dirk's chagrin, there _are_ other operating systems besides Debian linux ;-) I wouldn't want CRAN to honor said compiler switching inside src/Makevars but the way you're describing the perceived contract, you're implying they should. I totally understand your posit, but there's a larger universe to consider than internal package builds. In a sense you really want an R version of Rule of Acquisition #17 "A contract is a contract is a contract... but only on my internal systems." and that doesn't seem like a good idea to me when the larger universe of CRAN and the need for baseline standard package builds are considered. On Tue, Sep 27, 2016 at 11:19 AM, Kasper Daniel Hansen < kasperdanielhan...@gmail.com> wrote: > So now we have some important context for your original question. > > I understand why you want "symmetry" but because of the reasons Dirk > outlines, I personally think it is a bad idea, ie. I disagree with your > statement "anyway I still convinced that if R provides a mechanisn > hierachical way of variable overwrite pkg / user/ system it _SHOULD_ be > consistent at all levels". > > The mechanism for achieving what you want - overriding CC on a local > machine - is using a ~/.R/Makevars file. I am not sure that this can be > done in a package specific manner (I don;t do this very often), but you > could create this file ('this file" being ~/.R/Makevars), install the > package and then remove it. Remember, this is an install-time setting, so > you don't need the users of your system to be involved in this. That way > you could test your proposed fix. If the fix works, it seems to me like it > should be included in the package source by the package maintainer, perhaps > using a configure script, but that is ultimately something which is up to > the package maintainer. > > Best, > Kasper > > On Tue, Sep 27, 2016 at 10:52 AM, Eric Deveaud> wrote: > > > Le 27/09/16 à 16:17, Kasper Daniel Hansen a écrit : > > > >> As a package author, it is in my opinion irresponsible to override these > >> system settings (which is why it is also impossible). You have no idea > >> what system it is being deployed on, > >> > > > > as the it guy dedicated to install and maintain softs on our cluster I > > have a reasonable knowledge of the systems I work on. > > > > I don't want to distribute any of the piece of code I was asked to made > > available on the cluster. > > I just need (and succeded plus functional test succeded) to build the > > Rpackage requested by a specific software. > > > > > > I mean, you don't even know if the > >> compiler is gcc.If a user wants (say) heavy optimization they will > >> compile R with optimization. (For this reason I also don't think users > >> should modify their ~/.R/Makevars, unless for testing purposes). > >> > > > > my question was not in a R package developer context, but in the R user > > that grabs some piece of code and is not abble to compile it because of > > 1) a developper that mixed C and C++ code which is legit. > > 2) a silly interaction beetween C and C++ symbol generation because of > the > > use, in our case, of CC = gcc -std=gnu99 > > 3) a dev that answwer: "I have no clue, in debian it works" ;-( > > > > anyway I still convinced that if R provides a mechanisn hierachical way > of > > variable overwrite pkg / user/ system it _SHOULD_ be consistent at all > > levels > > > > my question was raised because of our install mechanism that (hopefully) > > does not allow > > modification of files like ~/.R/Makevars. > > I can only "play" with the sources of the software it is working on > > and//or environment variables. so I wanted to have a temporary way of > > setting CC to be plain gcc without ISO C99 language standard support just > > for this specific R library. > > > > > > The exception is (in my opinion) if you need to decrease the level of > >> optimization because you know or suspect the compiler to generate wrong > >> code. > >> > >> What you should do, is use > >> PKG_CFLAGS > >> as documented in R-exts, 1.2.1 under "Using Makevars". > >> > > > > in the documentation you pointed (and trust me I read it), keyword is > _set > > additional_ preprocessor options and//or compiler flags > > > > only way to _remove_ is to overwrite > > > > back to logic. > > > > either Makevars, whatever level, allow to overwrite CC definition > > either Makevars, whatever level, disable CC redefinition > > > > but not a mix > > > > > > Eric > > > > __ > > R-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel > > > > [[alternative HTML version deleted]] > > __ >
Re: [Rd] src/Makevars ignored ?
So now we have some important context for your original question. I understand why you want "symmetry" but because of the reasons Dirk outlines, I personally think it is a bad idea, ie. I disagree with your statement "anyway I still convinced that if R provides a mechanisn hierachical way of variable overwrite pkg / user/ system it _SHOULD_ be consistent at all levels". The mechanism for achieving what you want - overriding CC on a local machine - is using a ~/.R/Makevars file. I am not sure that this can be done in a package specific manner (I don;t do this very often), but you could create this file ('this file" being ~/.R/Makevars), install the package and then remove it. Remember, this is an install-time setting, so you don't need the users of your system to be involved in this. That way you could test your proposed fix. If the fix works, it seems to me like it should be included in the package source by the package maintainer, perhaps using a configure script, but that is ultimately something which is up to the package maintainer. Best, Kasper On Tue, Sep 27, 2016 at 10:52 AM, Eric Deveaudwrote: > Le 27/09/16 à 16:17, Kasper Daniel Hansen a écrit : > >> As a package author, it is in my opinion irresponsible to override these >> system settings (which is why it is also impossible). You have no idea >> what system it is being deployed on, >> > > as the it guy dedicated to install and maintain softs on our cluster I > have a reasonable knowledge of the systems I work on. > > I don't want to distribute any of the piece of code I was asked to made > available on the cluster. > I just need (and succeded plus functional test succeded) to build the > Rpackage requested by a specific software. > > > I mean, you don't even know if the >> compiler is gcc.If a user wants (say) heavy optimization they will >> compile R with optimization. (For this reason I also don't think users >> should modify their ~/.R/Makevars, unless for testing purposes). >> > > my question was not in a R package developer context, but in the R user > that grabs some piece of code and is not abble to compile it because of > 1) a developper that mixed C and C++ code which is legit. > 2) a silly interaction beetween C and C++ symbol generation because of the > use, in our case, of CC = gcc -std=gnu99 > 3) a dev that answwer: "I have no clue, in debian it works" ;-( > > anyway I still convinced that if R provides a mechanisn hierachical way of > variable overwrite pkg / user/ system it _SHOULD_ be consistent at all > levels > > my question was raised because of our install mechanism that (hopefully) > does not allow > modification of files like ~/.R/Makevars. > I can only "play" with the sources of the software it is working on > and//or environment variables. so I wanted to have a temporary way of > setting CC to be plain gcc without ISO C99 language standard support just > for this specific R library. > > > The exception is (in my opinion) if you need to decrease the level of >> optimization because you know or suspect the compiler to generate wrong >> code. >> >> What you should do, is use >> PKG_CFLAGS >> as documented in R-exts, 1.2.1 under "Using Makevars". >> > > in the documentation you pointed (and trust me I read it), keyword is _set > additional_ preprocessor options and//or compiler flags > > only way to _remove_ is to overwrite > > back to logic. > > either Makevars, whatever level, allow to overwrite CC definition > either Makevars, whatever level, disable CC redefinition > > but not a mix > > > Eric > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] src/Makevars ignored ?
Le 27/09/16 à 16:17, Kasper Daniel Hansen a écrit : As a package author, it is in my opinion irresponsible to override these system settings (which is why it is also impossible). You have no idea what system it is being deployed on, as the it guy dedicated to install and maintain softs on our cluster I have a reasonable knowledge of the systems I work on. I don't want to distribute any of the piece of code I was asked to made available on the cluster. I just need (and succeded plus functional test succeded) to build the Rpackage requested by a specific software. I mean, you don't even know if the compiler is gcc.If a user wants (say) heavy optimization they will compile R with optimization. (For this reason I also don't think users should modify their ~/.R/Makevars, unless for testing purposes). my question was not in a R package developer context, but in the R user that grabs some piece of code and is not abble to compile it because of 1) a developper that mixed C and C++ code which is legit. 2) a silly interaction beetween C and C++ symbol generation because of the use, in our case, of CC = gcc -std=gnu99 3) a dev that answwer: "I have no clue, in debian it works" ;-( anyway I still convinced that if R provides a mechanisn hierachical way of variable overwrite pkg / user/ system it _SHOULD_ be consistent at all levels my question was raised because of our install mechanism that (hopefully) does not allow modification of files like ~/.R/Makevars. I can only "play" with the sources of the software it is working on and//or environment variables. so I wanted to have a temporary way of setting CC to be plain gcc without ISO C99 language standard support just for this specific R library. The exception is (in my opinion) if you need to decrease the level of optimization because you know or suspect the compiler to generate wrong code. What you should do, is use PKG_CFLAGS as documented in R-exts, 1.2.1 under "Using Makevars". in the documentation you pointed (and trust me I read it), keyword is _set additional_ preprocessor options and//or compiler flags only way to _remove_ is to overwrite back to logic. either Makevars, whatever level, allow to overwrite CC definition either Makevars, whatever level, disable CC redefinition but not a mix Eric __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] src/Makevars ignored ?
Le 27/09/16 à 16:30, Dirk Eddelbuettel a écrit : On 27 September 2016 at 15:23, Eric Deveaud wrote: | so why ~/R/Makevars allows to change CC and not src/Makevars ? | | this is pretty confusing. It seems weird at first, but makes some sense when you think about it like this: -- src/Makevars is inside a package and cannot / should not alter "system" parameters like compiler brand or version as there may only be one compiler, the one R was built with it does allow however to set compilation flags, language standards, include directories etc as needed to build the package -- (user- or system-level) Makeconf allow compiler changes, but that is orthogonal to the per-package config and stays local to the machine yep I fully aggree with this. | in unix world mechanism of overwriting system default in user and | package order is something standadr and consistent given the level | | priority is package specific stuff overwrite user configuration that | overwrite system default | | mechanism of Makevars is NOT consistent Yes we sometimes wish we could override system Makeconf settings in a package, but we can't. | as a side note I solved the problem of overwritting CC for this | particular package using MAKEFLAGS="CC=gcc" R CMD INSTALL package That works and is documented, but is not "portable" to other machines. I realize based on Kasper's answer and your's that I was not clear on the context I was working. Eric __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] src/Makevars ignored ?
On 27 September 2016 at 15:23, Eric Deveaud wrote: | so why ~/R/Makevars allows to change CC and not src/Makevars ? | | this is pretty confusing. It seems weird at first, but makes some sense when you think about it like this: -- src/Makevars is inside a package and cannot / should not alter "system" parameters like compiler brand or version as there may only be one compiler, the one R was built with it does allow however to set compilation flags, language standards, include directories etc as needed to build the package -- (user- or system-level) Makeconf allow compiler changes, but that is orthogonal to the per-package config and stays local to the machine | in unix world mechanism of overwriting system default in user and | package order is something standadr and consistent given the level | | priority is package specific stuff overwrite user configuration that | overwrite system default | | mechanism of Makevars is NOT consistent Yes we sometimes wish we could override system Makeconf settings in a package, but we can't. | as a side note I solved the problem of overwritting CC for this | particular package using MAKEFLAGS="CC=gcc" R CMD INSTALL package That works and is documented, but is not "portable" to other machines. Hope this helps, Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] src/Makevars ignored ?
As a package author, it is in my opinion irresponsible to override these system settings (which is why it is also impossible). You have no idea what system it is being deployed on, I mean, you don't even know if the compiler is gcc. If a user wants (say) heavy optimization they will compile R with optimization. (For this reason I also don't think users should modify their ~/.R/Makevars, unless for testing purposes). The exception is (in my opinion) if you need to decrease the level of optimization because you know or suspect the compiler to generate wrong code. What you should do, is use PKG_CFLAGS as documented in R-exts, 1.2.1 under "Using Makevars". Best, Kasper On Tue, Sep 27, 2016 at 9:23 AM, Eric Deveaudwrote: > Le 27/09/16 à 13:31, Dirk Eddelbuettel a écrit : > > >> On 27 September 2016 at 09:37, Eric Deveaud wrote: >> | Hello, >> | >> | I'm tring to install a Rpackage that holds some C//C++ code >> | >> | as far as I understood the R library generic compilation mechanism, >> | compilation of C//C++ sources is controled >> | >> | 1) at system level by the ocntentos RHOME/etc/Makeconf >> | 2) at user level by the content of ~/.R/Makevars >> | 3) at package level by the content of src/Makevars >> | >> | Problem I have is that src/Makevars is ignored >> | >> | >> | see following example: >> | >> | R is compiled and use the following CC and CFLAGS definition >> | >> | bigmess:epactsR/src > R CMD config CC >> | gcc -std=gnu99 >> | bigmess:epactsR/src > R CMD config CFLAGS >> | -Wall -g >> | >> | so building C sources lead to the following >> | >> | bigmess:epactsR/src > R CMD SHLIB index.c >> | gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG >> | -I/usr/local/include-fpic -Wall -g -c index.c -o index.o >> | >> | normal, it uses defintion from RHOME/etc/Makeconf >> | >> | >> | when I set upp a ~/.R/Makevars that overwrite CC and CFLAGS definition. >> | >> | bigmess:epactsR/src > cat ~/.R/Makevars >> | CC=gcc >> | CFLAGS=-O3 >> | bigmess:epactsR/src > R CMD SHLIB index.c >> | gcc -I/local/gensoft2/adm/lib64/R/include -DNDEBUG >> -I/usr/local/include >> | -fpic -O3 -c index.c -o index.o >> | gcc -std=gnu99 -shared -L/usr/local/lib64 -o index.so index.o >> | >> | >> | OK CC and CFLAGS are honored and set accordingly to ~/.R/Makevars >> | >> | >> | but when I try to use src/Makevars, it is ignored >> | >> | bigmess:epactsR/src > cat ~/.R/Makevars >> | cat: /home/edeveaud/.R/Makevars: No such file or directory >> | bigmess:epactsR/src > cat ./Makevars >> | CC = gcc >> | CFLAGS=-O3 >> | bigmess:epactsR/src > R CMD SHLIB index.c >> | gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG >> | -I/usr/local/include-fpic -Wall -g -c index.c -o index.o >> | >> | >> | is there something I have missed, misunderstood or is there something >> | wrong ? >> >> You have not demonstrated that src/Makevars is ignored -- as it clearly >> isn't, given how thousands of CRAN packages use it. >> >> What you have done is demonstrate that you _cannot change CC and CXX_ in >> src/Makevars. And I think that was known, though maybe not as widely. >> > > > Hello Dirk, > > > so why ~/R/Makevars allows to change CC and not src/Makevars ? > > this is pretty confusing. > > in unix world mechanism of overwriting system default in user and package > order is something standadr and consistent given the level > > priority is package specific stuff overwrite user configuration that > overwrite system default > > mechanism of Makevars is NOT consistent > > > furthermore diggng in $RHOME/bin/config one can see the following > > > if test "${site}" = "yes"; then > : ${R_MAKEVARS_SITE="${R_HOME}/etc${R_ARCH}/Makevars.site"} > if test -f "${R_MAKEVARS_SITE}"; then > makefiles="${makefiles} -f ${R_MAKEVARS_SITE}" > fi > fi > if test "${personal}" = "yes"; then > if test "${R_OSTYPE}" = "windows"; then > if test -f "${R_MAKEVARS_USER}"; then > makefiles="${makefiles} -f ${R_MAKEVARS_USER}" > elif test ${R_ARCH} = "/x64" -a -f "${HOME}/.R/Makevars.win64"; then > makefiles="${makefiles} -f ${HOME}/.R/Makevars.win64" > elif test -f "${HOME}/.R/Makevars.win"; then > makefiles="${makefiles} -f ${HOME}/.R/Makevars.win" > elif test -f "${HOME}/.R/Makevars"; then > makefiles="${makefiles} -f ${HOME}/.R/Makevars" > fi > else > . ${R_HOME}/etc${R_ARCH}/Renviron > if test -f "${R_MAKEVARS_USER}"; then > makefiles="${makefiles} -f ${R_MAKEVARS_USER}" > elif test -f "${HOME}/.R/Makevars-${R_PLATFORM}"; then > makefiles="${makefiles} -f ${HOME}/.R/Makevars-${R_PLATFORM}" > elif test -f "${HOME}/.R/Makevars"; then > makefiles="${makefiles} -f ${HOME}/.R/Makevars" > fi > fi > fi > > it checks and honours R_MAKEVARS_USER, ~/.R/Makevars, but not src/Makevars > I'm not really familiar with the R policy about code comilation, but I > consider this a bug but maybee I'm wrong > > > as a side note I
Re: [Rd] src/Makevars ignored ?
Le 27/09/16 à 13:31, Dirk Eddelbuettel a écrit : On 27 September 2016 at 09:37, Eric Deveaud wrote: | Hello, | | I'm tring to install a Rpackage that holds some C//C++ code | | as far as I understood the R library generic compilation mechanism, | compilation of C//C++ sources is controled | | 1) at system level by the ocntentos RHOME/etc/Makeconf | 2) at user level by the content of ~/.R/Makevars | 3) at package level by the content of src/Makevars | | Problem I have is that src/Makevars is ignored | | | see following example: | | R is compiled and use the following CC and CFLAGS definition | | bigmess:epactsR/src > R CMD config CC | gcc -std=gnu99 | bigmess:epactsR/src > R CMD config CFLAGS | -Wall -g | | so building C sources lead to the following | | bigmess:epactsR/src > R CMD SHLIB index.c | gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG | -I/usr/local/include-fpic -Wall -g -c index.c -o index.o | | normal, it uses defintion from RHOME/etc/Makeconf | | | when I set upp a ~/.R/Makevars that overwrite CC and CFLAGS definition. | | bigmess:epactsR/src > cat ~/.R/Makevars | CC=gcc | CFLAGS=-O3 | bigmess:epactsR/src > R CMD SHLIB index.c | gcc -I/local/gensoft2/adm/lib64/R/include -DNDEBUG -I/usr/local/include | -fpic -O3 -c index.c -o index.o | gcc -std=gnu99 -shared -L/usr/local/lib64 -o index.so index.o | | | OK CC and CFLAGS are honored and set accordingly to ~/.R/Makevars | | | but when I try to use src/Makevars, it is ignored | | bigmess:epactsR/src > cat ~/.R/Makevars | cat: /home/edeveaud/.R/Makevars: No such file or directory | bigmess:epactsR/src > cat ./Makevars | CC = gcc | CFLAGS=-O3 | bigmess:epactsR/src > R CMD SHLIB index.c | gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG | -I/usr/local/include-fpic -Wall -g -c index.c -o index.o | | | is there something I have missed, misunderstood or is there something | wrong ? You have not demonstrated that src/Makevars is ignored -- as it clearly isn't, given how thousands of CRAN packages use it. What you have done is demonstrate that you _cannot change CC and CXX_ in src/Makevars. And I think that was known, though maybe not as widely. Hello Dirk, so why ~/R/Makevars allows to change CC and not src/Makevars ? this is pretty confusing. in unix world mechanism of overwriting system default in user and package order is something standadr and consistent given the level priority is package specific stuff overwrite user configuration that overwrite system default mechanism of Makevars is NOT consistent furthermore diggng in $RHOME/bin/config one can see the following if test "${site}" = "yes"; then : ${R_MAKEVARS_SITE="${R_HOME}/etc${R_ARCH}/Makevars.site"} if test -f "${R_MAKEVARS_SITE}"; then makefiles="${makefiles} -f ${R_MAKEVARS_SITE}" fi fi if test "${personal}" = "yes"; then if test "${R_OSTYPE}" = "windows"; then if test -f "${R_MAKEVARS_USER}"; then makefiles="${makefiles} -f ${R_MAKEVARS_USER}" elif test ${R_ARCH} = "/x64" -a -f "${HOME}/.R/Makevars.win64"; then makefiles="${makefiles} -f ${HOME}/.R/Makevars.win64" elif test -f "${HOME}/.R/Makevars.win"; then makefiles="${makefiles} -f ${HOME}/.R/Makevars.win" elif test -f "${HOME}/.R/Makevars"; then makefiles="${makefiles} -f ${HOME}/.R/Makevars" fi else . ${R_HOME}/etc${R_ARCH}/Renviron if test -f "${R_MAKEVARS_USER}"; then makefiles="${makefiles} -f ${R_MAKEVARS_USER}" elif test -f "${HOME}/.R/Makevars-${R_PLATFORM}"; then makefiles="${makefiles} -f ${HOME}/.R/Makevars-${R_PLATFORM}" elif test -f "${HOME}/.R/Makevars"; then makefiles="${makefiles} -f ${HOME}/.R/Makevars" fi fi fi it checks and honours R_MAKEVARS_USER, ~/.R/Makevars, but not src/Makevars I'm not really familiar with the R policy about code comilation, but I consider this a bug but maybee I'm wrong as a side note I solved the problem of overwritting CC for this particular package using MAKEFLAGS="CC=gcc" R CMD INSTALL package best regards Eric __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] src/Makevars ignored ?
On 27 September 2016 at 09:37, Eric Deveaud wrote: | Hello, | | I'm tring to install a Rpackage that holds some C//C++ code | | as far as I understood the R library generic compilation mechanism, | compilation of C//C++ sources is controled | | 1) at system level by the ocntentos RHOME/etc/Makeconf | 2) at user level by the content of ~/.R/Makevars | 3) at package level by the content of src/Makevars | | Problem I have is that src/Makevars is ignored | | | see following example: | | R is compiled and use the following CC and CFLAGS definition | | bigmess:epactsR/src > R CMD config CC | gcc -std=gnu99 | bigmess:epactsR/src > R CMD config CFLAGS | -Wall -g | | so building C sources lead to the following | | bigmess:epactsR/src > R CMD SHLIB index.c | gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG | -I/usr/local/include-fpic -Wall -g -c index.c -o index.o | | normal, it uses defintion from RHOME/etc/Makeconf | | | when I set upp a ~/.R/Makevars that overwrite CC and CFLAGS definition. | | bigmess:epactsR/src > cat ~/.R/Makevars | CC=gcc | CFLAGS=-O3 | bigmess:epactsR/src > R CMD SHLIB index.c | gcc -I/local/gensoft2/adm/lib64/R/include -DNDEBUG -I/usr/local/include | -fpic -O3 -c index.c -o index.o | gcc -std=gnu99 -shared -L/usr/local/lib64 -o index.so index.o | | | OK CC and CFLAGS are honored and set accordingly to ~/.R/Makevars | | | but when I try to use src/Makevars, it is ignored | | bigmess:epactsR/src > cat ~/.R/Makevars | cat: /home/edeveaud/.R/Makevars: No such file or directory | bigmess:epactsR/src > cat ./Makevars | CC = gcc | CFLAGS=-O3 | bigmess:epactsR/src > R CMD SHLIB index.c | gcc -std=gnu99 -I/local/gensoft2/adm/lib64/R/include -DNDEBUG | -I/usr/local/include-fpic -Wall -g -c index.c -o index.o | | | is there something I have missed, misunderstood or is there something | wrong ? You have not demonstrated that src/Makevars is ignored -- as it clearly isn't, given how thousands of CRAN packages use it. What you have done is demonstrate that you _cannot change CC and CXX_ in src/Makevars. And I think that was known, though maybe not as widely. Dirk | PS I tested the ssame behaviour with various version of R from R/2.15 to | R/3.3 | | best regards | | Eric | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel