[Rd] Building packages
I've started a new package and I'm trying to work out the best way to do it. I'm managing my package source directory with SVN, but "R CMD build" likes to dump things in the inst/doc directory when making vignette PDF files. I don't want to keep these in SVN (they aren't strictly 'source'), so it set me thinking. One of the other projects I work with has an out-of-source build system. You make a 'build' directory, run a config system (cmake-based) and then 'make' does everything in the build directory without touching the source tree. Very nice and neat. How much work would it take to have something similar for building R packages? At present I've just got some svn:ignore settings to stop SVN bothering me. I also hit the problem of vignettes needing the package to be installed before being able to build them, but not being able to install the package because the vignettes wouldn't build without the package already being installed. The fix is to build with --no-vignettes, then install the package, then build with the vignettes enabled. Seems kludgy, plus it means that vignettes are always built with the currently installed package and not the currently-being-installed package. So I install and do a second pass to get it all right again. Or am I doing it wrong? Once I get smooth running of R package development and SVN I might write it up for R-newsletter - there's a couple of other tricks I've had to employ... Barry __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages
Oleg Sklyar wrote: > If I am not mistaken R CMD build builds the package temporarily and uses > that build to build the vignette, so where is the problem? All my > vignettes build fine on both Linux and Windows and on Windows you > actually see that running R CMD build --binary builds the source code > two times - exactly for the above purposes. Ah ha. I'm building as a user and so I've been installing into a private library: ~/Rlibs. Hence my vignette has had library(foo,lib="~/Rlibs"). I was unaware that it would get the currently-being-built package in a library of its own! Thanks! www.doingitwrong.com Barry __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages
These files in the SVN tree does not harm the things that are checked in. However it is indeed reasonable to keep the rubbish out, so: > I've started a new package and I'm trying to work out the best way to do > it. I'm managing my package source directory with SVN, but "R CMD build" > likes to dump things in the inst/doc directory when making vignette PDF > files. I don't want to keep these in SVN (they aren't strictly > 'source'), so it set me thinking. Solution 1: copy the package SVN dir elsewhere and build/install from there Solution 2: a better one, make a 2-liner shell script that runs solution 1 (what I do) This will also prevent gcc from populating your svn src directory with .o, .so, .d, .dll files. > One of the other projects I work with has an out-of-source build system. > You make a 'build' directory, run a config system (cmake-based) and then > 'make' does everything in the build directory without touching the > source tree. Very nice and neat. How much work would it take to have > something similar for building R packages? At present I've just got some > svn:ignore settings to stop SVN bothering me. R does understand 'configure' which is more reasonable then to require cmake to be installed. Think of multiplatform builds etc. > I also hit the problem of vignettes needing the package to be > installed before being able to build them, but not being able to install > the package because the vignettes wouldn't build without the package > already being installed. The fix is to build with --no-vignettes, then > install the package, then build with the vignettes enabled. Seems > kludgy, plus it means that vignettes are always built with the currently > installed package and not the currently-being-installed package. So I > install and do a second pass to get it all right again. If I am not mistaken R CMD build builds the package temporarily and uses that build to build the vignette, so where is the problem? All my vignettes build fine on both Linux and Windows and on Windows you actually see that running R CMD build --binary builds the source code two times - exactly for the above purposes. > > Or am I doing it wrong? > > Once I get smooth running of R package development and SVN I might > write it up for R-newsletter - there's a couple of other tricks I've had > to employ... What exactly? > > Barry > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Dr Oleg Sklyar * EBI-EMBL, Cambridge CB10 1SD, UK * +44-1223-494466 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages
Gabor Grothendieck wrote: > An svn checkout directory can contain a mix of files that > are mirrored in the svn and not mirrored. In particular, if you > add a new file into your checkout directory it will not automatically > go into the repository on your next commit unless you specifically > place that file under svn control so junk files remain local. True, but 'svn status' will keep annoying you with: ? inst/doc/foo.eps until you tell it to ignore it ["svn propedit svn:ignore ." and then enter some expressions]. Barry __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages
An svn checkout directory can contain a mix of files that are mirrored in the svn and not mirrored. In particular, if you add a new file into your checkout directory it will not automatically go into the repository on your next commit unless you specifically place that file under svn control so junk files remain local. You can exclude files from R CMD build using the .Rbuildignore file. See the Writing Extensions manual. On Dec 7, 2007 11:07 AM, Barry Rowlingson <[EMAIL PROTECTED]> wrote: > I've started a new package and I'm trying to work out the best way to do > it. I'm managing my package source directory with SVN, but "R CMD build" > likes to dump things in the inst/doc directory when making vignette PDF > files. I don't want to keep these in SVN (they aren't strictly > 'source'), so it set me thinking. > > One of the other projects I work with has an out-of-source build system. > You make a 'build' directory, run a config system (cmake-based) and then > 'make' does everything in the build directory without touching the > source tree. Very nice and neat. How much work would it take to have > something similar for building R packages? At present I've just got some > svn:ignore settings to stop SVN bothering me. > > I also hit the problem of vignettes needing the package to be > installed before being able to build them, but not being able to install > the package because the vignettes wouldn't build without the package > already being installed. The fix is to build with --no-vignettes, then > install the package, then build with the vignettes enabled. Seems > kludgy, plus it means that vignettes are always built with the currently > installed package and not the currently-being-installed package. So I > install and do a second pass to get it all right again. > > Or am I doing it wrong? > > Once I get smooth running of R package development and SVN I might > write it up for R-newsletter - there's a couple of other tricks I've had > to employ... > > Barry > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages
On 12/7/07, Barry Rowlingson <[EMAIL PROTECTED]> wrote: > Gabor Grothendieck wrote: > > An svn checkout directory can contain a mix of files that > > are mirrored in the svn and not mirrored. In particular, if you > > add a new file into your checkout directory it will not automatically > > go into the repository on your next commit unless you specifically > > place that file under svn control so junk files remain local. > > True, but 'svn status' will keep annoying you with: > > ? inst/doc/foo.eps > > until you tell it to ignore it ["svn propedit svn:ignore ." and then > enter some expressions]. Yes, but that's completely normal svn operation - you ignore the non source files so that they don't interfere with your view of the source files. You particularly need this when working with latex. I have alias svnignore='svn pe svn:ignore' in my .profile to save a little typing. Hadley -- http://had.co.nz/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Building packages on Windows fails
What has changed in R for Windows from version 1.7.1 to 2.2.0 that won't allow me to build binary packages? On 1.7.1 I built a packge with this command: Rcmd build --force --binary iL04 On 2.2.0 this fails. First I had to copy sh.exe from d:\bin to c:\bin. This got me over the first hurdle. But then I got stuck at a later point. Here are the last lines of output: -- Making package iL04 latex: not found adding build stamp to DESCRIPTION latex: not found latex: not found latex: not found installing R files latex: not found installing demos installing data files latex: not found installing man source files installing indices cat: d:/R-2.2.0/library/*/CONTENTS: No such file or directory make[2]: *** [indices] Error 1 make[1]: *** [all] Error 2 make: *** [pkg-iL04] Error 2 *** Installation of iL04 failed *** -- Peter Kleiweg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages on Windows fails
On Fri, 7 Oct 2005, Peter Kleiweg wrote: > > What has changed in R for Windows from version 1.7.1 to > 2.2.0 that won't allow me to build binary packages? Many things have changed; I don't know which is causing the failure you see. One change is that instructions are now collected in the Installation and Administration manual. Try following the setup instructions there and see if it still fails. Duncan Murdoch > > On 1.7.1 I built a packge with this command: > >Rcmd build --force --binary iL04 > > On 2.2.0 this fails. First I had to copy sh.exe from d:\bin to > c:\bin. This got me over the first hurdle. But then I got stuck > at a later point. Here are the last lines of output: > > -- Making package iL04 > latex: not found > adding build stamp to DESCRIPTION > latex: not found > latex: not found > latex: not found > installing R files > latex: not found > installing demos > installing data files > latex: not found > installing man source files > installing indices > cat: d:/R-2.2.0/library/*/CONTENTS: No such file or directory > make[2]: *** [indices] Error 1 > make[1]: *** [all] Error 2 > make: *** [pkg-iL04] Error 2 > *** Installation of iL04 failed *** > > -- > Peter Kleiweg > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages on Windows fails
Duncan Murdoch schreef op de 6e dag van de wijnmaand van het jaar 2005: > On Fri, 7 Oct 2005, Peter Kleiweg wrote: > > > > > What has changed in R for Windows from version 1.7.1 to > > 2.2.0 that won't allow me to build binary packages? > > Many things have changed; I don't know which is causing the failure you see. > One change is that instructions are now collected in the Installation and > Administration manual. Try following the setup instructions there and see if > it still fails. I can't find anything on building packages for Windows in that manual. I did find a solution to the problem. On a Linux install, each package has a file CONTENTS. These are missing from the Windows install. I copied those files from my Linux install to my Windows install, and then I could build my own package. So I guess, these CONTENTS files should be included in the Windows install. I got another error message when I remove a package, but I couldn't see what effect this error has. When I run: d:\>Rcmd REMOVE iL04 The package gets removed, but I get: make: *** No rule to make target `indices'. Stop. > > -- Making package iL04 > > adding build stamp to DESCRIPTION > > installing R files > > installing demos > > installing data files > > installing man source files > > installing indices > > cat: d:/R-2.2.0/library/*/CONTENTS: No such file or directory > > make[2]: *** [indices] Error 1 > > make[1]: *** [all] Error 2 > > make: *** [pkg-iL04] Error 2 > > *** Installation of iL04 failed *** -- Peter Kleiweg http://www.let.rug.nl/~kleiweg/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages on Windows fails
On Fri, 7 Oct 2005, Peter Kleiweg wrote: > Duncan Murdoch schreef op de 6e dag van de wijnmaand van het jaar 2005: > >> On Fri, 7 Oct 2005, Peter Kleiweg wrote: >> >>> >>> What has changed in R for Windows from version 1.7.1 to >>> 2.2.0 that won't allow me to build binary packages? >> >> Many things have changed; I don't know which is causing the failure you see. >> One change is that instructions are now collected in the Installation and >> Administration manual. Try following the setup instructions there and see if >> it still fails. > > I can't find anything on building packages for Windows in that > manual. Your problems was installing, so the section on `Installing Packages' should help you. A binary build is an install plus wrapping-up. > I did find a solution to the problem. On a Linux install, each > package has a file CONTENTS. These are missing from the Windows > install. I copied those files from my Linux install to my > Windows install, and then I could build my own package. So I > guess, these CONTENTS files should be included in the Windows > install. And indeed they are, as the presence of 500+ packages on CRAN for Windows will show you. The recommended way to build a binary package on Windows is R CMD INSTALL --build since that is able to get HTML links made (on Windows, you need to install in the main library tree to do so). > > I got another error message when I remove a package, but I > couldn't see what effect this error has. When I run: > >d:\>Rcmd REMOVE iL04 > > The package gets removed, but I get: > >make: *** No rule to make target `indices'. Stop. That does seem a recent bug, soon to be fixed in R-patched. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages on Windows fails
Prof Brian Ripley schreef op de 7e dag van de wijnmaand van het jaar 2005: > On Fri, 7 Oct 2005, Peter Kleiweg wrote: > > > Duncan Murdoch schreef op de 6e dag van de wijnmaand van het jaar 2005: > > > > > On Fri, 7 Oct 2005, Peter Kleiweg wrote: > > > > > > > What has changed in R for Windows from version 1.7.1 to > > > > 2.2.0 that won't allow me to build binary packages? > > > > > > Many things have changed; I don't know which is causing > > > the failure you see. One change is that instructions are > > > now collected in the Installation and Administration > > > manual. Try following the setup instructions there and see > > > if it still fails. > > > > I can't find anything on building packages for Windows in > > that manual. > > Your problems was installing, so the section on `Installing > Packages' should help you. Installing worked fine. Building a binary distribution (with compiled help files) is what didn't work. This worked fine: Rcmd build iL04 But that just gave a gzip'ed tarfile, not a zip-file, and without the compiled helpfiles. This didn't work: Rcmd build --force --binary iL04 > > I did find a solution to the problem. On a Linux install, > > each package has a file CONTENTS. These are missing from the > > Windows install. I copied those files from my Linux install > > to my Windows install, and then I could build my own > > package. So I guess, these CONTENTS files should be included > > in the Windows install. > > And indeed they are, as the presence of 500+ packages on CRAN > for Windows will show you. Well, I just ran the install program for Windows, with compiled html help, but without the ordinary html help files. In that case, no CONTENTS files get installed. > The recommended way to build a binary package on Windows is > > R CMD INSTALL --build Yes, that works. Even without the CONTENTS files. And this is recommended in the manual "Creating R packages", another manual than Duncan Murdoch was referring to. I was using a method that was recommended in earlier versions. Perhaps that method should just be disabled, with a message about the current method, instead of having it fail for obscure reasons. -- Peter Kleiweg http://www.let.rug.nl/~kleiweg/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages on Windows fails
On Fri, 7 Oct 2005, Peter Kleiweg wrote: > Prof Brian Ripley schreef op de 7e dag van de wijnmaand van het jaar 2005: > >> On Fri, 7 Oct 2005, Peter Kleiweg wrote: >> >>> Duncan Murdoch schreef op de 6e dag van de wijnmaand van het jaar 2005: >>> On Fri, 7 Oct 2005, Peter Kleiweg wrote: > What has changed in R for Windows from version 1.7.1 to > 2.2.0 that won't allow me to build binary packages? Many things have changed; I don't know which is causing the failure you see. One change is that instructions are now collected in the Installation and Administration manual. Try following the setup instructions there and see if it still fails. >>> >>> I can't find anything on building packages for Windows in >>> that manual. >> >> Your problems was installing, so the section on `Installing >> Packages' should help you. > > Installing worked fine. Building a binary distribution (with > compiled help files) is what didn't work. > > This worked fine: > >Rcmd build iL04 > > But that just gave a gzip'ed tarfile, not a zip-file, and > without the compiled helpfiles. > > This didn't work: > >Rcmd build --force --binary iL04 > > >>> I did find a solution to the problem. On a Linux install, >>> each package has a file CONTENTS. These are missing from the >>> Windows install. I copied those files from my Linux install >>> to my Windows install, and then I could build my own >>> package. So I guess, these CONTENTS files should be included >>> in the Windows install. >> >> And indeed they are, as the presence of 500+ packages on CRAN >> for Windows will show you. > > Well, I just ran the install program for Windows, with compiled > html help, but without the ordinary html help files. In that > case, no CONTENTS files get installed. Ah, that's the clue. People normally build complete binary distributions: let's see if we can track that down. >> The recommended way to build a binary package on Windows is >> >> R CMD INSTALL --build > > Yes, that works. Even without the CONTENTS files. And this is > recommended in the manual "Creating R packages", another manual > than Duncan Murdoch was referring to. > > I was using a method that was recommended in earlier versions. > Perhaps that method should just be disabled, with a > message about the current method, instead of having it fail for > obscure reasons. It is not quite the same thing. I have been in favour of removing it, but others have differed in their opinions. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Building packages on Windows fails
Peter Kleiweg wrote: > Prof Brian Ripley schreef op de 7e dag van de wijnmaand van het jaar 2005: > > >>On Fri, 7 Oct 2005, Peter Kleiweg wrote: >> >> >>>Duncan Murdoch schreef op de 6e dag van de wijnmaand van het jaar 2005: >>> >>> On Fri, 7 Oct 2005, Peter Kleiweg wrote: >What has changed in R for Windows from version 1.7.1 to >2.2.0 that won't allow me to build binary packages? Many things have changed; I don't know which is causing the failure you see. One change is that instructions are now collected in the Installation and Administration manual. Try following the setup instructions there and see if it still fails. >>> >>>I can't find anything on building packages for Windows in >>>that manual. >> >>Your problems was installing, so the section on `Installing >>Packages' should help you. > > > Installing worked fine. Building a binary distribution (with > compiled help files) is what didn't work. > > This worked fine: > > Rcmd build iL04 > > But that just gave a gzip'ed tarfile, not a zip-file, and > without the compiled helpfiles. > > This didn't work: > > Rcmd build --force --binary iL04 > > > >>>I did find a solution to the problem. On a Linux install, >>>each package has a file CONTENTS. These are missing from the >>>Windows install. I copied those files from my Linux install >>>to my Windows install, and then I could build my own >>>package. So I guess, these CONTENTS files should be included >>>in the Windows install. >> >>And indeed they are, as the presence of 500+ packages on CRAN >>for Windows will show you. > > > Well, I just ran the install program for Windows, with compiled > html help, but without the ordinary html help files. In that > case, no CONTENTS files get installed. > > >>The recommended way to build a binary package on Windows is >> >>R CMD INSTALL --build > > > Yes, that works. Even without the CONTENTS files. And this is > recommended in the manual "Creating R packages", another manual > than Duncan Murdoch was referring to. > > I was using a method that was recommended in earlier versions. > Perhaps that method should just be disabled, with a > message about the current method, instead of having it fail for > obscure reasons. I agree there should be only one method, and it should work. The problem is: - Users with limited permissions can't run INSTALL, because it needs write access to R_HOME/library. - Because Windows doesn't have soft links, the only way we have to get cross references in the help files to work is to install to R_HOME/library. I'd prefer to fix "Rcmd build --binary" so that it got the help links right and then have "Rcmd INSTALL --build" use it first, then install the .zip file, but I haven't taken the time to do this. I don't like having two parallel tracks of very similar code (because of problems like yours, where the one I don't personnally use falls out of date and stops working). Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
