Re: [Bioc-devel] Custom build of packages
Hi, On Fr, 2016-04-22 at 10:32 -0400, Kasper Daniel Hansen wrote: > ... > 2. To others: I do agree that having preconfigured VMs for specific > platforms would be very worthwhile, especially for Windows, as we have just > seen with mzR. Thanks again for your heroic help there ! > But it would also take effort and there may be licensing > issues related to having an OS ready for download (especially on Windows). Yes, download will not be possible due to licensing. But what about providing Amazon AWS Windows instances (or whatever provider) with BioC installed ? I'd then log into AWS with my own credentials, pay ~20c per hour (IIRC that includes the windows license), connect via RDP or to an rstudio server running on that instance, and fix stuff. Bonus points if the chosen provider also offers MacOS. Yours, Steffen -- IPB HalleAG Massenspektrometrie & Bioinformatik Dr. Steffen Neumann http://www.IPB-Halle.DE Weinberg 3 http://msbi.bic-gh.de 06120 Halle Tel. +49 (0) 345 5582 - 1470 +49 (0) 345 5582 - 0 sneumann(at)IPB-Halle.DE Fax. +49 (0) 345 5582 - 1409 ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Custom build of packages
1. To Ruiz: right now there are tons of changes happening all the time because we are close to a release. So what you're asking is much harder than what you think it is. Basically, it'll not happen. What you first and foremost need to do, is to setup a build on your local machine (which is seems you have done). That will help you with many, many problems - but not problems which are package specific. 2. To others: I do agree that having preconfigured VMs for specific platforms would be very worthwhile, especially for Windows, as we have just seen with mzR. But it would also take effort and there may be licensing issues related to having an OS ready for download (especially on Windows). I am not up to date on the docker efforts, perhaps that is the way to go (but I don't know if that'll help us on Windows). Perhaps having a detailed discussion with potential solutions will help us understand to what extend it is possible and how much effort it requires. On Fri, Apr 22, 2016 at 9:42 AM, Ruiz Arenas, Carlos wrote: > Thanks Leonardo and Steffen, > > I have just realized that Martin also mentioned travis in his reply but I > didn't know what he was talking about. I will try it. > > Thank you again, > > Carlos Ruiz > > - Mensaje original - > De: "Leonardo Collado Torres" > Para: "Steffen Neumann" > CC: "Martin Morgan" , "Ruiz Arenas, > Carlos" , bioc-devel@r-project.org > Enviados: Viernes, 22 de Abril 2016 15:16:27 > Asunto: Re: [Bioc-devel] Custom build of packages > > You could also run tests on Travis CI. That's what I do for my > packages. See https://travis-ci.org/lcolladotor/derfinder and > https://github.com/lcolladotor/derfinder/blob/master/.travis.yml for > an example. If you want to use Travis, check the main docs > https://docs.travis-ci.com/user/languages/r > > On Fri, Apr 22, 2016 at 7:59 AM, Steffen Neumann > wrote: > > Hi, > > > > On Fr, 2016-04-22 at 10:53 +, Morgan, Martin wrote: > >> All packages are built nightly, in both release and devel, and across > >> three architectures. The build system is more complicated than > >> building individual packages -- reverse dependencies need to be > >> built, too, and successful builds synchronized with public > >> repositories -- so the nightly builds will not change in the near > >> future. > >> > >> The problem is likely that your local configuration is using the > >> current version of R and Bioconductor, whereas as a developer you > >> should be using the version of R and Bioconductor used in the 'devel' > >> branch. The version of R in use on the build machines is given at the > >> top of the build page > > > > I too have wanted the different build environments > > as close to the "official" BioC machines as possible > > for debugging. Especially preconfigured VMs or docker images > > would be great. > > > > For the latter, you can use > > > > docker run -ti bioconductor/devel_base bash > > > > but I think even that is a slightly different setup > > compared to the official build animals. > > > > I wouldn't even mind if I have to pay amazon (or google or whoever) > > a few $$$ to fire-up a clone of the build machines, > > log in remotely, and have a clone of the official environment. > > Bonus points if that covered windows, mac and Linux alike. > > > > Just my 2c, > > Yours, > > Steffen > > > > > > > > > > -- > > IPB HalleAG Massenspektrometrie & Bioinformatik > > Dr. Steffen Neumann http://www.IPB-Halle.DE > > Weinberg 3 http://msbi.bic-gh.de > > 06120 Halle Tel. +49 (0) 345 5582 - 1470 > > +49 (0) 345 5582 - 0 > > sneumann(at)IPB-Halle.DE Fax. +49 (0) 345 5582 - 1409 > > > > ___ > > Bioc-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/bioc-devel > > ___ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel > [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Custom build of packages
Thanks Leonardo and Steffen, I have just realized that Martin also mentioned travis in his reply but I didn't know what he was talking about. I will try it. Thank you again, Carlos Ruiz - Mensaje original - De: "Leonardo Collado Torres" Para: "Steffen Neumann" CC: "Martin Morgan" , "Ruiz Arenas, Carlos" , bioc-devel@r-project.org Enviados: Viernes, 22 de Abril 2016 15:16:27 Asunto: Re: [Bioc-devel] Custom build of packages You could also run tests on Travis CI. That's what I do for my packages. See https://travis-ci.org/lcolladotor/derfinder and https://github.com/lcolladotor/derfinder/blob/master/.travis.yml for an example. If you want to use Travis, check the main docs https://docs.travis-ci.com/user/languages/r On Fri, Apr 22, 2016 at 7:59 AM, Steffen Neumann wrote: > Hi, > > On Fr, 2016-04-22 at 10:53 +, Morgan, Martin wrote: >> All packages are built nightly, in both release and devel, and across >> three architectures. The build system is more complicated than >> building individual packages -- reverse dependencies need to be >> built, too, and successful builds synchronized with public >> repositories -- so the nightly builds will not change in the near >> future. >> >> The problem is likely that your local configuration is using the >> current version of R and Bioconductor, whereas as a developer you >> should be using the version of R and Bioconductor used in the 'devel' >> branch. The version of R in use on the build machines is given at the >> top of the build page > > I too have wanted the different build environments > as close to the "official" BioC machines as possible > for debugging. Especially preconfigured VMs or docker images > would be great. > > For the latter, you can use > > docker run -ti bioconductor/devel_base bash > > but I think even that is a slightly different setup > compared to the official build animals. > > I wouldn't even mind if I have to pay amazon (or google or whoever) > a few $$$ to fire-up a clone of the build machines, > log in remotely, and have a clone of the official environment. > Bonus points if that covered windows, mac and Linux alike. > > Just my 2c, > Yours, > Steffen > > > > > -- > IPB HalleAG Massenspektrometrie & Bioinformatik > Dr. Steffen Neumann http://www.IPB-Halle.DE > Weinberg 3 http://msbi.bic-gh.de > 06120 Halle Tel. +49 (0) 345 5582 - 1470 > +49 (0) 345 5582 - 0 > sneumann(at)IPB-Halle.DE Fax. +49 (0) 345 5582 - 1409 > > ___ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Custom build of packages
You could also run tests on Travis CI. That's what I do for my packages. See https://travis-ci.org/lcolladotor/derfinder and https://github.com/lcolladotor/derfinder/blob/master/.travis.yml for an example. If you want to use Travis, check the main docs https://docs.travis-ci.com/user/languages/r On Fri, Apr 22, 2016 at 7:59 AM, Steffen Neumann wrote: > Hi, > > On Fr, 2016-04-22 at 10:53 +, Morgan, Martin wrote: >> All packages are built nightly, in both release and devel, and across >> three architectures. The build system is more complicated than >> building individual packages -- reverse dependencies need to be >> built, too, and successful builds synchronized with public >> repositories -- so the nightly builds will not change in the near >> future. >> >> The problem is likely that your local configuration is using the >> current version of R and Bioconductor, whereas as a developer you >> should be using the version of R and Bioconductor used in the 'devel' >> branch. The version of R in use on the build machines is given at the >> top of the build page > > I too have wanted the different build environments > as close to the "official" BioC machines as possible > for debugging. Especially preconfigured VMs or docker images > would be great. > > For the latter, you can use > > docker run -ti bioconductor/devel_base bash > > but I think even that is a slightly different setup > compared to the official build animals. > > I wouldn't even mind if I have to pay amazon (or google or whoever) > a few $$$ to fire-up a clone of the build machines, > log in remotely, and have a clone of the official environment. > Bonus points if that covered windows, mac and Linux alike. > > Just my 2c, > Yours, > Steffen > > > > > -- > IPB HalleAG Massenspektrometrie & Bioinformatik > Dr. Steffen Neumann http://www.IPB-Halle.DE > Weinberg 3 http://msbi.bic-gh.de > 06120 Halle Tel. +49 (0) 345 5582 - 1470 > +49 (0) 345 5582 - 0 > sneumann(at)IPB-Halle.DE Fax. +49 (0) 345 5582 - 1409 > > ___ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Custom build of packages
Hi, On Fr, 2016-04-22 at 10:53 +, Morgan, Martin wrote: > All packages are built nightly, in both release and devel, and across > three architectures. The build system is more complicated than > building individual packages -- reverse dependencies need to be > built, too, and successful builds synchronized with public > repositories -- so the nightly builds will not change in the near > future. > > The problem is likely that your local configuration is using the > current version of R and Bioconductor, whereas as a developer you > should be using the version of R and Bioconductor used in the 'devel' > branch. The version of R in use on the build machines is given at the > top of the build page I too have wanted the different build environments as close to the "official" BioC machines as possible for debugging. Especially preconfigured VMs or docker images would be great. For the latter, you can use docker run -ti bioconductor/devel_base bash but I think even that is a slightly different setup compared to the official build animals. I wouldn't even mind if I have to pay amazon (or google or whoever) a few $$$ to fire-up a clone of the build machines, log in remotely, and have a clone of the official environment. Bonus points if that covered windows, mac and Linux alike. Just my 2c, Yours, Steffen -- IPB HalleAG Massenspektrometrie & Bioinformatik Dr. Steffen Neumann http://www.IPB-Halle.DE Weinberg 3 http://msbi.bic-gh.de 06120 Halle Tel. +49 (0) 345 5582 - 1470 +49 (0) 345 5582 - 0 sneumann(at)IPB-Halle.DE Fax. +49 (0) 345 5582 - 1409 ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Custom build of packages
Thanks Martin for your quick reply.
I really appreciate your comments to help me improve my package. As you comment
in the first lines, I have already downloaded the devel version of R and Bioc.
This is how I found out the changes in the dependencies of rowRanges function.
Regarding the other point, I had absolutely no idea of how to solve it and now
I will correct it. Hopefully, the package will have no problems in the next
build.
Anyway, my previous suggestions were ideas that I thought could be interesting
not only for me but for other developers, so I thought it was interesting to
share them with the whole community. Surely due to my lack of knowledge of
computer science, I don't still see why a build similar to what is done for non
accepted packages cannot be done in devel or release branches, only for those
packages with changes. My idea is not changing the general nightly build but to
add a complementary system to help solving problems more efficiently.
Bests,
Carlos Ruiz
- Mensaje original -
De: "Martin Morgan"
Para: "Ruiz Arenas, Carlos" , bioc-devel@r-project.org
Enviados: Viernes, 22 de Abril 2016 12:53:10
Asunto: Re: Custom build of packages
I guess asking on the devel mailing list rather than sending mail to a
particular individual means that you get the benefit of whatever global wisdom
is available.
All packages are built nightly, in both release and devel, and across three
architectures. The build system is more complicated than building individual
packages -- reverse dependencies need to be built, too, and successful builds
synchronized with public repositories -- so the nightly builds will not change
in the near future.
The problem is likely that your local configuration is using the current
version of R and Bioconductor, whereas as a developer you should be using the
version of R and Bioconductor used in the 'devel' branch. The version of R in
use on the build machines is given at the top of the build page
http://bioconductor.org/checkResults/3.3/bioc-LATEST/index.html
so R-3.3.0-beta, available from cran.r-project.org. Once you have the correct
version of R installed, then
source("https://bioconductor.org/biocLite.R";)
BiocInstaller::useDevel()
sets you up to install bioc devel packages (the second step is not always
necessary, depending on the version of R and Bioconductor; it is not necessary
in the current 'devel', so the error -- 'devel' version already in use -- can
be ignored). See a variant of these instructions at
http://bioconductor.org/developers/how-to/useDevel/. The specific commands used
for building are given in each section of the build report, so the specific
build command (editing paths, of course) can also be invoked.
I guess you are talking about MultiDataSet, with the build report at
http://bioconductor.org/checkResults/3.3/bioc-LATEST/MultiDataSet/zin2-buildsrc.html
The error
** R
** preparing package for lazy loading
Error : object ‘rowRanges’ is not exported by 'namespace:GenomicRanges'
ERROR: lazy loading failed for package ‘MultiDataSet’
* removing ‘/tmp/RtmpzWv6vq/Rinst36c3606d6b5a/MultiDataSet’
is because in bioc devel rowRanges is now in the SummarizedExperiment package.
I see from SVN that you have discovered this, too.
The warning
* installing *source* package 'MultiDataSet' ...
Warning in .write_description(db, file.path(outDir, "DESCRIPTION")) :
Unknown encoding with non-ASCII data: converting to ASCII
is probably complaining that the DESCRIPTION file does not declare a particular
encoding (so defaults to ASCII) but contains non-ASCII characters. These can be
spotted with
> tools::showNonASCIIfile("DESCRIPTION")
8: person("Juan R.", "Gonzlez", ,"jrgonza...@creal.cat", role =
"aut"))
and you could either forgo the accents or declare the encoding, probably by
adding the tag
Encoding: UTF-8
to the DESCRIPTION file.
People will also point to travis for use with github, but for me this
introduces two tools that play only sort-of ok with the Bioconductor build
infrastructure (trying to bridge git via github with svn in particular seems to
cause problems that can be very tedious and time consuming to solve).
Martin
From: Bioc-devel on behalf of Ruiz Arenas,
Carlos
Sent: Friday, April 22, 2016 6:06 AM
To: bioc-devel@r-project.org
Subject: [Bioc-devel] Custom build of packages
Hi Bioconductor community,
I am a recent Bioc developer. I uploaded a package just before the deadline of
this BioC 3.3. During the admission process, the build machine was run with my
packages almost every two hours which allowed me to improve the package more
efficiently. However, now that I am in the devel trunk, the build machine is
only run nightly. I am experiencing some problems due to some changes and I
have to wait until the following day to check that the changes had worked. One
of the people of Bioc tried to help me running the machine with my
Re: [Bioc-devel] Custom build of packages
I guess asking on the devel mailing list rather than sending mail to a
particular individual means that you get the benefit of whatever global wisdom
is available.
All packages are built nightly, in both release and devel, and across three
architectures. The build system is more complicated than building individual
packages -- reverse dependencies need to be built, too, and successful builds
synchronized with public repositories -- so the nightly builds will not change
in the near future.
The problem is likely that your local configuration is using the current
version of R and Bioconductor, whereas as a developer you should be using the
version of R and Bioconductor used in the 'devel' branch. The version of R in
use on the build machines is given at the top of the build page
http://bioconductor.org/checkResults/3.3/bioc-LATEST/index.html
so R-3.3.0-beta, available from cran.r-project.org. Once you have the correct
version of R installed, then
source("https://bioconductor.org/biocLite.R";)
BiocInstaller::useDevel()
sets you up to install bioc devel packages (the second step is not always
necessary, depending on the version of R and Bioconductor; it is not necessary
in the current 'devel', so the error -- 'devel' version already in use -- can
be ignored). See a variant of these instructions at
http://bioconductor.org/developers/how-to/useDevel/. The specific commands used
for building are given in each section of the build report, so the specific
build command (editing paths, of course) can also be invoked.
I guess you are talking about MultiDataSet, with the build report at
http://bioconductor.org/checkResults/3.3/bioc-LATEST/MultiDataSet/zin2-buildsrc.html
The error
** R
** preparing package for lazy loading
Error : object ‘rowRanges’ is not exported by 'namespace:GenomicRanges'
ERROR: lazy loading failed for package ‘MultiDataSet’
* removing ‘/tmp/RtmpzWv6vq/Rinst36c3606d6b5a/MultiDataSet’
is because in bioc devel rowRanges is now in the SummarizedExperiment package.
I see from SVN that you have discovered this, too.
The warning
* installing *source* package 'MultiDataSet' ...
Warning in .write_description(db, file.path(outDir, "DESCRIPTION")) :
Unknown encoding with non-ASCII data: converting to ASCII
is probably complaining that the DESCRIPTION file does not declare a particular
encoding (so defaults to ASCII) but contains non-ASCII characters. These can be
spotted with
> tools::showNonASCIIfile("DESCRIPTION")
8: person("Juan R.", "Gonzlez", ,"jrgonza...@creal.cat", role =
"aut"))
and you could either forgo the accents or declare the encoding, probably by
adding the tag
Encoding: UTF-8
to the DESCRIPTION file.
People will also point to travis for use with github, but for me this
introduces two tools that play only sort-of ok with the Bioconductor build
infrastructure (trying to bridge git via github with svn in particular seems to
cause problems that can be very tedious and time consuming to solve).
Martin
From: Bioc-devel on behalf of Ruiz Arenas,
Carlos
Sent: Friday, April 22, 2016 6:06 AM
To: bioc-devel@r-project.org
Subject: [Bioc-devel] Custom build of packages
Hi Bioconductor community,
I am a recent Bioc developer. I uploaded a package just before the deadline of
this BioC 3.3. During the admission process, the build machine was run with my
packages almost every two hours which allowed me to improve the package more
efficiently. However, now that I am in the devel trunk, the build machine is
only run nightly. I am experiencing some problems due to some changes and I
have to wait until the following day to check that the changes had worked. One
of the people of Bioc tried to help me running the machine with my package.
However, he is in Seatle while I am in Barcelona, so when I receive the mail he
is not working anymore and the other way round.
I am wondering if it could be possible for developers to generate a build
report for our package. Otherwise, I think it could be interesting that the
build machine was run more often during the last days of Bioc release, at least
for packages with changes.
Are any of these options feasible?
Thanks,
Carlos Ruiz
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
This email message may contain legally privileged and/or confidential
information. If you are not the intended recipient(s), or the employee or
agent responsible for the delivery of this message to the intended
recipient(s), you are hereby notified that any disclosure, copying,
distribution, or use of this email message is prohibited. If you have received
this message in error, please notify the sender immediately by e-mail and
delete this email message from your computer. Thank you.
___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mai
