[R-sig-Fedora] R 4.0.0 rebuild status
Over the last several days, I've been working hard to get all of the Fedora R packages rebuilt against R 4.0 in rawhide (in the F33-R-4 side tag). With the exception of R-biomaRt, R-BSgenome, R-GenomicAlignments, and R-rtracklayer, I believe everything is built and updated to the latest versions. And of those packages, they're all ready to go when Fedora infrastructure is working reliably again (the great datacenter migration has started and I can no longer git push). I'll also push R 4.0.1 into the tag when that's possible. There are two new packages that I created that are needed for R-biomaRt to be updated: R-AnnotationDbi https://bugzilla.redhat.com/show_bug.cgi?id=1845360 R-BiocFileCache https://bugzilla.redhat.com/show_bug.cgi?id=1845362 They are relatively simple noarch packages and should be quick reviews. Help in getting these reviewed quickly would be appreciated. Given the huge amount of builds (and rebuilds) in this process, I am strongly disinclined to attempt this work for Fedora 32 (the idea of hundreds of bodhi overrides does not fill me with joy), but I would not prevent someone else who wished to try to do so. Thanks, Tom P.S. I noticed that R-pls and R-statmod were orphaned (and as a result, they did not get rebuilt for R 4). Nothing seems to depend on them in Fedora, so this seems fine, but if you were using those packages, you may wish to revive them. [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 9 Jun 2020 at 04:42, Tom Callaway wrote: > > Over the last several days, I've been working hard to get all of the Fedora > R packages rebuilt against R 4.0 in rawhide (in the F33-R-4 side tag). With > the exception of R-biomaRt, R-BSgenome, R-GenomicAlignments, and > R-rtracklayer, I believe everything is built and updated to the latest > versions. And of those packages, they're all ready to go when Fedora > infrastructure is working reliably again (the great datacenter migration > has started and I can no longer git push). I'll also push R 4.0.1 into the > tag when that's possible. Much appreciated. > There are two new packages that I created that are needed for R-biomaRt to > be updated: > > R-AnnotationDbi > https://bugzilla.redhat.com/show_bug.cgi?id=1845360 > > R-BiocFileCache > https://bugzilla.redhat.com/show_bug.cgi?id=1845362 > > They are relatively simple noarch packages and should be quick reviews. > Help in getting these reviewed quickly would be appreciated. I took them. In progress. > Given the huge amount of builds (and rebuilds) in this process, I am > strongly disinclined to attempt this work for Fedora 32 (the idea of > hundreds of bodhi overrides does not fill me with joy), but I would not > prevent someone else who wished to try to do so. Note that this is no longer needed. It is possible to use a side tag for F32 too, which is much easier and more appropriate for a massive update like this. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 9 Jun 2020 at 10:21, Iñaki Ucar wrote: > > > Given the huge amount of builds (and rebuilds) in this process, I am > > strongly disinclined to attempt this work for Fedora 32 (the idea of > > hundreds of bodhi overrides does not fill me with joy), but I would not > > prevent someone else who wished to try to do so. > > Note that this is no longer needed. It is possible to use a side tag > for F32 too, which is much easier and more appropriate for a massive > update like this. Any plan on doing this in a side tag for F32? I would happily volunteer, but I'm not a provenpackager. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
At this point, I simply don't have the time. Tom On Tue, Jun 23, 2020, 6:06 AM Iñaki Ucar wrote: > On Tue, 9 Jun 2020 at 10:21, Iñaki Ucar wrote: > > > > > Given the huge amount of builds (and rebuilds) in this process, I am > > > strongly disinclined to attempt this work for Fedora 32 (the idea of > > > hundreds of bodhi overrides does not fill me with joy), but I would not > > > prevent someone else who wished to try to do so. > > > > Note that this is no longer needed. It is possible to use a side tag > > for F32 too, which is much easier and more appropriate for a massive > > update like this. > > Any plan on doing this in a side tag for F32? I would happily > volunteer, but I'm not a provenpackager. > > -- > Iñaki Úcar > > ___ > R-SIG-Fedora mailing list > R-SIG-Fedora@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
Maybe Elliott? On Tue, 23 Jun 2020 at 15:01, Tom Callaway wrote: > > At this point, I simply don't have the time. > > Tom > > On Tue, Jun 23, 2020, 6:06 AM Iñaki Ucar wrote: >> >> On Tue, 9 Jun 2020 at 10:21, Iñaki Ucar wrote: >> > >> > > Given the huge amount of builds (and rebuilds) in this process, I am >> > > strongly disinclined to attempt this work for Fedora 32 (the idea of >> > > hundreds of bodhi overrides does not fill me with joy), but I would not >> > > prevent someone else who wished to try to do so. >> > >> > Note that this is no longer needed. It is possible to use a side tag >> > for F32 too, which is much easier and more appropriate for a massive >> > update like this. >> >> Any plan on doing this in a side tag for F32? I would happily >> volunteer, but I'm not a provenpackager. >> >> -- >> Iñaki Úcar >> >> ___ >> R-SIG-Fedora mailing list >> R-SIG-Fedora@r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-fedora -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tuesday, 23 June 2020 14.01.39 WEST Tom Callaway wrote: > At this point, I simply don't have the time. > > Tom What needs to be done and how can the work be streamlined? I asked this since this procedure will happen for other updates (in one year I know but time flies). I am a provenpackager so that I can drive the process. Regards, -- Jos� Ab�lio [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 23 Jun 2020 at 17:01, José Abílio Matos wrote: > > On Tuesday, 23 June 2020 14.01.39 WEST Tom Callaway wrote: > > At this point, I simply don't have the time. > > > > Tom > > What needs to be done and how can the work be streamlined? > > I asked this since this procedure will happen for other updates (in one year I > know but time flies). > > I am a provenpackager so that I can drive the process. Great, thanks! :) Basically, we need (correct me if I'm wrong, Tom): 1) A new user side tag. 2) For R, merge master in F32 and send a build to that side tag. 3) For all packages, either merge master into F32 or just increase the release version and send builds to that side tag *in order*. 4) When everything is built, create an update from that side tag. I have tooling in my Copr repo to resolve CRAN dependencies and create a list of lists of builds in order. I can give you that. But note that Bioc packages (and others) are not included. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tuesday, 23 June 2020 16.10.07 WEST I�aki Ucar wrote: > 3) For all packages, either merge master into F32 or just increase the > release version and send builds to that side tag *in order*. The part that worries me is the *in order*. :-) Do we need to do bootstrap builds that are later replaced by the real builds? Because other than that it is business as usual. :-) -- Jos� Ab�lio [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
There are a few of those, but not many. On Tue, Jun 23, 2020 at 11:31 AM José Abílio Matos wrote: > On Tuesday, 23 June 2020 16.10.07 WEST I�aki Ucar wrote: > > 3) For all packages, either merge master into F32 or just increase the > > release version and send builds to that side tag *in order*. > > The part that worries me is the *in order*. :-) > > Do we need to do bootstrap builds that are later replaced by the real > builds? > > Because other than that it is business as usual. :-) > -- > Jos� Ab�lio > > [[alternative HTML version deleted]] > > ___ > R-SIG-Fedora mailing list > R-SIG-Fedora@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
I could do so, but it wouldn't be until this weekend. Also, Tom mixed in some version bumps to the rebuild, so we'd have to check whether those would actually be okay in a released version. I would also appreciate some review on new (mostly test) dependencies for these bumps. On Tue, 23 Jun 2020 at 09:43, Iñaki Ucar wrote: > > Maybe Elliott? > > On Tue, 23 Jun 2020 at 15:01, Tom Callaway wrote: > > > > At this point, I simply don't have the time. > > > > Tom > > > > On Tue, Jun 23, 2020, 6:06 AM Iñaki Ucar wrote: > >> > >> On Tue, 9 Jun 2020 at 10:21, Iñaki Ucar wrote: > >> > > >> > > Given the huge amount of builds (and rebuilds) in this process, I am > >> > > strongly disinclined to attempt this work for Fedora 32 (the idea of > >> > > hundreds of bodhi overrides does not fill me with joy), but I would not > >> > > prevent someone else who wished to try to do so. > >> > > >> > Note that this is no longer needed. It is possible to use a side tag > >> > for F32 too, which is much easier and more appropriate for a massive > >> > update like this. > >> > >> Any plan on doing this in a side tag for F32? I would happily > >> volunteer, but I'm not a provenpackager. > >> > >> -- > >> Iñaki Úcar > >> > >> ___ > >> R-SIG-Fedora mailing list > >> R-SIG-Fedora@r-project.org > >> https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > > > > -- > Iñaki Úcar -- Elliott ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
Thanks, José and Elliott. I can help with reviews. I attach here a list of batches of CRAN packages to be rebuilt in order (batches separated by a blank line), and the script that generates it. Hope it helps. Iñaki On Wed, 24 Jun 2020 at 11:35, Elliott Sales de Andrade wrote: > > I could do so, but it wouldn't be until this weekend. > > Also, Tom mixed in some version bumps to the rebuild, so we'd have to > check whether those would actually be okay in a released version. I > would also appreciate some review on new (mostly test) dependencies > for these bumps. > > On Tue, 23 Jun 2020 at 09:43, Iñaki Ucar wrote: > > > > Maybe Elliott? > > > > On Tue, 23 Jun 2020 at 15:01, Tom Callaway wrote: > > > > > > At this point, I simply don't have the time. > > > > > > Tom > > > > > > On Tue, Jun 23, 2020, 6:06 AM Iñaki Ucar wrote: > > >> > > >> On Tue, 9 Jun 2020 at 10:21, Iñaki Ucar wrote: > > >> > > > >> > > Given the huge amount of builds (and rebuilds) in this process, I am > > >> > > strongly disinclined to attempt this work for Fedora 32 (the idea of > > >> > > hundreds of bodhi overrides does not fill me with joy), but I would > > >> > > not > > >> > > prevent someone else who wished to try to do so. > > >> > > > >> > Note that this is no longer needed. It is possible to use a side tag > > >> > for F32 too, which is much easier and more appropriate for a massive > > >> > update like this. > > >> > > >> Any plan on doing this in a side tag for F32? I would happily > > >> volunteer, but I'm not a provenpackager. > > >> > > >> -- > > >> Iñaki Úcar > > >> > > >> ___ > > >> R-SIG-Fedora mailing list > > >> R-SIG-Fedora@r-project.org > > >> https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > > > > > > > > -- > > Iñaki Úcar > > > > -- > Elliott -- Iñaki Úcar AUC AsioHeaders BH Cairo DBI FMStable NISTunits R.methodsS3 R6 RColorBrewer RUnit Rcpp RhpcBLASctl XML abind acepack argon2 assertthat backports base64enc bindr bit bitops brew chron clipr clisymbols coda colorspace combinat commonmark crayon curl data.table date deldir dichromat digest disposables errors evaluate expm fansi farver fastmap fastmatch filehash fontBitstreamVera fontLiberation formatR fortunes fs futile.options gamlss.dist gee generics ggplot2movies git2r globals glue gmp gsl gss gtable gtools hexbin highlight highr import ini inline iterators jpeg jsonlite labeling lazyeval listenv lmodel2 mAr magrittr maps matrixStats measurements microbenchmark mime mlbench mvtnorm ncdf4 nws packrat pbdRPC pbdZMQ pkgconfig plogr png polyclip polynom praise prettyunits ps qcc qtl quadprog rappdirs rematch remotes rlang rlecuyer rstudioapi rsvg scatterplot3d sciplot sfsmisc snow sodium sourcetools sp stringdist stringi sys sysfonts systemfonts testit timeDate tinytest tkrplot udunits2 unix utf8 uuid viridisLite waveslim wavethresh webp wesanderson whisker withr xfun xml2 xmlparsedata xtable yaml zeallot zoo R.oo RCurl RInside RcppCCTZ Rmpfr V8 ape askpass biglm bindrcpp bit64 caTools car cli corpus debugme diffobj ellipsis foreach fts future gdata gdtools getPass htmltools hunspell igraph itertools jqr lambda.r later lifecycle lmtest lokern lubridate mapproj markdown memoise mockr msm munsell plyr poLCA prettycode processx purrr reticulate rex rgdal rgeos rprintf rprojroot rversions sandwich showtextdb simmer statnet.common stringr tikzDevice timeSeries tinytex tweenr units webutils Bessel R.utils RM2 callr desc doParallel futile.logger gplots here htmlwidgets knitr multcomp nanotime openssl orcutt pingr promises qpdf quantities reshape reshape2 scales selectr sessioninfo showtext svglite systemfit unitizer vctrs websocket xopen R.cache R.devices blob cyclocomp hms httpuv httr jose pdftools pillar pkgbuild profvis rmarkdown rsconnect spelling tidyselect R.rsp RSQLite gargle gh pkgload prettydoc progress rcmdcheck repr reprex rvest shiny tibble tufte whoami IRdisplay cellranger cliapp dplyr foghorn forcats gapminder ggplot2 miniUI nycflights13 readr rematch2 repurrrsive roxygen2 testthat IRkernel dbplyr dtplyr gmailr haven lintr mockery parsedate pkgdown readxl styler tidyr usethis broom devtools rhub geepack modelroptions(repos = "https://cloud.r-project.org";) get_build_list <- function(pkgs, cran=available.packages()) { base <- rownames(installed.packages(priority="high")) pkgs <- lapply(tools::package_dependencies(pkgs, db=cran), setdiff, base) pkgs <- lapply(Filter(Negate(is.null), pkgs), intersect, names(pkgs)) build <- list() while (length(pkgs)) { x <- names(Filter(function(i) all(i %in% unlist(build)), pkgs)) build[[length(build)+1]] <- x pkgs <- pkgs[!names(pkgs) %in% x] } build } cran <- available.packages() pkgs <- system2( "dnf", c("list", "available", "R-*", "--repo=fedora", "--repo=updates"), stdout=TRUE) pkgs <- grep("^R-", pkgs, value=TRUE) pkgs <- sapply(strsplit(pkgs, "[[:space:]]"), "[", 1) pkgs <- gsub("R-|\\.i686|\\.x86_64|\\.noarch|-devel", "",
Re: [R-sig-Fedora] R 4.0.0 rebuild status
Oh, and maybe in this process we could add to all packages the requirement on R(ABI) = 4 that Tom implemented. On Wed, 24 Jun 2020 at 11:42, Iñaki Ucar wrote: > > Thanks, José and Elliott. I can help with reviews. > > I attach here a list of batches of CRAN packages to be rebuilt in > order (batches separated by a blank line), and the script that > generates it. Hope it helps. > > Iñaki > > On Wed, 24 Jun 2020 at 11:35, Elliott Sales de Andrade > wrote: > > > > I could do so, but it wouldn't be until this weekend. > > > > Also, Tom mixed in some version bumps to the rebuild, so we'd have to > > check whether those would actually be okay in a released version. I > > would also appreciate some review on new (mostly test) dependencies > > for these bumps. > > > > On Tue, 23 Jun 2020 at 09:43, Iñaki Ucar wrote: > > > > > > Maybe Elliott? > > > > > > On Tue, 23 Jun 2020 at 15:01, Tom Callaway wrote: > > > > > > > > At this point, I simply don't have the time. > > > > > > > > Tom > > > > > > > > On Tue, Jun 23, 2020, 6:06 AM Iñaki Ucar > > > > wrote: > > > >> > > > >> On Tue, 9 Jun 2020 at 10:21, Iñaki Ucar > > > >> wrote: > > > >> > > > > >> > > Given the huge amount of builds (and rebuilds) in this process, I > > > >> > > am > > > >> > > strongly disinclined to attempt this work for Fedora 32 (the idea > > > >> > > of > > > >> > > hundreds of bodhi overrides does not fill me with joy), but I > > > >> > > would not > > > >> > > prevent someone else who wished to try to do so. > > > >> > > > > >> > Note that this is no longer needed. It is possible to use a side tag > > > >> > for F32 too, which is much easier and more appropriate for a massive > > > >> > update like this. > > > >> > > > >> Any plan on doing this in a side tag for F32? I would happily > > > >> volunteer, but I'm not a provenpackager. > > > >> > > > >> -- > > > >> Iñaki Úcar > > > >> > > > >> ___ > > > >> R-SIG-Fedora mailing list > > > >> R-SIG-Fedora@r-project.org > > > >> https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > > > > > > > > > > > > -- > > > Iñaki Úcar > > > > > > > > -- > > Elliott > > > > -- > Iñaki Úcar -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Wed, 24 Jun 2020 at 05:42, Iñaki Ucar wrote: > > Thanks, José and Elliott. I can help with reviews. > Thanks. I have a few open already: - https://bugzilla.redhat.com/show_bug.cgi?id=1839451 - R-servr - https://bugzilla.redhat.com/show_bug.cgi?id=1839456 - R-filelock - https://bugzilla.redhat.com/show_bug.cgi?id=1839474 - R-DBItest > I attach here a list of batches of CRAN packages to be rebuilt in > order (batches separated by a blank line), and the script that > generates it. Hope it helps. > > Iñaki > > On Wed, 24 Jun 2020 at 11:35, Elliott Sales de Andrade > wrote: > > > > I could do so, but it wouldn't be until this weekend. > > > > Also, Tom mixed in some version bumps to the rebuild, so we'd have to > > check whether those would actually be okay in a released version. I > > would also appreciate some review on new (mostly test) dependencies > > for these bumps. > > > > On Tue, 23 Jun 2020 at 09:43, Iñaki Ucar wrote: > > > > > > Maybe Elliott? > > > > > > On Tue, 23 Jun 2020 at 15:01, Tom Callaway wrote: > > > > > > > > At this point, I simply don't have the time. > > > > > > > > Tom > > > > > > > > On Tue, Jun 23, 2020, 6:06 AM Iñaki Ucar > > > > wrote: > > > >> > > > >> On Tue, 9 Jun 2020 at 10:21, Iñaki Ucar > > > >> wrote: > > > >> > > > > >> > > Given the huge amount of builds (and rebuilds) in this process, I > > > >> > > am > > > >> > > strongly disinclined to attempt this work for Fedora 32 (the idea > > > >> > > of > > > >> > > hundreds of bodhi overrides does not fill me with joy), but I > > > >> > > would not > > > >> > > prevent someone else who wished to try to do so. > > > >> > > > > >> > Note that this is no longer needed. It is possible to use a side tag > > > >> > for F32 too, which is much easier and more appropriate for a massive > > > >> > update like this. > > > >> > > > >> Any plan on doing this in a side tag for F32? I would happily > > > >> volunteer, but I'm not a provenpackager. > > > >> > > > >> -- > > > >> Iñaki Úcar > > > >> > > > >> ___ > > > >> R-SIG-Fedora mailing list > > > >> R-SIG-Fedora@r-project.org > > > >> https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > > > > > > > > > > > > -- > > > Iñaki Úcar > > > > > > > > -- > > Elliott > > > > -- > Iñaki Úcar -- Elliott ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Wednesday, 24 June 2020 10.42.10 WEST Iñaki Ucar wrote: > Thanks, José and Elliott. I can help with reviews. > > I attach here a list of batches of CRAN packages to be rebuilt in > order (batches separated by a blank line), and the script that > generates it. Hope it helps. > > Iñaki Sure it does. I am doing the rebuild in side tag f32-build-side-24797. -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Wednesday, 24 June 2020 10.44.14 WEST Iñaki Ucar wrote: > Oh, and maybe in this process we could add to all packages the > requirement on R(ABI) = 4 that Tom implemented. For that we need to start with rawhide and then change the R-rpm-macros package. Probably it should be enough to change the /usr/lib/rpm/R-deps.R script to add Requires: R(ABI)=4.0 I suggest to continue this as is and then implement that change in rawhide and bring it back to Fedora 32 as new updates are issued. What do you think? -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
This work is already complete in rawhide. Tom On Thu, Jun 25, 2020, 1:12 PM José Abílio Matos wrote: > On Wednesday, 24 June 2020 10.44.14 WEST Iñaki Ucar wrote: > > Oh, and maybe in this process we could add to all packages the > > requirement on R(ABI) = 4 that Tom implemented. > > For that we need to start with rawhide and then change the R-rpm-macros > package. > > Probably it should be enough to change the /usr/lib/rpm/R-deps.R script to > add > > Requires: R(ABI)=4.0 > > I suggest to continue this as is and then implement that change in rawhide > and > bring it back to Fedora 32 as new updates are issued. > > What do you think? > -- > José Abílio > > ___ > R-SIG-Fedora mailing list > R-SIG-Fedora@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Thursday, 25 June 2020 18.26.20 WEST Tom Callaway wrote: > This work is already complete in rawhide. > > Tom OK. Building R-rpm-macros now and then untag and rebuild the modules already done. After all this is an iterative procedure. :-( Thank you for the remark. :-) -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Wednesday, 24 June 2020 10.42.10 WEST Iñaki Ucar wrote: > Thanks, José and Elliott. I can help with reviews. > > I attach here a list of batches of CRAN packages to be rebuilt in > order (batches separated by a blank line), and the script that > generates it. Hope it helps. > > Iñaki R-acepack depends on R-testthat so I moved it the batch after R-testthat. :-) If I find a circular dependency I will need to do a bootstrap build. Fun lies ahead... :-) -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Thu, 25 Jun 2020 at 19:01, José Abílio Matos wrote: > > On Wednesday, 24 June 2020 10.42.10 WEST Iñaki Ucar wrote: > > Thanks, José and Elliott. I can help with reviews. > > > > I attach here a list of batches of CRAN packages to be rebuilt in > > order (batches separated by a blank line), and the script that > > generates it. Hope it helps. > > > > Iñaki > > R-acepack depends on R-testthat so I moved it the batch after R-testthat. :-) > > If I find a circular dependency I will need to do a bootstrap build. > > Fun lies ahead... :-) Thanks for starting off builds. However, please be careful merging to master, as some packages were bumped and have incompatibilities that should not be put in stable releases. I will try to come up with an exact list soon. > -- > José Abílio > -- Elliott ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Friday, 26 June 2020 00.45.46 WEST Elliott Sales de Andrade wrote: > Thanks for starting off builds. However, please be careful merging to > master, as some packages were bumped and have incompatibilities that > should not be put in stable releases. I will try to come up with an > exact list soon. I am doing the process manually to ensure that this does not happen. Basically my method is for a given package called appropriately pkg: cd R-pkg less R-pkg.spec # read file to ensure that it is safe to build it if it is safe then I run: $ fedpkg switch-branch f32 && git merge master && git push && fedpkg build -- target=f32-build-side-24797 && fedpkg switch-branch master; cd .. the idea, of course, is that if the merge fails then the case needs to be dealt manually. FWIW most of the credit goes to Tom (spot) since the heavy lifting was already done. :-) I will welcome your list as it becomes easier to find those cases. As I said before I am doing my best to avoid those cases. -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Thursday, 25 June 2020 18.26.20 WEST Tom Callaway wrote: > This work is already complete in rawhide. > > Tom BTW I suspect that now we can simplify the requires generator. As an example we have: $ rpm -qp --requires R-AUC-0.3.0-8.fc32.noarch.rpm R(ABI) = 4.0 R-core rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsZstd) <= 5.4.18-1 I guess that now R(ABI) and R-core are synonyms and thus we can remove the R- core requirement. Of course that this in minimal and mainly aesthetic. :-) -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tuesday, 23 June 2020 16.48.53 WEST Tom Callaway wrote: > There are a few of those, but not many. Hi Tom, I noticed that for example in R-assertthat you have used the bcond: %bcond_with check would not it be better to use bootstrap instead to take advantage of: https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping I am asking for curiosity since from now on we will need to do this dance once a year for the releases that we deem worth (I would expect that to be rawhide and the latest stable). Forgive me if the question does not make sense since I am still trying to make sense of this maze. :-) -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Fri, 26 Jun 2020 at 11:36, José Abílio Matos wrote: > > On Tuesday, 23 June 2020 16.48.53 WEST Tom Callaway wrote: > > There are a few of those, but not many. > > Hi Tom, > I noticed that for example in R-assertthat you have used the bcond: > > %bcond_with check > > would not it be better to use bootstrap instead to take advantage of: > https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping > > I am asking for curiosity since from now on we will need to do this dance once > a year for the releases that we deem worth (I would expect that to be rawhide > and the latest stable). > > Forgive me if the question does not make sense since I am still trying to make > sense of this maze. :-) I used bcond locally and wrongly assumed that fedpkg build would support --with BCOND and --without BCOND. Instead, the way to activate it is to change to "%bcond_with check" and then revert to "%bcond_without check". The only difference with bootstrap is that "bootstrap" is recognized and a suffix ~bootstrap can be added automatically to the resulting build. So the question is whether we want that suffix or not. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
Hi José, On Thu, 25 Jun 2020 at 20:05, José Abílio Matos wrote: > > On Friday, 26 June 2020 00.45.46 WEST Elliott Sales de Andrade wrote: > > Thanks for starting off builds. However, please be careful merging to > > master, as some packages were bumped and have incompatibilities that > > should not be put in stable releases. I will try to come up with an > > exact list soon. > > I am doing the process manually to ensure that this does not happen. > Ah, great, thanks for being careful. > Basically my method is for a given package called appropriately pkg: > > cd R-pkg > less R-pkg.spec # read file to ensure that it is safe to build it > > if it is safe then I run: > > $ fedpkg switch-branch f32 && git merge master && git push && fedpkg build -- > target=f32-build-side-24797 && fedpkg switch-branch master; cd .. > > the idea, of course, is that if the merge fails then the case needs to be > dealt manually. > For simple bumps that would not cause any merge issues, as it would just fast-forward. > FWIW most of the credit goes to Tom (spot) since the heavy lifting was already > done. :-) > > I will welcome your list as it becomes easier to find those cases. As I said > before I am doing my best to avoid those cases. It turns out the list is shorter than I expected; I went through their NEWS files to determine which ones are okay: masterf32 RCurl1.98.1.2 1.95.4.12 RSQLite 2.2.0 2.1.2 biglm 0.9.2 0.9.1 digest 0.6.25 0.6.22 fs 1.4.1 1.3.1- was missing dependencies before, I think? gargle 0.5.0 0.4.0 - was missing latest testthat; should be fine to update glue1.4.1 1.3.2 - requires new vctrs; do not bump haven 2.3.0 2.2.0 - requires new vctrs; do not bump httpuv 1.5.41.5.3.1 later 1.1.0.1 1.0.0 lubridate 1.7.8 1.7.4 - should be okay to build 1.7.8, but not 1.7.9, as it needs new vctrs matrixStats0.56.0 0.55.0- there are 'significant changes', but I'm not familiar with this package for whether that's okay multcomp 1.4.13 1.4.12 mvtnorm 1.1.0 1.0.12 pdftools2.3.12.2 - needs new dependencies, but I think check was turned off pkgload 1.1.0 1.0.2 prettyunits 1.1.1 1.0.2 qcc 2.72.2 quantities 0.1.4 0.1.3- should be okay to build 0.1.4, but not 0.1.5, as it needs new vctrs rgdal 1.5.8 1.4.8 rmarkdown 2.22.1 rsvg 2.11.3- probably okay, but missing test dependencies sandwich2.5.1 2.3.0 tidyr 1.1.0 1.0.2- requires new vctrs; do not bump tinytest1.2.1 1.1.0 tinytex 0.23 0.20 vctrs 0.3.0 0.2.4- many breaking changes; do not bump Feel free to double-check these. Also, let me know if you need to stop building and I can do so. > -- > José Abílio > -- Elliott ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Friday, 26 June 2020 10.47.13 WEST Iñaki Ucar wrote: > I used bcond locally and wrongly assumed that fedpkg build would > support --with BCOND and --without BCOND. Instead, the way to activate > it is to change to "%bcond_with check" and then revert to > "%bcond_without check". The only difference with bootstrap is that > "bootstrap" is recognized and a suffix ~bootstrap can be added > automatically to the resulting build. So the question is whether we > want that suffix or not. AFAIU the issue is that the bootstrap scheme does not requires to bump the release because with the suffix (~) is considered lower while changing from "%bcond_with check" to "%bcond_without check" requires to bump the release. The issue is that koji does not allow 2 builds with the same nvr. Since we should ensure the upgrade path that also requires to bump the release number in rawhide before applying it on f32 (in this particular case). -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Fri, 26 Jun 2020 at 12:09, José Abílio Matos wrote: > > On Friday, 26 June 2020 10.47.13 WEST Iñaki Ucar wrote: > > I used bcond locally and wrongly assumed that fedpkg build would > > support --with BCOND and --without BCOND. Instead, the way to activate > > it is to change to "%bcond_with check" and then revert to > > "%bcond_without check". The only difference with bootstrap is that > > "bootstrap" is recognized and a suffix ~bootstrap can be added > > automatically to the resulting build. So the question is whether we > > want that suffix or not. > > AFAIU the issue is that the bootstrap scheme does not requires to bump the > release because with the suffix (~) is considered lower while changing from > "%bcond_with check" to "%bcond_without check" requires to bump the release. > > The issue is that koji does not allow 2 builds with the same nvr. > > Since we should ensure the upgrade path that also requires to bump the release > number in rawhide before applying it on f32 (in this particular case). Then probably sticking to bootstrap is the best option. We should propose a change of the template SPEC in the guidelines to reflect that all packages should enclose their suggests and %check into a %{without boostrap} block, with a "%bcond_with bootstrap" defined on top. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Friday, 26 June 2020 11.17.56 WEST Iñaki Ucar wrote: > Then probably sticking to bootstrap is the best option. We should > propose a change of the template SPEC in the guidelines to reflect > that all packages should enclose their suggests and %check into a > %{without boostrap} block, with a "%bcond_with bootstrap" defined on > top. I agree. And probably such that the bootstrap stage exclude the testing part. So in the first instance we build the packages without testing and after the bootstrap we rebuild again those packages but with the tests on again. After finishing the second batch of packages (from your file) I have a huge list of packages that were not rebuild because they require either R-testthat or R-testit: # fails test stage: missing testthat or testit R-R-acepack R-R-bindr R-R-clipr R-R-clisymbols R-R-commonmark R-errors R-farver R-fastmap R-formatR R-fs R-glue R-ini R-jsonlite R-lazyeval R-packrat R-qtl R-rappdirs R-rlang R-systemfonts R-utf8 R-viridisLite R-xml2 R-zeallot R-askpass R-diffobj R-hunspell R-igraph R-later * R-lifecycle R-lubridate * R-mockr R-munsell R-plyr R-prettycode R-rex R-rgeos R-rprintf R-simmer R-stringr R-units # package fails because it requires dependencies above R-V8 R-bindrcpp R-cli R-corpus R-debugme R-ellipsis R-gdtools R-htmltools R-jqr R-lambda.r R-purrr R-reticulate R-rversions R-tweenr R-webutils The list of packages rebuilt until now is # first round/batch R R-rpm-macros rpy R-AUC R-AsioHeaders R-BH R-Cairo R-DBI R-FMStable R-NISTunits R-R.methodsS3 R-R6 R-RColorBrewer R-RUnit R-Rcpp R-RhpcBLASctl R-XML R-abind R-argon2 R-backports R-assertthat R-base64enc R-bit R-bitops R-brew R-chron R-coda R-colorspace R-combinat R-crayon R-curl R-date R-deldir R-dichromat R-digest R-disposables R-evaluate R-expm R-fansi R-fastmatch R-filehash R-fontBitstreamVera R-fontLiberation R-fortunes R-futile.options R-gamlss.dist R-gee R-generics R-ggplot2movies R-git2r R-globals R-gmp R-gsl R-gss R-gtable R-gtools R-highlight R-import R-inline R-iterators R-jpeg R-labeling R-listenv R-lmodel2 R-mAr R-magrittr R-maps R-matrixStats R-measurements R-microbenchmark R-mime R-mlbench R-mvtnorm R-ncdf4 R-nws R-pbdRPC R-pbdZMQ * R-pkgconfig R-plogr R-png R-polyclip R-polynom R-praise R-prettyunits R-ps R-quadprog R-rematch R-remotes R-rlecuyer R-rstudioapi R-rsvg R-scatterplot3d R-sciplot R-sfsmisc R-snow R-sodium R-sourcetools R-sp R-stringdist R-stringi R-sys R-sysfonts R-testit R-timeDate R-tinytest R-tkrplot R-udunits2 R-unix R-uuid R-waveslim R-wavethresh R-webp R-wesanderson R-whisker R-withr R-xfun R-xmlparsedata R-xtable R-yaml R-zoo # 2nd round/batch R-R.oo R-RCurl R-RInside R-RcppCCTZ R-Rmpfr R-V8 R-ape R-biglm R-bit64 R-caTools R-car R-foreach R-fts R-future R-gdata R-getPass R-itertools R-lmtest R-lokern R-mapproj R-markdown R-memoise R-msm R-poLCA R-processx R-rgdal R-rprojroot R-sandwich R-showtextdb R-statnet.common R-tikzDevice R-timeSeries R-tinytex # 3rd round/batch R-Bessel R-R.utils -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Wednesday, 24 June 2020 10.42.10 WEST I�aki Ucar wrote: > Thanks, Jos� and Elliott. I can help with reviews. > > I attach here a list of batches of CRAN packages to be rebuilt in > order (batches separated by a blank line), and the script that > generates it. Hope it helps. > > I�aki Hi I�aki, I noticed that you do not have TH.data, that in Fedora corresponds to R-TH-data in your results. I noticed it because multcomp depends on it and the compilation failed because R-TH-data was not yet ported. This questions is mainly directed to you, to Elliot and Tom, should we add to the rpm- macros something that puts Provides(R-rname) or since the naming is more or less the same we can add manually this for packages for which the R-rname and the Fedora package diverge... So in this case we would add, either directly or automatically, Provides(R-TH.data) What do you think? -- Jos� Ab�lio [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Mon, 29 Jun 2020 at 13:19, José Abílio Matos wrote: > > On Wednesday, 24 June 2020 10.42.10 WEST Iñaki Ucar wrote: > > Thanks, José and Elliott. I can help with reviews. > > > > I attach here a list of batches of CRAN packages to be rebuilt in > > order (batches separated by a blank line), and the script that > > generates it. Hope it helps. > > > > Iñaki > > Hi Iñaki, > I noticed that you do not have TH.data, that in Fedora corresponds to > R-TH-data in your > results. Yeah, sorry, that's a kind of bug in my script. I'm using CRAN names, and I forgot that some RPM packages change those names due to the dot to adhere to the guidelines. So TH-data is mistakenly dropped. > I noticed it because multcomp depends on it and the compilation failed > because R-TH-data > was not yet ported. > > This questions is mainly directed to you, to Elliot and Tom, should we add to > the rpm- > macros something that puts Provides(R-rname) or since the naming is more or > less the > same we can add manually this for packages for which the R-rname and the > Fedora > package diverge... > > > So in this case we would add, either directly or automatically, > > Provides(R-TH.data) > > What do you think? Now, we have $ sudo dnf repoquery --provides R-TH-data R(TH.data) = 1.0-10 R-TH-data = 1.0.10-4.fc3 which is what you are looking for I guess? I should be using that instead of the RPM package names. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Monday, 29 June 2020 12.35.35 WEST I�aki Ucar wrote: > Yeah, sorry, that's a kind of bug in my script. I'm using CRAN names, > and I forgot that some RPM packages change those names due to the dot > to adhere to the guidelines. So TH-data is mistakenly dropped. Thank you for feedback. I am trying to make this process as automatic as possible. > > I noticed it because multcomp depends on it and the compilation failed > > because R-TH-data was not yet ported. > > > > This questions is mainly directed to you, to Elliot and Tom, should we add > > to the rpm- macros something that puts Provides(R-rname) or since the > > naming is more or less the same we can add manually this for packages for > > which the R-rname and the Fedora package diverge... > > > > > > So in this case we would add, either directly or automatically, > > > > Provides(R-TH.data) > > > > What do you think? > > Now, we have > > $ sudo dnf repoquery --provides R-TH-data > R(TH.data) = 1.0-10 > R-TH-data = 1.0.10-4.fc3 > > which is what you are looking for I guess? You guessed right. :-) Apologies for not verifying this before but while controlling the build processes my attention span becomes that of a toddler. :-) I do not know if the process will succeed or not. The extreme example was R-pbdZMQ that took more than a day building. The issue was with the testing stage in s390x where a test with ports were stuck. I guess that there was a long timeout. > I should be using that instead of the RPM package names. Again you guessed right, that was the main idea. :-) As I told above and to reiterate it, the idea of this work is to make it easier to automate the process. Similarly to how we do the mass builds for all the packages. BTW your idea to separate the builds in batches is a very useful idea. The reason is that it takes some time for the builds to be added to the repository associated with the side tag. I had already three cases where the build failed because the new build of a dependent package was not yet in the repository and so the resolver took an older build (made with R-3.6). Best regards, -- Jos� Ab�lio [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Mon, 29 Jun 2020 at 14:25, José Abílio Matos wrote: > > Again you guessed right, that was the main idea. :-) > As I told above and to reiterate it, the idea of this work is to make it > easier to automate the > process. Similarly to how we do the mass builds for all the packages. But the mass rebuild process is very different, because releng doesn't follow any particular order (they don't need to, because nothing really changed). The question is whether there is any tool to resolve dependencies and generate a list of builds, as I do with the CRAN database, for an arbitrary list of RPMs. That would be perfect. In theory, this should be possible with dnf, but when I tried, I failed (or my script would theoretically work, but it would take ages to complete). I also took a look at releng's repos on Pagure, but I didn't find any tool to do this either. We should bring this to devel, maybe ask Miro and other Python folks who may have The Method(TM), as they are rebuilding stuff on a daily basis. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Friday, 26 June 2020 10.52.44 WEST Elliott Sales de Andrade wrote: > Feel free to double-check these. > > Also, let me know if you need to stop building and I can do so. The first round is ready. I built all the cran packages using Iñaki's order. For the remainder those that failed I will implement the bootstrap in packages as described before. I intend to disable testing and building vignettes in bootstrap mode. After all the packages are rebuilt I will apply the same recipe but with the bootstrap mode disabled. Most of the version differences were minor and I have updated those package. The exceptions goes as you suggested to the dependences of vctrs. I pinned vctrs version in F32. The same should apply to all the dependences, I think that I made a mistake for some of the dependences that I will fix in the next stage, one example is haven were I have updated the spec to rawhide version. This mistake is harmless at this stage because those packages do not build since vectrs' version is unchanged. I will revert that in the next stage. Regards, -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Friday, 26 June 2020 10.52.44 WEST Elliott Sales de Andrade wrote: > glue1.4.1 1.3.2 - requires new vctrs; do not bump This process takes time but I think that I am on the right track (to the abyss ?). :-) I am now rebuilding packages that need bootstrap and/or from Bioconductor. An example of how this is done can be seen here: https://src.fedoraproject.org/rpms/R-glue/blob/master/f/R-glue.spec In this case the bootstrap mode is off to make it on it would be enough to change one from %bcond_with bootstrap to %bcond_without bootstrap And yes at first sight it looks like the it should be the other way around. The question is read it as _allow_ to build the file conditionally with/ without bootstrap. Since most of the packages do not need this I propose to only add this snippet where it is really needed and not everywhere as I have hinted initially. Or else the spec files becomes noisy unnecessarily. Regards, -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Monday, 29 June 2020 13.46.02 WEST Iñaki Ucar wrote: > But the mass rebuild process is very different, because releng doesn't > follow any particular order (they don't need to, because nothing > really changed). > > The question is whether there is any tool to resolve dependencies and > generate a list of builds, as I do with the CRAN database, for an > arbitrary list of RPMs. That would be perfect. In theory, this should > be possible with dnf, but when I tried, I failed (or my script would > theoretically work, but it would take ages to complete). I also took a > look at releng's repos on Pagure, but I didn't find any tool to do > this either. > > We should bring this to devel, maybe ask Miro and other Python folks > who may have The Method(TM), as they are rebuilding stuff on a daily > basis. I think that I meant the way that the python packages are rebuilt for each new python version. I should have said Python mass rebuild. I took your work as the basis, together with Tom's help and I used dnf to get a list of R packages (assuming that their name starts with R- ). The advantage of this approach is that it works for CRAN and Bioconductor. $ dnf list available --disablerepo=\* --enablerepo=rawhide-source R-* | tail - n +3 | awk -e '{print $1};' | sed -e 's/\.src$//' > r-packages.txt That gave the list of the (source) packages to rebuild. In order to get the list of dependencies I used a similar strategy: $ (for p in $(cat r-packages.txt); do echo -n $p, ; dnf -q repoquery -- disablerepo=\* --enablerepo=rawhide-source --requires $p | sed ':a;N;$!ba;s/ \n/,/g'; done) > r-packages-dependencies.csv The resulting file is a csv file with the first column as the package and the remaining columns as the dependencies. Now the idea was to process this file using a python script to create a direct graph of dependencies (using networkx). In the process non R packages are removed and some dependencies are cleaned (e.g. -devel packages direct to the source package). The advantage of this is that it allows to identify cycles. My only surprise in this process was to find that several packages require R- rpm-macros. That causes some dependency cycles because R-rpm-macros also requires R-rprintf,R-devel,R-knitr,R-stringi,R-testthat. Since R-rpm-macros is used for Fedora and epel 8 I suggest to remove the dependency. Do you agree? -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Fri, 3 Jul 2020 at 19:03, José Abílio Matos wrote: > > On Monday, 29 June 2020 13.46.02 WEST Iñaki Ucar wrote: > > But the mass rebuild process is very different, because releng doesn't > > follow any particular order (they don't need to, because nothing > > really changed). > > > > The question is whether there is any tool to resolve dependencies and > > generate a list of builds, as I do with the CRAN database, for an > > arbitrary list of RPMs. That would be perfect. In theory, this should > > be possible with dnf, but when I tried, I failed (or my script would > > theoretically work, but it would take ages to complete). I also took a > > look at releng's repos on Pagure, but I didn't find any tool to do > > this either. > > > > We should bring this to devel, maybe ask Miro and other Python folks > > who may have The Method(TM), as they are rebuilding stuff on a daily > > basis. > > I think that I meant the way that the python packages are rebuilt for each new > python version. I should have said Python mass rebuild. > > I took your work as the basis, together with Tom's help and I used dnf to get > a list of R packages (assuming that their name starts with R- ). The advantage > of this approach is that it works for CRAN and Bioconductor. > > $ dnf list available --disablerepo=\* --enablerepo=rawhide-source R-* | tail - > n +3 | awk -e '{print $1};' | sed -e 's/\.src$//' > r-packages.txt > > That gave the list of the (source) packages to rebuild. > > In order to get the list of dependencies I used a similar strategy: > $ (for p in $(cat r-packages.txt); do echo -n $p, ; dnf -q repoquery -- > disablerepo=\* --enablerepo=rawhide-source --requires $p | sed ':a;N;$!ba;s/ > \n/,/g'; done) > r-packages-dependencies.csv > > The resulting file is a csv file with the first column as the package and the > remaining columns as the dependencies. > > Now the idea was to process this file using a python script to create a direct > graph of dependencies (using networkx). In the process non R packages are > removed and some dependencies are cleaned (e.g. -devel packages direct to the > source package). Nice! What if we create a group "R" on Pagure and a repo "fedora-scripts" or something like that? > The advantage of this is that it allows to identify cycles. > > My only surprise in this process was to find that several packages require R- > rpm-macros. > > That causes some dependency cycles because R-rpm-macros also requires > R-rprintf,R-devel,R-knitr,R-stringi,R-testthat. Are these the packages that require R-rpm-macros or others? > Since R-rpm-macros is used for Fedora and epel 8 I suggest to remove the > dependency. Do you agree? The packages are most probably maintained by Elliott, right? He would know better why they have this dependency. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Fri, 3 Jul 2020 at 13:03, José Abílio Matos wrote: > > On Monday, 29 June 2020 13.46.02 WEST Iñaki Ucar wrote: > > But the mass rebuild process is very different, because releng doesn't > > follow any particular order (they don't need to, because nothing > > really changed). > > > > The question is whether there is any tool to resolve dependencies and > > generate a list of builds, as I do with the CRAN database, for an > > arbitrary list of RPMs. That would be perfect. In theory, this should > > be possible with dnf, but when I tried, I failed (or my script would > > theoretically work, but it would take ages to complete). I also took a > > look at releng's repos on Pagure, but I didn't find any tool to do > > this either. > > > > We should bring this to devel, maybe ask Miro and other Python folks > > who may have The Method(TM), as they are rebuilding stuff on a daily > > basis. > > I think that I meant the way that the python packages are rebuilt for each new > python version. I should have said Python mass rebuild. > > I took your work as the basis, together with Tom's help and I used dnf to get > a list of R packages (assuming that their name starts with R- ). The advantage > of this approach is that it works for CRAN and Bioconductor. > > $ dnf list available --disablerepo=\* --enablerepo=rawhide-source R-* | tail - > n +3 | awk -e '{print $1};' | sed -e 's/\.src$//' > r-packages.txt > > That gave the list of the (source) packages to rebuild. > > In order to get the list of dependencies I used a similar strategy: > $ (for p in $(cat r-packages.txt); do echo -n $p, ; dnf -q repoquery -- > disablerepo=\* --enablerepo=rawhide-source --requires $p | sed ':a;N;$!ba;s/ > \n/,/g'; done) > r-packages-dependencies.csv > > The resulting file is a csv file with the first column as the package and the > remaining columns as the dependencies. > > Now the idea was to process this file using a python script to create a direct > graph of dependencies (using networkx). In the process non R packages are > removed and some dependencies are cleaned (e.g. -devel packages direct to the > source package). > > > The advantage of this is that it allows to identify cycles. > > My only surprise in this process was to find that several packages require R- > rpm-macros. > > That causes some dependency cycles because R-rpm-macros also requires > R-rprintf,R-devel,R-knitr,R-stringi,R-testthat. > Your process must have an error, because R-rpm-macros only Requires R-core. It also BuildRequires nothing. R-devel depends on R-rpm-macros, and this is intended, in order to pull in automated dependency generators. If anything else directly requires R-rpm-macros, that is an error, and these are all in packages I don't own. > Since R-rpm-macros is used for Fedora and epel 8 I suggest to remove the > dependency. Do you agree? > > > -- > José Abílio > -- Elliott ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
[Apologies if this message is a dupe but I do not find evidence of the other message I wrote] On Friday, 3 July 2020 23.06.43 WEST Elliott Sales de Andrade wrote: > Your process must have an error, because R-rpm-macros only Requires > R-core. It also BuildRequires nothing. You are right. This caught a blind spot in my shell script. If there are no dependencies the script does not change line and thus R-rpm-macros and R-rprintf entries were merged. > R-devel depends on R-rpm-macros, and this is intended, in order to > pull in automated dependency generators. If anything else directly > requires R-rpm-macros, that is an error, and these are all in packages > I don't own. There are several Rccp related packages that require R-rpm-macros. Currently it is no-op so not a problem although it is unnecessary. Regards, -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tuesday, 9 June 2020 03.40.52 WEST Tom Callaway wrote: > Over the last several days, I've been working hard to get all of the Fedora > R packages rebuilt against R 4.0 in rawhide (in the F33-R-4 side tag). With > the exception of R-biomaRt, R-BSgenome, R-GenomicAlignments, and > R-rtracklayer, I believe everything is built and updated to the latest > versions. And of those packages, they're all ready to go when Fedora > infrastructure is working reliably again (the great datacenter migration > has started and I can no longer git push). I'll also push R 4.0.1 into the > tag when that's possible. In line with what we discussed here I have the updates for R 4.0.2 ready. The exception is R-BiocFileCache and in that line R-biomaRt. They have an asterisk in front of them because of that. The packages that were rebuilt are in the list attached, together with the respective build order. I pushed an update to updates-testing where I disabled the automatic push (both karma and time based). Report problems either here or on bodhi: https://bodhi.fedoraproject.org/updates/FEDORA-2020-4c0ce71810 Best regards, -- José Abílio # round 0 R R-rpm-macros rpy # round 1 R-AUC R-AsioHeaders R-BH R-BiocGenerics R-BufferedMatrix R-DBI R-DynDoc R-FMStable R-GenomeInfoDbData R-NISTunits R-R.methodsS3 R-R6 R-RColorBrewer R-RODBC R-Rcompression R-RhpcBLASctl R-Rhtslib R-Rsolid R-abind R-affyio R-argon2 R-ascii R-assertthat R-backports R-base64enc R-bit R-bitops R-brew R-car R-chron R-coda R-colorspace R-combinat R-crayon R-curl R-date R-dichromat R-digest R-disposables R-evaluate R-expm R-fansi R-fastmatch R-filehash R-fontBitstreamVera R-fontLiberation R-fortunes R-futile.options R-gamlss.dist R-gee R-generics R-ggplot2movies R-globals R-gmp R-gsl R-gss R-gtable R-gtools R-highlight R-import R-inline R-iterators R-jpeg R-labeling R-listenv R-lmodel2 R-mAr R-magrittr R-matrixStats R-measurements R-microbenchmark R-mime R-mlbench R-mnormt R-mvtnorm R-ncdf4 R-nws R-pbdRPC R-pbdZMQ R-pkgconfig R-png R-polyclip R-polynom R-praise R-preprocessCore R-prettyunits R-ps R-quadprog R-rematch R-remotes R-rlecuyer R-rstudioapi R-rsvg R-scatterplot3d R-sciplot R-sfsmisc R-snow R-sodium R-sourcetools R-sp R-stringi R-sys R-sysfonts R-tinytest R-tkrplot R-udunits2 R-unix R-uuid R-waveslim R-wavethresh R-wesanderson R-widgetTools R-withr R-xfun R-xmlparsedata R-xtable R-zoo # round 2 R-Cairo R-R.oo R-RCurl R-Rcpp R-Rmpfr R-S4Vectors R-V8 R-ape R-biglm R-bit64 R-caTools R-cli R-deldir R-foreach R-fts R-future R-getPass R-lmtest R-lokern R-maps R-markdown R-memoise R-msm R-poLCA R-processx R-rgdal R-rprojroot R-sandwich R-showtextdb R-statnet.common R-stringdist R-testit R-tikzDevice R-tkWidgets R-webp # round 3 R-Bessel R-Biobase R-R.utils R-RInside R-RM2 R-TH-data R-RcppCCTZ R-XML R-callr R-desc R-git2r R-here R-highr R-itertools R-mapproj R-multcomp R-orcutt R-qcc R-scales R-showtext R-plogr R-systemfit R-tinytex R-whisker # round 4 R-R.cache R-R.devices R-httr R-RUnit # round 5 R-IRanges R-R.rsp R-doParallel R-gdata R-timeDate R-yaml # round 6 R-GenomeInfoDb R-XVector R-timeSeries # round 7 R-Biostrings R-GenomicRanges R-glue (bootstrap) R-rprintf (bootstrap) R-rlang (bootstrap) # round 8 R-ellipsis R-pkgbuild # round 9 R-pkgload R-vctrs (bootstrap) # round 10 R-blob R-testthat # round 11 R-RSQLite R-acepack R-askpass R-bindr R-clisymbols R-cyclocomp R-errors R-farver R-fastmap R-htmltools R-igraph R-ini R-mockery R-munsell R-pingr R-plyr R-qtl R-rgeos R-simmer R-units R-utf8 R-xopen # round 12 R-AnnotationDbi R-bindrcpp R-debugme R-jsonlite R-openssl R-pillar R-prettycode R-qpdf R-quantities R-reshape R-sessioninfo R-tweenr R-xml2 # round 13 R-commonmark R-gh R-htmlwidgets R-pdftools R-rversions R-whoami # round 14 R-stringr # round 15 R-knitr R-reshape2 R-selectr # round 16 R-corpus R-data.table R-gplots R-hexbin R-littler R-lubridate R-rmarkdown # round 17 R-clipr R-diffobj R-formatR R-hms R-hunspell R-later R-lazyeval R-lifecycle R-nanotime R-packrat R-prettydoc R-purrr R-rcmdcheck R-systemfonts R-tufte R-viridisLite # round 18 R-gdtools R-lambda.r R-mockr R-progress R-promises R-repr R-rex R-roxygen2 R-spelling R-unitizer R-websocket R-zeallot # round 19 R-IRdisplay R-cliapp R-futile.logger R-httpuv R-jose R-jqr R-lintr R-rappdirs R-rvest R-svglite R-tibble # round 20 R-BiocParallel R-IRkernel R-cellranger R-fs R-readr R-rematch2 R-reticulate R-tidyselect R-webutils # round 21 R-DelayedArray R-Rsamtools R-dplyr (bootstrap) R-gargle R-parsedate R-readxl # round 22 R-SummarizedExperiment R-dtplyr R-foghorn R-ggplot2 R-gmailr R-nycflights13 R-rhub R-styler # round 23 R-GenomicAlignments R-dbplyr R-forcats R-gapminder R-qvalue R-shiny # round 24 R-haven R-miniUI R-reprex R-repurrrsive R-rsconnect R-rtracklayer # round 25 R-BSgenome R-BiocFileCache * R-pkgdown R-tidyr # round 26 R-biomaRt * R-broom R-usethis # round 27 R-devtools R-geepack R-modelr # round 28 R-profvis ___ R-SIG-Fedora mailin
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Mon, 6 Jul 2020 at 16:40, José Abílio Matos wrote: > > On Tuesday, 9 June 2020 03.40.52 WEST Tom Callaway wrote: > > Over the last several days, I've been working hard to get all of the Fedora > > R packages rebuilt against R 4.0 in rawhide (in the F33-R-4 side tag). With > > the exception of R-biomaRt, R-BSgenome, R-GenomicAlignments, and > > R-rtracklayer, I believe everything is built and updated to the latest > > versions. And of those packages, they're all ready to go when Fedora > > infrastructure is working reliably again (the great datacenter migration > > has started and I can no longer git push). I'll also push R 4.0.1 into the > > tag when that's possible. > > In line with what we discussed here I have the updates for R 4.0.2 ready. > > The exception is R-BiocFileCache and in that line R-biomaRt. They have an > asterisk in front of them because of that. > > The packages that were rebuilt are in the list attached, together with the > respective build order. > > I pushed an update to updates-testing where I disabled the automatic > push (both karma and time based). > > Report problems either here or on bodhi: > https://bodhi.fedoraproject.org/updates/FEDORA-2020-4c0ce71810 [Sorry for the dupe, I didn't hit "reply all"] Wonderful! Now I have 15k packages to rebuild. :) Have you thought about creating an "R" group in Pagure and toss the scripts you used into a repo? -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
R-BiocFileCache is now branched for f32 (finally). You should be able to build it if/when the PDC comes back up. Lotta random outages right now. Tom On Mon, Jul 6, 2020 at 10:40 AM José Abílio Matos wrote: > On Tuesday, 9 June 2020 03.40.52 WEST Tom Callaway wrote: > > Over the last several days, I've been working hard to get all of the > Fedora > > R packages rebuilt against R 4.0 in rawhide (in the F33-R-4 side tag). > With > > the exception of R-biomaRt, R-BSgenome, R-GenomicAlignments, and > > R-rtracklayer, I believe everything is built and updated to the latest > > versions. And of those packages, they're all ready to go when Fedora > > infrastructure is working reliably again (the great datacenter migration > > has started and I can no longer git push). I'll also push R 4.0.1 into > the > > tag when that's possible. > > In line with what we discussed here I have the updates for R 4.0.2 ready. > > The exception is R-BiocFileCache and in that line R-biomaRt. They have an > asterisk in front of them because of that. > > The packages that were rebuilt are in the list attached, together with the > respective build order. > > I pushed an update to updates-testing where I disabled the automatic > push (both karma and time based). > > Report problems either here or on bodhi: > https://bodhi.fedoraproject.org/updates/FEDORA-2020-4c0ce71810 > > Best regards, > -- > José Abílio > ___ > R-SIG-Fedora mailing list > R-SIG-Fedora@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Monday, 6 July 2020 21.08.53 WEST Tom Callaway wrote: > R-BiocFileCache is now branched for f32 (finally). You should be able to > build it if/when the PDC comes back up. Lotta random outages right now. > > Tom I have re/built them using the side tag but I do not see how to add them to the (bodhi) update. I would like to add R-BiocFileCache and R-biomaRt to the update to finish the process. I see that Elliot added three new packages there (thank you) but when I edit the update I do not see any field to add new packages, or even to change the existing. -- Jos� Ab�lio [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 7 Jul 2020 at 12:38, José Abílio Matos wrote: > > On Monday, 6 July 2020 21.08.53 WEST Tom Callaway wrote: > > R-BiocFileCache is now branched for f32 (finally). You should be able to > > build it if/when the PDC comes back up. Lotta random outages right now. > > > > Tom > > I have re/built them using the side tag but I do not see how to add them to > the > (bodhi) update. > I would like to add R-BiocFileCache and R-biomaRt to the update to finish the > process. > > I see that Elliot added three new packages there (thank you) but when I edit > the > update I do not see any field to add new packages, or even to change the > existing. Try with the CLI (see "man bodhi"): $ bodhi updates edit --addbuilds BTW, could we coordinate to merge this into stable when I finish rebuilding stuff in Copr? -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tuesday, 7 July 2020 11.44.48 WEST Iñaki Ucar wrote: > Try with the CLI (see "man bodhi"): > > $ bodhi updates edit --addbuilds I found that the best call in this case is instead of --addbuilds to use --from-tag since then "this will update the builds to the latest ones in the tag." > BTW, could we coordinate to merge this into stable when I finish > rebuilding stuff in Copr? Sure. I am in no rush to submit this to stable. :-) -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 7 Jul 2020 at 15:58, José Abílio Matos wrote: > > On Tuesday, 7 July 2020 11.44.48 WEST Iñaki Ucar wrote: > > Try with the CLI (see "man bodhi"): > > > > $ bodhi updates edit --addbuilds > > I found that the best call in this case is instead of --addbuilds to use > --from-tag since then "this will update the builds to the latest ones in the > tag." > > > BTW, could we coordinate to merge this into stable when I finish > > rebuilding stuff in Copr? > > Sure. I am in no rush to submit this to stable. :-) Great, I already started. I'll keep you posted. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 7 Jul 2020 at 09:58, José Abílio Matos wrote: > > On Tuesday, 7 July 2020 11.44.48 WEST Iñaki Ucar wrote: > > Try with the CLI (see "man bodhi"): > > > > $ bodhi updates edit --addbuilds > > I found that the best call in this case is instead of --addbuilds to use > --from-tag since then "this will update the builds to the latest ones in the > tag." > Note, if the side tag was turned into an update, you need to tag new builds into f32-updates-candidate first. I've done that for: * R-biomaRt-2.44.0-1.fc32 * R-BiocFileCache-1.12.0-2.fc32 * rpy-3.3.5-1.fc32 > > BTW, could we coordinate to merge this into stable when I finish > > rebuilding stuff in Copr? > > Sure. I am in no rush to submit this to stable. :-) > > -- > José Abílio > -- Elliott ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Thursday, 9 July 2020 08.41.39 WEST Elliott Sales de Andrade wrote: > Note, if the side tag was turned into an update, you need to tag new > builds into f32-updates-candidate first. I've done that for: > * R-biomaRt-2.44.0-1.fc32 > * R-BiocFileCache-1.12.0-2.fc32 > * rpy-3.3.5-1.fc32 Thank you Elliot. -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 7 Jul 2020 at 16:15, Iñaki Ucar wrote: > > On Tue, 7 Jul 2020 at 15:58, José Abílio Matos wrote: > > > > On Tuesday, 7 July 2020 11.44.48 WEST Iñaki Ucar wrote: > > > Try with the CLI (see "man bodhi"): > > > > > > $ bodhi updates edit --addbuilds > > > > I found that the best call in this case is instead of --addbuilds to use > > --from-tag since then "this will update the builds to the latest ones in the > > tag." > > > > > BTW, could we coordinate to merge this into stable when I finish > > > rebuilding stuff in Copr? > > > > Sure. I am in no rush to submit this to stable. :-) > > Great, I already started. I'll keep you posted. CRAN rebuilt on Copr. Ready to push to stable. -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Thursday, 9 July 2020 21.25.05 WEST Iñaki Ucar wrote: > CRAN rebuilt on Copr. Ready to push to stable. The update also has the karma necessary to be pushed to stable. Does any one has any objection for this to be pushed to stable? If I do not hear until then I will push the update Monday night (Western Europe time zone, and yes we are in the Summer and the days are larger) in time for the daily batch update. Regards, -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Saturday, 11 July 2020 11.32.32 WEST Jos� Ab�lio Matos wrote: > If I do not hear until then I will push the update Monday night (Western > Europe time zone. Well I tried but I did not succeeded both using the web interface and cli interfaces: b"504 Gateway Time-out\nThe server didn't respond in time.\n\n" -- Jos� Ab�lio [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 14 Jul 2020 at 01:19, José Abílio Matos wrote: > > On Saturday, 11 July 2020 11.32.32 WEST José Abílio Matos wrote: > > If I do not hear until then I will push the update Monday night (Western > > Europe time zone. > > Well I tried but I did not succeeded both using the web interface and cli > interfaces: > > b"504 Gateway Time-out\nThe server didn't > respond in time.\n\n" Maybe it's due to the size of the update? File an issue here: https://github.com/fedora-infra/bodhi/issues -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tuesday, 14 July 2020 09.47.55 WEST Iñaki Ucar wrote: > Maybe it's due to the size of the update? File an issue here: > https://github.com/fedora-infra/bodhi/issues Before resorting to this I tried again and this time it worked. Elliot before had already tried, I got an email message saying: bodhi - 2020-07-14 04:47:02.353824 (karma: 0) This update has been submitted for stable by qulogic. Now I got another one saying: bodhi - 2020-07-14 16:11:38.021756 (karma: 0) This update has been submitted for stable by jamatos. Let us see if it works this time. :-) -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 14 Jul 2020 at 18:24, José Abílio Matos wrote: > > On Tuesday, 14 July 2020 09.47.55 WEST Iñaki Ucar wrote: > > Maybe it's due to the size of the update? File an issue here: > > https://github.com/fedora-infra/bodhi/issues > > Before resorting to this I tried again and this time it worked. > > Elliot before had already tried, I got an email message saying: > bodhi - 2020-07-14 04:47:02.353824 (karma: 0) > This update has been submitted for stable by qulogic. > > Now I got another one saying: > bodhi - 2020-07-14 16:11:38.021756 (karma: 0) > This update has been submitted for stable by jamatos. > > Let us see if it works this time. :-) Great, thanks for all the work! FYI, anyway I opened an issue, because this is likely to hit us in the future unless it's resolved: https://github.com/fedora-infra/bodhi/issues/4083 -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Friday, 3 July 2020 18.36.17 WEST Iñaki Ucar wrote: > Nice! What if we create a group "R" on Pagure and a repo > "fedora-scripts" or something like that? I would like to improve the scripts but FWIW here it comes a rough version of the script I used. The python script loads the csv file and constructs a directed graph and it starts to remove the leave packages (pruning or trimming if you take a gardening analogy). The csv files has the dependency of the packages. It is always possible to go to the packages that remain after the first stage and using a bootstrap mode to remove most of the dependencies and then to run the code again. This procedure was enough to process all the packages. -- José Abílio# -*- coding: utf-8 -*- """ Process csv files removing several files outside of R modules: R-devel non R related packages """ # from pprint import pprint import networkx as nx RCORE = ['R', 'R-core', 'R-java', 'R-KernSmooth', 'R-MASS', 'R-Matrix', 'R-base', 'R-boot', 'R-class', 'R-cluster', 'R-codetools', 'R-compiler', 'R-datasets', 'R-foreign', 'R-grDevices', 'R-graphics', 'R-grid', 'R-lattice', 'R-methods', 'R-mgcv', 'R-nlme', 'R-nnet', 'R-parallel', 'R-rpart', 'R-spatial', 'R-splines', 'R-stats', 'R-stats4', 'R-survival', 'R-tcltk', 'R-tools', 'R-translations', 'R-utils', 'R-rpm-macros'] depnet = {}; DG = nx.DiGraph() for line in open('r-packages-dependencies.csv'): line = line[:-1]; fields = line.split(','); package = fields[0]; DG.add_node(package) pkgd = [] for pkg in fields[1:]: if not pkg.startswith('R-'): continue pkg = pkg.split(' ')[0].replace('-devel', '') if pkg in RCORE: continue pkgd.append(pkg) DG.add_edge(package, pkg) depnet[package]=pkgd # pprint(depnet) # Try to identify batches of builds iter = 0 while True: batch = [x for x in DG if len(DG[x]) == 0] if len(batch) == 0: break iter += 1 print(f'# round {iter}') for x in sorted(batch): print(x) DG.remove_node(x) if len(DG): print("#Unresolved dependencies: {}".format(len(DG))) print(list(DG)) print() print("#Number of cycles {}".format(len(list(nx.simple_cycles(DG) print("Cycles:") print(list(nx.simple_cycles(DG)))R-AUC,R-devel,tex(latex) R-AnnotationDbi,R-Biobase >= 1.17.0,R-BiocGenerics >= 0.29.2,R-DBI,R-IRanges-devel,R-RSQLite,R-S4Vectors-devel >= 0.9.25,R-devel >= 2.7.0,R-methods,R-stats4,R-utils,tetex-latex R-AsioHeaders,R-devel,tex(latex) R-BH,R-devel >= 3.0.0,tetex-latex R-BSgenome,R-BiocGenerics >= 0.13.8,R-Biostrings-devel >= 2.47.6,R-GenomeInfoDb >= 1.23.9,R-GenomicRanges >= 1.31.10,R-IRanges-devel >= 2.13.16,R-Rsamtools,R-S4Vectors-devel >= 0.17.28,R-XVector-devel,R-devel >= 3.0.0,R-matrixStats,R-methods,R-rtracklayer >= 1.39.7,R-stats,R-utils,tex(latex) R-Bessel,R-Rmpfr,R-devel,R-gsl,R-methods,R-sfsmisc,tex(latex) R-Biobase,R-BiocGenerics >= 0.27.1,R-devel >= 3.4.0,R-methods,R-tkWidgets,R-utils,tex(latex) R-BiocFileCache,R-DBI,R-RSQLite,R-curl,R-dbplyr >= 1.0.0,R-devel >= 3.4.0,R-dplyr,R-httr,R-knitr,R-methods,R-rappdirs,R-rmarkdown,R-rtracklayer,R-stats,R-testthat,R-utils,tetex-latex R-BiocGenerics,R-devel >= 3.6.0,R-graphics,R-methods,R-parallel,R-stats,R-utils,tex(latex) R-BiocParallel,R-BH-devel,R-devel >= 3.0.0,R-futile.logger,R-methods,R-parallel,R-snow,R-stats,R-utils,autoconf,automake,gcc,gcc-c++ R-Biostrings,R-Biobase,R-BiocGenerics >= 0.31.5,R-IRanges-devel >= 2.21.6,R-S4Vectors-devel >= 0.25.14,R-XVector-devel >= 0.27.2,R-crayon,R-devel >= 3.5.0,R-grDevices,R-graphics,R-methods,R-stats,R-utils,tex(latex) R-BufferedMatrix,R-devel >= 3.4.0,R-methods,tex(latex) R-Cairo,R-devel,R-grDevices,R-graphics,R-png,cairo-devel >= 1.2,libXt-devel,tex(latex) R-DBI,R-devel >= 3.4.0,R-methods,texlive-latex R-DelayedArray,R-BiocGenerics >= 0.31.5,R-BiocParallel,R-IRanges-devel >= 2.17.3,R-S4Vectors-devel >= 0.25.15,R-devel >= 3.4.0,R-matrixStats,R-methods,R-stats4,tetex-latex R-DynDoc,R-devel >= 3.0.0,R-methods,R-utils,tex(latex) R-FMStable,R-devel,tex(latex) R-GenomeInfoDb,R-BiocGenerics >= 0.13.8,R-GenomeInfoDbData,R-IRanges-devel >= 2.13.12,R-RCurl,R-S4Vectors-devel >= 0.25.12,R-devel >= 3.1.0,R-methods,R-stats,R-stats4,R-utils R-GenomeInfoDbData,R-devel >= 3.5.0,tetex-latex R-GenomicAlignments,R-BiocGenerics >= 0.15.3,R-BiocParallel,R-Biostrings >= 2.47.6,R-GenomeInfoDb >= 1.13.1,R-GenomicRanges >= 1.37.2,R-IRanges-devel >= 2.15.12,R-Rsamtools-devel >= 1.31.2,R-S4Vectors-devel >= 0.23.19,R-SummarizedExperiment >= 1.9.13,R-devel >= 3.5.0,R-methods,R-stats,R-utils,tetex-latex R-GenomicRanges,R >= 3.4.0,R-BiocGenerics >= 0.25.3,R-GenomeInfoDb >= 1.15.2,R-IRanges-devel >= 2.21.6,R-S4Vectors-devel >= 0.25.14,R-XVector-devel >= 0.27.2,R-devel,R-methods,R-stats,R-stats4,R-utils,tex(latex) R-IRanges,R-BiocGenerics >= 0.25.3,R-RUnit,R-S4Vectors-devel >= 0.25.14,R-de
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 21 Jul 2020 at 17:05, José Abílio Matos wrote: > > On Friday, 3 July 2020 18.36.17 WEST Iñaki Ucar wrote: > > Nice! What if we create a group "R" on Pagure and a repo > > "fedora-scripts" or something like that? > > I would like to improve the scripts but FWIW here it comes a rough version of > the script I used. > > The python script loads the csv file and constructs a directed graph and it > starts to remove the leave packages (pruning or trimming if you take a > gardening analogy). > > The csv files has the dependency of the packages. It is always possible to go > to the packages that remain after the first stage and using a bootstrap mode > to remove most of the dependencies and then to run the code again. > > This procedure was enough to process all the packages. Looks great! I've created an R group on Pagure and added you all (jamatos, qulogic, spot) to it (feel free to add anyone else that may be interested). Please, create a repo there with these tools. I'll transfer https://pagure.io/fedora-r-packages there too (if that's possible; or just mirror it if not). -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Tue, 21 Jul 2020 at 11:05, José Abílio Matos wrote: > > On Friday, 3 July 2020 18.36.17 WEST Iñaki Ucar wrote: > > Nice! What if we create a group "R" on Pagure and a repo > > "fedora-scripts" or something like that? > > I would like to improve the scripts but FWIW here it comes a rough version of > the script I used. > > The python script loads the csv file and constructs a directed graph and it > starts to remove the leave packages (pruning or trimming if you take a > gardening analogy). > I'm not exactly fluent in Haskell, but Jens Petersen has a tool called fbrnch [1], which is able to calculate the build order, and then submit them, and even handles override for non-Rawhide builds. It does need you to handle some of the bootstrap conditions yourself, but with a defined workflow, it might be possible to request that it handle that too. > The csv files has the dependency of the packages. It is always possible to go > to the packages that remain after the first stage and using a bootstrap mode > to remove most of the dependencies and then to run the code again. > > This procedure was enough to process all the packages. > -- > José Abílio [1] https://github.com/juhp/fbrnch/ -- Elliott ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Mon, 27 Jul 2020 at 09:09, Elliott Sales de Andrade wrote: > > On Tue, 21 Jul 2020 at 11:05, José Abílio Matos wrote: > > > > On Friday, 3 July 2020 18.36.17 WEST Iñaki Ucar wrote: > > > Nice! What if we create a group "R" on Pagure and a repo > > > "fedora-scripts" or something like that? > > > > I would like to improve the scripts but FWIW here it comes a rough version > > of > > the script I used. > > > > The python script loads the csv file and constructs a directed graph and it > > starts to remove the leave packages (pruning or trimming if you take a > > gardening analogy). > > > > I'm not exactly fluent in Haskell, but Jens Petersen has a tool called > fbrnch [1], which is able to calculate the build order, and then > submit them, and even handles override for non-Rawhide builds. It does > need you to handle some of the bootstrap conditions yourself, but with > a defined workflow, it might be possible to request that it handle > that too. Very interesting! I poked around a bit and discovered that they use a Haskell library [1] for that, and a quick search led me to a package called rpmbuild-order [2]. Never heard of it. So $ rpmbuild-order sort pkg1 pkg2 pkg3 ... works nicely. It works by scanning for the packages' SPEC files in the current working directory, so you need to clone all the repos locally in the first place. [1] https://github.com/juhp/fbrnch/blob/d37825c1012d35ae3277d0d0185bd55213fdecd2/src/Cmd/Local.hs#L10 [2] https://hackage.haskell.org/package/rpmbuild-order -- Iñaki Úcar ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Monday, 27 July 2020 08.08.58 WEST Elliott Sales de Andrade wrote: > I'm not exactly fluent in Haskell, but Jens Petersen has a tool called > fbrnch [1], which is able to calculate the build order, and then > submit them, and even handles override for non-Rawhide builds. It does > need you to handle some of the bootstrap conditions yourself, but with > a defined workflow, it might be possible to request that it handle > that too. Jens gave a presentation at NEST and it seems quite interesting in what it does. FWIW the package is available in copr to install: https://copr.fedorainfracloud.org/coprs/petersen/fbrnch/ -- Jos� Ab�lio [[alternative HTML version deleted]] ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora