Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Friday, 26 June 2020 10.52.44 WEST Elliott Sales de Andrade wrote: > glue1.4.1 1.3.2 - requires new vctrs; do not bump This process takes time but I think that I am on the right track (to the abyss ?). :-) I am now rebuilding packages that need bootstrap and/or from Bioconductor. An example of how this is done can be seen here: https://src.fedoraproject.org/rpms/R-glue/blob/master/f/R-glue.spec In this case the bootstrap mode is off to make it on it would be enough to change one from %bcond_with bootstrap to %bcond_without bootstrap And yes at first sight it looks like the it should be the other way around. The question is read it as _allow_ to build the file conditionally with/ without bootstrap. Since most of the packages do not need this I propose to only add this snippet where it is really needed and not everywhere as I have hinted initially. Or else the spec files becomes noisy unnecessarily. Regards, -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Monday, 29 June 2020 13.46.02 WEST Iñaki Ucar wrote:
> But the mass rebuild process is very different, because releng doesn't
> follow any particular order (they don't need to, because nothing
> really changed).
>
> The question is whether there is any tool to resolve dependencies and
> generate a list of builds, as I do with the CRAN database, for an
> arbitrary list of RPMs. That would be perfect. In theory, this should
> be possible with dnf, but when I tried, I failed (or my script would
> theoretically work, but it would take ages to complete). I also took a
> look at releng's repos on Pagure, but I didn't find any tool to do
> this either.
>
> We should bring this to devel, maybe ask Miro and other Python folks
> who may have The Method(TM), as they are rebuilding stuff on a daily
> basis.
I think that I meant the way that the python packages are rebuilt for each new
python version. I should have said Python mass rebuild.
I took your work as the basis, together with Tom's help and I used dnf to get
a list of R packages (assuming that their name starts with R- ). The advantage
of this approach is that it works for CRAN and Bioconductor.
$ dnf list available --disablerepo=\* --enablerepo=rawhide-source R-* | tail -
n +3 | awk -e '{print $1};' | sed -e 's/\.src$//' > r-packages.txt
That gave the list of the (source) packages to rebuild.
In order to get the list of dependencies I used a similar strategy:
$ (for p in $(cat r-packages.txt); do echo -n $p, ; dnf -q repoquery --
disablerepo=\* --enablerepo=rawhide-source --requires $p | sed ':a;N;$!ba;s/
\n/,/g'; done) > r-packages-dependencies.csv
The resulting file is a csv file with the first column as the package and the
remaining columns as the dependencies.
Now the idea was to process this file using a python script to create a direct
graph of dependencies (using networkx). In the process non R packages are
removed and some dependencies are cleaned (e.g. -devel packages direct to the
source package).
The advantage of this is that it allows to identify cycles.
My only surprise in this process was to find that several packages require R-
rpm-macros.
That causes some dependency cycles because R-rpm-macros also requires
R-rprintf,R-devel,R-knitr,R-stringi,R-testthat.
Since R-rpm-macros is used for Fedora and epel 8 I suggest to remove the
dependency. Do you agree?
--
José Abílio
___
R-SIG-Fedora mailing list
R-SIG-Fedora@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Fri, 3 Jul 2020 at 19:03, José Abílio Matos wrote:
>
> On Monday, 29 June 2020 13.46.02 WEST Iñaki Ucar wrote:
> > But the mass rebuild process is very different, because releng doesn't
> > follow any particular order (they don't need to, because nothing
> > really changed).
> >
> > The question is whether there is any tool to resolve dependencies and
> > generate a list of builds, as I do with the CRAN database, for an
> > arbitrary list of RPMs. That would be perfect. In theory, this should
> > be possible with dnf, but when I tried, I failed (or my script would
> > theoretically work, but it would take ages to complete). I also took a
> > look at releng's repos on Pagure, but I didn't find any tool to do
> > this either.
> >
> > We should bring this to devel, maybe ask Miro and other Python folks
> > who may have The Method(TM), as they are rebuilding stuff on a daily
> > basis.
>
> I think that I meant the way that the python packages are rebuilt for each new
> python version. I should have said Python mass rebuild.
>
> I took your work as the basis, together with Tom's help and I used dnf to get
> a list of R packages (assuming that their name starts with R- ). The advantage
> of this approach is that it works for CRAN and Bioconductor.
>
> $ dnf list available --disablerepo=\* --enablerepo=rawhide-source R-* | tail -
> n +3 | awk -e '{print $1};' | sed -e 's/\.src$//' > r-packages.txt
>
> That gave the list of the (source) packages to rebuild.
>
> In order to get the list of dependencies I used a similar strategy:
> $ (for p in $(cat r-packages.txt); do echo -n $p, ; dnf -q repoquery --
> disablerepo=\* --enablerepo=rawhide-source --requires $p | sed ':a;N;$!ba;s/
> \n/,/g'; done) > r-packages-dependencies.csv
>
> The resulting file is a csv file with the first column as the package and the
> remaining columns as the dependencies.
>
> Now the idea was to process this file using a python script to create a direct
> graph of dependencies (using networkx). In the process non R packages are
> removed and some dependencies are cleaned (e.g. -devel packages direct to the
> source package).
Nice! What if we create a group "R" on Pagure and a repo
"fedora-scripts" or something like that?
> The advantage of this is that it allows to identify cycles.
>
> My only surprise in this process was to find that several packages require R-
> rpm-macros.
>
> That causes some dependency cycles because R-rpm-macros also requires
> R-rprintf,R-devel,R-knitr,R-stringi,R-testthat.
Are these the packages that require R-rpm-macros or others?
> Since R-rpm-macros is used for Fedora and epel 8 I suggest to remove the
> dependency. Do you agree?
The packages are most probably maintained by Elliott, right? He would
know better why they have this dependency.
--
Iñaki Úcar
___
R-SIG-Fedora mailing list
R-SIG-Fedora@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
On Fri, 3 Jul 2020 at 13:03, José Abílio Matos wrote:
>
> On Monday, 29 June 2020 13.46.02 WEST Iñaki Ucar wrote:
> > But the mass rebuild process is very different, because releng doesn't
> > follow any particular order (they don't need to, because nothing
> > really changed).
> >
> > The question is whether there is any tool to resolve dependencies and
> > generate a list of builds, as I do with the CRAN database, for an
> > arbitrary list of RPMs. That would be perfect. In theory, this should
> > be possible with dnf, but when I tried, I failed (or my script would
> > theoretically work, but it would take ages to complete). I also took a
> > look at releng's repos on Pagure, but I didn't find any tool to do
> > this either.
> >
> > We should bring this to devel, maybe ask Miro and other Python folks
> > who may have The Method(TM), as they are rebuilding stuff on a daily
> > basis.
>
> I think that I meant the way that the python packages are rebuilt for each new
> python version. I should have said Python mass rebuild.
>
> I took your work as the basis, together with Tom's help and I used dnf to get
> a list of R packages (assuming that their name starts with R- ). The advantage
> of this approach is that it works for CRAN and Bioconductor.
>
> $ dnf list available --disablerepo=\* --enablerepo=rawhide-source R-* | tail -
> n +3 | awk -e '{print $1};' | sed -e 's/\.src$//' > r-packages.txt
>
> That gave the list of the (source) packages to rebuild.
>
> In order to get the list of dependencies I used a similar strategy:
> $ (for p in $(cat r-packages.txt); do echo -n $p, ; dnf -q repoquery --
> disablerepo=\* --enablerepo=rawhide-source --requires $p | sed ':a;N;$!ba;s/
> \n/,/g'; done) > r-packages-dependencies.csv
>
> The resulting file is a csv file with the first column as the package and the
> remaining columns as the dependencies.
>
> Now the idea was to process this file using a python script to create a direct
> graph of dependencies (using networkx). In the process non R packages are
> removed and some dependencies are cleaned (e.g. -devel packages direct to the
> source package).
>
>
> The advantage of this is that it allows to identify cycles.
>
> My only surprise in this process was to find that several packages require R-
> rpm-macros.
>
> That causes some dependency cycles because R-rpm-macros also requires
> R-rprintf,R-devel,R-knitr,R-stringi,R-testthat.
>
Your process must have an error, because R-rpm-macros only Requires
R-core. It also BuildRequires nothing.
R-devel depends on R-rpm-macros, and this is intended, in order to
pull in automated dependency generators. If anything else directly
requires R-rpm-macros, that is an error, and these are all in packages
I don't own.
> Since R-rpm-macros is used for Fedora and epel 8 I suggest to remove the
> dependency. Do you agree?
>
>
> --
> José Abílio
>
--
Elliott
___
R-SIG-Fedora mailing list
R-SIG-Fedora@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
Re: [R-sig-Fedora] R 4.0.0 rebuild status
[Apologies if this message is a dupe but I do not find evidence of the other message I wrote] On Friday, 3 July 2020 23.06.43 WEST Elliott Sales de Andrade wrote: > Your process must have an error, because R-rpm-macros only Requires > R-core. It also BuildRequires nothing. You are right. This caught a blind spot in my shell script. If there are no dependencies the script does not change line and thus R-rpm-macros and R-rprintf entries were merged. > R-devel depends on R-rpm-macros, and this is intended, in order to > pull in automated dependency generators. If anything else directly > requires R-rpm-macros, that is an error, and these are all in packages > I don't own. There are several Rccp related packages that require R-rpm-macros. Currently it is no-op so not a problem although it is unnecessary. Regards, -- José Abílio ___ R-SIG-Fedora mailing list R-SIG-Fedora@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
