Re: archive test rebuilds and reports for bullseye?
On Fri, Nov 08, 2019 at 06:37:29PM +0100, Lucas Nussbaum wrote: > UDD uses > https://tests.reproducible-builds.org/debian/reproducible-tracker.json, > but that only includes results for suite='bullseye'. Is that expected? Yes, it's expected. reproducible-tracker.json has only the data that is supposed to be "useful" for maintainers without being distracting. For example, it also doesn't show blacklisted packages, or in other weird states. If you want all the data, that would be in reproducible.json with a slight different structure; however in the past many times we received complaints about unnecessary noise, so if you change I at least urge you to limit what's displayed in DMD. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. More about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Re: archive test rebuilds and reports for bullseye?
On 08/11/19 at 16:39 +, Holger Levsen wrote: > On Fri, Nov 08, 2019 at 05:29:33PM +0100, Lucas Nussbaum wrote: > > How often do packages get test-built thanks to that? (It looks like the > > answer is: "once per month"?) > > it depends - see the graphs on the bottom of > https://tests.reproducible-builds.org/debian/index_performance.html > > So, for unstable/amd64 it's currently once a month, but we also were at > twice a month in the past. > > > Do you have an estimate of how many failures without a corresponding bug > > there currently is? > > no. UDD uses https://tests.reproducible-builds.org/debian/reproducible-tracker.json, but that only includes results for suite='bullseye'. Is that expected? Anyway, UDD sees 387 source packages that are failing to build on bullseye, but don't have an open bug with 'FTBFS' in the title: select distinct source from reproducible where status = 'FTBFS' and architecture='amd64' and source not in ( select source from bugs where title ~ 'FTBFS' and status='pending'); (It might be OK if the bug has been closed and the package hasn't migrated yet, hence my question about the results above.) Lucas
Re: archive test rebuilds and reports for bullseye?
On 03/11/19 at 15:24 +, Holger Levsen wrote: > On Sun, Nov 03, 2019 at 03:14:38PM +0100, Matthias Klose wrote: > > As part of general QA we did some test rebuilds during the last release > > cycle, and filing the ftbfs reports as RC issues. Afaics there were no such > > test rebuilds and RC filing for bullseye after the release of buster. > > FWIW (and I mean that very literally), reproducible.debian.net does > continous rebuilds of experimental, sid and bullseye. And some people > even file bugs based on this. Hi Holger, How often do packages get test-built thanks to that? (It looks like the answer is: "once per month"?) Do you have an estimate of how many failures without a corresponding bug there currently is? Actually this question could probably be answered by UDD. Lucas
Re: archive test rebuilds and reports for bullseye?
On Fri, Nov 08, 2019 at 05:29:33PM +0100, Lucas Nussbaum wrote: > How often do packages get test-built thanks to that? (It looks like the > answer is: "once per month"?) it depends - see the graphs on the bottom of https://tests.reproducible-builds.org/debian/index_performance.html So, for unstable/amd64 it's currently once a month, but we also were at twice a month in the past. > Do you have an estimate of how many failures without a corresponding bug > there currently is? no. -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: archive test rebuilds and reports for bullseye?
On 03.11.19 16:24, Holger Levsen wrote: On Sun, Nov 03, 2019 at 03:14:38PM +0100, Matthias Klose wrote: As part of general QA we did some test rebuilds during the last release cycle, and filing the ftbfs reports as RC issues. Afaics there were no such test rebuilds and RC filing for bullseye after the release of buster. FWIW (and I mean that very literally), reproducible.debian.net does continous rebuilds of experimental, sid and bullseye. And some people even file bugs based on this. (Contrary to some people's outdated beliefs) there are also no modified packages involved. (We just test with usrmerge installed in one build and not the other. And with the other (legit) variations described in https://tests.reproducible-builds.org/debian/index_variations.html ) it would be nice to make this information more exposed, especially because you are covering four architectures, and not just one which is usually rebuilt for a test rebuild in Debian. Now looking at https://tracker.debian.org/pkg/linbox https://tracker.debian.org/pkg/python-xarray I have difficulties finding a build log showing the build failure. The link to the tracker leads to a page with (for me) non-obvious information. For the purpose of tracking build failures I'd like to see a direct link to a build log of a failing build on tracker.debian.org, however even looking at https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/linbox.html https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/python-xarray.html I have difficulties finding the log of the failed build. Also please note that the linbox ftbfs is non-amd64, so the tracker link is misleading. It would also be nice to generate a list of for builds that ftbfs and don't have any RC issues yet in the BTS, so that could be used for QA people to file not-yet-filed ftbfs issues. For comparative test rebuilds, would it be possible to add another configuration (you already have experimental, sid, ...) to use a new GCC version? I assume the LLVM maintainers also would be interested to test the LLVM toolchain the same way. Matthias
Re: archive test rebuilds and reports for bullseye?
On Sun, Nov 03, 2019 at 03:14:38PM +0100, Matthias Klose wrote: > As part of general QA we did some test rebuilds during the last release > cycle, and filing the ftbfs reports as RC issues. Afaics there were no such > test rebuilds and RC filing for bullseye after the release of buster. FWIW (and I mean that very literally), reproducible.debian.net does continous rebuilds of experimental, sid and bullseye. And some people even file bugs based on this. (Contrary to some people's outdated beliefs) there are also no modified packages involved. (We just test with usrmerge installed in one build and not the other. And with the other (legit) variations described in https://tests.reproducible-builds.org/debian/index_variations.html ) -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
archive test rebuilds and reports for bullseye?
As part of general QA we did some test rebuilds during the last release cycle, and filing the ftbfs reports as RC issues. Afaics there were no such test rebuilds and RC filing for bullseye after the release of buster. People only have a limited amount of time, however it would be nice if these rebuilds would start again, maybe having that work done by more people than before. I know I am a consumer of those test rebuilds as well, asking for comparative test rebuilds with new compiler versions, and lacking time to commit on regular test builds, otoh the lack of those is hurting. Matthias
