Re: Test gating enabled in Bodhi
On 03/01/2018 11:01 AM, mcatanz...@gnome.org wrote: > I know Pierre-Yves warned us about the bug that the tests will be > reported as a failure if the tests are still being run, but there was > absolutely no indication that there were more tests that were still > being run. So this could definitely stand to be improved. I think you may have hit this issue: https://github.com/fedora-infra/bodhi/issues/2124 signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Wed, Feb 28, 2018 at 7:49 PM, Randy Barlow wrote: It does not appear to be blocked currently. There is a bug. Yesterday it was red and said tests failed; the result changed itself. I see there are many more test results now than there were yesterday. It looks like the tests had not finished running when I checked yesterday. I know Pierre-Yves warned us about the bug that the tests will be reported as a failure if the tests are still being run, but there was absolutely no indication that there were more tests that were still being run. So this could definitely stand to be improved. Michael ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On 02/28/2018 07:05 PM, mcatanz...@gnome.org wrote: > I have an update here: > > https://bodhi.fedoraproject.org/updates/epiphany-3.26.6-1.fc27 > > that is blocked due to some harmless translation-related warnings from > desktop-file-validate. It does not appear to be blocked currently. signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
I have an update here: https://bodhi.fedoraproject.org/updates/epiphany-3.26.6-1.fc27 that is blocked due to some harmless translation-related warnings from desktop-file-validate. I'm not the translation police and do not plan to make any changes to the desktop file. * Who can help unblock this update? waiverdb looks too hard for me. * Who can fix this so that it does not cause update failures in the future? Thanks, Michael { "module" : "DesktopLint", "order" : 93, "results" : [ { "arch" : "armv7hl,i686,x86_64", "code" : "DesktopFileValidation", "context" : { "path" : "/usr/share/applications/org.gnome.Epiphany.desktop" }, "diag" : "warning: value \"???-???\" for key \"Comment[ru]\" in group \"Desktop Entry\" looks redundant with value \"???-???\" of key \"Name[ru]\"", "subpackage" : "epiphany" }, { "arch" : "armv7hl,i686,x86_64", "code" : "DesktopFileValidation", "context" : { "path" : "/usr/share/applications/org.gnome.Epiphany.desktop" }, "diag" : "warning: value \" ???\" for key \"Comment[tg]\" in group \"Desktop Entry\" looks redundant with value \" ???\" of key \"GenericName[tg]\"", "subpackage" : "epiphany" }, { "arch" : "armv7hl,i686,x86_64", "code" : "DesktopFileValidation", "context" : { "path" : "/usr/share/applications/org.gnome.Epiphany.desktop" }, "diag" : "warning: value \"Veb brauzer\" for key \"Comment[uz]\" in group \"Desktop Entry\" looks redundant with value \"Veb brauzer\" of key \"GenericName[uz]\"", "subpackage" : "epiphany" }, { "arch" : "armv7hl,i686,x86_64", "code" : "DesktopFileValidation", "context" : { "path" : "/usr/share/applications/org.gnome.Epiphany.desktop" }, "diag" : "warning: value \"??? ???\" for key \"Comment[uz@cyrillic]\" in group \"Desktop Entry\" looks redundant with value \"??? ???\" of key \"Name[uz@cyrillic]\"", "subpackage" : "epiphany" } ], "run_time" : 0, "status" : "completed" } ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Feb 20, 2018 at 11:10 PM, Rex Dieter wrote: > Alexander Ploumistos wrote: >> I am asking because the rpm documentation leaves quite a lot to be >> desired. If I went and changed all my "Requires: foo" to "Requires: >> foo%{_isa}" in all my non-noarch packages, would I be plain wrong, or >> is it justifiable - albeit an overkill? > > the only place I recall seeing recommendation to use %{_isa} is in subpkg > dependencies. > > IMO, It's wrong to use in general, unless you have good reason to do so. Do > you? Well, if I did, I'd know why it was a good reason and we wouldn't be having this conversation :) > Another alternative: don't make -devel depend on the main package (which is > ok for headers-only situations like this) That's a good point to discuss with the upstream developer as well, because I think he intends the libraries to be shipped (and work) that way. Thank you for all that Rex! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Alexander Ploumistos wrote: > Well, with some delay, the waiver worked and I was able to push the > f26 package to batched. > > > On Tue, Feb 20, 2018 at 8:47 PM, Rex Dieter wrote: >> Alexander Ploumistos wrote: >>> OpenBabel is a runtime dependency for some optional features of >>> Molsketch. The %{_isa} macro got added during the review >> >> I think the reviewer in this case was wrong to suggest that, just use >> Requires: openbabel > > I am asking because the rpm documentation leaves quite a lot to be > desired. If I went and changed all my "Requires: foo" to "Requires: > foo%{_isa}" in all my non-noarch packages, would I be plain wrong, or > is it justifiable - albeit an overkill? the only place I recall seeing recommendation to use %{_isa} is in subpkg dependencies. IMO, It's wrong to use in general, unless you have good reason to do so. Do you? > We had some long discussions with the reviewer and the upstream > developer as to what could/should be in the -devel subpackage and I > ended up with what's there. I was wondering why the subpackage was not > to be noarch, but then I found this in our guidelines: > > Do not use noarch > > It may be tempting to make the header library package noarch, since > the header files themselves are simply text. However, a library should > have tests which should be run on all architectures. Also, the install > process may modify the installed headers depending on the build > architecture. For these reasons, header-only packages must not be > marked noarch. > > > Upstream is working on a testsuite, so at some point down the road I > will (probably) need it as it is. That's fair. Another alternative: don't make -devel depend on the main package (which is ok for headers-only situations like this) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Well, with some delay, the waiver worked and I was able to push the f26 package to batched. On Tue, Feb 20, 2018 at 8:47 PM, Rex Dieter wrote: > Alexander Ploumistos wrote: >> OpenBabel is a runtime dependency for some optional features of >> Molsketch. The %{_isa} macro got added during the review > > I think the reviewer in this case was wrong to suggest that, just use > Requires: openbabel I am asking because the rpm documentation leaves quite a lot to be desired. If I went and changed all my "Requires: foo" to "Requires: foo%{_isa}" in all my non-noarch packages, would I be plain wrong, or is it justifiable - albeit an overkill? > > FYI, the package is multilib'd because it has a -devel subpkg, which depends > on the main one (-devel pkgs are automatically multilib). the -devel subpkg > is headers only, you could consider either making it noarch, or drop it > altogether. We had some long discussions with the reviewer and the upstream developer as to what could/should be in the -devel subpackage and I ended up with what's there. I was wondering why the subpackage was not to be noarch, but then I found this in our guidelines: Do not use noarch It may be tempting to make the header library package noarch, since the header files themselves are simply text. However, a library should have tests which should be run on all architectures. Also, the install process may modify the installed headers depending on the build architecture. For these reasons, header-only packages must not be marked noarch. Upstream is working on a testsuite, so at some point down the road I will (probably) need it as it is. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Alexander Ploumistos wrote: > OpenBabel is a runtime dependency for some optional features of > Molsketch. The %{_isa} macro got added during the review I think the reviewer in this case was wrong to suggest that, just use Requires: openbabel FYI, the package is multilib'd because it has a -devel subpkg, which depends on the main one (-devel pkgs are automatically multilib). the -devel subpkg is headers only, you could consider either making it noarch, or drop it altogether. -- Rex ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, 2018-02-20 at 19:15 +0100, Alexander Ploumistos wrote: > On Tue, Feb 20, 2018 at 5:13 PM, Rex Dieter wrote: > > Alexander Ploumistos wrote: > > > First question, why "arch" and "scenario" are x86_64, but the error > > > concerns the 32-bit build? Since OpenBabel exists on all arches, why > > > do I get this particular error message? > > > > Your package is multilib'd? > > No, I don't think it is. > > > > > I have removed and reinstalled molsketch on both i686 and x86_64 with > > > no errors from dnf > > > > and openbable is not multilib'd, that's the problem. > > > > Why do you have an explicit dep? Why is it arch'd? > > > > If you sure it's still needed, a simple: > > Requires: openbabel > > > > should suffice. > > OpenBabel is a runtime dependency for some optional features of > Molsketch. The %{_isa} macro got added during the review, I had asked > about it, but completely forgot it while we were trying to figure out > some other stuff. Is this technically wrong or is dist.rpmdeplint > being overzealous here? We did have a similar case of cross-arch oddness in rpmdeplint which I think turned out to be something extremely non-obvious: https://pagure.io/taskotron/task-rpmdeplint/issue/9 CCing kparal. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Feb 20, 2018 at 5:13 PM, Rex Dieter wrote: > Alexander Ploumistos wrote: >> First question, why "arch" and "scenario" are x86_64, but the error >> concerns the 32-bit build? Since OpenBabel exists on all arches, why >> do I get this particular error message? > > Your package is multilib'd? No, I don't think it is. >> I have removed and reinstalled molsketch on both i686 and x86_64 with >> no errors from dnf > > and openbable is not multilib'd, that's the problem. > > Why do you have an explicit dep? Why is it arch'd? > > If you sure it's still needed, a simple: > Requires: openbabel > > should suffice. OpenBabel is a runtime dependency for some optional features of Molsketch. The %{_isa} macro got added during the review, I had asked about it, but completely forgot it while we were trying to figure out some other stuff. Is this technically wrong or is dist.rpmdeplint being overzealous here? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Alexander Ploumistos wrote: > Hi, > > I've been having a problem with dist.rpmdeplint checks for one of my > packages (molsketch). > This is a new package and it has a run-time dependency on OpenBabel: > > Requires: openbabel%{?_isa} > > The failure is the same on both f26 & f27: > > results: > - arch: x86_64 > item: molsketch-0.5.1-7.fc26 > outcome: FAILED > scenario: x86_64 > type: koji_build > > > nothing provides openbabel(x86-32) needed by molsketch-0.5.1-7.fc26.i686 > nothing provides openbabel(x86-32) needed by molsketch-0.5.1-7.fc26.i686 > > First question, why "arch" and "scenario" are x86_64, but the error > concerns the 32-bit build? Since OpenBabel exists on all arches, why > do I get this particular error message? Your package is multilib'd? > I have removed and reinstalled molsketch on both i686 and x86_64 with > no errors from dnf and openbable is not multilib'd, that's the problem. Why do you have an explicit dep? Why is it arch'd? If you sure it's still needed, a simple: Requires: openbabel should suffice. -- Rex ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Hi, I've been having a problem with dist.rpmdeplint checks for one of my packages (molsketch). This is a new package and it has a run-time dependency on OpenBabel: Requires: openbabel%{?_isa} The failure is the same on both f26 & f27: results: - arch: x86_64 item: molsketch-0.5.1-7.fc26 outcome: FAILED scenario: x86_64 type: koji_build nothing provides openbabel(x86-32) needed by molsketch-0.5.1-7.fc26.i686 nothing provides openbabel(x86-32) needed by molsketch-0.5.1-7.fc26.i686 First question, why "arch" and "scenario" are x86_64, but the error concerns the 32-bit build? Since OpenBabel exists on all arches, why do I get this particular error message? I have removed and reinstalled molsketch on both i686 and x86_64 with no errors from dnf, with the correct version of openbabel getting pulled in every time, so I decided to submit waivers for the failing results. That worked for the f27 package, but not for the f26 one. I do get a "Created waiver 24 for result 19678515" from waiverdb-cli, but even though the packages has spent a week in testing, I do not have the option to push it to batched or stable and I do not think this is a matter of time, since the f26 package got pushed to testing earlier than that for f27. I'd really appreciate any insights. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 10:28 AM, Pierre-Yves Chibon wrote: > Good Morning Fedorans! > > On Thursday, a new version of Bodhi was deployed that enabled Bodhi to > gate updates based on test results. You may notice a "Test Gating > Status" message in the right have side of the page. > > One thing to know about this is that there is currently a confusing > issue where Bodhi will say that the tests have failed when the tests > haven't finished running[0]. We are working on solving that issue, but > for now you can just wait a while and it should report the result once > all the tests have finished. > > There are three tests that must pass in order for updates to go to stable: > > 0. dist.depcheck - to make sure the update's dependencies are available. > Since I haven't seen anyone clarifying this, I'll clarify - the task is named dist.rpmdeplint, not dist.depcheck. I fixed the wiki as well. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Rex Dieter wrote: > Pierre-Yves Chibon wrote: >> 1. dist.abicheck - to make sure the update's ABI remains stable in a >>given Fedora release. > > I think it unwise to make item 1 a mandatory item at this point. I'd > argue a large number of packages do not provide public api/abi that's > worth worrying about in this regard. +1 See: https://pagure.io/fesco/issue/1810#comment-488673 https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/UVMM7O4OEGCNMZ47HN7QYPQDIV2IJZFR/ I wrote there: > Uh, `dist.abicheck` produces a lot of false positives on: > > * libraries that are internal and that nothing should depend on (e.g., in > QupZilla, package `qupzilla`), > * APIs explicitly documented as "private, can change at any version", as > common in all Qt modules (e.g., in QtWebEngine, package > `qt5-qtwebengine`). > > My packages often fail `dist.abicheck`. It is absolutely not realistic to > expect it to pass for all updates. I don't think gating on this test will EVER be a reasonable thing to do. It has just too many false positives. There is no automated way to figure out which ABIs are intended to be public and which aren't. And even an API "public" at package level can be private at distro level. (Consider a library with exactly one package using it. It is always reasonable to bump those together in a grouped update. We have several such examples in Fedora.) Adam Williamson wrote: > Additionally, as Ralph wrote, he has removed abicheck from the list of > gating tests for now, so you shouldn't need to worry about abicheck > failures blocking updates, as soon as Bodhi starts applying that > change. FYI, it looks like this is already working. (The other thread said it'd take only up to 6 hours for Bodhi to pick it up, and indeed, qt5-qtwebengine is now showing as passing the gating even though it "fails" the bogus abicheck test.) But why was this check enabled to begin with, when the issues were already known (see the above links)? I get the feeling that I am talking to a wall! Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 10:33:01PM +0100, Dominik 'Rathann' Mierzejewski wrote: > On Tuesday, 23 January 2018 at 20:16, Matthew Miller wrote: > > On Tue, Jan 23, 2018 at 12:25:56PM -0600, Rex Dieter wrote: > > > > There are three tests that must pass in order for updates to go to > > > > stable: > > > > 0. dist.depcheck - to make sure the update's dependencies are available. > > > > 1. dist.abicheck - to make sure the update's ABI remains stable in a > > > >given Fedora release. > > > ... > > > > Finally, if it turns out you need to push an update through despite of > > > > the > > > > test results, you can do so using waiver-cli (dnf install waiverdb-cli). > > > > We are working on integrating this into Bodhi itself, making this > > > > easier. > > > I think it unwise to make item 1 a mandatory item at this point. I'd > > > argue > > > a large number of packages do not provide public api/abi that's worth > > > worrying about in this regard. > > > > I think we should definine a set of packages where we care about this, > > and enable it on a case-by-case basis and make it advisory otherwise. > > That makes sense. How about we start with critical path packages? > Alternatively, libraries which a lot of of other packages depend on > would be good candidates. A thought - we need a defined process for changing and proposing changes to the greenwave policy. Right now, it reflects a proof of concept list that "we just made up" to get this up and running. At the moment, anyone in the 'sysadmin-qa' group is able to make changes to it. Should FESCo be involved? QA? FPC? I want to fix tooling issues, but don't want to be the policy arbiter. Let's get ourselves unblocked for now and then start figuring out some process around this stuff. signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Wed, Jan 24, 2018 at 03:25:48AM -0500, Randy Barlow wrote: > On 01/23/2018 05:12 PM, Adam Williamson wrote: > > The initial thought we - we being Tim, Kamil, Josef, Ralph and I - had > > is to simply invert the policy, if we can, so it becomes "the update > > passes *unless* we can find a 'fail' for any of the required tests". So > > all updates would be push-able (so far as this mechanism is concerned, > > ignoring all the old ones) until one of the gating tests definitely > > failed. > > This sounds like a reasonable workaround to me. > > > If we can't do that, we're going to just have to disable the gating > > again until this is sorted out; we're definitely of the opinion that > > Taskotron doesn't yet provide enough of a solid guarantee that all the > > tests will be run for a policy which *assumes* that will be the case to > > be viable. > > I agree, the gating is a bit too unreliable as is to stay in its current > state. Well, we can remove dist.rpmdeplint but we have not had complain about the AtomicCI results gating so far, so let's not turn off everything entirely. Pierre signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On 01/23/2018 05:12 PM, Adam Williamson wrote: > The initial thought we - we being Tim, Kamil, Josef, Ralph and I - had > is to simply invert the policy, if we can, so it becomes "the update > passes *unless* we can find a 'fail' for any of the required tests". So > all updates would be push-able (so far as this mechanism is concerned, > ignoring all the old ones) until one of the gating tests definitely > failed. This sounds like a reasonable workaround to me. > If we can't do that, we're going to just have to disable the gating > again until this is sorted out; we're definitely of the opinion that > Taskotron doesn't yet provide enough of a solid guarantee that all the > tests will be run for a policy which *assumes* that will be the case to > be viable. I agree, the gating is a bit too unreliable as is to stay in its current state. signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, 2018-01-23 at 21:25 +, Philip Kovacs wrote: >> Can someone please elaborate on how I can control the abi tests >> directly?Where exactly can I access these and refine them on a per- >> package basis? >That text isn't talking about "fixing the tests", but about fixing the >*bugs*. It assumes that the test indicates a real issue that needs >fixing, therefore you can fix the issue in your package and cause the >test to pass. OK, thanks. The abi check canvases everything in the package and is sensitive to changes that are internal to the package. Thoser internal changes are not bugs for us to manage.Take, for example, a c/c++ package with many .so plugins or other internal libraries that provide services to the application, but are not user-facing -- the abi check sees these internal changes as noteworthy, but they really are not and should be filtered. I guess I'll have to use the waiver for now. On Tuesday, January 23, 2018 5:44 PM, Adam Williamson wrote: On Tue, 2018-01-23 at 21:25 +, Philip Kovacs wrote: > Can someone please elaborate on how I can control the abi tests > directly?Where exactly can I access these and refine them on a per- > package basis? > How to fix the tests? That text isn't talking about "fixing the tests", but about fixing the *bugs*. It assumes that the test indicates a real issue that needs fixing, therefore you can fix the issue in your package and cause the test to pass. It doesn't really cover the scenario where what the test is reporting isn't really a problem and doesn't need to be fixed. You can't resolve that from your package git repository, but you *can* submit a waiver, as discussed upthread. Additionally, as Ralph wrote, he has removed abicheck from the list of gating tests for now, so you shouldn't need to worry about abicheck failures blocking updates, as soon as Bodhi starts applying that change. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, 23 Jan 2018 11:06:59 +0100 Pierre-Yves Chibon wrote: > On Tue, Jan 23, 2018 at 09:42:50AM +, Richard W.M. Jones wrote: > > On Tue, Jan 23, 2018 at 10:28:14AM +0100, Pierre-Yves Chibon > > wrote: > > > Good Morning Fedorans! > > > > > > On Thursday, a new version of Bodhi was deployed that enabled > > > Bodhi to gate updates based on test results. You may notice a > > > "Test Gating Status" message in the right have side of the page. > > > > > > One thing to know about this is that there is currently a > > > confusing issue where Bodhi will say that the tests have failed > > > when the tests haven't finished running[0]. We are working on > > > solving that issue, but for now you can just wait a while and it > > > should report the result once all the tests have finished. > > > > > > There are three tests that must pass in order for updates to go > > > to stable: > > > > > > 0. dist.depcheck - to make sure the update's dependencies are > > > available. 1. dist.abicheck - to make sure the update's ABI > > > remains stable in a given Fedora release. > > > 2. AtomicCI pipeline - packages that are part of the Atomic Host > > > *and* include in their dist-git tests, must have these tests > > > passed in the AtomicCI pipeline > > > > This update cannot be pushed: > > > > https://bodhi.fedoraproject.org/updates/FEDORA-2018-932548462e > > > > When I click on the "automated tests" it says: > > > > "Automated Test Results > > Failed to talk to Greenwave. > > The update can not be pushed: 1 of 2 required tests not found" > > > > and then the only "red" test (ie. failure) is rpmlint which is just > > a bunch of rpmlint being wrong. Nothing about "dist.depcheck" or > > "dist.abicheck" is mentioned anywhere. > > abicheck is the first test results shown in the automated tests tab > but indeed, no depcheck mentioned. Maybe someone from taskotron could > help us on this? In this particular case, I think that the update in question went from updates-testing-pending to updates-testing while taskotron was down for an updgrade on the 9th. There are no rpmdeplint results in resultsdb which is what I assume greenwave is complaining about. To be honest, we've always assumed that this situation was unlikely, verging on impossible due to the way that rpmdeplint is scheduled and run. Obviously, it's more likely than we thought and is an issue that we'll need to address going forward before gating can be enabled for everything. Actually, all of these issues are a bit surprising and there are several things which have gone from "eh, nobody cares. we'll get around to it someday" to "we should probably get that figured out now-ish" due to the issues raised around the gating in bodhi. Tim pgp4aAexCmlR9.pgp Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, 2018-01-23 at 18:12 +, Richard W.M. Jones wrote: > The problem for the OCaml packages is missing tests or tests that > haven't been run: > > https://bodhi.fedoraproject.org/updates/FEDORA-2018-932548462e > https://bodhi.fedoraproject.org/updates/FEDORA-2018-ecd3541af9 > > BTW these both really need to be pushed as soon as possible because > they fix a license violation. Yes, we're looking into that too - see https://github.com/fedora-infra/bodhi/issues/2133 for some discussion. Basically sometimes Taskotron tests for some package or other just don't run for some transient reason - some bit of the process is down or not working correctly, and because it's all fedmsg-driven, there's really only one shot and if the tests don't trigger correctly at the time they should, nothing goes back and re-triggers them later. Improving this has never been a high priority because it hasn't really mattered a lot to anyone...until now. The policy as currently implemented basically means "the update *only* passes if we can find a 'pass' for each of the required tests". So if one isn't run, the update fails. The initial thought we - we being Tim, Kamil, Josef, Ralph and I - had is to simply invert the policy, if we can, so it becomes "the update passes *unless* we can find a 'fail' for any of the required tests". So all updates would be push-able (so far as this mechanism is concerned, ignoring all the old ones) until one of the gating tests definitely failed. If we can't do that, we're going to just have to disable the gating again until this is sorted out; we're definitely of the opinion that Taskotron doesn't yet provide enough of a solid guarantee that all the tests will be run for a policy which *assumes* that will be the case to be viable. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, 2018-01-23 at 21:25 +, Philip Kovacs wrote: > Can someone please elaborate on how I can control the abi tests > directly?Where exactly can I access these and refine them on a per- > package basis? > How to fix the tests? That text isn't talking about "fixing the tests", but about fixing the *bugs*. It assumes that the test indicates a real issue that needs fixing, therefore you can fix the issue in your package and cause the test to pass. It doesn't really cover the scenario where what the test is reporting isn't really a problem and doesn't need to be fixed. You can't resolve that from your package git repository, but you *can* submit a waiver, as discussed upthread. Additionally, as Ralph wrote, he has removed abicheck from the list of gating tests for now, so you shouldn't need to worry about abicheck failures blocking updates, as soon as Bodhi starts applying that change. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, 2018-01-23 at 14:16 -0500, Matthew Miller wrote: > On Tue, Jan 23, 2018 at 12:25:56PM -0600, Rex Dieter wrote: > > > There are three tests that must pass in order for updates to go to stable: > > > 0. dist.depcheck - to make sure the update's dependencies are available. > > > 1. dist.abicheck - to make sure the update's ABI remains stable in a > > >given Fedora release. > > > > ... > > > Finally, if it turns out you need to push an update through despite of the > > > test results, you can do so using waiver-cli (dnf install waiverdb-cli). > > > We are working on integrating this into Bodhi itself, making this easier. > > > > I think it unwise to make item 1 a mandatory item at this point. I'd argue > > a large number of packages do not provide public api/abi that's worth > > worrying about in this regard. > > I think we should definine a set of packages where we care about this, > and enable it on a case-by-case basis and make it advisory otherwise. I think we could probably do something a *bit* less icky than a completely manual package list. What I thought of as a very simple initial heuristic would be to only care about changes in "any shared object installed to /usr/lib or /usr/lib64", optionally we could restrict that further to "in a critpath package". That's installed *directly to those directories*, not to any subdirectories of them. This would certainly be less than the set of things we *should* care about, because there are various mechanisms by which a shared object that's not directly in %{libdir} can have a public ABI of some kind (ldconfig snippets, etc). But it would at least be something to start with...unless I'm missing something. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tuesday, 23 January 2018 at 20:16, Matthew Miller wrote: > On Tue, Jan 23, 2018 at 12:25:56PM -0600, Rex Dieter wrote: > > > There are three tests that must pass in order for updates to go to stable: > > > 0. dist.depcheck - to make sure the update's dependencies are available. > > > 1. dist.abicheck - to make sure the update's ABI remains stable in a > > >given Fedora release. > > ... > > > Finally, if it turns out you need to push an update through despite of the > > > test results, you can do so using waiver-cli (dnf install waiverdb-cli). > > > We are working on integrating this into Bodhi itself, making this easier. > > I think it unwise to make item 1 a mandatory item at this point. I'd argue > > a large number of packages do not provide public api/abi that's worth > > worrying about in this regard. > > I think we should definine a set of packages where we care about this, > and enable it on a case-by-case basis and make it advisory otherwise. That makes sense. How about we start with critical path packages? Alternatively, libraries which a lot of of other packages depend on would be good candidates. Regards, Dominik -- Fedora https://getfedora.org | RPMFusion http://rpmfusion.org There should be a science of discontent. People need hard times and oppression to develop psychic muscles. -- from "Collected Sayings of Muad'Dib" by the Princess Irulan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Can someone please elaborate on how I can control the abi tests directly?Where exactly can I access these and refine them on a per-package basis? How to fix the tests? The tests are all in your hand, you can fix the dist.depcheck and dist.abicheck by adjusting the update or the build and you can fix the package level testing since the tests are stored in the dist-git repository of the package itself. You have the control on the tests. On Tuesday, January 23, 2018 2:16 PM, Matthew Miller wrote: On Tue, Jan 23, 2018 at 12:25:56PM -0600, Rex Dieter wrote: > > There are three tests that must pass in order for updates to go to stable: > > 0. dist.depcheck - to make sure the update's dependencies are available. > > 1. dist.abicheck - to make sure the update's ABI remains stable in a > > given Fedora release. > ... > > Finally, if it turns out you need to push an update through despite of the > > test results, you can do so using waiver-cli (dnf install waiverdb-cli). > > We are working on integrating this into Bodhi itself, making this easier. > I think it unwise to make item 1 a mandatory item at this point. I'd argue > a large number of packages do not provide public api/abi that's worth > worrying about in this regard. I think we should definine a set of packages where we care about this, and enable it on a case-by-case basis and make it advisory otherwise. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 12:25:56PM -0600, Rex Dieter wrote: > > There are three tests that must pass in order for updates to go to stable: > > 0. dist.depcheck - to make sure the update's dependencies are available. > > 1. dist.abicheck - to make sure the update's ABI remains stable in a > >given Fedora release. > ... > > Finally, if it turns out you need to push an update through despite of the > > test results, you can do so using waiver-cli (dnf install waiverdb-cli). > > We are working on integrating this into Bodhi itself, making this easier. > I think it unwise to make item 1 a mandatory item at this point. I'd argue > a large number of packages do not provide public api/abi that's worth > worrying about in this regard. I think we should definine a set of packages where we care about this, and enable it on a case-by-case basis and make it advisory otherwise. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On 23 January 2018 at 19:22, Ralph Bean wrote: > On Tue, Jan 23, 2018 at 06:35:45PM +0100, Rafael dos Santos wrote: > > I get this error after clicking the authorization link: > > > > [16450:16491:0123/182437.407698:ERROR:browser_gpu_ > channel_host_factory.cc(120)] > > Failed to launch GPU process. > > Created new window in existing browser session. > > 127.0.0.1 - - [23/Jan/2018 18:24:37] "GET > > /?error_description=Unknown+client+ID&error=unauthorized_client > HTTP/1.1" > > 200 47 > > Traceback (most recent call last): > > File "/usr/bin/waiverdb-cli", line 11, in > > load_entry_point('waiverdb==0.4.0', 'console_scripts', > 'waiverdb-cli')() > > File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in > > __call__ > > return self.main(*args, **kwargs) > > File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in > main > > rv = self.invoke(ctx) > > File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in > invoke > > return ctx.invoke(self.callback, **ctx.params) > > File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in > invoke > > return callback(*args, **kwargs) > > File "/usr/lib/python2.7/site-packages/waiverdb/cli.py", line 102, in > cli > > if not resp.ok: > > AttributeError: 'NoneType' object has no attribute 'ok' > > ACK. Just resolved this one now. Can you try again? > > (You may need to visit https://id.fedoraproject.org/logout first.) > Yes, it's working fine now. Thanks. Att. -- Rafael Fonseca ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
Pierre-Yves Chibon wrote: > On Thursday, a new version of Bodhi was deployed that enabled Bodhi to > gate updates based on test results. You may notice a "Test Gating > Status" message in the right have side of the page. ... > There are three tests that must pass in order for updates to go to stable: > > 0. dist.depcheck - to make sure the update's dependencies are available. > 1. dist.abicheck - to make sure the update's ABI remains stable in a >given Fedora release. ... > Finally, if it turns out you need to push an update through despite of the > test results, you can do so using waiver-cli (dnf install waiverdb-cli). > We are working on integrating this into Bodhi itself, making this easier. I think it unwise to make item 1 a mandatory item at this point. I'd argue a large number of packages do not provide public api/abi that's worth worrying about in this regard. -- Rex ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 06:35:45PM +0100, Rafael dos Santos wrote: > I get this error after clicking the authorization link: > > [16450:16491:0123/182437.407698:ERROR:browser_gpu_channel_host_factory.cc(120)] > Failed to launch GPU process. > Created new window in existing browser session. > 127.0.0.1 - - [23/Jan/2018 18:24:37] "GET > /?error_description=Unknown+client+ID&error=unauthorized_client HTTP/1.1" > 200 47 > Traceback (most recent call last): > File "/usr/bin/waiverdb-cli", line 11, in > load_entry_point('waiverdb==0.4.0', 'console_scripts', 'waiverdb-cli')() > File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in > __call__ > return self.main(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main > rv = self.invoke(ctx) > File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke > return ctx.invoke(self.callback, **ctx.params) > File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke > return callback(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/waiverdb/cli.py", line 102, in cli > if not resp.ok: > AttributeError: 'NoneType' object has no attribute 'ok' ACK. Just resolved this one now. Can you try again? (You may need to visit https://id.fedoraproject.org/logout first.) signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 05:18:42PM +0100, Adam Williamson wrote: > On Tue, 2018-01-23 at 08:42 -0700, Jerry James wrote: > > Where are the instructions? Why is informing packagers, the group > > most affected by this change, an afterthought? We should have been > > told about all of this, in detail, prior to the thing being turned on, > > *well* before it was turned on. > > So, my answer to this is second-hand - apologies if any of it is wrong, > and the folks involved will no doubt correct me. But as I understand > it, I think they weren't really expecting the tests that were turned on > to have false positives; the tests that were chosen were intended to be > ones that wouldn't cause this kind of problem, so there'd be more time > to get all the waiverdb integrations in place. I don't think they > actually anticipated that false positives would happen and people would > need to use waiverdb-cli like this, which is why instructions for it > weren't part of the plan. I'm sure no-one intended to cause disruption > and inconvenience, and we're sorry about it. I'll try to ask the > relevant folks if perhaps we should stop gating on the abidiff test for > now as a short-term measure, or something like that. The problem for the OCaml packages is missing tests or tests that haven't been run: https://bodhi.fedoraproject.org/updates/FEDORA-2018-932548462e https://bodhi.fedoraproject.org/updates/FEDORA-2018-ecd3541af9 BTW these both really need to be pushed as soon as possible because they fix a license violation. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On 01/23/2018 01:28 AM, Pierre-Yves Chibon wrote: > There are three tests that must pass in order for updates to go to stable: > > 0. dist.depcheck - to make sure the update's dependencies are available. > 1. dist.abicheck - to make sure the update's ABI remains stable in a >given Fedora release. > 2. AtomicCI pipeline - packages that are part of the Atomic Host *and* include >in their dist-git tests, must have these tests passed in the AtomicCI > pipeline > > This last requirement only concern packages that are in the Atomic Host while > the first two is enforced for all packages. In my rust+cargo updates, I'm getting "1 of 4 required tests not found". f27: https://bodhi.fedoraproject.org/updates/FEDORA-2018-d5eb234853 f26: https://bodhi.fedoraproject.org/updates/FEDORA-2018-ae3b642c47 It looks like it ran lots of tests for cargo, but only one on rust. What can I do to resolve this? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
I get this error after clicking the authorization link: [16450:16491:0123/182437.407698:ERROR:browser_gpu_channel_host_factory.cc(120)] Failed to launch GPU process. Created new window in existing browser session. 127.0.0.1 - - [23/Jan/2018 18:24:37] "GET /?error_description=Unknown+client+ID&error=unauthorized_client HTTP/1.1" 200 47 Traceback (most recent call last): File "/usr/bin/waiverdb-cli", line 11, in load_entry_point('waiverdb==0.4.0', 'console_scripts', 'waiverdb-cli')() File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke return callback(*args, **kwargs) File "/usr/lib/python2.7/site-packages/waiverdb/cli.py", line 102, in cli if not resp.ok: AttributeError: 'NoneType' object has no attribute 'ok' Att. -- Rafael Fonseca ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On 23/01/18 16:18, Adam Williamson wrote: On Tue, 2018-01-23 at 08:42 -0700, Jerry James wrote: Where are the instructions? Why is informing packagers, the group most affected by this change, an afterthought? We should have been told about all of this, in detail, prior to the thing being turned on, *well* before it was turned on. So, my answer to this is second-hand - apologies if any of it is wrong, and the folks involved will no doubt correct me. But as I understand it, I think they weren't really expecting the tests that were turned on to have false positives; the tests that were chosen were intended to be ones that wouldn't cause this kind of problem, so there'd be more time to get all the waiverdb integrations in place. I don't think they actually anticipated that false positives would happen and people would need to use waiverdb-cli like this, which is why instructions for it weren't part of the plan. I'm sure no-one intended to cause disruption and inconvenience, and we're sorry about it. I'll try to ask the relevant folks if perhaps we should stop gating on the abidiff test for now as a short-term measure, or something like that. There was one that came up on IRC yesterday that had failed abidiff entirely because of added functions as far as I could see. I don't know if that counts as a false positive or not because I don't know what the design goals are - added functions should mean it's not going to break any existing programs but new programs built against it might not work against old versions of the library. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, 2018-01-23 at 08:42 -0700, Jerry James wrote: > > There are no instructions on how to use it on this wiki page. Okay, > we'll try using --help: > > $ waiverdb-cli --help > Usage: waiverdb-cli [OPTIONS] > > Creates new waivers against test results. > > Examples: > > waiverdb-cli -r 123 -r 456 -p "fedora-26" -c "It's dead!" > > Options: > -C, --config-file PATH Specify a config file to use > -r, --result-id INTEGER Specify one or more results to be waived > -p, --product-version TEXT Specify one of PDC's product version > identifiers. > --waived / --no-waived Whether or not the result is waived > -c, --comment TEXT A comment explaining why the result is waived > -h, --help Show this message and exit. > > > Ookaay, so what is a result-id and how do I figure out which > one I need to supply for my particular case? It's how resultsdb identifies a single specific result. Each result in resultsdb has one, and they're unique. However, I see an obvious problem here: I can't see any easy way to figure out the result ID of the failure from the Bodhi web UI. The URL you get taken to when you click on the test result in Bodhi doesn't include it, so far as I can see. You *can* actually find the failed test quite 'easily' from the resultsdb web UI, but you have to know how to do it: go to the search page - https://taskotron.fedoraproject.org/resultsdb/results - and enter the package NVR (gap-pkg-io-4.5.1-1.fc27) as the 'item', and dist.abicheck as the 'testcase', then search, and you'll find it. Then click 'details' and you can see the ID is 18913262. But that's obviously not an ideal workflow! As Pierre said, they're working on hooking this all up from the Bodhi web UI so waiving will be a much easier process. > I can guess what a > product-version is, but why is it in a different format from every > other Fedora tool that takes such a thing? Because that's the form PDC uses; waiverdb works with PDC, and PDC chose its format a while ago, so that's kind of plumbed in now. > How do I point this tool > at the update of mine that is currently being blocked? At least AIUI, you don't actually have to: all that matching logic is performed by the various systems involved. Basically, Bodhi asks Greenwave if the update is good to go; Greenwave asks ResultsDB for tests associated with the update, and asks WaiverDB for waivers associated with those tests. So its answer to Bodhi will take the waiver into account. All you have to do is submit the waiver. > Where are the instructions? Why is informing packagers, the group > most affected by this change, an afterthought? We should have been > told about all of this, in detail, prior to the thing being turned on, > *well* before it was turned on. So, my answer to this is second-hand - apologies if any of it is wrong, and the folks involved will no doubt correct me. But as I understand it, I think they weren't really expecting the tests that were turned on to have false positives; the tests that were chosen were intended to be ones that wouldn't cause this kind of problem, so there'd be more time to get all the waiverdb integrations in place. I don't think they actually anticipated that false positives would happen and people would need to use waiverdb-cli like this, which is why instructions for it weren't part of the plan. I'm sure no-one intended to cause disruption and inconvenience, and we're sorry about it. I'll try to ask the relevant folks if perhaps we should stop gating on the abidiff test for now as a short-term measure, or something like that. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 2:28 AM, Pierre-Yves Chibon wrote: > Finally, if it turns out you need to push an update through despite of the > test > results, you can do so using waiver-cli (dnf install waiverdb-cli). We are > working on integrating this into Bodhi itself, making this easier. Good! I need this. So let's apply it to my update. H, there is no man page in the waiverdb-cli package. There are no instructions on how to use it in this email. > We have started a wiki page to store all of this information and that we will > keep up to date as improvements are done or for any frequent questions coming > up: > > https://fedoraproject.org/wiki/CI/gating_updates There are no instructions on how to use it on this wiki page. Okay, we'll try using --help: $ waiverdb-cli --help Usage: waiverdb-cli [OPTIONS] Creates new waivers against test results. Examples: waiverdb-cli -r 123 -r 456 -p "fedora-26" -c "It's dead!" Options: -C, --config-file PATH Specify a config file to use -r, --result-id INTEGER Specify one or more results to be waived -p, --product-version TEXT Specify one of PDC's product version identifiers. --waived / --no-waived Whether or not the result is waived -c, --comment TEXT A comment explaining why the result is waived -h, --help Show this message and exit. Ookaay, so what is a result-id and how do I figure out which one I need to supply for my particular case? I can guess what a product-version is, but why is it in a different format from every other Fedora tool that takes such a thing? How do I point this tool at the update of mine that is currently being blocked? Where are the instructions? Why is informing packagers, the group most affected by this change, an afterthought? We should have been told about all of this, in detail, prior to the thing being turned on, *well* before it was turned on. -- Jerry James http://www.jamezone.org/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 09:47:44AM +, Tom Hughes wrote: > On 23/01/18 09:42, Richard W.M. Jones wrote: > > >When I click on the "automated tests" it says: > > > > "Automated Test Results > > Failed to talk to Greenwave. > > The update can not be pushed: 1 of 2 required tests not found" > > > >and then the only "red" test (ie. failure) is rpmlint which is just a > >bunch of rpmlint being wrong. Nothing about "dist.depcheck" or > >"dist.abicheck" is mentioned anywhere. > > If you're using uMatrix or similar then check it isn't getting in > the way - the "Failed to talk to Greenwave" is a message that I was > getting yesterday until I trained uMatrix to allow it. > > I'm not getting that message but I am getting the rest so it looks > like there is an issue here. I enabled Greenwave, but still I don't see what the error is. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 09:42:50AM +, Richard W.M. Jones wrote: > On Tue, Jan 23, 2018 at 10:28:14AM +0100, Pierre-Yves Chibon wrote: > > Good Morning Fedorans! > > > > On Thursday, a new version of Bodhi was deployed that enabled Bodhi to > > gate updates based on test results. You may notice a "Test Gating > > Status" message in the right have side of the page. > > > > One thing to know about this is that there is currently a confusing > > issue where Bodhi will say that the tests have failed when the tests > > haven't finished running[0]. We are working on solving that issue, but > > for now you can just wait a while and it should report the result once > > all the tests have finished. > > > > There are three tests that must pass in order for updates to go to stable: > > > > 0. dist.depcheck - to make sure the update's dependencies are available. > > 1. dist.abicheck - to make sure the update's ABI remains stable in a > >given Fedora release. > > 2. AtomicCI pipeline - packages that are part of the Atomic Host *and* > > include > >in their dist-git tests, must have these tests passed in the AtomicCI > > pipeline > > This update cannot be pushed: > > https://bodhi.fedoraproject.org/updates/FEDORA-2018-932548462e > > When I click on the "automated tests" it says: > > "Automated Test Results > Failed to talk to Greenwave. > The update can not be pushed: 1 of 2 required tests not found" > > and then the only "red" test (ie. failure) is rpmlint which is just a > bunch of rpmlint being wrong. Nothing about "dist.depcheck" or > "dist.abicheck" is mentioned anywhere. abicheck is the first test results shown in the automated tests tab but indeed, no depcheck mentioned. Maybe someone from taskotron could help us on this? Pierre ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On 23/01/18 09:42, Richard W.M. Jones wrote: When I click on the "automated tests" it says: "Automated Test Results Failed to talk to Greenwave. The update can not be pushed: 1 of 2 required tests not found" and then the only "red" test (ie. failure) is rpmlint which is just a bunch of rpmlint being wrong. Nothing about "dist.depcheck" or "dist.abicheck" is mentioned anywhere. If you're using uMatrix or similar then check it isn't getting in the way - the "Failed to talk to Greenwave" is a message that I was getting yesterday until I trained uMatrix to allow it. I'm not getting that message but I am getting the rest so it looks like there is an issue here. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Test gating enabled in Bodhi
On Tue, Jan 23, 2018 at 10:28:14AM +0100, Pierre-Yves Chibon wrote: > Good Morning Fedorans! > > On Thursday, a new version of Bodhi was deployed that enabled Bodhi to > gate updates based on test results. You may notice a "Test Gating > Status" message in the right have side of the page. > > One thing to know about this is that there is currently a confusing > issue where Bodhi will say that the tests have failed when the tests > haven't finished running[0]. We are working on solving that issue, but > for now you can just wait a while and it should report the result once > all the tests have finished. > > There are three tests that must pass in order for updates to go to stable: > > 0. dist.depcheck - to make sure the update's dependencies are available. > 1. dist.abicheck - to make sure the update's ABI remains stable in a >given Fedora release. > 2. AtomicCI pipeline - packages that are part of the Atomic Host *and* include >in their dist-git tests, must have these tests passed in the AtomicCI > pipeline This update cannot be pushed: https://bodhi.fedoraproject.org/updates/FEDORA-2018-932548462e When I click on the "automated tests" it says: "Automated Test Results Failed to talk to Greenwave. The update can not be pushed: 1 of 2 required tests not found" and then the only "red" test (ie. failure) is rpmlint which is just a bunch of rpmlint being wrong. Nothing about "dist.depcheck" or "dist.abicheck" is mentioned anywhere. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Test gating enabled in Bodhi
Good Morning Fedorans! On Thursday, a new version of Bodhi was deployed that enabled Bodhi to gate updates based on test results. You may notice a "Test Gating Status" message in the right have side of the page. One thing to know about this is that there is currently a confusing issue where Bodhi will say that the tests have failed when the tests haven't finished running[0]. We are working on solving that issue, but for now you can just wait a while and it should report the result once all the tests have finished. There are three tests that must pass in order for updates to go to stable: 0. dist.depcheck - to make sure the update's dependencies are available. 1. dist.abicheck - to make sure the update's ABI remains stable in a given Fedora release. 2. AtomicCI pipeline - packages that are part of the Atomic Host *and* include in their dist-git tests, must have these tests passed in the AtomicCI pipeline This last requirement only concern packages that are in the Atomic Host while the first two is enforced for all packages. We are also working on another CI pipeline that will allow running tests stored in dist-git for all packages without discrimination. You are in control of these three tests. The first two can be fixed by making changes in the spec file, and the latter test is controlled via the tests in your repo. Finally, if it turns out you need to push an update through despite of the test results, you can do so using waiver-cli (dnf install waiverdb-cli). We are working on integrating this into Bodhi itself, making this easier. We have started a wiki page to store all of this information and that we will keep up to date as improvements are done or for any frequent questions coming up: https://fedoraproject.org/wiki/CI/gating_updates Please accept my apology in the delay in writing this message. It should have been sent last week when it was enabled, and it got lost. Thanks for your attention, Pierre [0] https://github.com/fedora-infra/bodhi/issues/2124 signature.asc Description: PGP signature ___ devel-announce mailing list -- devel-annou...@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org