Hello, po 15. 7. 2024 o 10:57 Zbigniew Jędrzejewski-Szmek <zbys...@in.waw.pl> napísal(a):
> On Mon, Jul 15, 2024 at 10:39:37AM +0200, Cristian Le via devel wrote: > > Hi Zbyszek, > > > > On 2024/07/14 20:04, Zbigniew Jędrzejewski-Szmek wrote: > > > I'm looking for a solution which doesn't just skip the installability > > > tests altogether. > > > > On PRs with zuul or FedoraCI automation, the same instability tests that > are > > done for Bodhi are performed. But what would help is to make these tests > as > > required to pass unless they are manually waved. Manually that can be > done > > by setting `gating.yaml`. There was some discussion on making some of > these > > tests as gating by default. > > > > Another issue specific to installability is that the issue is often > further > > down the stream, particularly with the SELinux test. Definitely these > need > > to be tracked down and fixed. > > I fully agree. But a test that just does 'dnf install > rpms-from-update/*.rpm' > will predicatably fail. > > > > A second problem is that when the tests fail, it's just soooo hard to > > > find out why they failed.From the bodhi status page, one has to > > > go to the Jenkins status page, guess that it's useful to look at > > > Console Output, scroll over a few pages of incomrehensible JSON > > > gibberish, guess that it's worth clicking on Testing Farm Artifacts > URL, > > > click that, scroll pages of output to see > > > "guest setup failed: Test environment installation failed: Install > packages". > > > > Weird, when the test is finished, you should have only the final > > testing-farm results page. Here's an example [1]. Maybe in your case it > > encountered an internal failure? > > > > [1]: https://bodhi.fedoraproject.org/updates/FEDORA-2024-57f489c90d > > Maybe I'm doing things wrong. I'd be happy to learn. > > I do the following: > 1. Look at the bodhi update page ( > https://bodhi.fedoraproject.org/updates/FEDORA-2024-3aafcac6a8) > 2. Click on 'Automated Tests' > (There seems to be no URL for the view. This means that it's always > and extra click after every page load or reload.) > 3. I click on one of the pinkish lines, e.g. the first one. > (Another usability problem here is that the click open a new page in > new tab/window. Why, oh why? I want to use left-click to open a link > in the existing tab, and middle-click to open a new tab. The current > UI breaks navigation.) > 4. I switch to the new tab and see > > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/398487/ > > (BTW, I mentioned unrelated scary-looking warnings in my OP. > Here: > The following steps that have been detected may have insecure > interpolation of sensitive variables (click here for an explanation): > > httpRequest: [TESTING_FARM_API_KEY] > ) > 5. Click on 'Console Output' > 6. Click on 'Testing Farm Artifacts URL: > https://artifacts.dev.testing-farm.io/25316385-50d8-42b4-b4c1-3eff059034eb > ' > 7. Click on 'build installation' ( > https://artifacts.dev.testing-farm.io/25316385-50d8-42b4-b4c1-3eff059034eb/guest-setup-09b7edc6-0b7e-431b-ae68-afac2527fbb1/artifact-installation-09b7edc6-0b7e-431b-ae68-afac2527fbb1 > ) > 8. Click on 60-Install-packages.txt > > So 8 steps to get to the actual result… > This is not actually the installability test. This is a functional test where Testing Farm couldn't prepare the machine and thus the actual test didn't even start. But it's true that figuring out what went wrong and where is an impossible task for mere mortals. Thanks, Michal > > Zbyszek > > -- > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue >
-- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
