On Tue, 9 Sept 2025 at 09:39, Alex Bennée <[email protected]> wrote: > > Thomas Huth <[email protected]> writes: > > > On 08/09/2025 23.18, Alex Bennée wrote: > >> This is a script designed to collect data from multiple pipelines and > >> analyse the failure modes they have. > >> Signed-off-by: Alex Bennée <[email protected]> > >> --- > >> scripts/ci/gitlab-failure-analysis | 65 ++++++++++++++++++++++++++++++ > >> 1 file changed, 65 insertions(+) > >> create mode 100755 scripts/ci/gitlab-failure-analysis > > > > You already get a nice overview by visiting a page like > > https://gitlab.com/qemu-project/qemu/-/pipelines/2019002986 ... what's > > the advantage of this script? > > Not having to click every link when I want to see what the pattern of > failures is and what might be a candidate for making flaky.
What I would like for finding flaky tests is to find every case where: * a job failed on commit hash X * we also have the same job succeeding on the same commit X Those are the flaky tests, where we hit retry and it just passed the second time, and it rules out the cases where we had a genuine "job failed because the code being tested for merge had a problem". When we find those jobs that only failed because of a flaky test then we can analyse their logs to identify what the exact failures were. Can we find those with this script ? (You can't do it with the gitlab web UI, whose search and filter capabilities are extremely limited.) thanks -- PMM
