> On Jun 16, 2019, at 11:14, Darin Adler <[email protected]> wrote: > > If we want to augment it, we should think of what we are aiming at. I do find > it useful to see which tests are failing, and when I click on the red bubble > I don’t see that information. I have to click once to see the “log of > activities” then click on “results”, then see a confusing giant file with > lots of other information. At the bottom of that file the one thing I want to > know.
We might want to also start turning those failure into action items. We could have an automatic mechanism that gathers the failures, records them in a database, and then — with sufficient data — makes determinations about the flakiness or other status of the test. It could then mark the test as flaky or raise it as an issue to some responsible (and responsive) party. We could also have a relatively manual process. The failures are surfaced in Bugzilla or in a Bugzilla-accessible page. The engineer posting the patch could then review the failures and mark them as “Flag as flaky”, “Flag as failing and should be fixed by someone else”, “Flag as failing and should be ignored”, etc. These responses could then be turned into action items for some responsible (and responsive) party to address. As Michael says, there’s a big issue with ignoring test results. Putting a frictionless process in place to address test results would help make them more effective. When I make a change to an Xcode project and Windows builds throw up errors, that’s not something caused by my immediate patch, but I would like to see the flaky test fixed. — Keith _______________________________________________ webkit-dev mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-dev
