I think a lot of the problem is not necessarily a technical issue, meaning I am not sure that tooling will solve the problem, but it is more of a social problem.
I think the problem is making sure that items are triaged and placed into peoples workflow sooner would solve this problem but I also appreciate the difficulty when there are competing priorities within teams. This was a problem I started working on last quarter, mostly for web platform tests, and would like to continue it this quarter. I am going to be reaching out to more people over the quarter to see if we can solve this. If you would like to be part of the process please let me know and I will schedule an interview with you. David On Thu, 9 Jan 2020 at 00:28, Andrew Sutherland <asutherl...@asutherland.org> wrote: > On 1/8/20 12:50 PM, Geoffrey Brown wrote: > > Instead of changing the reviewers, how about: > > - we remind the sheriffs to needinfo > > - #intermittent-reviewers check that needinfo is in place when > > reviewing disabling patches. > > To try and help address the visibility issue, we could also make > searchfox consume the test-info-disabled-by-os taskcluster task[1] and: > > - put banners at the top of test files that say "Hey! This is > (sometimes) disabled on [android/linux/mac/windows]" > > - put collapsible sections at the top of the directory listings that > explicitly call out the disabled tests in that directory. The idea would > be to avoid people needing to scroll through the potentially long list > of files to know which are disabled and provide some ambient awareness > of disabled tests. > > If there's a way to get a similarly pre-built[2] mapping that would > provide information about the orange factor of tests[3] or that it's > been marked as needswork, that could also potentially be surfaced. > > Andrew > > 1: > > https://searchfox.org/mozilla-central/rev/be7d1f2d52dd9474ca2df145190a817614c924e4/taskcluster/ci/source-test/file-metadata.yml#62 > > 2: Emphasis on pre-built in the sense that searchfox's processing > pipeline really doesn't want to be issuing a bunch of dynamic REST > queries that would add to its processing latency. It would want a > taskcluster job that runs as part of the nightly build process so it can > fetch a JSON blob at full network speed. > > 3: I guess test-info-all at > > https://searchfox.org/mozilla-central/rev/be7d1f2d52dd9474ca2df145190a817614c924e4/taskcluster/ci/source-test/file-metadata.yml#91 > does provide the "failed runs" count and "total runs" which could > provide the orange factor? The "total run time, seconds" scaled by the > "total runs" would definitely be interesting to surface in searchfox. > > _______________________________________________ > firefox-dev mailing list > firefox-...@mozilla.org > https://mail.mozilla.org/listinfo/firefox-dev > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
