On Fri, Nov 17, 2017 at 5:23 PM, Maciej Stachowiak <[email protected]> wrote: > > >> On Nov 17, 2017, at 7:53 AM, Frédéric WANG <[email protected]> wrote: >> >> On 17/11/2017 16:26, Maciej Stachowiak wrote: >>> WebKit has a lot of tests that were regression tests for a specific >>> bug fix, not as conformance tests (though they might b useful for that >>> too). In such cases, the association with the bugs.webkit.org bug is >>> more important than the spec URL. That’s particularly the case when >>> the test has been changed multiple times to reflect further WebKit >>> behavior changes. I know that I’ve personally found it very useful to >>> look at revision history of tests, or search for bug numbers or >>> keywords in LayoutTests/ChangeLog to find related tests. >>> Not saying this is the sole purpose of tests, but losing this ability or making it more awkward would be a downside to deleting tests from the WebKit repo. >> Well, I think we agree here, that's why "conformance" tests was >> specified in my message. For such tests, connecting the history of tests >> with the development of the specs is actually more important. For >> example I recently noticed that some bugs with WOFF2 (on Apple's ports >> only) and WebVTT have been ignored in WebKit because we don't >> import/sync WPT tests for these specs. > > I think importing new test suites is a different question than upstreaming/removing tests. In general importing more test suites is probably good, but we probably need to tackle the WPT performance problem before we pull in too many new WPT suites. > > I should also note that pulling in the test suite won't automatically get the bugs fixed or prioritized, or even filed in the bug tracker necessarily.
This is the case in Chromium as well, although Robert Ma is working on import notifications: https://docs.google.com/document/d/1W3V81l94slAC_rPcTKWXgv3YxRxtlSIAxi3yj6NsbBw/edit?usp=sharing We don't know yet what's going to work well, but I think that somehow, before too long, we need to at least triage all new failures. Far from all will be worth prioritizing of course, but sometimes the existence of the test can reduce the effort and make it easier to prioritize.
_______________________________________________ webkit-dev mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-dev
