On Fri, May 12, 2017 at 12:04 PM, Alexey Proskuryakov <[email protected]> wrote: > > 12 мая 2017 г., в 11:52, Ben Kelly <[email protected]> написал(а): > > On Fri, May 12, 2017 at 2:26 PM, Rick Byers <[email protected]> wrote: >> >> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov <[email protected]> >> wrote: >>> >>> Since imported WPT tests are very flaky, and are not necessarily written >>> to defend against important regressions, investigating issues with them is >>> relatively lower priority than investigating issues observed with WebKit >>> tests. So I would recommend not mixing tests for WebKit regressions with WPT >>> tests - if your test eventually ends up in LayoutTests/imported, it will >>> become a lot less effective. >> >> >> FWIW this is absolutely NOT how we're treating this in chromium. If this >> is how things end up in practice then we will have failed massively in this >> effort. >> >> We figure if we want the web to behave consistently, we really have no >> choice but to treat web-platform-tests as first class with all the >> discipline we give to our own tests. As such we are actively moving many of >> our LayoutTests to web-platform-tests and depending entirely on the >> regression prevention they provide us from there. Obviously there will be >> hiccups, but because our product quality will depend on web-platform-tests >> being an effective and non-flaky testsuite (and because we're starting to >> require most new features have web-platform-tests before they ship), I'm >> confident that we've got the incentives in place to lead to constant >> ratcheting up the engineering discipline and quality of the test suite. > > > FWIW, mozilla also treats WPT as first class tests. We're not actively > moving old tests to WPT like google, but all new tests (at least in DOM) are > being written in WPT. Of course, we do have exceptions for some tests that > require gecko-specific features (forcing GC, etc). > > > We don't have a concept of "first class", but I hope that when choosing > between looking into a simple test that was added for a known important bug, > and looking into an imported test whose importance is unclear, any WebKit > engineer will pick the former. And since no one can fix all the things, such > prioritization makes imported tests less effective.
This is absolutely not how I operate at all. Since almost all custom elements and shadow DOM API tests I wrote are written using testharness.js and upstreamed to web-platform-tests, they have been deleted from LayoutTests/fast/shadow-dom & LayoutTests/fast/custom-elements. As such, if any new shadow DOM or custom elements tests under LayoutTests/imported/ start to fail, then we must fix them since we idon'thave any other test coverage. - R. Niwa _______________________________________________ webkit-dev mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-dev
