Le ven. 12 mai 2017 à 11:07, Alexey Proskuryakov <a...@webkit.org> a écrit :
> > 9 мая 2017 г., в 11:27, Simon Fraser <simon.fra...@apple.com> написал(а): > > > Another consideration here is "would my test be useful for other browser > vendors". I don't think the answer is a unanimous "yes", so I think we > should only use WPT for tests that will think are worth sharing. > > > Since imported WPT tests are very flaky, and are not necessarily written > to defend against important regressions, investigating issues with them is > relatively lower priority than investigating issues observed with WebKit > tests. So I would recommend not mixing tests for WebKit regressions with > WPT tests - if your test eventually ends up in LayoutTests/imported, it > will become a lot less effective. > WPT tests are flaky in WebKit because WPT stability bots do not run yet Safari, and most tests are written in non-WebKit environment. Often, it is due to the fact that tests are not passing and flakiness is only seen with failing assertions. >From my experience with fetch API tests, I disagree that broken WPT tests are lower priority. I think it will change as more WebKit contributors will author WPT tests. I agree that tests doing subtle WebKit-specific regression checks are not good candidates for WPT upstream. When the test is all about conformance with a spec, it is a very good candidate. > Using the complicated harness has a similar consequence - if you use any > WPT goodies like templates or server side scripts, the cost of > investigating issues on the test increases, and makes the test less > valuable. > It is true that WPT put some emphasis on easing authoring of tests. I guess there is a learning curve here in WPT test debugging. If you have a file with 20 tests, it is harder to debug. It is also increasing the chances for flakiness/timeouts. Maybe we could send that feedback. WPT infra could also be improved: more verbose debug-only output, enabling running selected subtest only... testharness.js is actively maintained and is continuously improving. Since we have specific requirements as you are describing, we should push them. I don't know if there is any way to adopt WPT that won't make testing less > effective. WPT tests may be useful in very rare cases, where we actively > want to communicate certain subtle behavior details to other vendors - but > even so, I imagine that other vendors may not put high priority on those, > for the same reasons. > My own experience is that WPT tests are actually very valuable, at least when we are talking about interoperability/spec conformance. I also see WPT as an efficient way to author tests. > > - Alexey > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev >
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev