On Mon, Aug 20, 2012 at 6:03 PM, Maciej Stachowiak <m...@apple.com> wrote:
> Here's how I imagine the workflow when a sheriff or just innocent > bystander notices a deterministically failing test. Follow this two-step > algorithm: > > 1) Are you confident that the new result is an improvement or no worse? If > so, then simply update -expected.txt. > 2) Otherwise, copy the old result to > -<whatever-we-call-the-unexpected-pass-result>.txt, and check in the new > result as -<whatever-we-call-the-expected-failure-result.txt>. > I think we should do this. I don't care much about the naming. > This replaces all other approaches to marking expected failures, including > the Skipped list, overwriting -expected even you know the result is a > regression, marking the test in TestExpectations as Skip, Wontfix, Image, > Text, or Text+Image, or any of the other legacy techniques for marking an > expected failure reult. > Don't forget suffixing the test with "-disabled"! We have 109 such tests at the moment according to http://code.google.com/searchframe#search/&exact_package=chromium&q=file:third_party/WebKit/LayoutTests/.*%5C-disabled$&type=cs. I think we should also get rid of this. If we need a way to disable a test across ports (e.g. because it crashes in cross-platform code), we should make a Skipped/TestExpectations file in LayoutTest/platform instead of renaming the test file.
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
