On Jul 1, 2011, at 2:54 PM, Dirk Pranke wrote:
> Does that apply to -expected.txt files in the base directories, or just
> platform-specific exceptions?
Base directories.
Expected files contain output reflecting the behavior of WebKit at the time the
test was checked in. The expected result when we re-run a test. Many expected
files contain text that says “FAIL” in them. The fact that these expected
results are not successes, but rather expected failures does not seem to me to
be a subtle point, but one of the basic things about how these tests are set up.
> I wonder how it is that I've been working (admittedly, mostly on tooling) in
> WebKit for more that two years and this is the first I'm hearing about this.
I’m guessing it’s because you have been working on Chrome.
The Chrome project came up with a different system for testing layered on top
of the original layout test machinery based on different concepts. I don’t
think anyone ever discussed that system with me; I was the one who created the
original layout test system, to help Dave Hyatt originally, and then later the
rest of the team started using it.
> Are there reasons we [are] doing things this way[?]
Sure. The idea of the layout test framework is to check if the code is still
behaving as it did when the test was created and last run; we want to detect
any changes in behavior that are not expected. When there are expected changes
in behavior, we change the contents of the expected results files.
It seems possibly helpful to augment the test system with editorial comments
about which tests show bugs that we’d want to fix. But I wouldn’t want to stop
running all regression tests where the output reflects the effects of a bug or
missing feature.
-- Darin
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
