I think we should just switch the default for text vs. pixel tests. Make tests require a <script> layoutTestController.dumpPixels()</script> to dump pixels.
We could also white-list certain directories.[1] -eric 1. If we did add white-listing, I'd like to move away from our hacky-in-run-webkit-tests white-lists to an explict on-disk per-directory config file or similar. Understanding why "fast/loader" turns on the loader callbacks when you're writing a test has driven more than one would-be-webkit-hacker crazy. :) On Wed, Dec 8, 2010 at 12:43 PM, Simon Fraser <[email protected]>wrote: > On Dec 8, 2010, at 12:13 PM, Peter Kasting wrote: > > If you never write layout tests, you can stop reading. > > Right now few ports run pixel tests (a shame). One reason is because we > frequently need different reference images for each port, and creating > hundreds or thousands of these is a hassle. Maintenance burden also > increases. > > We could greatly decrease the number of these baselines by following a > simple rule: don't display text unless you need to. For example, let's say > I have a test that is supposed to display a green square. If the test > output is: > > A green square means this test passes. If there is any red below, fail. > [green square here] > > ...then for each platform that has different font rendering, whether that > be subpixel AA differences, font size differences, or anything else, we'll > need new baselines. By contrast, if that explanation was in an HTML comment > inside the test code, and the output was a simple green square, one baseline > would serve for all ports. > > I think this doesn't really hinder tracking down test failures, for a > couple of reasons: > * Most tests are "green = pass, red = failure" by convention, so frequently > a pixel result that differs from the baseline is easily classifiable at a > quick glance. > * When this isn't the case, one of the first steps in understanding the > test output is to read the test, at which point the HTML comment will > explain things. > > Obviously, some tests need to display text, but this seems to me to be a > good rule of thumb. > > Am I missing anything? If not, is anyone interested in helping to make > this change on existing tests where possible, to trim the number of > baselines in the tree and make it easier for new ports to bring up pixel > tests? > > > I totally agree, and have been avoiding test in pixel tests as much as > possible recently. > > One useful strategy is to add explanatory notes to the test as HTML > comments, so the purpose of the test is easily determined by someone looking > at the source. > > Simon > > > > _______________________________________________ > webkit-dev mailing list > [email protected] > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > >
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
