On Wed, Jun 6, 2012 at 10:36 PM, Peter Kasting <pkast...@chromium.org> wrote: > On Wed, Jun 6, 2012 at 10:22 PM, Ryosuke Niwa <rn...@webkit.org> wrote: >> >> Now that everyone knows the problem, I propose to rename FAIL to DIFF. >> >> FAIL should mean that the test fails, not that it fails with image, text, >> or image and text failures. >> >> DIFF, on the other hand, has no ambiguity. It can't be interpreted as >> timeout, crash, or pass but can easily be associated with image and text >> differences. > > > I don't think DIFF is any better. It sounds like it means the output is > "different than what we wanted", thus it effectively means "didn't pass", > and one would expect it to match MISSING/CRASH/TIMEOUT as much as one would > expect FAIL to. > > Personally I'd prefer to resolve this -- if we need to -- by removing FAIL > entirely. Being explicit about your expectations isn't a bad thing. Plus, > the number of cases that are truly TEXT IMAGE IMAGE+TEXT seems likely to be > small.
FAIL was supposed to have been removed long ago in favor of TEXT/IMAGE/IMAGE+TEXT as necessary as Peter suggests, but I never got around to it :) I agree with his reasoning, and have not been convinced that not having to be specific about what is failing is a good thing. I also agree that we shouldn't TEXT/IMAGE/IMAGE+TEXT in the file at all for extended periods of time since you can just check in updated baselines. -- Dirk _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
