On Tue, Nov 13, 2012 at 12:02 PM, Tony Chang <t...@chromium.org> wrote: > On Tue, Nov 13, 2012 at 11:35 AM, Dirk Pranke <dpra...@chromium.org> wrote: >> >> On Tue, Nov 13, 2012 at 11:29 AM, Glenn Adams <gl...@skynav.com> wrote: >> > >> > On Tue, Nov 13, 2012 at 12:09 PM, Dirk Pranke <dpra...@chromium.org> >> > wrote: >> >> >> >> We don't currently support port-specific reftests (or at least, not >> >> very well), and we certainly should be trying to minimize where they >> >> occur. >> > >> > >> > Hmm, I actually used port specific reftest expectation files in a recent >> > patch [1] (since rolled out), and it appeared to work (as a way to >> > effectively rebaseline those expectations). So something seems to be >> > working. >> > >> > [1] http://trac.webkit.org/changeset/133529 >> > >> >> I expect it'll sort of work, but it won't be robust; you may hit weird >> behavior and/or bugs. We really haven't beaten on this aspect of >> things, and I don't know yet how much we want to. > > > I don't think we should support port specific ref test results. That kind > of misses the point of using a ref test in the first place. I mean, you may > as well check in port specific pixel results which are easier to review for > correctness. > > It may be the case that a ref test is not appropriate for what you're trying > to test.
I think that there are probably cases where we will have differences in results because of (legal and entirely correct or permissible) differences in the implementations and in this case a reftest can still be an improvement over maintaining N platform-specific pixel versions. The obvious example is when we haven't implemented features yet (or have bugs). The W3C's spec for handling reftests also gives you a way to say "a test passes if any of these N references may match", which is fairly consistent with the idea that platform specific references are okay in some cases. As to whether pixel-tests are easier to review for correctness than reftests or not, I think it depends on the test. -- Dirk _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev