On Thu, Mar 21, 2013 at 5:18 PM, Robert Hogan <li...@roberthogan.net> wrote:
> > > On Thursday, 21 March 2013, Ryosuke Niwa wrote: > >> On Thu, Mar 21, 2013 at 1:31 AM, Robert Hogan <li...@roberthogan.net>wrote: >> >>> On Thursday, 21 March 2013, Ryosuke Niwa wrote: >>> >>>> I used to pull results from the bots where possible but creating >>>>> inconsistency between png/text results is not good. >>>>> >>>> >>>> It is unfortunate but it's much better than losing the complete test >>>> coverage. >>>> >>> >>> If that's the case then I'm happy to land whatever garden-o-matic pulls >>> in or I can sweep from the bots, even if it means that png results for Mac, >>> Qt, et al. go bad as a result. >>> >>> I guess we will always have ports whose bots do not run pixel tests so >>> if those ports are happy to live with the downsides of doing that then >>> there really is no obstacle to authors owning the job of updating the >>> baselines for all ports when they land a change. >>> >>> IMHO ports who don't run pixel tests would be better off deleting any >>> png results they have in the tree. Is there a reason Mac hasn't done that? >>> Don't you get lots of failures when you run pixel tests locally? >>> >> >> Yes, but I'd argue that it's better than losing the test coverage. >> >> By the way, we can easily address this problem by always generating pixel >> results for unexpectedly failing tests. Namely, we can force --pixel when >> we're retrying tests. >> >> > Perhaps NRWT could produce txt and png results for all tests marked with > REBASELINE or similar in TestExpectations. That would avoid the need to > turn the bots red on each platform for at least one build cycle. > I like this specific proposal. There's already a similar expectation planned, 'NeedsRebaseline'. https://bugs.webkit.org/show_bug.cgi?id=100415 > > Or is that something we can live with? > > > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev > >
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
