We've had cases of a patch destroying the buildbots by rampaging through thousands of test failures. The 20 limit is a fail-stop to keep the bots rolling.
I agree that try servers would be a better way to handle this case than lighting the buildbot on fire. I'd like to extend the EWS to run tests, which could then make the new baselines available. Some other folks have expressed interest in having that work too. If you're interested in helping to make that happen, let me know. Adam On Mon, Feb 8, 2010 at 6:51 PM, Nikolas Zimmermann <zimmerm...@physik.rwth-aachen.de> wrote: > Good morning crowd, > > can anyone explain the "--exit-after-n-failures=20" default behaviour for > the build bots? > > Background: > Today I had to update a lot of SVG *expected.txt files including all > platform specific variations (in LayoutTests/platform/<gtk/qt/win/mac-*>). > I landed my patch with updated mac results and planned to wait for the build > bots to show me all differences, so I could update all baselines > for the individual platforms needing custom layout test results differing > from the mac baseline. > > Though because the bots exit early after 20 test failures it took a long > time to identify all differences because I could only fix a maximum of > 20 failures per commit - that's odd. I'm aware it's a rare case to update > 900+ test results in one shot but I thought it's worth to mention. > > I think the best way to resolve this problem is to run layout tests on the > try bots, so any layout test differences can be identified before comitting. > I recall some talk about that in the past, what's the status? > > Cheers, > Niko > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev