We've had cases of a patch destroying the buildbots by rampaging
through thousands of test failures. The 20 limit is a fail-stop to
keep the bots rolling.
I agree that try servers would be a better way to handle this case
than lighting the buildbot on fire. I'd like to extend the EWS to run
tests, which could then make the new baselines available. Some other
folks have expressed interest in having that work too. If you're
interested in helping to make that happen, let me know.
Adam
On Mon, Feb 8, 2010 at 6:51 PM, Nikolas Zimmermann
zimmerm...@physik.rwth-aachen.de wrote:
Good morning crowd,
can anyone explain the --exit-after-n-failures=20 default behaviour for
the build bots?
Background:
Today I had to update a lot of SVG *expected.txt files including all
platform specific variations (in LayoutTests/platform/gtk/qt/win/mac-*).
I landed my patch with updated mac results and planned to wait for the build
bots to show me all differences, so I could update all baselines
for the individual platforms needing custom layout test results differing
from the mac baseline.
Though because the bots exit early after 20 test failures it took a long
time to identify all differences because I could only fix a maximum of
20 failures per commit - that's odd. I'm aware it's a rare case to update
900+ test results in one shot but I thought it's worth to mention.
I think the best way to resolve this problem is to run layout tests on the
try bots, so any layout test differences can be identified before comitting.
I recall some talk about that in the past, what's the status?
Cheers,
Niko
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
