Title: [90571] trunk/Tools
- Revision
- 90571
- Author
- commit-qu...@webkit.org
- Date
- 2011-07-07 10:18:09 -0700 (Thu, 07 Jul 2011)
Log Message
Completely ignore too-many-failures builds in TestFailures in most circumstances
Because a semi-arbitrary subset of tests are run in a too-many-failures build, we can't
really use them to perform regression analysis. The only time we want to pay attention to
too-many-failures builds is when we're trying to explain when the current bout of
too-many-failures started.
Fixes <http://webkit.org/b/64106> TestFailures page sometimes claims a test started failing
in a build that didn't even run it (because it exited early due to too many failues)
Patch by Adam Roben <aro...@apple.com> on 2011-07-07
Reviewed by David Kilzer.
* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/LayoutTestHistoryAnalyzer.js:
(LayoutTestHistoryAnalyzer.prototype._incorporateBuildHistory): Removed old, broken
too-many-failures handling that would cause us to blame builds that didn't even run a given
test for breaking it. Instead, skip over all too-many-failures builds unless the most recent
build was itself a too-many-failures build.
Modified Paths
Diff
Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/LayoutTestHistoryAnalyzer.js (90570 => 90571)
--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/LayoutTestHistoryAnalyzer.js 2011-07-07 17:17:18 UTC (rev 90570)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/LayoutTestHistoryAnalyzer.js 2011-07-07 17:18:09 UTC (rev 90571)
@@ -118,6 +118,20 @@
var self = this;
self._loader.start(nextBuildName, function(tests, tooManyFailures) {
+ if (tooManyFailures) {
+ var firstBuildName = Object.keys(self._history)[0];
+ // If the first (i.e., current or most recent) build exited early due to too many
+ // failures, we want to process other too-many-failures builds normally to try to
+ // figure out when the too-many-failures started occurring. If the first/current
+ // build did not exit due to too many failures, then too-many-failures builds will
+ // only confuse our analysis (since they run a semi-arbitrary subset of tests), so
+ // we should just skip them entirely.
+ if (firstBuildName && !self._history[firstBuildName].tooManyFailures) {
+ callback(true);
+ return;
+ }
+ }
+
++self._testRunsSinceLastInterestingChange;
var historyItem = {
@@ -150,16 +164,6 @@
historyItem.tests[testName] = tests[testName];
}
- if (tooManyFailures && previousHistoryItem) {
- // Not all tests were run due to too many failures. Just assume that all the tests
- // that failed in the last build would still have failed in this build had they been
- // run.
- for (var testName in previousHistoryItem.tests) {
- historyItem.tests[testName] = previousHistoryItem.tests[testName];
- delete previousHistoryItem.tests[testName];
- }
- }
-
var previousUnexplainedFailuresCount = previousBuildName ? Object.keys(self._history[previousBuildName].tests).length : 0;
var unexplainedFailuresCount = Object.keys(self._history[nextBuildName].tests).length;
Modified: trunk/Tools/ChangeLog (90570 => 90571)
--- trunk/Tools/ChangeLog 2011-07-07 17:17:18 UTC (rev 90570)
+++ trunk/Tools/ChangeLog 2011-07-07 17:18:09 UTC (rev 90571)
@@ -1,5 +1,25 @@
2011-07-07 Adam Roben <aro...@apple.com>
+ Completely ignore too-many-failures builds in TestFailures in most circumstances
+
+ Because a semi-arbitrary subset of tests are run in a too-many-failures build, we can't
+ really use them to perform regression analysis. The only time we want to pay attention to
+ too-many-failures builds is when we're trying to explain when the current bout of
+ too-many-failures started.
+
+ Fixes <http://webkit.org/b/64106> TestFailures page sometimes claims a test started failing
+ in a build that didn't even run it (because it exited early due to too many failues)
+
+ Reviewed by David Kilzer.
+
+ * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/LayoutTestHistoryAnalyzer.js:
+ (LayoutTestHistoryAnalyzer.prototype._incorporateBuildHistory): Removed old, broken
+ too-many-failures handling that would cause us to blame builds that didn't even run a given
+ test for breaking it. Instead, skip over all too-many-failures builds unless the most recent
+ build was itself a too-many-failures build.
+
+2011-07-07 Adam Roben <aro...@apple.com>
+
Teach webkitpy's Checkout class to use commit-log-editor to create commit messages
Fixes <http://webkit.org/b/26755> webkit-patch's commit messages are less readable than
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-changes
