Title: [158093] trunk/Websites/test-results
- Revision
- 158093
- Author
- rn...@webkit.org
- Date
- 2013-10-26 17:47:21 -0700 (Sat, 26 Oct 2013)
Log Message
New flakiness dashboard shouldn't treat tests with right expectations as failing
https://bugs.webkit.org/show_bug.cgi?id=123385
Reviewed by Alexey Proskuryakov.
We define failing tests to be tests with wrong expectations whose actual results are not PASS
since tests with TEXT, IMAGE, etc... failures do not turn the bots red as long as the expectation
of the same type is specified in TestExpectation files.
* public/include/test-results.php:
(FailingResultsJSONWriter): Inherit from WrongExpectationsResultsJSONWriter.
Modified Paths
Diff
Modified: trunk/Websites/test-results/ChangeLog (158092 => 158093)
--- trunk/Websites/test-results/ChangeLog 2013-10-26 20:42:28 UTC (rev 158092)
+++ trunk/Websites/test-results/ChangeLog 2013-10-27 00:47:21 UTC (rev 158093)
@@ -1,3 +1,17 @@
+2013-10-26 Ryosuke Niwa <rn...@webkit.org>
+
+ New flakiness dashboard shouldn't treat tests with right expectations as failing
+ https://bugs.webkit.org/show_bug.cgi?id=123385
+
+ Reviewed by Alexey Proskuryakov.
+
+ We define failing tests to be tests with wrong expectations whose actual results are not PASS
+ since tests with TEXT, IMAGE, etc... failures do not turn the bots red as long as the expectation
+ of the same type is specified in TestExpectation files.
+
+ * public/include/test-results.php:
+ (FailingResultsJSONWriter): Inherit from WrongExpectationsResultsJSONWriter.
+
2013-10-25 Ryosuke Niwa <rn...@webkit.org>
Build fix. The query results weren't sorted by the latest commit time,
Modified: trunk/Websites/test-results/public/include/test-results.php (158092 => 158093)
--- trunk/Websites/test-results/public/include/test-results.php 2013-10-26 20:42:28 UTC (rev 158092)
+++ trunk/Websites/test-results/public/include/test-results.php 2013-10-27 00:47:21 UTC (rev 158093)
@@ -137,13 +137,6 @@
abstract protected function pass_for_failure_type(&$results);
}
-class FailingResultsJSONWriter extends ResultsJSONWriter {
- public function __construct($fp) { parent::__construct($fp); }
- protected function pass_for_failure_type(&$results) {
- return $results[0]['actual'] == 'PASS';
- }
-}
-
class FlakyResultsJSONWriter extends ResultsJSONWriter {
public function __construct($fp) { parent::__construct($fp); }
protected function pass_for_failure_type(&$results) {
@@ -175,6 +168,13 @@
}
}
+class FailingResultsJSONWriter extends WrongExpectationsResultsJSONWriter {
+ public function __construct($fp) { parent::__construct($fp); }
+ protected function pass_for_failure_type(&$results) {
+ return $results[0]['actual'] == 'PASS' || parent::pass_for_failure_type($results);
+ }
+}
+
class ResultsJSONGenerator {
private $db;
private $builder_id;
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
