Title: [281216] trunk/Tools
- Revision
- 281216
- Author
- jbed...@apple.com
- Date
- 2021-08-18 17:20:00 -0700 (Wed, 18 Aug 2021)
Log Message
[run-webkit-tests] Remember failures when repeating test
https://bugs.webkit.org/show_bug.cgi?id=229217
<rdar://problem/81919223>
Reviewed by Ryan Haddad.
* Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py:
(LayoutTestRunner.update_summary_with_result): Replace existing result with unexpected result so
that unexpected results always take precedence over expected ones.
* Scripts/webkitpy/layout_tests/models/test_run_results.py:
(TestRunResults.add): Do not replace existing result.
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (281215 => 281216)
--- trunk/Tools/ChangeLog 2021-08-18 23:14:15 UTC (rev 281215)
+++ trunk/Tools/ChangeLog 2021-08-19 00:20:00 UTC (rev 281216)
@@ -1,3 +1,17 @@
+2021-08-18 Jonathan Bedard <jbed...@apple.com>
+
+ [run-webkit-tests] Remember failures when repeating test
+ https://bugs.webkit.org/show_bug.cgi?id=229217
+ <rdar://problem/81919223>
+
+ Reviewed by Ryan Haddad.
+
+ * Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py:
+ (LayoutTestRunner.update_summary_with_result): Replace existing result with unexpected result so
+ that unexpected results always take precedence over expected ones.
+ * Scripts/webkitpy/layout_tests/models/test_run_results.py:
+ (TestRunResults.add): Do not replace existing result.
+
2021-08-18 Lauro Moura <lmo...@igalia.com>
REGRESSION(r279169) [WebDriver] WPT Tools now requires python3 to launch the server
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py (281215 => 281216)
--- trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py 2021-08-18 23:14:15 UTC (rev 281215)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py 2021-08-19 00:20:00 UTC (rev 281216)
@@ -227,6 +227,7 @@
if result.type == test_expectations.SKIP:
exp_str = got_str = 'SKIP'
expected = True
+ expectations = None
else:
expectations = self._expectations.filtered_expectations_for_test(result.test_name, self._options.pixel_tests or bool(result.reftest_type), self._options.world_leaks)
expected = self._expectations.matches_an_expected_result(result.test_name, result.type, expectations)
@@ -233,7 +234,11 @@
exp_str = self._expectations.model().expectations_to_string(expectations)
got_str = self._expectations.model().expectation_to_string(result.type)
+ existing = self._current_run_results.results_by_name.get(result.test_name)
self._current_run_results.add(result, expected)
+ if existing and not expected:
+ existing_expectation = self._expectations.matches_an_expected_result(result.test_name, existing.type, expectations)
+ self._current_run_results.change_result_to_failure(existing, result, existing_expectation, expected)
self.printer.print_finished_test(result, expected, exp_str, got_str)
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py (281215 => 281216)
--- trunk/Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py 2021-08-18 23:14:15 UTC (rev 281215)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py 2021-08-19 00:20:00 UTC (rev 281216)
@@ -69,7 +69,7 @@
def add(self, test_result, expected):
self.tests_by_expectation[test_result.type].add(test_result.test_name)
- self.results_by_name[test_result.test_name] = test_result
+ self.results_by_name[test_result.test_name] = self.results_by_name.get(test_result.test_name, test_result)
if test_result.is_other_crash:
return
if test_result.type != test_expectations.SKIP:
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
