Title: [90823] trunk/Tools
- Revision
- 90823
- Author
- [email protected]
- Date
- 2011-07-12 10:49:27 -0700 (Tue, 12 Jul 2011)
Log Message
cr-linux-ews complains about tests that aren't actually failing
https://bugs.webkit.org/show_bug.cgi?id=64367
The underlying problem here is that full_results.json doesn't have
enough information to determine whether a given test result was
expected because whether an actual result is expected depends on
whether full_results.json was generated duing a test run that included
pixel tests.
The right long-term solution is to make full_results.json a complete
description of what happened durning a test run. In this patch
however, to stop the spam, this patch makes the jsonresultsparser
ignore pixel failures.
(I'm landing this patch unreviewed in order to stop the bots from
spamming. I'll happily address any review comments in a follow-up
patch.)
* Scripts/webkitpy/common/net/resultsjsonparser.py:
* Scripts/webkitpy/common/net/resultsjsonparser_unittest.py:
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (90822 => 90823)
--- trunk/Tools/ChangeLog 2011-07-12 17:46:16 UTC (rev 90822)
+++ trunk/Tools/ChangeLog 2011-07-12 17:49:27 UTC (rev 90823)
@@ -1,5 +1,28 @@
2011-07-12 Adam Barth <[email protected]>
+ cr-linux-ews complains about tests that aren't actually failing
+ https://bugs.webkit.org/show_bug.cgi?id=64367
+
+ The underlying problem here is that full_results.json doesn't have
+ enough information to determine whether a given test result was
+ expected because whether an actual result is expected depends on
+ whether full_results.json was generated duing a test run that included
+ pixel tests.
+
+ The right long-term solution is to make full_results.json a complete
+ description of what happened durning a test run. In this patch
+ however, to stop the spam, this patch makes the jsonresultsparser
+ ignore pixel failures.
+
+ (I'm landing this patch unreviewed in order to stop the bots from
+ spamming. I'll happily address any review comments in a follow-up
+ patch.)
+
+ * Scripts/webkitpy/common/net/resultsjsonparser.py:
+ * Scripts/webkitpy/common/net/resultsjsonparser_unittest.py:
+
+2011-07-12 Adam Barth <[email protected]>
+
Re-work garden-o-matic UI to begin to look like the mocks
https://bugs.webkit.org/show_bug.cgi?id=64334
Modified: trunk/Tools/Scripts/webkitpy/common/net/resultsjsonparser.py (90822 => 90823)
--- trunk/Tools/Scripts/webkitpy/common/net/resultsjsonparser.py 2011-07-12 17:46:16 UTC (rev 90822)
+++ trunk/Tools/Scripts/webkitpy/common/net/resultsjsonparser.py 2011-07-12 17:49:27 UTC (rev 90823)
@@ -73,6 +73,9 @@
def did_run_as_expected(self):
actual_results = self._actual_as_tokens()
expected_results = self._expected_as_tokens()
+ # FIXME: We should only remove_pixel_failures this JSONResult can from a run without pixel tests!
+ if not test_expectations.has_pixel_failures(actual_results):
+ expected_results = test_expectations.remove_pixel_failures(expected_results)
for actual_result in actual_results:
if not test_expectations.result_was_expected(actual_result, expected_results, False, False):
return False
@@ -82,7 +85,7 @@
tokens = map(test_expectations.TestExpectations.expectation_from_string, results_string.split(' '))
if None in tokens:
log("Unrecognized result in %s" % results_string)
- return tokens
+ return set(tokens)
@memoized
def _actual_as_tokens(self):
Modified: trunk/Tools/Scripts/webkitpy/common/net/resultsjsonparser_unittest.py (90822 => 90823)
--- trunk/Tools/Scripts/webkitpy/common/net/resultsjsonparser_unittest.py 2011-07-12 17:46:16 UTC (rev 90822)
+++ trunk/Tools/Scripts/webkitpy/common/net/resultsjsonparser_unittest.py 2011-07-12 17:49:27 UTC (rev 90823)
@@ -58,6 +58,10 @@
"prototype-strawberry.html": {
"expected": "PASS",
"actual": "TEXT PASS"
+ },
+ "prototype-peach.html": {
+ "expected": "IMAGE+TEXT",
+ "actual": "TEXT"
}
}
},
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py (90822 => 90823)
--- trunk/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py 2011-07-12 17:46:16 UTC (rev 90822)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py 2011-07-12 17:49:27 UTC (rev 90823)
@@ -84,6 +84,10 @@
return expected_results
+def has_pixel_failures(actual_results):
+ return IMAGE in actual_results or IMAGE_PLUS_TEXT in actual_results
+
+
def strip_comments(line):
"""Strips comments from a line and return None if the line is empty
or else the contents of line with leading and trailing spaces removed
_______________________________________________
webkit-changes mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo.cgi/webkit-changes
