Title: [277406] trunk/PerformanceTests
- Revision
- 277406
- Author
- mmaxfi...@apple.com
- Date
- 2021-05-12 16:18:49 -0700 (Wed, 12 May 2021)
Log Message
[MotionMark] Temporarily remove the focus test
https://bugs.webkit.org/show_bug.cgi?id=225714
Reviewed by Geoffrey Garen.
The focus test is causing unacceptably high variance on test results. Initial investigation
indicates that this is related to secondary and tertiary interactions between the browser's
runloop and the compositor's runloop. Not only that, but the variable cost of the focus test
seems to affect the variance on the subtests which run after it.
This patch temporarily removes the focus test, just so we can get a stronger signal of
browsers' true performance, while we do a deeper analysis of the focus test.
Because the focus test's score is lower than the average score of the subtests, removing it
causes scores in all browsers to increase:
| WebKit | Chrome | Firefox
================================================
Score with Focus | 1020 | 550 | 962
Score without Focus | 1623 | 874 | 1101
Stddev with Focus | 9.3% | 2.3% | 2.9%
Stddev without Focus | 0.7% | 0.5% | 7.9%
* MotionMark/resources/runner/tests.js:
Modified Paths
Diff
Modified: trunk/PerformanceTests/ChangeLog (277405 => 277406)
--- trunk/PerformanceTests/ChangeLog 2021-05-12 23:16:43 UTC (rev 277405)
+++ trunk/PerformanceTests/ChangeLog 2021-05-12 23:18:49 UTC (rev 277406)
@@ -1,3 +1,30 @@
+2021-05-12 Myles C. Maxfield <mmaxfi...@apple.com>
+
+ [MotionMark] Temporarily remove the focus test
+ https://bugs.webkit.org/show_bug.cgi?id=225714
+
+ Reviewed by Geoffrey Garen.
+
+ The focus test is causing unacceptably high variance on test results. Initial investigation
+ indicates that this is related to secondary and tertiary interactions between the browser's
+ runloop and the compositor's runloop. Not only that, but the variable cost of the focus test
+ seems to affect the variance on the subtests which run after it.
+
+ This patch temporarily removes the focus test, just so we can get a stronger signal of
+ browsers' true performance, while we do a deeper analysis of the focus test.
+
+ Because the focus test's score is lower than the average score of the subtests, removing it
+ causes scores in all browsers to increase:
+
+ | WebKit | Chrome | Firefox
+ ================================================
+ Score with Focus | 1020 | 550 | 962
+ Score without Focus | 1623 | 874 | 1101
+ Stddev with Focus | 9.3% | 2.3% | 2.9%
+ Stddev without Focus | 0.7% | 0.5% | 7.9%
+
+ * MotionMark/resources/runner/tests.js:
+
2021-04-19 Kimmo Kinnunen <kkinnu...@apple.com>
Enable -Wthread-safety, add attributes to custom lock classes, and provide macros to declare guards
Modified: trunk/PerformanceTests/MotionMark/resources/runner/tests.js (277405 => 277406)
--- trunk/PerformanceTests/MotionMark/resources/runner/tests.js 2021-05-12 23:16:43 UTC (rev 277405)
+++ trunk/PerformanceTests/MotionMark/resources/runner/tests.js 2021-05-12 23:18:49 UTC (rev 277406)
@@ -52,10 +52,6 @@
name: "Canvas Lines"
},
{
- url: "master/focus.html",
- name: "Focus"
- },
- {
url: "master/image-data.html",
name: "Images"
},
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
