Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 7dc7c1f5a8abc337fb5cc365f03161e2c536480e
https://github.com/WebKit/WebKit/commit/7dc7c1f5a8abc337fb5cc365f03161e2c536480e
Author: Sihui Liu <[email protected]>
Date: 2026-06-02 (Tue, 02 Jun 2026)
Changed paths:
M Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
M Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py
M Tools/Scripts/webkitpy/port/driver.py
M Tools/WebKitTestRunner/Options.cpp
M Tools/WebKitTestRunner/Options.h
M Tools/WebKitTestRunner/TestController.cpp
Log Message:
-----------
Introduce --site-isolation-enabled-by-default to run-webkit-tests to simulate
SI-on-by-default
https://bugs.webkit.org/show_bug.cgi?id=315857
rdar://178255472
Reviewed by Per Arne Vollan.
Adds a `--site-isolation-enabled-by-default` flag that runs every test with
SiteIsolationEnabled=true as the default
preference, simulating the environment we'd get if site isolation were turned
on by default in
UnifiedWebPreferences.yaml. This lets a bot surface problems ahead of that
change.
The flag is different from `--site-isolation` because it runs tests with Site
Isolation on and compares the result
against the -expected.txt file, while `--site-isolation` compares the results
with SI off and on.
The flag is different from `--additional-header=SiteIsolationEnabled=true`
because it injects SiteIsolationEnabled=true
into the test runner's global features, so an individual test can still opt out
with a SiteIsolationEnabled=false
header, while `--additional-header` would override the test's own header and
force SI on even for tests that ask to
disable it (e.g. the test wants to verify a non-SI behavior). Another
difference is expected-result lookup prefers the
mac-site-isolation / ios-site-isolation platform baselines, falling back to the
generic expected files. This would be
useful if there is intended behavior change under Site Isolation (e.g. console
log may not include full url for remote
frame under SI).
For imported WPT tests, there is a known issue that subtest results in test
output might be out-of-order under SI.
Specifically, all subtests are still PASS, but the order of the result is
different compared to when SI is off. This is
because when frames are hosted in different processes, the ordering of events
and operations could change, and it can
impact when a subtest is created. WPT treats testharness subtest results as an
unordered set matched by name (see
tools/wptrunner/.../manifestexpected.py get_subtest), so the test still passes
under SI; but WebKit test infra treats
subtest results as an ordered list (must match the order in -expected.txt), so
the test is mistakenly reported as FAIL.
To fix this issue, this patch adds a fallback in the result comparison under
`--site-isolation-enabled-by-default` for
WPT tests: when exact comparison between test output and the -expected.txt file
fails, the script sorts the subtest
result lines in the outputs and compares them again (performing an
order-insensitive comparison).
* Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
(SingleTestRunner._compare_text):
(SingleTestRunner._is_site_isolation_enabled_by_default_mode):
(SingleTestRunner):
(SingleTestRunner._is_wpt_test):
(SingleTestRunner._normalize_testharness_result_order):
* Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py:
(parse_args):
(_set_up_derived_options):
* Tools/Scripts/webkitpy/port/driver.py:
(Driver.cmd_line):
* Tools/WebKitTestRunner/Options.cpp:
(WTR::handleOptionSiteIsolationEnabledByDefault):
(WTR::OptionsHandler::OptionsHandler):
* Tools/WebKitTestRunner/Options.h:
* Tools/WebKitTestRunner/TestController.cpp:
(WTR::TestController::initialize):
Canonical link: https://commits.webkit.org/314431@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
