Re: [blink-dev] RuntimeEnabledFeatures flags that we might be able to remove

[Xianzhu Wang]

It seems that some old public features can also be removed. They have
dedicated --disable-* command line switches or are bound to base features,
but are not much different from the non-public ones which are also
controllable with command line flags
(--disable-features/--disable-blink-features) or through automatically
defined base features. They are public perhaps just because they had
existed before we added new ways to control blink runtime features, such as
WebRuntimeFeatures::EnableFeatureFromString (instead of dedicated
functions) and automatically defined base features.

On Fri, Jan 12, 2024 at 8:39 AM David Baron  wrote:

> Since we've recently been more careful about creating feature flags for
> changes that have the possibility of breaking content, we've also been
> creating more feature flags than before.  This means that we're also
> creating a larger number of feature flags that have shipped to stable, been
> shown to be safe, and have served their purpose.  Many of these flags (and
> associated flag-controlled code) can hopefully be removed.
>
> I made a spreadsheet of feature flags that have been shipped in stable
> 
>  along
> with the stable milestone they shipped in.  The sheet should be publicly
> viewable and editable by any Chromium committer.  The sheet itself has
> notes about how I made it (briefly: mostly with
> third_party/blink/tools/list_stable_features.py).
>
> This sheet is presented as data to help folks remember to remove flags
> that they were intending to remove.  I'm sure there are a bunch of flags
> listed that shouldn't be removed, but also plenty that can be removed
> (either now or soon).
>
> Feel free to add notes to the sheet, update owners as needed, and to write
> CLs to remove flags that we don't need anymore.
>
> -David
>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAG0MU3ia7vybzAWOADVmqdLhN%2BT7VJsfNcmaLcXAutwZRSpVTQ%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxridoK3ZO8AQYDM73M9OAB8aRHx-3Y7bD0DCmsqyavhC9_Q%40mail.gmail.com.

Re: [blink-dev] Blink-rel try bots do not run unit tests

[Xianzhu Wang]

I think the blink try bots are mainly for rebaselining web tests. The CQ
bots win-rel, linux-rel, mac-rel run blink unit tests. Running blink unit
tests on all versions of platforms may be a rare requirement.

Weizhong, is there an easy way to force blink_unittests and
blink_platform_unittests on the blink-rel try bots when needed?

On Fri, Dec 1, 2023 at 7:37 AM Stephen Chenney 
wrote:

> I am surprised to discover that the various blink-rel try bots, say
> linux-blink-rel or win11-blink-rel, do not run blink_unittests or
> blink_platform_unittests or any other unit tests.
>
> Is this deliberate?
>
> I ask because I've been writing unit tests that depend on font metrics
> that differ across platforms, and I want to verify the tests on all those
> platforms. I recall being able to do this with the blink rel bots.
>
> Stephen.
>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGsbWzS1e1nPWQcvahwVfonA20zYO1Z2Z9o4KYG14toyWs9VEA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxric%2Bkt3HD%3D3xoiPq31huSLrYA6bsFwOnW47pycDjgsW4%3DA%40mail.gmail.com.

Re: [blink-dev] flag-specific + FlagExpectations + virtual

['Xianzhu Wang' via blink-dev]

This document

describes the relationship between flag-specifc and virtual suites.

Perhaps we should
- move FlagExpectations/flag-name to
flag-specific/flag-name/TestExpectations
- require README.md under each flag-specifc/flag-name
- add web_tests/README.md, web_tests/flag-specific/README.md, which could
just reference the above document.


On Fri, Jul 14, 2023 at 11:03 AM Vladimir Levin  wrote:

> Hi,
>
> Is there any documentation for the following directories or any planned
> work to structure them better?
>
> blink/third_party/web_tests/flag-specific
> blink/third_party/web_tests/FlagExpectations
> blink/third_party/web_tests/virtual
>
> I know that `FlagExpectations` mirrors TestExpectations for each flag (aka
> virtual test suite?)
>
> `virtual` seems to be READMEs that describe the particular virtual test
> suite (I'm not sure where this is used -- is this useful?)
>
> `flag-specific` is where flag specific expectations are (not to be
> confused with FlagExpectations), and it's also a directory that takes me an
> embarrassingly long amount of time to find every time I have to look in
> there
>
> I don't have any specific proposals, but I was just wondering if others
> felt that it is currently not the best structure. Maybe we can come up with
> something better, or at least help me frame it in a way that I can remember
> :)
>
> Thanks,
> vmpstr
>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADsXd2Okx6LPqbOJR8irh7JN%2BcKcEAeOSbgszrEJmgu3eurbSg%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxridJeV9%2BCS1HD%2Bc%2BmLEwHLQ8nHrcdXjrjSLTNgHcaxaoqg%40mail.gmail.com.

Re: [blink-dev] Re: Intent to Ship: overflow:overlay aliases overflow:auto

[Xianzhu Wang]

On Tue, Jun 20, 2023 at 1:51 PM fri tz  wrote:

> hi,
> I agree with May b. We have been using this feature in our web agency for
> years and are not happy about this change... The scrollbar track harms
> modern layouts especially with fullscreen images or horizontal overflow
> elements like sliders. A scroll function is not always required, e.g. in
> fullscreen navigations, modals etc... removing the scrollbar by overflow
> hidden in a modal causes a jump of the content. Overflow overlay prevent
> this issue. The scrollbar on Mac Safari or Win Firefox are overlayed by
> default. Why does Chrome go an other way?
>

I just tried Safari, Firefox and Chrome on my Mac. All of them follow the
system scrollbar appearance setting (System Settings > Appearance > Show
scrollbars). They all use overlay scrollbars if the setting is "When
scrolling", or "Automatically based on mouse or trackpad" with trackpad.
Otherwise they all use non-overlay scrollbars.

Overlay scrollbar has advantages and disadvantages (see crbug.com/801671),
so we would like to let users control it based on their preference. For now
users can force overlay or non-overlay scrollbars by enabling/disabling
chrome://flags#overlay-scrollbars. It's not an official feature though. We
prefer the system setting if it's available.


> Chris Harrelson schrieb am Mittwoch, 7. Juni 2023 um 18:43:09 UTC+2:
>
>> Hi,
>>
>> On Wed, Jun 7, 2023 at 7:59 AM may b  wrote:
>>
>>> Hello,
>>>
>>> I noticed today while working on our product that the overflow:overlay
>>> feature is no longer working. I finally stumbled upon this thread that it
>>> has been aliased into auto. I think it's really unfortunate that this
>>> feature is no longer available, and for people building web applications,
>>> having the scrollbar gutter present can ruin designs, which is why overlay
>>> was a nice solution.
>>
>>
>> In general I think the browser needs to be in charge of this, because
>> scrollbars are ultimately a browser feature to help users see content. Over
>> time I hope overlay scrollbars for Chromium will appear on more
>> platforms than just Mac, Android and ChromeOS.
>>
>> Note also that this feature is not supported in Safari or Firefox, so in
>> both cases the behavior in your video should be present on those browsers
>> when non-overlay scrollbars are used by the browser.
>>
>>
>>> A use case would be that we have modal windows in our application, and
>>> so we hide the scrollbar and show it when content is being hovered on. Our
>>> intention eventually was to update it to show overlayed scrollbars only if
>>> scrolling (similar to desktop applications like Slack, or the default
>>> overlay scrollbars on Mac OS). With the new change, content in flex boxes
>>> are completely being shifted. Having the autonomy to choose when and how
>>> the user sees the scrollbar would be nice, and for us, this change will
>>> require a big workaround to fix every instance in the application.
>>
>>
>> In Chromium or Webkit-based browsers, I think you can achieve almost all
>> of this with:
>>
>> #my-scroller:not(:hover)::-webkit-scrollbar { visibility:hidden }
>> #my-scroller { scrollbar-gutter: stable }
>>
>> This will avoid the content jumping like in your video. (It will not
>> allow use of the gutter area for scrolling content though, which I think is
>> good because it avoids obscuring information.)
>>
>>
>>> Thank you
>>> On Thursday, April 6, 2023 at 9:02:01 AM UTC+9 Chris Harrelson wrote:
>>>
 Contact emailschri...@chromium.org

 Specification
 https://drafts.csswg.org/css-overflow-3/#valdef-overflow-auto

 Summary

 Removes the overflow:overlay scrolling mode, and makes overlay a legacy
 alias of auto. overflow:overlay is the same as overflow:auto, except that
 it does not prevent content from extending into the scrollbar gutter, in
 cases where non-overlay OS scrollbars are present. (If overlay scrollbars
 are present, there is no effect.) Example: With overflow:overlay:
 https://output.jsbin.com/yujenuq/quiet With overflow:auto:
 https://output.jsbin.com/ruzogaf/quiet


 Blink componentBlink>Scroll
 

 TAG reviewNone

 TAG review statusNot applicable

 Risks


 Interoperability and Compatibility

 Developers currently relying on content overlapping the scrollbar
 gutter would instead see some additional line wrapping. Users, on the other
 hand, would be able to see more content that is currently invisible
 underneath a scrollbar. On platform configurations with overlay scrollbars
 in the OS, this change has no effect; it only applies to situations where a
 non-overlay scrollbar is configured by the browser. Use counter:
 https://chromestatus.com/metrics/feature/timeline/popularity/2995
 Adoption is more than 2% of page loads. However: * I don't think any sites

Re: [blink-dev] Re: Looking For People Familiar With Web Tests

[Xianzhu Wang]

Brian, please add me, too, if needed.

On Wed, Nov 9, 2022 at 3:39 PM Brian Sheedy  wrote:

> I've received a number of volunteers privately, so I think I should be all
> set now - thanks everyone!
>
> On Tuesday, November 8, 2022 at 2:18:05 PM UTC-8 Brian Sheedy wrote:
>
>> Hi all,
>>
>> TL;DR: I'm looking for a handful of people who are familiar with web
>> tests/web test expectations that would be willing to occasionally be CCed
>> on generated CLs. You shouldn't need to do anything, but we need a human
>> CCed on generated CLs that are being autosubmitted.
>>
>> I'm in the process of automating the Blink unexpected pass finder/stale
>> expectation remover
>> 
>>  to
>> help increase the amount of usable test coverage we have at any given time.
>> The bot appears to be fully functional and can generate CLs like this
>> . All
>> that's really left is to enable autosubmit, but there doesn't appear to be
>> a good list of people that we could CC on the CLs.
>>
>> The recipe we're using on the bot supports grabbing someone from a
>> rotation, but AFAIK there isn't a suitable Blink rotation we could use. The
>> other option is to have a list of people to randomly select from, which is
>> what we'll be doing, but there doesn't seem to be a good source for a list
>> of suitable people - the OWNERS file for the relevant directory
>> 
>> just lists *.
>>
>> So, I'm looking for volunteers to occasionally be CCed on these generated
>> CLs. The expected frequency is <1 CL per day - the bot will run daily, but
>> will only CC one person from the list each time. You should not need to do
>> anything, as the CLs will be set to autosubmit and will be approved by the
>> rubber stamper bot. If the CL ends up getting falsely rejected by the CQ,
>> it would be nice if the CCed person could trigger another run, but that is
>> not critical. If a CL fails to submit, a new one with similar changes will
>> just be generated the next day.
>>
>> Any volunteers?
>>
>> Thanks,
>> Brian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/4ca5c154-4c9e-47e0-a228-d1225f9b6cb6n%40chromium.org
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxridE-Z9F80ZLE81G7OTYhug_Mvu2ajPGH3wS7FeQgxXksQ%40mail.gmail.com.

Re: [blink-dev] What are the criteria for passing the blink tests on LUCI

[Xianzhu Wang]

(I thought I had answered the question in another thread, but I just found
it's still in my Draft box. Sorry.)

For now we do have many blin_web_tests and blink_wpt_tests that don't pass.
A few are expected not to pass, e.g. harness-tests/crash.html. Most of them
are due to bugs. When a test starts to fail, and we can't find the culprit
change to revert or fix the failure immediately, we file a bug and add an
entry in TestExpectations
<https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/TestExpectations;l=10?q=TestExpectations&ss=chromium>
to
suppress the failure. Now these "expected failures" cause loss of several
percent of loss of test coverage. We are constantly fixing them to recover
test coverage.

On Wed, Aug 31, 2022 at 4:13 AM fang yiwei 
wrote:

> As we can see from the results.html, There are some failure cases. What
> kind of cases will not affect the passing criteria of the CI system。[image:
> 20220831190722.jpg]
>
> On Wednesday, August 31, 2022 at 12:17:54 AM UTC+8 Xianzhu Wang wrote:
>
>> You can see these tests in results.html
>> <https://test-results.appspot.com/data/layout_results/Linux_Tests/119982/blink_web_tests/layout-test-results/results.html>
>>  by
>> clicking "Unexpected Flaky". The link to the results.html is available
>> under the "archive results for blink_web_tests" (or "archive results for
>> blink_wpt_tests") step.
>>
>> On Tue, Aug 30, 2022 at 9:14 AM Xianzhu Wang 
>> wrote:
>>
>>> The tests are unexpected flakes, i.e. they failed in the first round for
>>> various reasons, and succeeded in retry.
>>>
>>> For now we don't report them as errors, but collect the flaky data and
>>> try to fix highly flaky tests.
>>>
>>> On Tue, Aug 30, 2022 at 9:05 AM fang yiwei  wrote:
>>>
>>>>  Form luci platform
>>>> <https://ci.chromium.org/ui/p/chromium/builders/ci/Linux%20Tests/119982/overview>,
>>>> blink_web_tests also run on Ubuntu-18.04, and some test cases also didn't
>>>> run as expected, however the test step is judged to be passed. What are the
>>>> criteria for the CI system to judge whether the test step pass or not.
>>>> [image: 20220830210239.jpg]
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "blink-dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to blink-dev+...@chromium.org.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/aafa4b0f-fa81-4bf1-aa6b-f3ee7bbbe84cn%40chromium.org
>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/aafa4b0f-fa81-4bf1-aa6b-f3ee7bbbe84cn%40chromium.org?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxric2Wpx2k3q0Ub3oBbffU6sCqPt7vAqwsu2AH7zvAScTHg%40mail.gmail.com.

Re: [blink-dev] Developer pain as a result of "introducing blink_wpt_tests"

[Xianzhu Wang]

Hi Blink devs,

The move has landed (https://crrev.com/1039971).

If you have any pending CLs that add/remove/modify files under
web_tests/platform/generic, please sync, and add/remove/modify the
corresponding file under web_tests/ instead.

Thanks,
Xianzhu

On Fri, Aug 26, 2022 at 10:57 AM Weizhong Xia  wrote:

> Hi Blink devs
>
> FYI we are finally ready to move generic baselines back to their original
> places: the test folders. I plan to land the CL today. To avoid any merge
> conflict during this process, we will add an OWNERS file to
> //third_party/blink/web_tests/platform/generic. Any change to that folder
> will require an additional +1, and will not be approved. Once the move is
> done, the "platform/generic" folder will be removed.
>
> thanks, Weizhong
>
> On Thu, Jun 23, 2022 at 11:00 AM Weizhong Xia  wrote:
>
>> Hi blink devs
>>
>> Thanks to those who joined the survey at
>> https://forms.gle/ju45qciS5VTR4ywN7. Most of you expressed the desire to
>> put baselines at the same place of the tests. Your voice is heard, and here
>> is the plan for the next step.
>>
>> In Q3 we will work on to completely separate legacy layout tests and wpt
>> tests, to put them under `third_party/blink/web_tests` and
>> `third_party/blink/wpt_tests` respectively. Generic baselines (including
>> generic virtual baselines) will be moved back to their previous place.
>> Rebaseline tool will be updated to work with this structure, and update
>> baselines for legacy layout tests and wpt tests in a single run. We will
>> have different copies of *TestExpectations*, *FlagSpecificConfig*,
>> *VirtualTestSuites* etc for legacy tests and wpt tests. When working on
>> those files, we will need to make sure we are updating the correct copy of
>> the file. (We will investigate if we need some presubmit check for such a
>> scenario).
>>
>> The reason for this is two fold: as requested we want to put baselines
>> side to side to the tests, and we want to make the directory structure
>> right to speed up the switch to wptrunner.
>>
>> Thoughts? Feel free to leave a comment in crbug/1299834
>> <https://crbug.com/1299834>.
>>
>> Thanks, Weizhong
>>
>>
>>
>> On Mon, Jun 6, 2022 at 5:00 PM Xianzhu Wang 
>> wrote:
>>
>>> On Mon, Jun 6, 2022 at 3:52 PM Weizhong Xia  wrote:
>>>
>>>> Xianzhu, yes 'baselines' is the name we agreed on previously. The
>>>> reason I later changed back to use 'platform' is because that will make the
>>>> CL smaller, and make it easier for gerrit to handle it. We can make one
>>>> round rename when everything is stabilized. (I left you a message when you
>>>> are OOO. I am not sure if that message lived long enough for you to catch
>>>> it.)
>>>>
>>>
>>> I caught the message. I guessed that the name 'platform' might be one of
>>> the reasons for the surprise to blink developers after the change, and the
>>> name 'baselines' might make the change easier to explain :)
>>>
>>> On Mon, Jun 6, 2022 at 3:52 PM Domenic Denicola 
>>> wrote:
>>>
>>>> I'm not sure I understand this logic. -expected.txt files are just
>>>> more-convenient, more in-your-face versions of TestExpectations files.
>>>> Surely you're not suggesting we get rid of TestExpectations files?
>>>>
>>>
>>> Sorry, "-expected.txt ... should be ... eventually be removed" in my
>>> previous email was not clear. I meant for each individual -expected.txt we
>>> should eventually remove it because we should fix the failure. The same
>>> logic applies to TestExpectations. At any time we may allow a certain
>>> number of failures but we should keep the number as small as possible.
>>>
>>> I think we should prefer TestExpectations to -expected.txt for WPT tests
>>> because the entries in TestExpectations have associated bugs which track
>>> the fixing process, unless we find a better way to track the fixing of the
>>> failures in -expected.txt. -expected.txt files do have their values, e.g.
>>> for partially-passing tests we can discover regressions and progressions of
>>> individual sub tests, but they should be rare.
>>>
>>> I think separating -expected.txt from the tests has the following
>>> benefits:
>>> - It makes it clear to blink developers that the files are not a part of
>>> WPT.
>>> - It simplifies the WPT export/import process and others by reducing
&g

[blink-dev] run_web_tests.py --additional-driver-flag no longer matches flag-specific expectations or baselines

[Xianzhu Wang]

Hi,

After this CL
,
run_web_tests.py --additional-driver-flag no longer matches additional
expectations under web_tests/FlagExpectations

or additional baselines under web_tests/flag-specific
.
Please use --flag-specific instead if you need that behavior.

The reason for the change is that the expectations file / baseline
directory matching rule is complicated and sometimes problematic when
--additional-driver-flag specifies multiple flags. --flag-specific can
avoid the problems. Some new tools, e.g. remove_stale_expectations.py
,
support --flag-specific only.

Note virtual suites are more suitable for testing small sets of web tests
with flags. See this doc

for more details. Also note that for a blink runtime enabled feature that
just adds an API without skipping any existing code path, setting status:
"test" or status: "experimental" in runtime_enabled_features.json5

is
preferred to a virtual suite or a flag-specific configuration.

Xianzhu

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxridWn9rzQz8FgtkuxRDUe0EYdWm8GygDZy%3DzDFmHG3%2BXog%40mail.gmail.com.

Re: [blink-dev] A quick survey on setting up fuzzy match rules to identify resolvable flaky web tests

[Xianzhu Wang]

On Mon, Aug 1, 2022 at 10:36 AM Vivian Zhi (支文文）  wrote:

> Thanks for valuable feedback! Stephen, Xianzhu, will see if we can add a
> filter in result.html to grab those tests in range.
>

The CL <https://chromium-review.googlesource.com/c/chromium/src/+/3803707>
adding pixel diff filter in results.html has landed. Thanks Thorben!

In this example results.html
<https://test-results.appspot.com/data/layout_results/linux-rel/1086271/blink_web_tests%20%28with%20patch%29/layout-test-results/results.html>,
you can examine the pixel results of tests that produced pixel differences
matching a particular fuzzy rule in the following steps:
1. Enter pixel difference filter e.g. "channel_max:1-1" in the filter input
box;
2. Click "All" button (as we show regressions only by default).
You might want to switch to "side-by-side view" and click the image to
examine the pixel values.

With "channel_max:1-1" we can see all tests that produced pixel differences
that can be tolerated with a fuzzy rule like . There are 70 such tests in the example
results.html. All of them look benign to me. So perhaps a universal rule
(for non wpt tests) is proper?

On the other hand, even if we have such a universal rule, we can only
recover 70 tests. Instead of applying the rule automatically, we can also
manually modify these tests to include a meta fuzzy rule.


> On Mon, Aug 1, 2022 at 8:40 AM Xianzhu Wang 
> wrote:
>
>> On Mon, Aug 1, 2022 at 4:25 AM Stephen Chenney 
>> wrote:
>>
>>> Thanks for investigating the potential for fuzzy matching.
>>>
>>> Rendering Core continues to oppose a single fuzzy match rule across all
>>> web_tests. We have some tests where single pixel differences matter
>>> (related to pixel snapping, for example) and a universal fuzzy match would
>>> fail to identify problems with those. This came up in practice recently
>>> when the GPU team enabled fuzzy matching without telling us, and expected
>>> failing tests started passing when they shouldn't.
>>>
>>
>> I think a key difference between the original fuzzy matching rule and the
>> rule proposed by Vivian is the ranges. With maxDifference=0-1, we should be
>> able to catch most visible single pixel differences. What I'm not sure is
>> whether a difference like rgb(1, 0, 0) vs rgb(0, 0, 0) (each component in
>> the range of 0-255) should be treated as a failure in some cases.
>>
>> Maybe specific sub teams have directories they could apply default fuzzy
>>> matching to. My guess is that the same directories where it will work will
>>> be directories with few failing tests, limiting the impact of a
>>> per-directory approach.
>>>
>>> Is there a way to reproduce the sampling below with a side-by-side
>>> comparison of the images? I would find it helpful to look through some of
>>> the cases that would pass with ,
>>> for example.
>>>
>>
>> A filter by actual maxDifference and totalPixels in results.html will be
>> helpful. I can add it when I get time.
>>
>> Stephen.
>>>
>>> On Fri, Jul 29, 2022 at 8:20 PM 'Vivian Zhi (支文文）' via blink-dev <
>>> blink-dev@chromium.org> wrote:
>>>
>>>> Hi blink-dev
>>>>
>>>> I would like to let you know that blink-engprod has added feature
>>>> support for non-WPT fuzzy tests. It now allows both non-WPT reftests and
>>>> pixel tests to use the same fuzzy matching meta-tags as WPT tests.It also
>>>> shows max color channel difference and total number of different pixels
>>>> image diff stats in results.html
>>>> <https://test-results.appspot.com/data/layout_results/linux-rel/1073794/blink_web_tests%20%28with%20patch%29/layout-test-results/results.html>.
>>>> With these capabilities in place, we like to research further to see if we
>>>> can set up some general fuzzy match rules, help blink dev identify flaky
>>>> tests that can be potentially resolved by adjusting fuzzy matching rules.
>>>> Currently there are quite some web tests that are flaky due to a slight
>>>> image mismatch, which should have been tolerated. If we setup a general
>>>> fuzzy matching rule , something like:
>>>>
>>>>  
>>>>
>>>> Instruct the image comparison web tests that if color channel and pixel
>>>> diff fall within the range of the rule, we can ignore the diff and pass the
>>>> test.This way we can reduce test flakiness while still maintain test
>>>> accuracy without missing a real bug.
>>>>
>>

Re: [blink-dev] A quick survey on setting up fuzzy match rules to identify resolvable flaky web tests

[Xianzhu Wang]

On Mon, Aug 1, 2022 at 4:25 AM Stephen Chenney 
wrote:

> Thanks for investigating the potential for fuzzy matching.
>
> Rendering Core continues to oppose a single fuzzy match rule across all
> web_tests. We have some tests where single pixel differences matter
> (related to pixel snapping, for example) and a universal fuzzy match would
> fail to identify problems with those. This came up in practice recently
> when the GPU team enabled fuzzy matching without telling us, and expected
> failing tests started passing when they shouldn't.
>

I think a key difference between the original fuzzy matching rule and the
rule proposed by Vivian is the ranges. With maxDifference=0-1, we should be
able to catch most visible single pixel differences. What I'm not sure is
whether a difference like rgb(1, 0, 0) vs rgb(0, 0, 0) (each component in
the range of 0-255) should be treated as a failure in some cases.

Maybe specific sub teams have directories they could apply default fuzzy
> matching to. My guess is that the same directories where it will work will
> be directories with few failing tests, limiting the impact of a
> per-directory approach.
>
> Is there a way to reproduce the sampling below with a side-by-side
> comparison of the images? I would find it helpful to look through some of
> the cases that would pass with ,
> for example.
>

A filter by actual maxDifference and totalPixels in results.html will be
helpful. I can add it when I get time.

Stephen.
>
> On Fri, Jul 29, 2022 at 8:20 PM 'Vivian Zhi (支文文）' via blink-dev <
> blink-dev@chromium.org> wrote:
>
>> Hi blink-dev
>>
>> I would like to let you know that blink-engprod has added feature support
>> for non-WPT fuzzy tests. It now allows both non-WPT reftests and pixel
>> tests to use the same fuzzy matching meta-tags as WPT tests.It also shows
>> max color channel difference and total number of different pixels image
>> diff stats in results.html
>> .
>> With these capabilities in place, we like to research further to see if we
>> can set up some general fuzzy match rules, help blink dev identify flaky
>> tests that can be potentially resolved by adjusting fuzzy matching rules.
>> Currently there are quite some web tests that are flaky due to a slight
>> image mismatch, which should have been tolerated. If we setup a general
>> fuzzy matching rule , something like:
>>
>>  
>>
>> Instruct the image comparison web tests that if color channel and pixel
>> diff fall within the range of the rule, we can ignore the diff and pass the
>> test.This way we can reduce test flakiness while still maintain test
>> accuracy without missing a real bug.
>>
>> We want to ask you some quick survey questions to help us make design
>> decisions, whether it makes sense to set up an universal cross-the-board
>> fuzzy match tolerant rule for all blink web tests, or we should make the
>> rules more specific to individual test or test sets.
>>
>> 1.  Is an universal fuzzy match tolerant rule acceptable for the web
>> tests in your area?
>>
>> a). If the answer is yes, what is the acceptable range of max color
>> channel and pixel diff for your tests?
>> b) If the answer is no, pls share your reasons.
>>
>> 2. Do you prefer fuzzy matching rule adjustment at a per-test or per test
>> set level based on the pixel difference numbers shown in results.html?
>>
>> Here is some sample data help you make choice, we collected data recently
>> from blink_web_tests result on linux-test builder, the distribution of
>> color channel maxDifference and totalPixel diff for failing/flaky
>> blink_web_tests
>> ( Note: over 70% tests in color channel maxDifference 0-10 range have
>> maxDifference=1):
>>
>> Color Channel maxDifferenece
>> Range Fail test count
>> 0-10 98
>> 11-100 31
>> 101-200 28
>> 201-260 111
>> totalPixels
>> Diff Range
>> Fail test count
>> 0-100 30
>> 100-1000 57
>> 1000-10,000 99
>> 10,000-100,000 66
>> 100,000-1,000,000 16
>>
>> Let me know if you have any questions, looking forward to hearing from
>> you!
>>
>>
>> Vivian
>> on behalf of Chrome-Blink-EngProd
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "blink-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to blink-dev+unsubscr...@chromium.org.
>> To view this discussion on the web visit
>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPCqkTs-L5u22-Xp5U_LeBdLP%3D%2BTDH1KGv8MTmtKQFRcANCZJg%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to bli

[blink-dev] PSA: --ignore-testharness-expected-txt for run_web_tests.py

[Xianzhu Wang]

Hi,

We have more than 2000 WPT tests that actually partially fail but pass
under run_web_tests.py because they have -expected.txt containing failing
results. Now run_web_tests.py has a new option
--ignore-testharness-expected-text to show these failures by ignoring these
-expected.txt. For now this is for local usage (i.e. not on bots), e.g. for
an owner of a WPT subdirectory to know which tests are failing and need
fixing.

In the future we will have a better mechanism to suppress testharness test
failures with better bug tracking.

The flag also applies to non-WPT testharness tests.

Xianzhu

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxrifhOGwiUDkU-iLWmosznP4vhKNbdu%2BVMad3FrJunzk3qw%40mail.gmail.com.

Re: [blink-dev] Developer pain as a result of "introducing blink_wpt_tests"

[Xianzhu Wang]

On Mon, Jun 6, 2022 at 3:52 PM Weizhong Xia  wrote:

> Xianzhu, yes 'baselines' is the name we agreed on previously. The reason I
> later changed back to use 'platform' is because that will make the CL
> smaller, and make it easier for gerrit to handle it. We can make one round
> rename when everything is stabilized. (I left you a message when you are
> OOO. I am not sure if that message lived long enough for you to catch it.)
>

I caught the message. I guessed that the name 'platform' might be one of
the reasons for the surprise to blink developers after the change, and the
name 'baselines' might make the change easier to explain :)

On Mon, Jun 6, 2022 at 3:52 PM Domenic Denicola 
wrote:

> I'm not sure I understand this logic. -expected.txt files are just
> more-convenient, more in-your-face versions of TestExpectations files.
> Surely you're not suggesting we get rid of TestExpectations files?
>

Sorry, "-expected.txt ... should be ... eventually be removed" in my
previous email was not clear. I meant for each individual -expected.txt we
should eventually remove it because we should fix the failure. The same
logic applies to TestExpectations. At any time we may allow a certain
number of failures but we should keep the number as small as possible.

I think we should prefer TestExpectations to -expected.txt for WPT tests
because the entries in TestExpectations have associated bugs which track
the fixing process, unless we find a better way to track the fixing of the
failures in -expected.txt. -expected.txt files do have their values, e.g.
for partially-passing tests we can discover regressions and progressions of
individual sub tests, but they should be rare.

I think separating -expected.txt from the tests has the following benefits:
- It makes it clear to blink developers that the files are not a part of
WPT.
- It simplifies the WPT export/import process and others by reducing
blink-specific files under external/wpt.

It does make it more difficult to find -expected.txt, but we already have
the similar well-known logic for platform-specific baselines. Though
platform-specific baselines are rare, ignoring a platform baseline can
still cause surprises.

I think we can improve the test result viewer
<https://test-results.appspot.com/data/layout_results/Mac10_15_Tests/26148/blink_web_tests/layout-test-results/results.html>
- to better show -expected.txt for passing tests
- to show information about tests without actually running the tests
WDYT?


>
> On Mon, Jun 6, 2022 at 6:30 PM Xianzhu Wang 
> wrote:
>
>> I think we first need to answer a question: Why do we need *-expected.txt
>> for WPT tests?
>>
>> Upstream WPT doesn't have *-expected.txt. *-expected.txt is a
>> blink-specific thing to allow some failing WPT tests to pass on blink with
>> temporarily allowed failures. If a test has -expected.txt, it means Chrome
>> behaves differently than the standard web platform behavior (or the test
>> itself is wrong). The files are sometimes harmful because they hide
>> failures and web platform incompatibilities (e.g.
>> <http://crbug.com/772405>). So I think -expected.txt files for WPT tests
>> should be rare and eventually be removed. An -expected.txt should not be
>> treated as a part of the test itself because a) it doesn't exist in
>> upstream WPT and 2) it doesn't describe the standard expected behavior,
>> thus isn't necessarily placed besides the test.
>>
>> The directory name 'platform' may be misleading. 'baselines' may be a
>> better name (but we should not rename until we decide what to do for
>> generic baselines). For WPT tests, 'failures' is perhaps an even better
>> name.
>>
>> On Friday, June 3, 2022 at 6:55:49 AM UTC-7 Dominic Farolino wrote:
>>
>>> For the long term, we can definitely move this back once we have
>>>> separate legacy tests and wpt into different folders. I can also reach out
>>>> to you guys to better understand your needs.
>>>>
>>>> For the folders Dom mentioned, I can check to see if any of that can be
>>>> removed. I understand the README is needed for virtual test suites.
>>>>
>>>
>>> Can you define "long term" here? Is there a timeline? I was really
>>> hoping that we'd go back to how things were almost immediately. Still the
>>> need for the delay does not quite make sense to me, and I'm really
>>> hoping that for Blink developer experience we can revert this back to the
>>> previous setup ASAP.
>>>
>>> Under "web_tests/virtual" we have baselines for wpt and pure virtual
>>>> legacy tes

Re: [blink-dev] Developer pain as a result of "introducing blink_wpt_tests"

[Xianzhu Wang]

I think we first need to answer a question: Why do we need *-expected.txt 
for WPT tests?

Upstream WPT doesn't have *-expected.txt. *-expected.txt is a 
blink-specific thing to allow some failing WPT tests to pass on blink with 
temporarily allowed failures. If a test has -expected.txt, it means Chrome 
behaves differently than the standard web platform behavior (or the test 
itself is wrong). The files are sometimes harmful because they hide 
failures and web platform incompatibilities (e.g. ). 
So I think -expected.txt files for WPT tests should be rare and eventually 
be removed. An -expected.txt should not be treated as a part of the test 
itself because a) it doesn't exist in upstream WPT and 2) it doesn't 
describe the standard expected behavior, thus isn't necessarily placed 
besides the test.

The directory name 'platform' may be misleading. 'baselines' may be a 
better name (but we should not rename until we decide what to do for 
generic baselines). For WPT tests, 'failures' is perhaps an even better 
name.

On Friday, June 3, 2022 at 6:55:49 AM UTC-7 Dominic Farolino wrote:

> For the long term, we can definitely move this back once we have separate 
>> legacy tests and wpt into different folders. I can also reach out to you 
>> guys to better understand your needs.
>>
>> For the folders Dom mentioned, I can check to see if any of that can be 
>> removed. I understand the README is needed for virtual test suites.
>>
>
> Can you define "long term" here? Is there a timeline? I was really hoping 
> that we'd go back to how things were almost immediately. Still the need for 
> the delay does not quite make sense to me, and I'm really hoping that for 
> Blink developer experience we can revert this back to the previous setup 
> ASAP.
>
> Under "web_tests/virtual" we have baselines for wpt and pure virtual 
>> legacy tests, we can list the folders under "virtual/prefix" for each 
>> virtual test suite, which will make the dependency very large(yes), and we 
>> need make such change each time when we add new virtual suites. I don't 
>> think this is something blink devs want to do.
>>
>
> I'm sorry, but like Domenic I am not sure what to make of much of this. 
> What I do know is that Blink developers also don't want the current WPT 
> writing experience where expectation files are positioned far away from the 
> source test. I think this experience matters a lot.
>
> On Thu, Jun 2, 2022 at 4:25 PM Weizhong Xia  wrote:
>
>> Previously you don't need to specify anything in BUILD.gn is because we 
>> are downloading the whole "web_tests" folder. Now we want to run legacy 
>> tests and wpt in different steps. Under "web_tests/virtual" we have 
>> baselines for wpt and pure virtual legacy tests, we can list the folders 
>> under "virtual/prefix" for each virtual test suite, which will make the 
>> dependency very large(yes), and we need make such change each time when we 
>> add new virtual suites. I don't think this is something blink devs want to 
>> do.
>>
>> The discussion is at the beginning of the crbug. So pls scroll back to 
>> #c1, and read from there.
>>
>> thanks, Weizhong
>>
>> On Thu, Jun 2, 2022 at 12:29 PM Domenic Denicola  
>> wrote:
>>
>>>
>>>
>>> On Thu, Jun 2, 2022 at 3:21 PM Weizhong Xia  wrote:
>>>
 Folks

 I'm sorry to see this has caused inconvenience, and sorry for being 
 late in response to this, due to the same reason Dom had.

>>>
>>> Thanks for responding and listening to our concerns!
>>>  
>>>

 The reason to move baselines to one central place at this point is to 
 make it possible to specify dependency in BUILD.gn.

>>>
>>> I don't quite understand this. I've had to work with WPTs and WPT 
>>> expectations my entire time working on Chromium. I've never had to "specify 
>>> dependency in BUILD.gn"; I don't really know what that means. I'd love to 
>>> hear more (or be referred to a doc explaining the issue), so that I 
>>> understand why we're making the sacrifice we're making. (The linked bug 
>>> isn't very understandable for me, unfortunately. I can't even understand 
>>> enough to find the part you mentioned where you discussed different 
>>> approaches.)
>>>
>>> I'm also unsure how the decision was weighed. Is the population of 
>>> people specifying dependency in BUILD.gn very large, so that their needs 
>>> are outweighing those of the Chromium developers working with web platform 
>>> tests?
>>>  
>>>
 This is part of the work to use upstream wptrunner to run wpt tests. In 
 crbug/1299834  we have tried to discuss 
 some different approaches. I would say this is the least disruptive way. 
 For blink engprod, I think to make blink devs happy is our top priority.

 For the long term, we can definitely move this back once we have 
 separate legacy tests and wpt into different folders. I can also reach out 
 to you guys to better understand your needs.

Re: [blink-dev] Question : How to create *-expected.png for blink_web_tests

[Xianzhu Wang]

   - blink_tool.py rebaseline-cl: can rebaseline for all platforms.
   - run_web_tests.py --reset-results: can rebaseline for your current
   platform, or if the baseline is not platform-dependent.
   - rebaseline script in results.html: can rebaseline for a particular
   platform (which can be different from your current platform) at a time, or
   if the baseline is not platform-dependent.

See the how-to-rebaseline
document
for more details about the above options.

On Mon, Mar 14, 2022 at 9:01 PM Jihwan Kim  wrote:

> Thanks for help!
>
> I have one more question.
> There are 3 platforms on WPT. (Linux, Mac, Windows)
> In this case, to create new *-expected.png, should I have all these three
> devices?
>
> 2022년 3월 14일 월요일 오후 4시 41분 30초 UTC+9에 Xianzhu Wang님이 작성:
>
>> See also:
>> https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/web_test_expectations.md#How-to-rebaseline
>> for other options.
>>
>> On Mon, Mar 14, 2022 at 3:32 PM 'Mathias Bynens' via blink-dev <
>> blin...@chromium.org> wrote:
>>
>>> Use --reset-results:
>>> https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/web_tests.md#test-harness-options
>>>
>>> On Mon, Mar 14, 2022 at 7:39 AM Jihwan Kim  wrote:
>>>
 I'm fixing issue with pickers, but some tests failed.
 It was a failure due to image diff.

 Since the resource (css) file is modified, it is correct that the image
 diff appears,
 So I have to modify the some *-expected.png files.
 (ex: datetimelocal-month-year-selector-expected.png)

 And my question is -
 Are there any rules or tools to create *-expected.png?
 or is it manual work (crop after screen capture, etc.)?

 This is my first time about an image comparison test,
 so please understand even if the question is too basic.

 --
 You received this message because you are subscribed to the Google
 Groups "blink-dev" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to blink-dev+...@chromium.org.
 To view this discussion on the web visit
 https://groups.google.com/a/chromium.org/d/msgid/blink-dev/dee25422-6fd2-4d08-9883-d9b3d345c7fen%40chromium.org
 
 .

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "blink-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to blink-dev+...@chromium.org.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADizRgbbezvsx1iF5rqKgiPMAikZg74sTwiK1E4BNN4TtjAEWg%40mail.gmail.com
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxricvTvK1vR38dbCX7dPYpaqx3vRLkJAY_a7f1Dy2pDyy7w%40mail.gmail.com.

Re: [blink-dev] Question : How to create *-expected.png for blink_web_tests

[Xianzhu Wang]

See also:
https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/web_test_expectations.md#How-to-rebaseline
for other options.

On Mon, Mar 14, 2022 at 3:32 PM 'Mathias Bynens' via blink-dev <
blink-dev@chromium.org> wrote:

> Use --reset-results:
> https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/web_tests.md#test-harness-options
>
> On Mon, Mar 14, 2022 at 7:39 AM Jihwan Kim 
> wrote:
>
>> I'm fixing issue with pickers, but some tests failed.
>> It was a failure due to image diff.
>>
>> Since the resource (css) file is modified, it is correct that the image
>> diff appears,
>> So I have to modify the some *-expected.png files.
>> (ex: datetimelocal-month-year-selector-expected.png)
>>
>> And my question is -
>> Are there any rules or tools to create *-expected.png?
>> or is it manual work (crop after screen capture, etc.)?
>>
>> This is my first time about an image comparison test,
>> so please understand even if the question is too basic.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "blink-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to blink-dev+unsubscr...@chromium.org.
>> To view this discussion on the web visit
>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/dee25422-6fd2-4d08-9883-d9b3d345c7fen%40chromium.org
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADizRgbbezvsx1iF5rqKgiPMAikZg74sTwiK1E4BNN4TtjAEWg%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxrieotayGZT4aDvQh1Jpexe6Hd6jb35%3Dp-090aGyXSB1LBw%40mail.gmail.com.

[blink-dev] PSA: Blink is now using gfx int/float geometry types

[Xianzhu Wang]

Hi, blink-dev,

As the second part of crbug.com/738465, we have finished migrating blink to
use gfx int/float geometry types:

   - blink::IntRect -> gfx::Rect
   - blink::IntPoint -> gfx::Point (if it's a point) or gfx::Vector2d (if
   it's an offset)
   - blink::IntSize -> gfx::Size (if it's a size) or gfx::Vector2d (if it's
   an offset)
   - blink::FloatRect -> gfx::RectF
   - blink::FloatPoint -> gfx::PointF (if it's a point) or gfx::Vector2dF
   (if it's an offset)
   - blink::FloatSize -> gfx::SizeF (if it's a size) or gfx::Vector2dF (if
   it's an offset)
   - blink::FloatBox -> gfx::BoxF
   - blink::FloatQuad -> gfx::QuadF
   - blink::FloatPoint3D -> gfx::Point3F (if it's a point) or
   gfx::Vector3dF (if it's an offset)
   - blink::IntRectOutsets -> gfx::Outsets or gfx::Insets
   - blink::FloatRectOusets -> gfx::OutsetsF or gfx::Outsets

Most differences between the blink types and gfx types are straightforward,
but the following are notable:

   - gfx::Size, gfx::SizeF, gfx::Rect and gfx::RectF clamp negative
   width/height to zero.
   - All gfx integer geometry types clamp values to integer range. Some
   stored values are clamped to prevent overflow of calculated values. For
   example, gfx::Rect clamps also clamps width to prevent overflow of right
   (x+width).

Cheers,
Xianzhu

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxricqKD4J%2Bvm_d8bpcD9QX6Ux-3qSctX_YHN-bPU9Fc-zxw%40mail.gmail.com.

Re: [blink-dev] Questions about WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS

[Xianzhu Wang]

On Thu, Nov 4, 2021 at 11:38 AM Xianzhu Wang 
wrote:

> Actually we have only two remaining usages of such macros in blink
> geometry types:
> 1. float_size.h
> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/geometry/float_size.h;drc=134f570f0aff185007cd8a2757f4188567a7df9f;l=240>
> :
>   // Allows this class to be stored in a HeapVector.
>   WTF_ALLOW_CLEAR_UNUSED_SLOTS_WITH_MEM_FUNCTIONS(blink::FloatSize)
>   This seems unnecessary because FloatSize is not a garbage collected
> type, and we don't have any usage of Vector in blink.
> 2. int_rect.h
> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/geometry/int_rect.h;drc=134f570f0aff185007cd8a2757f4188567a7df9f;l=272>
> :
>   WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS(blink::IntRect)
>   We have a lot of Vector usages. Will try perf tests.
>

Perf tests didn't show an obvious difference with
WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS(blink::IntRect) removed.
The results actually show progressions of total rendering CPU usage, but
they are likely to be just noise. See the pinpoint results in
https://chromium-review.googlesource.com/c/chromium/src/+/3262556 for
details.

So I'm going forward to migrate blink geometry types to gfx types without
worrying about WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS on
geometry types.

About std::is_trivial, actually we don't allow particular trivial types to
> use such macros. For example, in
> WTF_ALLOW_MOVE_AND_INIT_WITH_MEM_FUNCTIONS, we have:
>static_assert(!std::is_trivially_default_constructible::value
> || \
> !std::is_trivially_move_assignable::value, \
> "macro not needed"); \
>
> Also most of our classes (including all geometry classes) are not
> trivially default constructible because they do need initialization. We
> could check if a default constructed object is all zero to see if it can be
> initialized by memset, but I'm not sure if this is possible at compile time.
>
> It seems that we can check std::is_trivially_copyable and/or
> std::is_trivially_move_assignable for kCan*WithMemcpy
> and kCanCompareWithMemcmp.
>

Actually we are already doing this
<https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/wtf/vector_traits.h;drc=c31166996f0c14c8d88f2ca2fdd1bbfce0f9f6f0;l=51>
:)


>
> On Thu, Nov 4, 2021 at 10:39 AM Xianzhu Wang 
> wrote:
>
>> One data point: the CL
>> <https://chromium-review.googlesource.com/c/chromium/src/+/3252774> that
>> replaces blink::IntPoint (declared with the macro) with gfx::Point was
>> landed 2 days ago, and I haven't received any performance bug yet. Perhaps
>> we don't use Vector in performance critical code.
>>
>> Will try std::is_trivial before converting other types.
>>
>> On Thu, Nov 4, 2021 at 10:19 AM Daniel Bratell 
>> wrote:
>>
>>> (see inline)
>>> On 2021-11-03 12:15, Kentaro Hara wrote:
>>>
>>> The VectorTraits::kCan{Copy,Move,Fill,Compare}... fields are used for
>>>> all Vectors though, no? This particular macro doesn't appear to
>>>> explicitly define/override any Oilpan-related fields AFAICS.
>>>
>>>
>>> You're right -- thanks for pointing it out :)
>>>
>>> a) The macros work as a performance optimization for all Vectors. I
>>> have no data about the performance numbers though.
>>>
>>> The traits have a big effect on micro-benchmarks but it's always hard to
>>> map that to user observable performance.
>>>
>>> Furthermore, is it possible to replace the macros with use of standard
>>> C++ traits such as std::is_trivial
>>> <http://www.cplusplus.com/reference/type_traits/is_trivial/>? It would
>>> require auditing current types, and possibly modifying them, but that is
>>> something that has to be done anyway before a complete type merge between
>>> Blink and
>>>
>>> /Daniel
>>>
>>>
>>> b) Some of the macros are mandatory to make HeapVector work correctly
>>> (e.g., see static_assert here
>>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/heap/collection_support/heap_vector_backing.h;l=81?q=kCanClearUnusedSlotsWithMemset&ss=chromium%2Fchromium%2Fsrc:third_party%2Fblink%2Frenderer%2Fplatform%2F>
>>> ).
>>>
>>> b) does not apply to the geometry types in question. Regarding a), maybe
>>> can we just measure performance using benchmarks and see if dropping the
>>> macros causes performance regr

Re: [blink-dev] Questions about WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS

[Xianzhu Wang]

Actually we have only two remaining usages of such macros in blink geometry
types:
1. float_size.h
<https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/geometry/float_size.h;drc=134f570f0aff185007cd8a2757f4188567a7df9f;l=240>
:
  // Allows this class to be stored in a HeapVector.
  WTF_ALLOW_CLEAR_UNUSED_SLOTS_WITH_MEM_FUNCTIONS(blink::FloatSize)
  This seems unnecessary because FloatSize is not a garbage collected type,
and we don't have any usage of Vector in blink.
2. int_rect.h
<https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/geometry/int_rect.h;drc=134f570f0aff185007cd8a2757f4188567a7df9f;l=272>
:
  WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS(blink::IntRect)
  We have a lot of Vector usages. Will try perf tests.

About std::is_trivial, actually we don't allow particular trivial types to
use such macros. For example, in
WTF_ALLOW_MOVE_AND_INIT_WITH_MEM_FUNCTIONS, we have:
   static_assert(!std::is_trivially_default_constructible::value
|| \
!std::is_trivially_move_assignable::value, \
"macro not needed"); \

Also most of our classes (including all geometry classes) are not trivially
default constructible because they do need initialization. We could check
if a default constructed object is all zero to see if it can be initialized
by memset, but I'm not sure if this is possible at compile time.

It seems that we can check std::is_trivially_copyable and/or
std::is_trivially_move_assignable for kCan*WithMemcpy
and kCanCompareWithMemcmp.


On Thu, Nov 4, 2021 at 10:39 AM Xianzhu Wang 
wrote:

> One data point: the CL
> <https://chromium-review.googlesource.com/c/chromium/src/+/3252774> that
> replaces blink::IntPoint (declared with the macro) with gfx::Point was
> landed 2 days ago, and I haven't received any performance bug yet. Perhaps
> we don't use Vector in performance critical code.
>
> Will try std::is_trivial before converting other types.
>
> On Thu, Nov 4, 2021 at 10:19 AM Daniel Bratell 
> wrote:
>
>> (see inline)
>> On 2021-11-03 12:15, Kentaro Hara wrote:
>>
>> The VectorTraits::kCan{Copy,Move,Fill,Compare}... fields are used for
>>> all Vectors though, no? This particular macro doesn't appear to
>>> explicitly define/override any Oilpan-related fields AFAICS.
>>
>>
>> You're right -- thanks for pointing it out :)
>>
>> a) The macros work as a performance optimization for all Vectors. I
>> have no data about the performance numbers though.
>>
>> The traits have a big effect on micro-benchmarks but it's always hard to
>> map that to user observable performance.
>>
>> Furthermore, is it possible to replace the macros with use of standard
>> C++ traits such as std::is_trivial
>> <http://www.cplusplus.com/reference/type_traits/is_trivial/>? It would
>> require auditing current types, and possibly modifying them, but that is
>> something that has to be done anyway before a complete type merge between
>> Blink and
>>
>> /Daniel
>>
>>
>> b) Some of the macros are mandatory to make HeapVector work correctly
>> (e.g., see static_assert here
>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/heap/collection_support/heap_vector_backing.h;l=81?q=kCanClearUnusedSlotsWithMemset&ss=chromium%2Fchromium%2Fsrc:third_party%2Fblink%2Frenderer%2Fplatform%2F>
>> ).
>>
>> b) does not apply to the geometry types in question. Regarding a), maybe
>> can we just measure performance using benchmarks and see if dropping the
>> macros causes performance regressions? (I hope not.)
>>
>>
>> On Wed, Nov 3, 2021 at 7:19 PM Fredrik Söderquist  wrote:
>>
>>> On Wed, Nov 3, 2021 at 12:29 AM Kentaro Hara 
>>> wrote:
>>>
>>>> The macro matters only for objects stored in HeapVector (i.e.
>>>> Oilpan). So you don't need to export the macro outside Blink :)
>>>>
>>>
>>> The VectorTraits::kCan{Copy,Move,Fill,Compare}... fields are used for
>>> all Vectors though, no? This particular macro doesn't appear to
>>> explicitly define/override any Oilpan-related fields AFAICS.
>>>
>>>
>>> /fs
>>>
>>>
>>>>
>>>>
>>>> 2021年11月3日(水) 8:04 Xianzhu Wang :
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm migrating blink to use gfx geometry types, and noticed
>>>>> that WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS is applied to 
>>>>> blink
>>>>> geometry types. It seems

Re: [blink-dev] Questions about WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS

[Xianzhu Wang]

One data point: the CL
<https://chromium-review.googlesource.com/c/chromium/src/+/3252774> that
replaces blink::IntPoint (declared with the macro) with gfx::Point was
landed 2 days ago, and I haven't received any performance bug yet. Perhaps
we don't use Vector in performance critical code.

Will try std::is_trivial before converting other types.

On Thu, Nov 4, 2021 at 10:19 AM Daniel Bratell  wrote:

> (see inline)
> On 2021-11-03 12:15, Kentaro Hara wrote:
>
> The VectorTraits::kCan{Copy,Move,Fill,Compare}... fields are used for
>> all Vectors though, no? This particular macro doesn't appear to
>> explicitly define/override any Oilpan-related fields AFAICS.
>
>
> You're right -- thanks for pointing it out :)
>
> a) The macros work as a performance optimization for all Vectors. I
> have no data about the performance numbers though.
>
> The traits have a big effect on micro-benchmarks but it's always hard to
> map that to user observable performance.
>
> Furthermore, is it possible to replace the macros with use of standard C++
> traits such as std::is_trivial
> <http://www.cplusplus.com/reference/type_traits/is_trivial/>? It would
> require auditing current types, and possibly modifying them, but that is
> something that has to be done anyway before a complete type merge between
> Blink and
>
> /Daniel
>
>
> b) Some of the macros are mandatory to make HeapVector work correctly
> (e.g., see static_assert here
> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/heap/collection_support/heap_vector_backing.h;l=81?q=kCanClearUnusedSlotsWithMemset&ss=chromium%2Fchromium%2Fsrc:third_party%2Fblink%2Frenderer%2Fplatform%2F>
> ).
>
> b) does not apply to the geometry types in question. Regarding a), maybe
> can we just measure performance using benchmarks and see if dropping the
> macros causes performance regressions? (I hope not.)
>
>
> On Wed, Nov 3, 2021 at 7:19 PM Fredrik Söderquist  wrote:
>
>> On Wed, Nov 3, 2021 at 12:29 AM Kentaro Hara 
>> wrote:
>>
>>> The macro matters only for objects stored in HeapVector (i.e.
>>> Oilpan). So you don't need to export the macro outside Blink :)
>>>
>>
>> The VectorTraits::kCan{Copy,Move,Fill,Compare}... fields are used for
>> all Vectors though, no? This particular macro doesn't appear to
>> explicitly define/override any Oilpan-related fields AFAICS.
>>
>>
>> /fs
>>
>>
>>>
>>>
>>> 2021年11月3日(水) 8:04 Xianzhu Wang :
>>>
>>>> Hi,
>>>>
>>>> I'm migrating blink to use gfx geometry types, and noticed
>>>> that WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS is applied to blink
>>>> geometry types. It seems to optimize performance of Vector.
>>>>
>>>> I have the following questions:
>>>>
>>>> 1. For a type defined out of blink, how do we force users to include a
>>>> header delaring WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS(Type)?
>>>> I'm thinking of putting it in vector_traits.h
>>>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/wtf/vector_traits.h>,
>>>> but I would like to know if there are better ways.
>>>>
>>>> 2. Do we have data showing the performance difference with and without
>>>> WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS?
>>>>
>>>> Thanks,
>>>> Xianzhu
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "blink-dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to blink-dev+unsubscr...@chromium.org.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxriepZahOVJfzQseAyFfkjUPUgLWovXrKUYH54UGY6K2mEw%40mail.gmail.com
>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxriepZahOVJfzQseAyFfkjUPUgLWovXrKUYH54UGY6K2mEw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "blink-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to blink-dev+unsubscr...@chromium.org.
>>> To view this discussion on the web visit
>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/C

[blink-dev] Questions about WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS

[Xianzhu Wang]

Hi,

I'm migrating blink to use gfx geometry types, and noticed
that WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS is applied to blink
geometry types. It seems to optimize performance of Vector.

I have the following questions:

1. For a type defined out of blink, how do we force users to include a
header delaring WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS(Type)?
I'm thinking of putting it in vector_traits.h
,
but I would like to know if there are better ways.

2. Do we have data showing the performance difference with and without
WTF_ALLOW_MOVE_INIT_AND_COMPARE_WITH_MEM_FUNCTIONS?

Thanks,
Xianzhu

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADBxriepZahOVJfzQseAyFfkjUPUgLWovXrKUYH54UGY6K2mEw%40mail.gmail.com.

Re: [blink-dev] Slow tests that timeouts

[Xianzhu Wang]

I see the CL changed the expectation of some tests from [ Timeout ] to [
Slow ], and the tests still timeout. To make the bot green, we need to
choose from:
a. Add back the [ Timeout ] entries in TestExpectations with the [ Slow ]
entries in SlowTests;
b. Revert the change for the tests (i.e. keep the [ Timeout ] entries in
TestExpectations without [ Slow ] entries in SlowTests);
c. Add [ Skip ] for the test. We can keep both [ Timeout ] and [ Slow ] as
they don't matter if there is [ Skip ], but they partly describe the reason
for the skip.

I think Weizhong's proposal is to handle #a which makes the tests
unnecessarily take a long time before timeout. Proposal #1 changes #a to
#c, and proposal #2 disallows #a.

For now it's not easy to check if a timing-out test is just slow or it
hangs indefinitely. @Brian Sheedy  is working on a
project  to automatically update TestExpectations
based on bot history by removing stale failure/flaky entries for passing
tests, to recover test coverage. I think the project could be extended in
the future to handle slow/timeout entries based on bot history. For
example, we can force running tests with Skip, and try bigger timeout for
Timeout, on some bots periodically, to check 1) if a timing-out test is
just slow, and 2) if a skipped test can be unskipped, and update test
expectations automatically.

On Thu, Aug 19, 2021 at 7:42 AM Stephen Chenney 
wrote:

>
>
> On Wed, Aug 18, 2021 at 3:06 PM Weizhong Xia  wrote:
>
>> >> I recently found a test that was TIMEOUT in TestExpectations, and
>> still timed out in Slow, so I left it in TestExpectations under the
>> assumption that it will timeout faster there.
>> What do you mean "timed out in Slow"? A test did not mark as slow, but
>> timeout after 60 seconds? Worth mentioning some slow wpt tests are given
>> the same timeout value as slow tests in the code.
>>
>
> See https://chromium-review.googlesource.com/c/chromium/src/+/3097709/1
> and the blink_win7 try job.
>
> I moved the expectation
> for external/wpt/css/css-fonts/test_font_family_parsing.html from the
> TestExpectations file to the SlowTests file and ran the try jobs. The test
> was still slower than the slow test timeout, and given the console output I
> strongly suspect the test hangs indefinitely.
>
>
>> But I did see some tests timed out at some abnormal value. For example
>> tests below timeouts at 22 second. Not sure how this could happen.
>> 19:13:50.493 20120 [3522/4178] 
>> external/wpt/webrtc-identity/idlharness.https.window.html
>> failed unexpectedly (test timed out) 22.3300s
>>
>> On Wed, Aug 18, 2021 at 11:26 AM Stephen Chenney 
>> wrote:
>>
>>>
>>>
>>> On Wed, Aug 18, 2021 at 12:01 PM 'Weizhong Xia' via blink-dev <
>>> blink-dev@chromium.org> wrote:
>>>
 Dear blink-devs,

 As another effort trying to speed up blink_web_tests on CQ, We are
 looking at slow tests that also timeouts. From the timing stats(example
 )
 and some research, there are about 0.25% such tests, but running such tests
 takes about 4% of the total run time. So we are thinking of
 doing something about it.

 We would like to propose as below, and get feedback on this:
 1. We add Skip to test expectations for slow tests that also timeouts.
 This will effectively skip all such tests on swarming bots, and save ~50s
 real time on linux-rel.
 2. We add a presubmit check to make any unskipped Slow+Timeout an
 error. (We can also make this a warning, but such warning will persist, so
 kind of annoying)
 3. Devs are encouraged to take actions to either fix the timeout, or
 break the slow tests into pieces to make it not slow.


>>> I recently found a test that was TIMEOUT in TestExpectations, and still
>>> timed out in Slow, so I left it in TestExpectations under the assumption
>>> that it will timeout faster there.
>>>
>>> The underlying issue is not a slow test but a test that hangs and we may
>>> or may not want to skip such a test. Would identifying these cases and
>>> moving them from Slow to TestExpectations have any noticeable impact?
>>>
>>>
 Please let us know your opinions on this.

 Thanks, Weizhong

 --
 You received this message because you are subscribed to the Google
 Groups "blink-dev" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to blink-dev+unsubscr...@chromium.org.
 To view this discussion on the web visit
 https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSipU3NDSp7tWd%3DqA%3DrZWMrmXjfT4NzTG7U8EoEAgsQ1QXw%40mail.gmail.com
 
 .

>>>

-- 
You received thi

Re: [blink-dev] PSA: Introduce collect_web_tests.py for web test

[Xianzhu Wang]

Can we make this a standalone step or run it in the "isolate test" step on
the bot? It's like a preparation before sharding blink_web_tests, to reduce
the duplicated work on the swarming bots.

On Tue, Aug 17, 2021 at 11:17 AM Domenic Denicola 
wrote:

>
>
> On Tue, Aug 17, 2021 at 2:08 PM 'Weizhong Xia' via blink-dev <
> blink-dev@chromium.org> wrote:
>
>> Running collect_web_tests on external/wpt is kind of slow, because we
>> need to generate wpt manifest first. Running that on other folders will be
>> fast. And I will take a look to see if there is room for improvement.
>>
>> One thing worth mentioning is that if the change in external/wpt is meant
>> to be exported to upstream ,
>> you can ignore the presubmit check warning. (Should have mentioned this in
>> the PSA).
>>
>
> Aren't all changes in external/wpt meant to be exported to upstream? From
> what I understand that's the purpose of the directory...
>
>
>>
>> On Tue, Aug 17, 2021 at 11:01 AM Dirk Pranke  wrote:
>>
>>> On Tue, Aug 17, 2021 at 10:48 AM Ian Kilpatrick <
>>> ikilpatr...@chromium.org> wrote:
>>>
 So I was a little surprised by this last night.

>>>
>>> Yeah, we probably should've had the discussion first, sorry :(.
>>>
>>>

 1. This was relatively slow on my macbook, running locally takes ~40s.
 From just now:

 ikilpatrick-macbookpro:src ikilpatrick$ touch
 third_party/blink/web_tests/external/wpt/css/css-tables/test.html

 ikilpatrick-macbookpro:src ikilpatrick$ time
 third_party/blink/tools/collect_web_tests.py external/wpt


 real 0m40.492s

 user 0m23.007s

 sys 0m41.715s

 2. I had a bunch of uncommitted tests in various wpt directories that I
 needed to move out to somewhere else (rather than the script just adding
 committed tests).

>>>
>>> One other concern is that we might end up with a lot of merge conflicts
>>> in a single file, and so we might want to shard by the top level
>>> directories or something ...
>>>
>>> (I'd also note that uploading patches is already relatively slow, do we
 have metrics for how long developers are waiting for upload scripts to run?
 - I suspect this is due to WPT presubmit scripts but unsure)

>>>
>>> The upload checks are just generally crazy slow these days. I'm not sure
>>> if we have collected metrics for them, but things are an order of magnitude
>>> slower than I'd like them to be.
>>>
>>> But that's partially a separate topic (apart from this check, of course).
>>>
>>> -- Dirk
>>>
>>>
 Ian




 On Tue, Aug 17, 2021 at 10:20 AM 'Dirk Pranke' via blink-dev <
 blink-dev@chromium.org> wrote:

>
>
> On Tue, Aug 17, 2021 at 10:08 AM 'Weizhong Xia' via blink-dev <
> blink-dev@chromium.org> wrote:
>
>> tl;dr: We introduced a mechanism that pre-collects test cases to
>> //third_party/blink/web_tests/AllTestsByDirectories.json, and uses a
>> presubmit check to ensure this file is up to date. You can stop here if 
>> you
>> don't write web test cases.
>>
>> Dear blink-devs,
>>
>> You may have noticed that when you try to upload a CL, you are
>> prompted to update AllTestsByDirectories.json if your CL changes web test
>> cases. Yes, today we landed a CL
>> 
>> that pre-collects test cases and saves that to the json file. The reason 
>> to
>> do this is that rwt tries to collect tests on every swarming bots, and 
>> that
>> takes about 30 seconds (the best case when running on SSD). This is not a
>> small amount of time as we are targeting a 10-min CQ. After this CL 
>> lands,
>> collecting tests now takes less than 2 seconds for all platforms.
>>
>> Impact to you: if your CL adds/deletes/renames web test cases, you
>> will be prompted. Follow the instructions there should be enough. But if
>> you still have an issue, (even though we have tested many different
>> scenarios we can think about), feel free to ping/email me.
>>
>> There is no change to how you run rwt locally. We added a new switch
>> to tell the swarming bots to load tests from the json file. This switch 
>> is
>> turned off by default. Developers usually don't need to turn this on,
>> unless all of your tests are saved to the json file and you are running a
>> full test so want to save about 30 seconds.
>>
>
> FWIW, I think this is an important and probably worthwhile change
> (disclaimer: I did review it and was involved in the design). The number 
> of
> tests we have now is substantial and the startup delay is a significant
> cost. We need to be looking into ways to reduce this.
>
> I think it's possible we should turn this on by default and/or do some
> hooking to make things aware of when you've added