Re: [webkit-dev] Review request time limit

[Michael Catanzaro via webkit-dev]

On Mon, Nov 1 2021 at 01:30:28 PM -0700, Alex Christensen via 
webkit-dev  wrote:
I just removed r? on all the bugs in http://webkit.org/pending-review 
that had requested review in 2018 or before and had been untouched 
since then.  I imagine that did not interrupt anyone’s work.  I was 
thinking of removing review requests on bugs that hadn’t been 
updated in one full year to maintain the usefulness of the review 
queue.  Does anyone have any strong opinions that that is too much 
time or too little time?  Most reviews are done in a few days or 
weeks, but occasionally something useful sits in the review queue for 
a few months.


$0.02:

Removing the flag acts as a nice reminder: "hey, you need to find a 
reviewer for this patch!" So the sooner the better IMO. If we were to 
automate this, I would do a warning comment after one month, then strip 
r? after two months. Something like that.


/$0.02


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Review request time limit

[Alex Christensen via webkit-dev]

I just removed r? on all the bugs in http://webkit.org/pending-review 
 that had requested review in 2018 or before 
and had been untouched since then.  I imagine that did not interrupt anyone’s 
work.  I was thinking of removing review requests on bugs that hadn’t been 
updated in one full year to maintain the usefulness of the review queue.  Does 
anyone have any strong opinions that that is too much time or too little time?  
Most reviews are done in a few days or weeks, but occasionally something useful 
sits in the review queue for a few months.___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Fuzzy Reftest Plans, and Metadata Locations

[Simon Fraser via webkit-dev]

> On Oct 30, 2021, at 10:45 AM, Ryosuke Niwa via webkit-dev 
>  wrote:
> 
> On Thu, Oct 28, 2021 at 10:24 AM Sam Sneddon via webkit-dev 
> mailto:webkit-dev@lists.webkit.org>> wrote:
> As part of the ongoing work on GPU Process, we’re interested in adding 
> support for reftest fuzzy matching (i.e., allowing a certain amount of 
> tolerance when comparing the generated images).
> 
> Our intention is to match the semantics of WPT’s reftests 
> (https://web-platform-tests.org/writing-tests/reftests.html#fuzzy-matching 
> ):
> 
> There are cases where we’ll want to apply these to the tests unconditionally, 
> for example where varying behaviour is expected across ports (such as 
> anti-aliasing differences), and in these cases for WPT tests these 
> annotations should probably be exported upstream.
> 
> The current plan, and work is underway to do this, is to support this syntax 
> via parsing the HTML in Python when there is a hash mismatch, which should 
> minimise the performance impact versus always reading this metadata.
> 
> However, this doesn’t entirely suffice. There are cases where we might want 
> to allow more tolerance on one platform or another, or vary based on GPU 
> model or driver. As such, this requires not only platform specific metadata 
> (i.e., similar to that which we have in TestExpectations files today), but 
> also expectations with finer granularity.
> 
> Are we sure we really need that? What are examples of tests that do warrant 
> such a mechanism?
> 
> Generally, we want to keep our testing infrastructure as simple as possible.
> 
> One option is to extend the meta content to encode conditional variants, 
> though this doesn’t work for WPT tests (unless we get buy-in to upstream 
> these annotations into the upstream repo, though that might be desirable for 
> the sake of results on wpt.fyi). We would need to be confident that this 
> wouldn’t become unwieldy however; we wouldn’t want to end up with something 
> like 
> (if:port=Apple)maxDifference=1;totalPixels=10,(if:platform=iOS)maxDifference=10;totalPixels=20,(if:port=GTK)maxDifference=10;totalPixels=300.
> 
> Another option is to extend TestExpectations to store more specific data 
> (though again this might become unwieldy, as we’re unlikely to add new 
> “platforms” based on every variable we might want to distinguish results on). 
> This also means the metadata is far away from the test itself, and the 
> TestExpectations files would continue to grow even further (and we already 
> have 34k lines of TestExpectations data!). TestExpectations is also a rather 
> horrible file format to modify the parser of.
> 
> I'm fine with either of the above options but I don't think we should 
> introduce this kind of micro syntax if we're going with meta.
> 
> We should probably specify a platform in a different attribute altogether. 
> e.g.
> 

I like this suggestion; WPT already allows multiple  because 
you can specify a per-reference fuzzy value:
.

> 
> I really hate that WPT is using a micro-syntax for this. Why isn't this 
> simply a different content attribute like this:
>  total-pixels="300">

Indeed. Maybe be should propose that change to avoid complicating the 
micro-syntax?

> 
> There is also test-options.json which has most of the same downsides as 
> TestExpectations, albeit without the pain in modifying the parser.
> 
> Finally, we could add per-test or per-directory files alongside the tests. 
> (Due to how things work, these could presumably also be in directories in 
> platform/.) This I think is probably the best option as it keeps the metadata 
> near the test, without needing to modify the test (which, per above, is 
> problematic for WPT as we move to automatically exporting changes). One could 
> imagine either a __dir__-metadata.json (to use a similar name to how WPT 
> names directory-level metadata files) or a -expected-fuzzy.json file 
> alongside each test.
> 
> Both of these two options seem worse than either encoding in the test or 
> putting in the test expectations. They invent a brand new mechanism to store 
> metadata for tests. We don't want to introduce yet another file / mechanism 
> people need to be aware of.

It may be that, for performance, we have a run-tests-time step that extracts 
fuzzy data from tests and puts it in a file somewhere, but that's orthogonal to 
where devs go to look for/edit fuzzy data.

Also something to consider: when importing WPT, we extract "slow" metadata and 
store it in a file. We should converge our solutions for all these WPT features 
that involve metadata in tests.

Simon

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev