Re: [dev-servo] Categorizing Stylo reftest failures

2017-08-10 Thread Boris Zbarsky
On 8/10/17 2:25 PM, Chris Peterson wrote: Besides fixing the regressions, we should review the test cases where Gecko fails but Stylo passes to make sure we're not unintentionally shipping new CSS features. Indeed. And any remaining such should probably be annotated with Gecko bugs or otherwi

Re: [dev-servo] Categorizing Stylo reftest failures

2017-08-10 Thread Chris Peterson
On 2017-08-10 1:55 AM, James Graham wrote: On 10/08/17 02:53, Manish Goregaokar wrote: https://gist.github.com/Manishearth/086118c940ff86a6cfc573f53c508279 FWIW I made a list of the differences between reftest results in web-platform-tests with stylo enabled vs not (based on the expectation

Re: [dev-servo] Categorizing Stylo reftest failures

2017-08-10 Thread James Graham
On 10/08/17 02:53, Manish Goregaokar wrote: https://gist.github.com/Manishearth/086118c940ff86a6cfc573f53c508279 FWIW I made a list of the differences between reftest results in web-platform-tests with stylo enabled vs not (based on the expectation metadata files). Paths are relative to the

Re: [dev-servo] Categorizing Stylo reftest failures

2017-08-09 Thread Bobby Holley
Great work Manish! On Wed, Aug 9, 2017 at 6:53 PM, Manish Goregaokar wrote: > https://gist.github.com/Manishearth/086118c940ff86a6cfc573f53c508279 > > I've now gone through the failures and categorized them as yes/no/maybe > based on whether they should block shipping. I've filed bugs for and >

Re: [dev-servo] Categorizing Stylo reftest failures

2017-08-09 Thread Manish Goregaokar
https://gist.github.com/Manishearth/086118c940ff86a6cfc573f53c508279 I've now gone through the failures and categorized them as yes/no/maybe based on whether they should block shipping. I've filed bugs for and partially investigated all of the "yes" ones (and fixed some of them). Some of these bug

Re: [dev-servo] Categorizing Stylo reftest failures

2017-07-24 Thread Manish Goregaokar
We aren't! I'm just not seeing that bug in the reftests outside of audio and video, and TYLin is already looking into it anyway. Thanks, -Manish Goregaokar On Mon, Jul 24, 2017 at 8:20 AM, Boris Zbarsky wrote: > On 7/24/17 1:53 AM, Manish Goregaokar wrote: >> >> and the styloVsGecko / issue (w

Re: [dev-servo] Categorizing Stylo reftest failures

2017-07-24 Thread Boris Zbarsky
On 7/24/17 1:53 AM, Manish Goregaokar wrote: and the styloVsGecko / issue (which does not block landing since it's a slight discrepancy between stylo and gecko) Looks like it's a difference in vertical alignment. How sure are we that this won't affect web pages outside of media controls? -

Re: [dev-servo] Categorizing Stylo reftest failures

2017-07-23 Thread Manish Goregaokar
Now that we've fixed most of these I went through it again and checked stuff. Filtering out scoped style and web components failures we only have a hundred tests left I recategorized most of them: https://gist.github.com/Manishearth/086118c940ff86a6cfc573f53c508279 The major issues right now are

Re: [dev-servo] Categorizing Stylo reftest failures

2017-05-11 Thread Manish Goregaokar
And, it's done! All reftest failures classified. Permalink to current revision: https://gist.github.com/Manishearth/086118c940ff86a6cfc573f53c508279/a181e327a4e2d83d75e28862d1fc01e121923bff Bear in mind that the try run this is based on is a few days old so some of the things there may be fixed i

Re: [dev-servo] Categorizing Stylo reftest failures

2017-05-10 Thread Manish Goregaokar
First link in the file! https://treeherder.mozilla.org/#/jobs?repo=try&revision=bff0878649f10db8946d50e760b0d13f0f3be9be Also, each section in the file already links to the analyzer results for that subset of the tests. -Manish Goregaokar ___ dev-servo

Re: [dev-servo] Categorizing Stylo reftest failures

2017-05-10 Thread Boris Zbarsky
On 5/10/17 8:14 PM, Manish Goregaokar wrote: So I took a try push with all expectations set as passing and went about categorizing the reftest failures. Do you mind linking to the try push? Looking at the actual reftest-analyzer results is helpful for categorizing things. -Boris ___

[dev-servo] Categorizing Stylo reftest failures

2017-05-10 Thread Manish Goregaokar
So I took a try push with all expectations set as passing and went about categorizing the reftest failures. I'm halfway done. This is pre-HasAuthorSpecifiedRules and there are lots of related failures. At some point I'll extract all the HASR-marked failures and have a second look at the ones which