[dev-servo] Roadmap Q2 goal : Pass Acid3
Hi, If I may weigh in, I wonder if passing Acid3 is a valuable goal in the short term. IE8 scores 20/100 at Acid3. All major websites and most websites have to support IE8, because it's the current "boat-anchor browser" [1]. So most websites don't need the features that make a 100 score. Specifically, SVG might be a big piece and since IE8 doesn't support it, usage is still very rare in websites. Aiming at IE8 parity (... or rather the intersection of IE8 and current standards, obviously) feels like a more practical goal if you want to support the web as it is today. An incomplete list of relevant features to get to IE8 parity [2]. Said differently, aiming at IE8 parity can only yield a much much better supported-website-count/effort ratio than passing Acid3. David [1] http://infrequently.org/2010/10/ie-8-is-the-new-ie-6/ [2] http://caniuse.com/#compare=ie+8 ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
I don't think we should be setting these kinds of goals for Servo in the short term. Passing Acid-style tests or meeting site-compatibility criteria require doing a lot of work that is not particularly risky and does not have much architectural impact on the engine ... but there's a lot of work to do that is risky and does have significant architectural impact, and that work should be higher priority, IMHO. Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
On 25/02/14 11:42, David Bruant wrote: Hi, If I may weigh in, I wonder if passing Acid3 is a valuable goal in the short term. IE8 scores 20/100 at Acid3. All major websites and most websites have to support IE8, because it's the current "boat-anchor browser" [1]. So most websites don't need the features that make a 100 score. Specifically, SVG might be a big piece and since IE8 doesn't support it, usage is still very rare in websites. Aiming at IE8 parity (... or rather the intersection of IE8 and current standards, obviously) feels like a more practical goal if you want to support the web as it is today. An incomplete list of relevant features to get to IE8 parity [2]. Said differently, aiming at IE8 parity can only yield a much much better supported-website-count/effort ratio than passing Acid3. I think I agree that passing Acid 3 in the next four months is not going to happen. But that's more because it requires a whole set of technologies that we don't have an implementation of in Servo (SVG, DOM Range+traversal, etc.) and which don't obviously seem like high priorities compared to getting the fundamental architecture right. I also think that the acid tests in general and acid 3 in particular are rather flawed tests; they make great PR pieces and that's a good reason to target them, but the actual things that they test are a grab-bag of random features and implementation bugs from the time they were written. Having said that, I don't think that using "IE8 parity" as a metric is very useful. IE8 takes a lot of non-standard codepaths on real sites, so merely targeting the standard features it implements is not going to give you an end result that is actually compatible with all the sites that work with IE8 (for example, I believe that IE8 still has the old attachEvent junk rather than the standard addEventListener API, so by a literal reading of the above you wouldn't implement events at all). I think a better way to proceed is to try to implement the parts that change the underlying architecture as early as possible (so, on the DOM side, this would be HTML parsing, page loading, script scheduling, history navigation, etc.) and as a minimum target as-good-as-gecko (or webkit or blink, or whatever) scores on the relevant web-platform-tests for those features (in practice we want to aim for 100% of course, but as long as we at least pass the tests that everyone else passes the compat. story should be OK). Once this is done we survey web content to see which higher-level features are most used and implement those in priority order using the same approach. My method isn't likely to produce the best short-term gains, but I think it's the best route to compatibility with real sites in the long term. ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
On 2/25/14 6:42 AM, David Bruant wrote: If I may weigh in, I wonder if passing Acid3 is a valuable goal in the short term. Absolutely not. It's not a valuable goal in any term, really, until we're trying to go for random "standards pr" points. -Boris ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
I forgot. I was referring to https://github.com/mozilla/servo/wiki/Roadmap
Le 25/02/2014 12:57, James Graham a écrit :
Having said that, I don't think that using "IE8 parity" as a metric is
very useful. IE8 takes a lot of non-standard codepaths on real sites,
so merely targeting the standard features it implements is not going
to give you an end result that is actually compatible with all the
sites that work with IE8 (for example, I believe that IE8 still has
the old attachEvent junk rather than the standard addEventListener
API, so by a literal reading of the above you wouldn't implement
events at all).
blah :-p Yes of course, don't do a literal reading. What I tried to
convey takes lots of words if aiming at accuracy and I feel "IE8 parity"
roughly conveys it. Among other things, I'm obviously not recommanding
to remove features that are already in Servo, but not in IE8 ;-)
My main point was about the supported-website-count/effort ratio. And
that aiming at some IE8 features may be more relevant than Acid3
features, especially looking at the second overarching goal ("render
most web pages faithfully") and Q4 goal.
I think a better way to proceed is to try to implement the parts that
change the underlying architecture as early as possible (so, on the
DOM side, this would be HTML parsing, page loading, script scheduling,
history navigation, etc.)
I agree with you and roc on this part and didn't mention it because I
agree. Only the Acid3 part felt a bit off (at least for now. I
acknowledge the PR value, but it doesn't seem like a concern for 2014)
David
___
dev-servo mailing list
dev-servo@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
>> If I may weigh in, I wonder if passing Acid3 is a valuable goal in the >> short term. > > Absolutely not. It's not a valuable goal in any term, really, until we're > trying to go for random "standards pr" points. Acid2 and Acid3 are the current goals of our partners, and the Servo team is helping them achieve those goals. This is helping design and reviewing code. It's not insignificant work, but we think having partners is important. You can ping me off list for more info and context if you would like. The MoCo employees working on Servo are currently focusing on the rest of those items in the list. jack. ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
Le 25/02/2014 17:14, Jack Moffitt a écrit : If I may weigh in, I wonder if passing Acid3 is a valuable goal in the short term. Absolutely not. It's not a valuable goal in any term, really, until we're trying to go for random "standards pr" points. Acid2 and Acid3 are the current goals of our partners, and the Servo team is helping them achieve those goals. This is helping design and reviewing code. It's not insignificant work, but we think having partners is important. Agreed. I wasn't aware of this context. Withdrawing my doubts about Acid3. Thanks, David ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
On 2/25/14 3:55 AM, Robert O'Callahan wrote: I don't think we should be setting these kinds of goals for Servo in the short term. Passing Acid-style tests or meeting site-compatibility criteria require doing a lot of work that is not particularly risky and does not have much architectural impact on the engine ... but there's a lot of work to do that is risky and does have significant architectural impact, and that work should be higher priority, IMHO. If I may defend the Acid tests just a little bit... From my point of view, especially with my work lately, one of the most important questions that Servo needs to answer is "how much parallelism can we gain on real-world Web sites?" To that end, Web site compatibility (which Acid2 is a rough proxy for a large subset of, if used sensibly [1]) is a useful thing to work on. Otherwise, we don't know whether the parallel gains that we are getting will scale up to real-world usage, or whether we're only getting them because we don't implement the difficult features. To give a concrete example, the work on block formatting contexts [2] and absolute positioning [3] has been hugely illustrative. Not only was this work a sea change to our layout algorithm, it helped us establish exactly what the parallel hazards with floats were. At the beginning before we implemented floats, we got massive parallel gains on all sites, but these were unrealistic. In Servo master, we're using a conservative algorithm to handle floats that completely erases any parallel gains on most sites. Only with this block formatting context work--which is also necessary for Web compat--do we start to see gains on most sites. Handling absolute positioning properly will let us perform further measurements on how much parallelism is impacted on sites: in particular, how often in the real world is an absolute frame's hypothetical box impacted by floats? We don't know the answer to that right now, but this seems to me to be a very important question. So I agree that we must not make architectural decisions that make it difficult to implement Web features. But I do think that Web compat is useful, because it helps us to identify the parallel hazards and know where we stand in terms of the Web of today, which helps us answer the big questions that Servo set out to answer. Patrick [1]: "Sensibly" means implementing the features that the test tests in a way that handles even edge cases that are not tested, and not accepting patches that would make it hard implement more complex features that are not tested, such as incremental reflow, bidi, etc. [2]: https://github.com/mozilla/servo/pull/1734 [3]: https://github.com/mozilla/servo/pull/1681 ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
On 25/02/2014 18:02, Patrick Walton wrote: So I agree that we must not make architectural decisions that make it difficult to implement Web features. But I do think that Web compat is useful, because it helps us to identify the parallel hazards and know where we stand in terms of the Web of today, which helps us answer the big questions that Servo set out to answer. Yes, Web compat is important. But Acid3 in particular is a poor proxy for Web compat, IMO. -- Simon Sapin ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
> From my point of view, especially with my work lately, one of the most > important questions that Servo needs to answer is "how much parallelism > can we gain on real-world Web sites?" To that end, Web site > compatibility (which Acid2 is a rough proxy for a large subset of, if > used sensibly [1]) is a useful thing to work on. Otherwise, we don't > know whether the parallel gains that we are getting will scale up to > real-world usage, or whether we're only getting them because we don't > implement the difficult features. Is it possible to programmatically identify CSS rules or page layouts which would benefit from the parallelism that Servo potentially offers? For the CSS un/prefixing debate I/we had some success in deploying web crawlers that parsed and summarized millions of CSS rules from thousands of sites. Could something similar be of use for quantifying the lowest hanging parallelism fruit for Servo? Said another way, is it possible to create a targeted benchmark for Servo compatibility and performance? John -- John Jensen | Director, Metrics | Mozilla Corporation ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
> Is it possible to programmatically identify CSS rules or page layouts which > would benefit from the parallelism that Servo potentially offers? For the CSS > un/prefixing debate I/we had some success in deploying web crawlers that > parsed and summarized millions of CSS rules from thousands of sites. Could > something similar be of use for quantifying the lowest hanging parallelism > fruit for Servo? > > Said another way, is it possible to create a targeted benchmark for Servo > compatibility and performance? I think the answer is yes, but I would be interested in talking about this more. We've done some preliminary work discussing and estimating this for specific sites, but it sounds like you are much farther down this road with tooling. We'd like to leverage what you've already done if possible. jack. ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
On 2/25/14 1:02 PM, Patrick Walton wrote: To that end, Web site compatibility (which Acid2 is a rough proxy for a large subset of, if used sensibly [1]) Acid3 is a _much_ worse proxy here than Acid2, even with your sensible use caveat, fwiw. -Boris ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
On Wed, Feb 26, 2014 at 7:02 AM, Patrick Walton wrote: > So I agree that we must not make architectural decisions that make it > difficult to implement Web features. But I do think that Web compat is > useful, because it helps us to identify the parallel hazards and know where > we stand in terms of the Web of today, which helps us answer the big > questions that Servo set out to answer. > That's a good point, but Acid3, and to a lesser extent Acid2, are about testing edge cases and the presence of obscure features. I don't think they tell you anything significant about parallelism in the mass of pages on the Web. No single page can, but Acid3 is probably even worse than picking a page at random. You'd be much better off picking, say, a Wikipedia page (which I know you've done!) or the HTML5 single-page spec. Otherwise, we don't know whether the parallel gains that we are getting > will scale up to real-world usage, or whether we're only getting them > because we don't implement the difficult features. > That's a good point too, but the problem is that key parts of the Web like GMail, Youtube, Facebook etc require so much work for full support that I don't think we draw broad conclusions until very far in the future, when it will be far too difficult to make architectural changes. If you must discover those conclusions earlier then we should probably piggyback some analysis on an existing browser engine instead of doing it in Servo. The good news is that I think CSS layout is by far the largest piece of the Web where we have implicit parallelism that is subject to unpredictable hazards. Other big chunks of work, like almost all DOM APIs, are either obviously not parallelizable or obviously parallelizable. Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Roadmap Q2 goal : Pass Acid3
On 2/25/14 3:08 PM, Robert O'Callahan wrote: That's a good point, but Acid3, and to a lesser extent Acid2, are about testing edge cases and the presence of obscure features. I don't think they tell you anything significant about parallelism in the mass of pages on the Web. No single page can, but Acid3 is probably even worse than picking a page at random. You'd be much better off picking, say, a Wikipedia page (which I know you've done!) or the HTML5 single-page spec. So I agree about Acid3 in particular not being particularly interesting for parallelism. (Well, except maybe the SVG stuff, but I don't think that the real-world risk if we have to make SVG slow impacts Servo too much.) Wikipedia is actually a bit tricky to use as a test case in my experience, because it makes heavy use of floats the way they were designed to be used--for floating objects--and as a result achieves poor parallelism right now. (Mobile Wikipedia is much faster, because it doesn't use floats.) The Alexa Top 50 pages that I've managed to be able to test actually generally achieve significantly better parallelism than Wikipedia does. This is why I think it's important to have some breadth in our tests. That's a good point too, but the problem is that key parts of the Web like GMail, Youtube, Facebook etc require so much work for full support that I don't think we draw broad conclusions until very far in the future, when it will be far too difficult to make architectural changes. If you must discover those conclusions earlier then we should probably piggyback some analysis on an existing browser engine instead of doing it in Servo. The good news is that I think CSS layout is by far the largest piece of the Web where we have implicit parallelism that is subject to unpredictable hazards. Other big chunks of work, like almost all DOM APIs, are either obviously not parallelizable or obviously parallelizable. Right, I'm more interested in the legacy CSS layout stuff that's so prevalent that we can't realistically push back on its use. You do have a good point about it being hard to test things like Gmail or Facebook due to sheer number of dynamic HTML features they use, but maybe we can leverage existing engines and produce static versions of their pages to test in Servo. (I actually already did this to test YouTube.) Based on my (limited) amount of experience trying to guess how much parallelism we'll gain by performing analyses in Gecko (in the previous thread about measuring parallel hazards) I think it's difficult to perform these analyses directly in existing engines without involving our actual implementation in Servo somehow. The chaotic nature of the parallel scheduling with work stealing makes Servo's parallel layout difficult to model and simulate accurately. Patrick ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
