On Jul 6, 2009, at 10:11 AM, Geoffrey Garen wrote:
So, what you end up with is after a couple of years, the slowest
test in the suite is the most significant part of the score.
Further, I'll predict that the slowest test will most likely be the
least relevant test, because the truly important parts of JS
engines were already optimized. This has happened with Sunspider
0.9 - the regex portions of the test became the dominant factor,
even though they were not nearly as prominent in the real world as
they were in the benchmark. This leads to implementors optimizing
for the benchmark - and that is not what we want to encourage.
How did you determine that regex performance is "not nearly as
prominent in the real world?"
For reference: in current JavaScriptCore, the one regexp-centric test
is about 4.6% of the score by time. 3 of the string tests also spend
their time in regexps, however, I think those are among the tests that
most closely resemble what Web sites do. I believe the situation is
roughly similar in other competitive JavaScript engines. This is
probably not exactly proportionate but it doesn't dominate the test. I
don't think any of this is a problem, unless one thinks the regexp
improvements in Nitro, V8 and TraceMonkey were a waste of resources.
What I have seen happen is that numeric processing and especially
integer math became a smaller and smaller proportion of the test,
looking at the best publicly available engines over time. I think that
turned out to be the case because math had much more room for
optimization in naive implementations than, say, string processing.
Regards,
Maciej
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
