On Fri, Mar 8, 2013 at 2:24 AM, Benjamin Poulain <benja...@webkit.org>wrote:
> On Fri, Mar 8, 2013 at 2:15 AM, Ryosuke Niwa <rn...@webkit.org> wrote: > >> Three major problems with existing WebKit performance tests are: >> >> 1. There are too many tests to run >> 2. Some tests have variances that are too high to be of any use >> 3. Some tests are too specific to be of any general use >> >> > If you find what differentiate the good tests from the bad tests, it would > be good to create a wiki page explaining what to be careful about to make a > good test. > Yeah. We need to spend more time analysing tests in categories 2 & 3. One thing that have bitten my ass in the past is the JIT being smart and > removing some "dead" test code because it has no side effect. I am sure > there are plenty of mistake like this to be avoided. > I was always curious about this. I wonder if there’s a good way to amortize the effect so that we can measure relialistic number. The last thing I want is to introduce a lot of measurement biases to lower the variance and end up measuring something unrealistic. - R. Niwa
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
