Re: [webkit-dev] BigO correlation tests

2016-04-13 Thread Darin Adler
> On Apr 13, 2016, at 12:18 PM, Ryosuke Niwa wrote: > > I think Darin likes these tests. I love the idea of them and I’d like to see us write more. Changing our algorithms to be efficient for large data sets is a tricky problem and an easy way to write a test that cleanly checks for that is qu

Re: [webkit-dev] BigO correlation tests

2016-04-13 Thread Ryosuke Niwa
On Wed, Apr 13, 2016 at 12:17 AM, Nikos Andronikos wrote: > >> >> However, what you proposed will only reduce the likelihood of type I errors >> (false positives). You should also examine how it affects the likelihood of >> type II errors (false negative). You might want to make some API >> a

Re: [webkit-dev] BigO correlation tests

2016-04-13 Thread Nikos Andronikos
> On 13 Apr 2016, at 4:22 PM, Ryosuke Niwa wrote: > > I don’t think order of magnitudes tests ever quite worked JSC (it was added > by Google so it obviously worked on V8). If adjusting the threshold was all > that was needed to make the tests pass, I see no reason how to change it. Adjusting

Re: [webkit-dev] BigO correlation tests

2016-04-12 Thread Ryosuke Niwa
I don’t think order of magnitudes tests ever quite worked JSC (it was added by Google so it obviously worked on V8).  If adjusting the threshold was all that was needed to make the tests pass, I see no reason how to change it. However, what you proposed will only reduce the likelihood of type I

[webkit-dev] BigO correlation tests

2016-04-12 Thread Nikos Andronikos
Hey all, We’ve noticed what we think is some flakiness with tests that attempt to determine the BigO characteristics (using magnitude-perf.js) of performance tests and would like some feedback from the community. In our particular case, we are seeing a failure on the iOS simulator of the followin