On 01/12/2012, at 1:42 AM, Simon Peyton-Jones wrote: > | > While writing a new nofib benchmark today I found myself wondering > | > whether all the nofib benchmarks are run just before each release, > > I think we could do with a GHC Performance Tsar. Especially now that Simon > has changed jobs, we need to try even harder to broaden the base of people > who help with GHC. It would be amazing to have someone who was willing to: > > * Run nofib benchmarks regularly, and publish the results > > * Keep baseline figures for GHC 7.6, 7.4, etc so we can keep > track of regressions > > * Investigate regressions to see where they come from; ideally > propose fixes. > > * Extend nofib to contain more representative programs (as Johan is > currently doing). > > That would help keep us on the straight and narrow.
I was running a performance regression buildbot for a while a year ago, but gave it up because I didn't have time to chase down the breakages. At the time we were primarily worried about the asymptotic performance of DPH, and fretting about a few percent absolute performance was too much of a distraction. However: if someone wants to pick this up then they may get some use out of the code I wrote for it. The dph-buildbot package in the DPH repository should still compile. This package uses http://hackage.haskell.org/package/buildbox-1.5.3.1 which includes code for running tests, collecting the timings, comparing against a baseline, making pretty reports etc. There is then a second package buildbox-tools which has a command line tool for listing the benchmarks that have deviated from the baseline by a particular amount. Here is an example of a report that dph-buildbot made: http://log.ouroborus.net/limitingfactor/dph/nightly-20110809_000147.txt Ben. _______________________________________________ Glasgow-haskell-users mailing list [email protected] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
