We have some benchmarks for Cloud Haskell and its underlying network-transport infrastructure that I'm in the process of trying to automate. I'd be very interested to see how these fare against various GHC releases, though I suspect we'll have to tweak the dependencies considerably in order to make the automation happen.
I don't know if that fits into the 'other regressions' category or not? Cheers, Tim On 5 Feb 2013, at 14:24, Nicolas Frisby wrote: > I'd like to investigate the "other regressions out there". > > Do you have more info? Perhaps a list? Maybe even benchmarking code? > > Thanks. > > > On Tue, Feb 5, 2013 at 4:22 AM, Austin Seipp <mad....@gmail.com> wrote: > I'm +1 for this. Eyal Lotem and I were just discussing this on IRC a > few minutes ago, and he suffered a rather large (~25%) performance hit > when upgrading to 7.6.1, which is unfortunate. > > Committers are typically very good about recording nofib results in > their commit and being performance-courteous, but I'm not sure there's > ever been a longer-scale view of GHC performance over multiple > releases like this - or even a few months. At least not recently. On > top of that, his application was a type checker, which may certainly > stress different performance points than what nofib might. Once we get > performance bots set up, I've got a small set of machines I'm willing > to throw at it. > > Thanks for the results, Johan! > > On Mon, Feb 4, 2013 at 4:33 PM, Johan Tibell <johan.tib...@gmail.com> wrote: > > Hi all, > > > > I haven't had much time to do performance tzar work yet, but I did run nofib > > on the last few GHC releases to see the current trend. The benchmarks where > > run on my 64-bit Core i7-3770 @ 3.40GHz Linux machine. Here are the results: > > > > 7.0.4 to 7.4.2: > > > > -------------------------------------------------------------------------------- > > Program Size Allocs Runtime Elapsed TotalMem > > -------------------------------------------------------------------------------- > > Min -1.6% -57.3% -39.1% -36.4% -25.0% > > Max +21.5% +121.5% +24.5% +25.4% +300.0% > > Geometric Mean +8.5% -0.7% -7.1% -5.2% +2.0% > > > > The big loser here in terms of runtime is "kahan", which I added to test > > tight loops involving unboxed arrays and floating point arithmetic. I > > believe there was a regression in fromIntegral RULES during this release, > > which meant that some conversions between fixed-width types went via > > Integer, causing unnecessary allocation. > > > > 7.4.2 to 7.6.1: > > > > -------------------------------------------------------------------------------- > > Program Size Allocs Runtime Elapsed TotalMem > > -------------------------------------------------------------------------------- > > Min -5.1% -23.8% -11.8% -12.9% -50.0% > > Max +5.3% +225.5% +7.2% +8.8% +200.0% > > Geometric Mean -0.4% +2.1% +0.3% +0.2% +0.7% > > > > The biggest loser here in terms of runtime is "integrate". I haven't looked > > into why yet. > > > > 7.6.1 to 7.6.2: > > > > -------------------------------------------------------------------------------- > > Program Size Allocs Runtime Elapsed TotalMem > > -------------------------------------------------------------------------------- > > Min -2.9% +0.0% -4.8% -4.4% -1.9% > > Max +0.0% +1.0% +4.5% +6.4% +20.8% > > Geometric Mean -1.7% +0.0% +0.1% +0.3% +0.2% > > > > I have two takeaways: > > > > * It's worthwhile running nofib before releases as it does find some > > programs that regressed. > > * There are some other regressions out there (i.e. in code on Hackage) that > > aren't reflected here, suggesting that we need to add more programs to > > nofib. > > > > Cheers, > > Johan > > > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://www.haskell.org/mailman/listinfo/ghc-devs > > > > > > -- > Regards, > Austin > > _______________________________________________ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs > > _______________________________________________ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs