Hello,
A while back, we had a discussion on #haskell about assembling a Haskell benchmark suite, that is suitable for doing performance tests. A preliminary page was erected athttp://www.haskell.org/haskellwiki/HaBench . In the meantime, Donald Steward extended the original nofib suite with some shootout benchmarks afaik, resulting in nobench. The code base for the latter currently resides at http://code.haskell.org/nobench/ .
I have been trying to get it running on GHC 6.10.1. For now, I added a number of type definitions to the code, causing the build/runtime system to compile. The same probably ought to be done for the benchmarks themselves, unless there is a cheat around this using some language extension. Anyhow, I'll post a patch against the current repository as soon as I have a number of benchmarks running.
The main issue that still remains is the availability of real life benchmarks. I agree with the fact that micro-benchmarks can be useful for testing purposes or measures the efficacy and effectiveness of certain optimisations, yet I firmly believe any community has need of a set of benchmarks that actually reflects the real life usage of the language. I am think along the lines of something alike to the DaCapo projects, which assembled a number of very good benchmarks for the Java language and its VM. So the question basically boils down to this. Is there anybody interested in making the move toward HaBench, and if so, do you know of real life benchmarks that can serve for this exact purpose?
The benchmarks should preferably execute for > 10s on modern machines, using a decent amount of RAM (say somewhere between 50 and 500MB), thus exercising all parts of a modern computing system. The code should not be trivial and the set of benchmarks should eventually cover the most uses of Haskell in the industry. Of course, the benchmarks themselves should be open source. If possible, they should come with multiple inputs, allowing a short (test) run as well as longer measurement runs.
If you are able and willing to help out, drop by at the HaBench page and drop a line,
-- Andy _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
