| I think that we should have, as David Roundy pointed out, a
| restriction to code that is actually used frequently. However, I
| think we should make a distinction between micro-benchmarks, that
| test some specific item, and real-life benchmarks.
As many of you will know, the nofib benchmark suite made just such a
distinction from the beginning. In fact, it has 3 sub-suites:
* Imaginary: very tiny programs whose merit is that they sometimes hit missing
optimisations very hard indeed. Useful for compiler writers.
* Spectral: these are often called "kernels" and are meant to be typical of the
inner loops of some real programs.
* Real: these are unadulterated real applications, of various sizes.
We found these categories to be useful and robust, and I think they'd be useful
for the new suite. In particular, the imaginary suite is useless for (say)
choosing a compiler, but fantastic for exposing particular weak spots. But if
the imaginary programs were mixed with the real ones, the whole thing would
lose credibility.
You can find more info here:
http://research.microsoft.com/~simonpj/Papers/nofib.ps
Simon
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
