-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I recently stumbled across this thread, related to the Shootout
(which I run) and wanted to add a few comments.
First, I agree that I am a Haskell novice; however, I rely on
people like yourselves to show me where I am doing things wrong.
If our solutions are horribly inefficient, please tell us what we
can do to resolve the problem. But note that in some cases, we are
purposely being inefficient so that we can measure the cost of
certain basic operations.
Doug Bagely started the Shootout as a comparison of scripting
languages. After I revived it, I started focussing more on
functional and logic languages, since I was curious how these
languages would fare in comparison with the more typically-used
languages like C/C++. Consequently, many of the tests show this
heritage and are highly imperative. We have been working to relax
some of these rules, and have added many new tests that try to be
more amenable to comparison of lazy and 'strict' languages.
In our defense I would like to plead that it is not easy to devise
short problems (e.g., that can be expressed in a page or so of code
- -- we try to limit it to 100 lines of C++ code as a rough
benchmark), and that also take enough run time for a meaningful
measurement. So, many of the tests initially suffered from foolish
looping and repetition as a means of extending the run times
sufficiently to allow measurement. We have been changing tests so
that fewer of them rely on these kinds of activities to generate
results. We are always interested in new ideas, and would consider
any benchmark ideas you have.
Finally, I don't agree at all that Haskell's scores show poorly.
The GHC compiler is one of the better performers, and regularly
trounces other languages when we let it do its job. For example,
we had to jump through many hoops to convince the compiler to
actually invoke all the loops required to do the "same way" tests
on methodcall and similar benchmarks, because the compiler was so
sharp at identifying wasted effort and optimizing it away. Perhaps
we don't say this strongly enough on the site, but I was truly
impressed at its ability to toss aside wasted calculations and
focus on the core problem. I'll admit it was frustrating because
we wanted to see how long it would take for Haskell to make N
recursive calls, or what have you, but the fact that the compiler
saw that there was no need to do the recursion to get the result is
quite amazing! I'm sure this would be extremely useful when
compiling real-world programs.
There have been many cases where the "obvious" solution in Haskell
is a great performer, but we discover that it's so fast because it
has tossed aside wasted cycles and just done the work necessary to
arrive at a solution. It's unfortunate, since we then have to
dirty the program with wasted steps to force it to exercise the
tasks required. But in a "real world" test I think Haskell would
show quite well.
As we evolve into the newer tests, I think you will see Haskell
make a better showing. Simon Marlow (and others) have been a huge
help in optimizing the existing tests in a "fair" fashion, and as
we provide more "same thing" type tests, I think Haskell will pull
further ahead.
Thanks,
- -Brent
P.S. I must say that I find Haskell to be the most esthetically
pleasing language -- it's quite beautiful when printed!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (Darwin)
iD8DBQFDuWIlzGDdrzfvUpURAnJuAJ90G9ltXH5m+jsz6D0QoucoY6anNACfcfkQ
Mg2K0NO0kbBYLy4ZX7LCoeM=
=uGnw
-----END PGP SIGNATURE-----
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
