Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-19 Thread Janek S.
Thank you very much Thomas. This is the kind of explanation I needed! Janek Dnia czwartek, 18 października 2012, Thomas Schilling napisał: > On 18 October 2012 13:15, Janek S. wrote: > >> Something like this might work, not sure what the canonical way is. > >> (...) > > > > This is basically the

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Thomas Schilling
On 18 October 2012 13:15, Janek S. wrote: >> Something like this might work, not sure what the canonical way is. >> (...) > > This is basically the same as the answer I was given on SO. My concerns about > this solutions are: > - rnf requires its parameter to belong to NFData type class. This is

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Thomas Schilling
Yes, Criterion always discards the time of the first evaluation. On 18 October 2012 15:06, Janek S. wrote: >> So the evaluation will be included in the benchmark, but if "bench" is >> doing enough trials it will be statistical noise. > When I intentionally delayed my dataBuild function (using del

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Antoine Latter
On Thu, Oct 18, 2012 at 9:06 AM, Janek S. wrote: >> So the evaluation will be included in the benchmark, but if "bench" is >> doing enough trials it will be statistical noise. > > When I intentionally delayed my dataBuild function (using delayThread > 100) the estimated time > of benchmark wa

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Janek S.
> So the evaluation will be included in the benchmark, but if "bench" is > doing enough trials it will be statistical noise. When I intentionally delayed my dataBuild function (using delayThread 100) the estimated time of benchmark was incorrect, but when I got the final results all runs were

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Antoine Latter
On Thu, Oct 18, 2012 at 4:23 AM, Janek S. wrote: > Dear list, > > during past few days I spent a lot of time trying to figure out how to write > Criterion benchmarks, > so that results don't get skewed by lazy evaluation. I want to benchmark > different versions of an > algorithm doing numerical

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Janek S.
> Something like this might work, not sure what the canonical way is. > (...) This is basically the same as the answer I was given on SO. My concerns about this solutions are: - rnf requires its parameter to belong to NFData type class. This is not the case for some data structures like Repa ar

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Alfredo Di Napoli
I don't know if you have already read them, but Tibell's slides on High Performance Haskell are pretty good: http://www.slideshare.net/tibbe/highperformance-haskell There is a section at the end where he runs several tests using Criterion. HTH, A. On 18 October 2012 11:45, Claude Heiland-Allen

Re: [Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Claude Heiland-Allen
Hi Janek, On 18/10/12 10:23, Janek S. wrote: during past few days I spent a lot of time trying to figure out how to write Criterion benchmarks, so that results don't get skewed by lazy evaluation. I want to benchmark different versions of an algorithm doing numerical computations on a vector.

[Haskell-cafe] How to correctly benchmark code with Criterion?

2012-10-18 Thread Janek S.
Dear list, during past few days I spent a lot of time trying to figure out how to write Criterion benchmarks, so that results don't get skewed by lazy evaluation. I want to benchmark different versions of an algorithm doing numerical computations on a vector. For that I need to create an inpu