On [many an occasion] "Many people" wrote: > user=> (time foo) ... > "Elapsed time: 245.152 msecs"
I hate to be a wet blanket, but how accurate is this? The doc doesn't even say whether it measures wall clock time or cpu time. Even if you knew that, it's at best a foundation to build a real benchmarking system on. By "real benchmarking" I mean one that does things like run the benchmark a few times to fill caches and let the JIT do it's thing, etc., makes sure it runs the target code enough times to get a chunk of time large enough to means something, and takes into account the overhead of making multiple runs. Now do that a dozen or so times so you can take some stats, so when you compare two timings you actually have enough information to decide if one is really faster than the other, or the difference is just fuzz in the measurement system. Ok, I started building something like that, and immediately started getting ludicrous results. Googling for Java benchmarks turned up exactly *one* such tool (which presumably works on any runnable or callable, but fails on Clojure functions) amongst the sets of benchmark code (most of which didn't talk about how they measured the time). First thing: a timer that returns the time so I can work with it: (defmacro nano-timer "Time body, returning nanoseconds and result" [body] `(do (System/runFinalization) (System/gc) (let [start# (System/nanoTime) res# ~body] [(- (System/nanoTime) start#) res#]))) Second thing: run the benchmark many times, subtracting out the loop overhead: (defmacro loop-timer "Time code in a loop, returning nanoseconds taken minus loop overhead" [code count] `(let [overhead# (first (nano-timer (dotimes [_# ~count]))) mytime# (first (nano-timer (dotimes [_# ~count] ~code)))] (- mytime# overhead#))) Which has the annoying habit of returning negative numbers :-(. I'd really like to know how the loop that calculates overhead takes more time than the loop that actually runs the code, no matter how trivial the code is! Is anyone using anything more sophisticated than clojure.core/time for benchmarking clojure code? <mike -- Mike Meyer <m...@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en