On Mon, Jul 26, 2010 at 17:45:57 +0000, Jason Dagit wrote: > The benchmark suite finished. It took over 24 hours. I have pasted the > output into the wiki, but it's somewhat suspect because I was using my > computer during that 24 hour period on and off. Should the benchmark > suite really take 30 hours? > > http://wiki.darcs.net/Benchmarks/ParserRefactor
We should be down to 15 hours thanks to dropping the non-op repos in darcs-benchmark 0.1.8.3, but still a pain. darcs-benchmark run --fast should also eliminate some of the slower benchmarks (full darcs get and darcs annotate). Hmm, patch parsing could play some role in darcs annotate. Alternatively, you can select which benchmarks you want to run by passing in the only flag, eg. darcs benchmark run darcs-before darcs-after / ghc-hashed --only annotate Also, you commented [1]: > From your ticket updates I get the impression it also doesn't > currently benchmark anything we care about. I thought that was the > point of darcs-benchmark. I believe you're referring to the comment in which I said it would be useful to think about some parser-intensive benchmarks [2]. We need to be more precise about "benchmark anything we care about". Right now, the benchmarking is very general, ie. we time some common operations, get, pull, record, wh. Some questions which would arise from your comment are * For general purposes: that *sounds* like something we care about, although if you have ideas on how to improve the benchmarks to be more meaningful from a darcs-user perspective, I'm all ears. * For parser-hacking purposes: is it worthwhile to benchmark something more highly targeted? The highly-targeted benchmarks could perhaps be artificial and have little real world consequence, but they could help us to tune the hell out of the parser. In other words "what we care about" depends on the context. Anyway, there are two ways I can think to extend the suite: either add new benchmarks (sequences of commands with some subsequence being timed), or add new repositories to the zoo which display interesting/pathological characteristics... [1] http://irclog.perlgeek.de/darcs/2010-07-26#i_2613228 [2] http://bugs.darcs.net/msg11869 -- Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow> For a faster response, please try +44 (0)1273 64 2905.
signature.asc
Description: Digital signature
_______________________________________________ darcs-users mailing list [email protected] http://lists.osuosl.org/mailman/listinfo/darcs-users
