Hi all, On Tuesday, 14 July 2015 02:49:57 UTC+2, Aaron Meurer wrote: > > - Get a benchmark machine and run airspeed velocity on it. We need to > catch performance regressions. The benchmark suite can be anything, > although obviously well-made benchmarks are better. >
I found a few hours and threw this together (motivated by having noticed some things I have been doing with sympy has been getting slower): https://github.com/bjodah/sympy_benchmarks_bjodah then, on a machine with asv 0.1.1, I ran: $ asv run ALL -s 100 which essensitally takes 100 equally spaced commits from the whle history and run the benchmarks one those. You can view the results here: http://hera.physchem.kth.se/~sympy_asv some commits failed, maybe I should be targetting the merge commits but I haven't figured out how to do that yet. Furthermore: the comparison isn't quite fair since I belive the later versions return more correct results (Piecewise solutions), I'm not sure how to deal with this fairly, maybe some kind of assertion should be made (like a tests) at the end of the individual benchmark, a passing assertion means it should be included, a failing one that it should be excluded. But then this assertion shouldn't be included in the timing. There could be something in asv for this but I haven't figured it out yet. I am stil a bit new to using asv so I haven't quite figured out best practives here. It would be nice to have a dedicated machine (or maybe several), maybe a Raspberry Pi 2 which would make it easy for people to get the exact same hardware for comparisons? Anyways, thought I would share. Feel free to fork/send pull requests, I'll happily run them on that machine if there is an interest. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/5a7e4b53-6daa-4fc2-a8ac-a3288a0270c7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
