You might look at how GAP does this. Its tst directory contains expected timings. One only compares relative times. GAP tests do not fail on a pentium 75mhz, since GAP users employ a wide range of hardware. Surely other software has similar features.
On Aug 10, 3:07 pm, Martin Albrecht <[EMAIL PROTECTED]> wrote: > > Yeah sure, I didn't mean to hijack the thread. The segfaults are > > obviously more important than the slownesses. I think your proposal is > > more likely to succeed if we don't get sidetracked :-) > > So let's open up a new thread then. > > To summarize: We need something to test speed regressions. Problems we face > are at least: > - we need tests > - we can only compare against previous results on the same machine > - we need to deal with some uncertainness of the results (e.g. through system > load) > - we need code to run the tests and compare with previous results > > These are just some random thoughts to get the thread rolling. > > The original message by David was: > > > > > I'd like to add something. I consider "slowness" a bug. When SAGE does > > something obviously way too slowly, that's almost as bad as a genuine > > bug. Some slownesses are easy to fix, some are not. > > > One of the things that really worries me is that someone who doesn't > > know the code really intimately, can change something in a way that > > they think improves the code, but that has a dramatic negative effect > > on performance in apparently unrelated areas. A common culprit is > > adding "type-checking" to the beginning of a low-level function. We > > currently don't have any systematic way of even *noticing* these kinds > > of things. > > > It would be great if we had some coherent way of managing this. The > > first thing that occurs to me is some kind of "docprofile" (like > > doctests, but for checking speed of operations). If this works, it > > might be a good way of spotting "speed regressions". This is probably > > quite hard to set up.... for lots of reasons.... but I'm wondering > > whether people think something like this is feasible. > > Martin > > -- > name: Martin Albrecht > _pgp:http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99 > _www:http://www.informatik.uni-bremen.de/~malb > _jab: [EMAIL PROTECTED] --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/ -~----------~----~----~----~------~----~------~--~---
