Joe Buck wrote: > It would be possible to detect performance regression after fact, but > soon enough to look at reverting patches. For example, given multiple > machines doing SPEC benchmark runs every night, the alarm could be raised > if a significant performance regression is detected.
Right; I think we do some of that at present. I was hoping that having it there when people did test runs would change the psychology; instead of having already checked in a patch, which we're then looking to revert, we'd be making ourselves aware of performance impact before check-in, even for patches that we don't expect to have performance impact. (For major new optimizations, we already expect people to do some benchmarking.) But, yes, this is a definite alternative: we could further automate the SPEC testers, or try to set up more of them. >>As a strawman, perhaps we could add a small integer program (bzip?) and >>a small floating-point program to the testsuite, and have DejaGNU print >>out the number of iterations of each that run in 10 seconds. > > Would that really catch much? I really don't know. That's why it's a strawman. :-) -- Mark Mitchell CodeSourcery, LLC [EMAIL PROTECTED] (916) 791-8304