On Tue, Mar 6, 2012 at 8:27 PM, pcrews <[email protected]> wrote: > 2) What do we want to do with the data on a failure? Do we want to just > remove the run? (I need to refresh my memory on how drizzle-automation > currently handles things)
For performance regressions, you of course want to keep the data especially for failures. (I assume you mean the output from the benchmark?) I haven't followed these closely, but from superficial browsing it seems that the variability between normal runs is only 1-2 percent. The recent regression seems to have been like 15 percent. So clearly visible. The threshold should probably be configurable, but in any case will be something between 3 and 5 %. henrik -- [email protected] +358-40-8211286 skype: henrik.ingo irc: hingo www.openlife.cc My LinkedIn profile: http://www.linkedin.com/profile/view?id=9522559 _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
