On Wed, 2008-06-25 at 12:56 -0400, Kent Johnson wrote: > On Wed, Jun 25, 2008 at 12:05 PM, Lie Ryan <[EMAIL PROTECTED]> wrote: > > > t_a = min(t_A, t_a) > > t_b = min(t_A, t_b) > > t_c = min(t_A, t_c) > > t_d = min(t_A, t_d) > > What is this for? It should at least be t_B, t_C, t_D.
A common pitfall in benchmarking is averaging the benchmark result. That is WRONG, FLAT WRONG. Why? Variations of how long a code is executed is caused by the environment, not the code itself. The correct way to benchmark is to use the one with the lowest time (i.e. min() function), since the lowest one is the one that is least interfered by the environment. > > ## OUTPUT > > # 1.02956604958 > > # 1.02956604958 > > # 1.02956604958 > > # 1.02956604958 > > It's *very easy* to write bogus timing tests, as this thread > demonstrates. Some protections: > - when comparing different implementations of a function, make sure > each implementation returns the correct result by checking the return > value. Since the purpose of the test is to benchmark the difference of where the code is located, we should use a very simple function, that doesn't even do much of anything, thus 'a = 1'. If that simple code is substituted with anything else, I'm still confident that the result won't be far off. > You probably want to make this check outside the actual timing > test. Actually the timing is all equal because of the timer's resolution. I don't have a high-precision timer on hand. > - when your results don't make sense, suspect your tests. > Kent > Kent _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
