Steven D'Aprano <steve+pyt...@pearwood.info> added the comment:
What sort of statistics, and why do you think they are going to be meaningful?
Measurement errors in the timings aren't two-sided, they are only one-sided,
which means calculating the mean and standard deviation isn't appropriate. This
is documented in the Timer.repeat() method:
Note: it's tempting to calculate mean and standard deviation
from the result vector and report these. However, this is not
very useful. In a typical case, the lowest value gives a lower
bound for how fast your machine can run the given code snippet;
higher values in the result vector are typically not caused by
variability in Python's speed, but by other processes interfering
with your timing accuracy. So the min() of the result is
probably the only number you should be interested in.
The purpose of the mean is to estimate the actual time taken by the
computation, less any random errors ("noise") in the measurement values.
If random noise is centred around zero, then the positive errors and the
negative errors tend to cancel, and the mean is the best estimate of the actual
computation time. But the noise isn't centred on zero, there are no negative
noise, and the mean estimates the computation time plus the average measurement
error, not the computation time.
In other words, given a bunch of measurements:
timer = Timer(code, setup)
values = timer.repeat()
the closest estimate to the true computation time is min(values), not
mean(values).
If you still insist on generating statistics, they're not difficult to collect
using the statistics module:
statistics.mean(values)
statistics.stdev(values)
If there are additional statistics you would like to see, they should be added
to the statistics module, not timeit.
----------
nosy: +steven.daprano, tim.peters
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32589>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
