[issue23552] Have timeit warn about runs that are not independent of each other

Changes by Robert Collins : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.6 -Python 3.5 ___ Python tracker ___ _

[issue23552] Have timeit warn about runs that are not independent of each other

Roundup Robot added the comment: New changeset 2e9cf58c891d by Robert Collins in branch 'default': Issue #23552: Timeit now warns when there is substantial (4x) variance https://hg.python.org/cpython/rev/2e9cf58c891d -- nosy: +python-dev ___ Python tr

[issue23552] Have timeit warn about runs that are not independent of each other

Changes by Serhiy Storchaka : Removed file: http://bugs.python.org/file38532/timeit_python_warning.diff ___ Python tracker ___ ___ Python-bugs

[issue23552] Have timeit warn about runs that are not independent of each other

Serhiy Storchaka added the comment: Implemented Robert's suggestion. $ ./python -m timeit -n1 -r 10 -s "import time, random" -- "time.sleep(random.random())" 1 loops, best of 10: 30.2 msec per loop :0: UserWarning: The test results are likely unreliable. The worst time (946 msec) was more than

[issue23552] Have timeit warn about runs that are not independent of each other

Serhiy Storchaka added the comment: Here is a patch that emits a warning using the warnings module. The warning is output to stderr and can be suppressed with the -Wignore option, as all other warnings. $ ./python -m timeit -n1 -r 10 -s "import time, random" -- "time.sleep(random.random())" 1

[issue23552] Have timeit warn about runs that are not independent of each other

Robert Collins added the comment: Reviewed on rietvald. -- stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing

[issue23552] Have timeit warn about runs that are not independent of each other

Robert Collins added the comment: I think for PyPI its actually important here - the JIT'd state of the code is essentially global state being mutated - you can't assess how fast the code is without first warming up the JIT, and if it warms up half way through your fastest run, you're still no

[issue23552] Have timeit warn about runs that are not independent of each other

Serhiy Storchaka added the comment: May be add an option to control a warning (e.g. turn it off when Python run with -Wignore)? May be write warning to stderr? -- ___ Python tracker ___

[issue23552] Have timeit warn about runs that are not independent of each other

Alex Gaynor added the comment: This seems like it probably will report something useless (and ultimately be disabled) on PyPy, where runs before and after the JIT will display significant variance. -- nosy: +alex, arigo, fijall ___ Python tracker <

[issue23552] Have timeit warn about runs that are not independent of each other

Raymond Hettinger added the comment: See attached -- keywords: +patch Added file: http://bugs.python.org/file38294/timeit_warning.diff ___ Python tracker ___

[issue23552] Have timeit warn about runs that are not independent of each other

Serhiy Storchaka added the comment: How it looks? -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue23552] Have timeit warn about runs that are not independent of each other

New submission from Raymond Hettinger: IPython 3.0 added a useful feature that we ought to consider for inclusion either in timeit.repeat() or in the command-line interface: """Using %timeit prints warnings if there is at least a 4x difference in timings between the slowest and fastest runs, s