2016-06-10 17:09 GMT+02:00 Paul Moore <[email protected]>: > Also, the way people commonly use > micro-benchmarks ("hey, look, this way of writing the expression goes > faster than that way") doesn't really address questions like "is the > difference statistically significant".
If you use the "python3 -m perf compare method1.json method2.json", perf will checks that the difference is significant using the is_significant() method: http://perf.readthedocs.io/en/latest/api.html#perf.is_significant "This uses a Student’s two-sample, two-tailed t-test with alpha=0.95." FYI at the beginning, this function comes from the Unladen Swallow benchmark suite ;-) We should design a CLI command to do timeit+compare at once. Victor _______________________________________________ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
