Hi, Your document doesn't explain how you configured the host to run benchmarks. Maybe you didn't tune Linux or anything else? Be careful with modern hardware which can make funny (or not) surprises. See my recent talk at FOSDEM (last month):
"How to run a stable benchmark" https://fosdem.org/2017/schedule/event/python_stable_benchmark/ Factors impacting Python benchmarks: * Linux Address Space Layout Randomization (ASRL), /proc/sys/kernel/randomize_va_space * Python random hash function: PYTHONHASHSEED * Command line arguments and environmnet variables: enabling ASLR helps here (?) * CPU power saving and performance features: disable Intel Turbo Boost and/or use a fixed CPU frequency. * Temperature: temperature has a limited impact on benchmarks. If the CPU is below 95°C, Intel CPUs still run at full speed. With a correct cooling system, temperature is not an issue. * Linux perf probes: /proc/sys/kernel/perf_event_max_sample_rate * Code locality, CPU L1 instruction cache (L1c): Profiled Guided Optimization (PGO) helps here * Other processes and the kernel, CPU isolation (CPU pinning) helps here: use isolcpus=cpu_list and rcu_nocbs=cpu_list on the * Linux kernel command line * ... Reboot? Sadly, other unknown factors may still impact benchmarks. Sometimes, it helps to reboot to restore standard performances. https://haypo-notes.readthedocs.io/microbenchmark.html#factors-impacting-benchmarks Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
