On 6/22/20 5:07 PM, Antoine Pitrou wrote:
Le 22/06/2020 à 06:27, Micah Kornfield a écrit :
There has been significant effort recently trying to optimize our C++
code. One thing that seems to come up frequently is different benchmark
results between GCC and Clang. Even different versions of the same
compiler can yield significantly different results on the same code.
I would like to propose that we choose a specific compiler and version on
Linux for evaluating performance related PRs. PRs would only be accepted
if they improve the benchmarks under the selected version.
Would this be a hard rule or just a guideline? There are many ways in
which benchmark numbers can be improved or deteriorated by a PR, and in
some cases that doesn't matter (benchmarks are not always realistic, and
they are not representative of every workload).
I agree that microbenchmark is not always useful, focusing too much on
improving microbenchmark result gives me feeling of "overfit" (to some
specific microarchitecture, compiler, or use case).
Regards
Antoine.
