Hi, Neil Jerram <n...@ossau.homelinux.net> skribis:
> l...@gnu.org (Ludovic Courtès) writes: > >>> My proposal is to rebase the iteration count in 0-reference.bm to run >>> for 0.5s on some modern machine, and adjust all benchmarks to match, >>> removing those benchmarks that do not measure anything useful. >> >> Sounds good. However, adjusting iteration counts of the benchmarks >> themselves should be done rarely, as it breaks performance tracking like >> <http://ossau.homelinux.net/~neil/bm_master_i.html>. >> >>> Finally we should perhaps enable automatic scaling of the iteration >>> count. What do folks think about that? >>> >>> On the positive side, all of our benchmarks are very clear that they are >>> a time per number of iterations, and so this change should not affect >>> users that measure time per iteration. >> >> If the reported time is divided by the global iteration count, then >> automatic scaling of the global iteration count would be good, yes. > > For http://ossau.homelinux.net/~neil I do still have all of the raw data > including iteration counts, so I could easily implement dividing by the > iteration count, and hence allow for future iteration count changes. > > Is there any downside from doing that? (I don't think so.) No, I guess. And as you show, having raw data instead of synthesized figures gives more freedom. Thanks, Ludo’.