[email protected] (Ludovic Courtès) writes: >> My proposal is to rebase the iteration count in 0-reference.bm to run >> for 0.5s on some modern machine, and adjust all benchmarks to match, >> removing those benchmarks that do not measure anything useful. > > Sounds good. However, adjusting iteration counts of the benchmarks > themselves should be done rarely, as it breaks performance tracking like > <http://ossau.homelinux.net/~neil/bm_master_i.html>. > >> Finally we should perhaps enable automatic scaling of the iteration >> count. What do folks think about that? >> >> On the positive side, all of our benchmarks are very clear that they are >> a time per number of iterations, and so this change should not affect >> users that measure time per iteration. > > If the reported time is divided by the global iteration count, then > automatic scaling of the global iteration count would be good, yes.
For http://ossau.homelinux.net/~neil I do still have all of the raw data including iteration counts, so I could easily implement dividing by the iteration count, and hence allow for future iteration count changes. Is there any downside from doing that? (I don't think so.) Regards, Neil
