Hi, I'm trying to understand the benchstat's output. The benchstat that I have is fresh from golang.org/x/perf/cmd/, installed hours ago.
This is understandable: name old time/op new time/op delta FormatEmoji-2 1.79µs ± 0% 1.82µs ± 1% ~ (p=0.100 n=3+3) FormatEmoji-4 1.99µs ± 1% 1.91µs ± 4% ~ (p=0.200 n=3+3) FormatEmoji-8 1.97µs ± 3% 1.99µs ± 1% ~ (p=0.700 n=3+3) FormatEmoji-12 2.05µs ± 1% 2.03µs ± 1% ~ (p=0.400 n=3+3) name old alloc/op new alloc/op delta FormatEmoji-2 384B ± 0% 384B ± 0% ~ (p=1.000 n=3+3) FormatEmoji-4 386B ± 0% 385B ± 0% ~ (p=0.700 n=3+3) FormatEmoji-8 387B ± 0% 386B ± 0% ~ (p=0.400 n=3+3) FormatEmoji-12 388B ± 0% 388B ± 0% ~ (p=1.000 n=3+3) name old allocs/op new allocs/op delta FormatEmoji-2 7.00 ± 0% 7.00 ± 0% ~ (all equal) FormatEmoji-4 7.00 ± 0% 7.00 ± 0% ~ (all equal) FormatEmoji-8 7.00 ± 0% 7.00 ± 0% ~ (all equal) FormatEmoji-12 7.00 ± 0% 7.00 ± 0% ~ (all equal) However, Why benchstat thinks there is no differences between the following two runs? $ benchstat /tmp/old.log /tmp/new.log name old time/op new time/op delta FormatEmojiString-2 1.79µs ± 0% 6.81µs ± 2% ~ (p=0.100 n=3+3) FormatEmojiString-4 1.99µs ± 1% 7.22µs ± 1% ~ (p=0.100 n=3+3) FormatEmojiString-8 1.97µs ± 3% 7.55µs ± 2% ~ (p=0.100 n=3+3) FormatEmojiString-12 2.05µs ± 1% 7.64µs ± 1% ~ (p=0.100 n=3+3) name old alloc/op new alloc/op delta FormatEmojiString-2 384B ± 0% 1459B ± 0% ~ (p=0.100 n=3+3) FormatEmojiString-4 386B ± 0% 1463B ± 0% ~ (p=0.100 n=3+3) FormatEmojiString-8 387B ± 0% 1468B ± 0% ~ (p=0.100 n=3+3) FormatEmojiString-12 388B ± 0% 1473B ± 0% ~ (p=0.100 n=3+3) name old allocs/op new allocs/op delta FormatEmojiString-2 7.00 ± 0% 25.00 ± 0% ~ (p=0.100 n=3+3) FormatEmojiString-4 7.00 ± 0% 25.00 ± 0% ~ (p=0.100 n=3+3) FormatEmojiString-8 7.00 ± 0% 25.00 ± 0% ~ (p=0.100 n=3+3) FormatEmojiString-12 7.00 ± 0% 25.00 ± 0% ~ (p=0.100 n=3+3) I'd think that the delta will be quite big. Now take a look at this. benchstat thinks there is no differences between the following two runs is beyond imagination: $ benchstat /tmp/old.log /tmp/new.log name old time/op new time/op delta FormatEmojiBlock-2 1.79µs ± 0% 13543.09µs ± 4% ~ (p=0.100 n=3+3) FormatEmojiBlock-4 1.99µs ± 1% 14597.09µs ± 1% ~ (p=0.100 n=3+3) FormatEmojiBlock-8 1.97µs ± 3% 16390.62µs ± 2% ~ (p=0.100 n=3+3) FormatEmojiBlock-12 2.05µs ± 1% 17409.37µs ± 2% ~ (p=0.100 n=3+3) name old alloc/op new alloc/op delta FormatEmojiBlock-2 384B ± 0% 15270330B ± 0% ~ (p=0.100 n=3+3) FormatEmojiBlock-4 386B ± 0% 15270671B ± 0% ~ (p=0.100 n=3+3) FormatEmojiBlock-8 387B ± 0% 15271206B ± 0% ~ (p=0.100 n=3+3) FormatEmojiBlock-12 388B ± 0% 15271727B ± 0% ~ (p=0.100 n=3+3) name old allocs/op new allocs/op delta FormatEmojiBlock-2 7.00 ± 0% 3467.00 ± 0% ~ (p=0.100 n=3+3) FormatEmojiBlock-4 7.00 ± 0% 3467.00 ± 0% ~ (p=0.100 n=3+3) FormatEmojiBlock-8 7.00 ± 0% 3467.00 ± 0% ~ (p=0.100 n=3+3) FormatEmojiBlock-12 7.00 ± 0% 3467.00 ± 0% ~ (p=0.100 n=3+3) What's happening? What I've done wrong? Thx -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a56a17b1-a8c3-4926-a27a-21cd86074f67n%40googlegroups.com.
