I've recently started using go test's benchmarks support and I'm
particularly interested in understanding the benchmark timer functions.
I've been getting results that I found surprising and I was wondering if
anyone could explain what's going on here.
The code below has three benchmarks that each invoke a single function. The
implementation of foo isn't important, it's just something to take up some
time:
- foo is called once per iteration in Benchmark1.
- It's called twice per iteration in Benchmark2 so I'd expect Benchmark2's
duration to be nominally twice that of Benchmark1.
- It's also called twice per iteration in Benchmark3 but the first call is
wrapped in b.StopTimer and b.startTimer calls. Because of this I'd have
expected Benchmark3 to be about the same duration as Benchmark1
Apologies for the length of the example but I didn't think it fair to ask
the question and leave anything out.
package demo_test
import (
"strconv"
"testing"
)
var Foo1 []string
var Foo2 []string
var Count int = 32767
func Benchmark1(b *testing.B) {
for i := 0; i < b.N; i++{
Foo1 = foo(Count)
}
}
func Benchmark2(b *testing.B) {
for i := 0; i < b.N; i++{
Foo1 = foo(Count)
Foo2 = foo(Count)
}
}
func Benchmark3(b *testing.B) {
for i := 0; i < b.N; i++{
b.StartTimer()
Foo1 = foo(Count)
b.StopTimer()
Foo2 = foo(Count)
}
}
func setup(count int) []string{
testData := []string{}
for i:= 0; i < count; i++ {
testData = append(testData, strconv.Itoa(i))
}
return testData
}
func foo(count int) []string{
testData := []string{}
for i:= 0; i < count; i++ {
testData = append(testData, strconv.Itoa(i))
}
return testData
}
When the benchmarks are run the results are as follows:
Benchmark1-4 432 2805980 ns/op
Benchmark2-4 199 5930258 ns/op
Benchmark3-4 406 2909451 ns/op
PASS
ok bar.com/benchmarks 6.618s
OK benchmark3 is a little slower than Benchmark1 but that's not what's
bothering me. It's this: if I now change Count to something much smaller
the results are a surprise, at least to me. Here are the results when Count
= 8:
Benchmark1-4 2640558 453 ns/op
Benchmark2-4 1330572 898 ns/op
Benchmark3-4 943780 1292 ns/op
PASS
ok bar.com/benchmarks 35.257s
The timings for Benchmark1 and Benchmark2 are roughly in line with
expectations but I was surprised to see that the timings for Benchmark3 are
now larger than those for Benchmark2.
Can anyone explain this?
TIA
Orson
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/acbd4ca8-4199-4662-aca7-abf87b48b5f2%40googlegroups.com.
