yes
On Tue, Mar 24, 2020 at 11:48 AM Orson Cart <objectivedynam...@gmail.com>
wrote:
> On Tuesday, 24 March 2020 18:31:29 UTC, Michael Jones wrote:
>>
>> You use them to stop the time charged against your benchmark.
>>
>> For example:
>>
>> bench:
>> stop timer
>> generate initial data
>> start timer
>> do test
>>
>
> Thanks Michael. What if the initial data has to be generated from scratch
> on each iteration? Is the following expected to work?
>
> bench:
> b.N Loop
> stop timer
> generate new data
> start timer
> do test
>
>
>
>
>>
>> On Tue, Mar 24, 2020 at 10:47 AM Orson Cart <objectiv...@gmail.com>
>> wrote:
>>
>>>
>>> On Tuesday, 24 March 2020 16:49:55 UTC, Robert Engels wrote:
>>>>
>>>> Can you please succinctly explain the problem?
>>>>
>>>
>>> Let's see. Benchmarks can yield incorrect results when b.StopTimer and
>>> b.StartTimer are used.
>>>
>>> My reasoning:
>>>
>>> 1/ I have a benchmark that calls a single function. The reported
>>> duration is T.
>>> 2/ I have another benchmark which invokes the same function twice. I'd
>>> expect the reported duration for this to be nominally 2T and within reason
>>> it appears to be the case.
>>> 3/ I have yet another benchmark which also invokes the same function
>>> twice but it stops the benchmark timer before the first call and then
>>> starts it after the the first call has completed. I'd expect the reported
>>> duration to be nominally T again. It isn't. It's closer to 3T on my
>>> hardware and I've seen much worse correlation on other hardware - almost 5T
>>>
>>> Now it's entirely possible that I'm misunderstanding something about how
>>> b.StopTimer and b.StartTimer are intended to be used, hence my post here :)
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>>
>>>> On Mar 24, 2020, at 11:24 AM, Orson Cart <objectiv...@gmail.com> wrote:
>>>>
>>>>
>>>> I posted this earlier but I realised that the code had a fundamental
>>>> error in it. I've corrected here it but the underlying problem still
>>>> exists.
>>>>
>>>> I've recently started using go test's benchmarks support and I'm
>>>> particularly interested in understanding the benchmark timer functions.
>>>> I've been getting results that I found surprising and I was wondering if
>>>> anyone could explain what's going on here.
>>>>
>>>> The code below has three benchmarks that each invoke a single function
>>>> (foo). The implementation of foo isn't important, it's just there to
>>>> consume some time:
>>>> - foo is called once per iteration in Benchmark1.
>>>> - It's called twice per iteration in Benchmark2 so I'd expect
>>>> Benchmark2's duration to be nominally twice that of Benchmark1.
>>>> - It's also called twice per iteration in Benchmark3 but the first call
>>>> is wrapped in b.StopTimer and b.startTimer calls. Because of this I'd have
>>>> expected Benchmark3 to be about the same duration as Benchmark1
>>>>
>>>> Apologies for the length of the example but I didn't think it fair to
>>>> ask the question and leave anything out.
>>>>
>>>> package demo_test
>>>>
>>>> import (
>>>> "strconv"
>>>> "testing"
>>>> )
>>>>
>>>> var Foo1 []string
>>>> var Foo2 []string
>>>> var Count int = 32767
>>>>
>>>> func Benchmark1(b *testing.B) {
>>>> for i := 0; i < b.N; i++{
>>>> Foo1 = foo(Count)
>>>> }
>>>> }
>>>>
>>>> func Benchmark2(b *testing.B) {
>>>> for i := 0; i < b.N; i++{
>>>> Foo1 = foo(Count)
>>>> Foo2 = foo(Count)
>>>> }
>>>> }
>>>>
>>>> func Benchmark3(b *testing.B) {
>>>> for i := 0; i < b.N; i++{
>>>> b.StopTimer()
>>>> Foo1 = foo(Count)
>>>> b.StartTimer()
>>>> Foo2 = foo(Count)
>>>> }
>>>> }
>>>>
>>>> func foo(count int) []string{
>>>> testData := []string{}
>>>> for i:= 0; i < count; i++ {
>>>> testData = append(testData, strconv.Itoa(i))
>>>> }
>>>>
>>>> return testData
>>>> }
>>>>
>>>>
>>>> When the benchmarks are run the results are as follows:
>>>>
>>>> Benchmark1-4 351 3345215 ns/op
>>>> Benchmark2-4 166 7206582 ns/op
>>>> Benchmark3-4 334 3457907 ns/op
>>>> PASS
>>>> ok bar.com/benchmarks 6.881s
>>>>
>>>> OK benchmark3 is a little slower than Benchmark1 but that's not what's
>>>> bothering me. It's this: if I now change Count to something much smaller
>>>> the results are a surprise, at least to me. Here are the results when Count
>>>> = 8:
>>>>
>>>> Benchmark1-4 2706196 442 ns/op
>>>> Benchmark2-4 1357482 873 ns/op
>>>> Benchmark3-4 840729 1387 ns/op
>>>> PASS
>>>> ok bar.com/benchmarks 23.547s
>>>>
>>>> The ratio of timings for Benchmark1 and Benchmark2 are roughly in line
>>>> with expectations but I was surprised to see that the timings for
>>>> Benchmark3 are now larger than those for Benchmark2.
>>>>
>>>> Can anyone explain this?
>>>>
>>>> TIA
>>>> Orson
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to golan...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/golang-nuts/38f10940-a770-4fc5-86a9-19cc1371152a%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/golang-nuts/38f10940-a770-4fc5-86a9-19cc1371152a%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/1d291cef-420a-4f80-b719-e0654585c25d%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/1d291cef-420a-4f80-b719-e0654585c25d%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
>>
>> --
>>
>> *Michael T. jonesmichae...@gmail.com*
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/c3fcc6c5-9aa2-489c-adc4-c6b070ea6617%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/c3fcc6c5-9aa2-489c-adc4-c6b070ea6617%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
*Michael T. jonesmichael.jo...@gmail.com <michael.jo...@gmail.com>*
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyLgHddM9p%3DkJO1R-LojUq2DwPAMx%2BS94Dt4_uR1ypVTA%40mail.gmail.com.
