On Tuesday, 24 March 2020 20:47:07 UTC, Robert Engels wrote:
>
> One way to handle this is to generate all of the data up front in an array
> and then just index into the array based on the run.
>
Yeah, I had thought of that before posting but then I'd have to decide on a
value for b.N. I was trying to roll with the idea of the framework
establishing a value. To be honest if it wasn't for the fact that I was
just curious as to why it wasn't working I'd just have settled for an array
with a fixed b.N.
All part of my golang learning experience
>
> On Mar 24, 2020, at 3:42 PM, Orson Cart <objectiv...@gmail.com
> <javascript:>> wrote:
>
>
>
>
> On Tuesday, 24 March 2020 20:27:39 UTC, Adrian Ratnapala wrote:
>>
>> ...
>
> So that sounds like the use-case is to call Stop-, StartTimer once
>> before you enter the main loop of the benchmark. They not efficient
>> enough for a tight inner loop.
>>
>
> Thanks for the input and I think that you are probably correct. It would
> be nice if the docs said that though.
>
> The code that I'm profiling wasn't written by me. It's fast but it
> modifies its input data so it needs new data on each iteration. The
> implementation is non-trivial and it might be a maintenance headache. I've
> been asked to benchmark it along with less complicated implementations. It
> was whilst trying to set up per-iteration test data that I came across this
> issue.
>
>
>
>
>
>
>
>
>
>>
>> On Wed, 25 Mar 2020 at 06:51, Jake Montgomery <jake...@gmail.com> wrote:
>> >
>> > Strange. I hope someone has a real answer for you.
>> >
>> > In the meantime, you can simplify your example to demonstrate the
>> issue:
>> >
>> > package demo_test
>> >
>> > import (
>> > "testing"
>> > )
>> >
>> > var Foo1 []string
>> > var Count int = 8
>> >
>> > func Benchmark1(b *testing.B) {
>> > for i := 0; i < b.N; i++ {
>> > Foo1 = foo(Count)
>> > }
>> > }
>> >
>> > func Benchmark2(b *testing.B) {
>> > b.StopTimer()
>> > for i := 0; i < b.N; i++ {
>> > // Hypothetical setup here
>> > b.StartTimer()
>> > Foo1 = foo(Count)
>> > b.StopTimer()
>> > }
>> > }
>> >
>> >
>> > func foo(count int) []string {
>> > testData := []string{}
>> > for i := 0; i < count; i++ {
>> > testData = append(testData, "a")
>> > }
>> >
>> > return testData
>> > }
>> >
>> > I get:
>> >
>> > goos: windows
>> > goarch: amd64
>> > Benchmark1-4 2101567 584 ns/op
>> > Benchmark2-4 1000000 1668 ns/op
>> > PASS
>> >
>> > So it appears that StopTimer() and StartTimer() are introducing some
>> overhead. I am surprised that it is this large.
>> >
>> > Good Luck
>> >
>> >
>> >
>> > On Tuesday, March 24, 2020 at 12:24:08 PM UTC-4, Orson Cart wrote:
>> >>
>> >> I posted this earlier but I realised that the code had a fundamental
>> error in it. I've corrected here it but the underlying problem still
>> exists.
>> >>
>> >> I've recently started using go test's benchmarks support and I'm
>> particularly interested in understanding the benchmark timer functions.
>> I've been getting results that I found surprising and I was wondering if
>> anyone could explain what's going on here.
>> >>
>> >> The code below has three benchmarks that each invoke a single function
>> (foo). The implementation of foo isn't important, it's just there to
>> consume some time:
>> >> - foo is called once per iteration in Benchmark1.
>> >> - It's called twice per iteration in Benchmark2 so I'd expect
>> Benchmark2's duration to be nominally twice that of Benchmark1.
>> >> - It's also called twice per iteration in Benchmark3 but the first
>> call is wrapped in b.StopTimer and b.startTimer calls. Because of this I'd
>> have expected Benchmark3 to be about the same duration as Benchmark1
>> >>
>> >> Apologies for the length of the example but I didn't think it fair to
>> ask the question and leave anything out.
>> >>
>> >> package demo_test
>> >>
>> >> import (
>> >> "strconv"
>> >> "testing"
>> >> )
>> >>
>> >> var Foo1 []string
>> >> var Foo2 []string
>> >> var Count int = 32767
>> >>
>> >> func Benchmark1(b *testing.B) {
>> >> for i := 0; i < b.N; i++{
>> >> Foo1 = foo(Count)
>> >> }
>> >> }
>> >>
>> >> func Benchmark2(b *testing.B) {
>> >> for i := 0; i < b.N; i++{
>> >> Foo1 = foo(Count)
>> >> Foo2 = foo(Count)
>> >> }
>> >> }
>> >>
>> >> func Benchmark3(b *testing.B) {
>> >> for i := 0; i < b.N; i++{
>> >> b.StopTimer()
>> >> Foo1 = foo(Count)
>> >> b.StartTimer()
>> >> Foo2 = foo(Count)
>> >> }
>> >> }
>> >>
>> >> func foo(count int) []string{
>> >> testData := []string{}
>> >> for i:= 0; i < count; i++ {
>> >> testData = append(testData, strconv.Itoa(i))
>> >> }
>> >>
>> >> return testData
>> >> }
>> >>
>> >>
>> >> When the benchmarks are run the results are as follows:
>> >>
>> >> Benchmark1-4 351 3345215 ns/op
>> >> Benchmark2-4 166 7206582 ns/op
>> >> Benchmark3-4 334 3457907 ns/op
>> >> PASS
>> >> ok bar.com/benchmarks 6.881s
>> >>
>> >> OK benchmark3 is a little slower than Benchmark1 but that's not what's
>> bothering me. It's this: if I now change Count to something much smaller
>> the results are a surprise, at least to me. Here are the results when Count
>> = 8:
>> >>
>> >> Benchmark1-4 2706196 442 ns/op
>> >> Benchmark2-4 1357482 873 ns/op
>> >> Benchmark3-4 840729 1387 ns/op
>> >> PASS
>> >> ok bar.com/benchmarks 23.547s
>> >>
>> >> The ratio of timings for Benchmark1 and Benchmark2 are roughly in line
>> with expectations but I was surprised to see that the timings for
>> Benchmark3 are now larger than those for Benchmark2.
>> >>
>> >> Can anyone explain this?
>> >>
>> >> TIA
>> >> Orson
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to golan...@googlegroups.com.
>> > To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/feda7e38-5d1f-43cf-b0cd-98db0a94d3c9%40googlegroups.com.
>>
>>
>>
>>
>>
>> --
>> Adrian Ratnapala
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golan...@googlegroups.com <javascript:>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/291af5de-8a01-4c30-98bb-d4765e3a5333%40googlegroups.com
>
> <https://groups.google.com/d/msgid/golang-nuts/291af5de-8a01-4c30-98bb-d4765e3a5333%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/ba489845-c602-440a-bce9-d99386355481%40googlegroups.com.
