Hi Mike, Thank you for your consideration. I think you exactly got the essence of my question: How do I wait on all go routines to finish (or be blocked on one or more channels) before advancing time.
A key thing I would like from such a solution is that it does not require too heavy modifications to the code to be tested or put restrictions on how it can do things. I think it may be possible to solve it with some explicit check in / check out as I think you also suggest. I guess in essence you will check out before you call select and check in again after select is done waiting. I think this will still not work if buffered channels are used. But maybe if buffered channels are mocked, it may be doable. I think I may want to make a feature request on this. I see several options: * Make a version of runtime.gosched that only returns when no other go routines can run * Make it possible to read the number of go routines that are ready to run. You could then make a loop where you call runtime.gosched until that value is 0. * Make it possible to start a special go routine when the system is deadlocked. One problem is what to do if the program is waiting on external IO such as the completion of an HTTP request. I guess in an ideal solution it would be possible for the program to decide if it will advance time in that situation or not. Please let me know if you have any ideas of other things to put into the feature request. Thanks, Christian On Fri, Jan 29, 2021 at 9:25 PM mspr...@us.ibm.com <mspre...@us.ibm.com> wrote: > Volker: injecting sleep is a nice idea, in the general vein that Jesper > said of injecting time. However, as soon as we zoom out a step and need to > test both that generator and the goroutine(s) consuming and acting upon > that channel activity, we get back to the essence of the original question: > how to test when we have a bunch of goroutines doing stuff and the test > needs to wait for them all to finish before advancing time? > > FYI, in Kubernetes we have done something similar to the Facebook clock > package --- but recently we have called out the narrower interface used by > code that only reads time. See PassiveClock in > https://github.com/kubernetes/utils/blob/master/clock/clock.go and > https://github.com/kubernetes/apimachinery/blob/master/pkg/util/clock/clock.go > (yeah, we have two forked lines of development of this clock thing, sigh). > > The pattern of using channel activity to coordinate asynchronous activity > is inherently inimical to what the original poster asked for. An > alternative is to define clocks that run procedures rather than do channel > sends. See the EventClock in > https://github.com/kubernetes/apiserver/blob/master/pkg/util/flowcontrol/fairqueuing/testing/clock/event_clock.go > . A mocked one of those could know when all the timed activities have > completed --- if all the timed activities were synchronously contained in > EventFuncs. Sadly this is too restrictive a pattern for a lot of real > code. You will see in that package an additional idea: explicitly tracking > (at "user level") when the goroutines in question block/unblock. This is > painful, but I see no better way (given the golang runtime interface as it > is defined today). > > Regards, > Mike > > On Friday, January 29, 2021 at 10:11:34 AM UTC-5 Volker Dobler wrote: > >> One way to do this is have an internal implementation like >> func generatorImpl(sleep func(time.Duration)) <-chan int >> and func generator just calls that one with time.Sleep. >> Tests are done against generatorImpl where you know have >> detailed control of how much (typically none) time is >> actually slept. >> >> Expiration of cookies is tested in that way, see e.g. >> https://golang.org/src/net/http/cookiejar/jar.go#L159 >> So while technically Jar.Cookies is never tested the >> risk is basically nil. >> >> V. >> On Thursday, 28 January 2021 at 22:15:50 UTC+1 Christian Worm Mortensen >> wrote: >> >>> Hi! >>> >>> Suppose I want to unit test this function: >>> >>> func generator() <-chan int { >>> ret := make(chan int) >>> go func() { >>> for i := 0; i < 10; i++ { >>> ret <- i >>> time.Sleep(time.Second) >>> } >>> }() >>> return ret >>> } >>> >>> What is a good way to do that? One way is to do it is like this: >>> >>> func testGenerator() { >>> start := time.Now() >>> g := generator() >>> for i := 0; i < 10; i++ { >>> v := <-g >>> if v != i { >>> panic("Wrong value") >>> } >>> } >>> elapsed := time.Now().Sub(start) >>> if elapsed < 9*time.Second || elapsed > 11*time.Second { >>> panic("Wrong execution time") >>> } >>> } >>> >>> However there are several issues with this: >>> >>> 1) The unit test takes a long time to run - 10 seconds. >>> 2) The unit test is fragile to fluctuations in CPU availability >>> 3) The unit test is not very accurate >>> >>> Of course this is a simple example. But what if I want to test a >>> complicated piece of code with many go routines interacting in complicated >>> ways and with long timeouts? >>> >>> In other programming languages, I have been able to implement a form of >>> virtual time which increases only when all threads are waiting for time to >>> increase. This allows functions like generator above to be tested basically >>> instantly and this has been extremely useful for me in many projects over >>> the years. >>> >>> Can I do something similar in Go? I would expect I would need to wrap >>> time.Now, time.Sleep and time.After which I will be happy to do. >>> >>> I can see that Go has a deadlock detector. If somehow it was possible to >>> have Go start a new Go routine when a deadlock was detected, I think it >>> would be pretty straight forward to implement virtual time as described. I >>> could then do something like: >>> >>> runtime.registerDeadlockCallback(func () { >>> // Increase virtual time and by that: >>> // * Make one or more wrapped time.Sleep calls return or >>> // * Write to one or more channels returned by wrapped time.After. >>> }) >>> >>> Obviously this would only be needed for test code, not production code. >>> >>> Thanks, >>> >>> Christian >>> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "golang-nuts" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/golang-nuts/Y9Ccen0uMcs/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/9f5fa53a-64ca-483a-8e63-bae5c061e569n%40googlegroups.com > <https://groups.google.com/d/msgid/golang-nuts/9f5fa53a-64ca-483a-8e63-bae5c061e569n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CABTkUoZcX8LHgmO1_NA3Bh9vN071s_H6dv-%3DMMrgTAXbji%2BpEA%40mail.gmail.com.
