I'll try to clarify as best as I can, thanks again to anyone looking at this.
The simple server implementation of "output <- input+1" is here and it is not "under our control" - it's what we have to work with: https://github.com/egonk/chandemo/blob/master/server.go The test runner or client is here: https://github.com/egonk/chandemo/blob/master/demo.go (it just pushes in ints and gets server replies back through a connection layer) The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong implementation of bidi-comm, which is what I'll be illustrating. I have three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can we remove the extra goroutine from 1_1.go and make the code nicer to read than 2_3.go and 2_4.go. The extra goroutine that I'd like to be removed is started here: https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14) What I mean by removed - no go statement, replaced presumably by some kind of for/select combination. On Saturday, December 7, 2019 at 7:02:50 AM UTC+1, robert engels wrote: > > I’m sorry but your design is not comprehendible by me, and I’ve done lots > of TCP based services. > > i think you only need to emulate classic TCP processing - a reader thread > (Go routine) on each side of the connection using range to read until > closed. The connection is represented by 2 channels - one for each > direction. > > I think you might be encountering a deadlock because the producer on one > end is not also reading the incoming - so either restructure, or use 2 more > threads for the producers. > > > > On Dec 6, 2019, at 10:38 PM, Egon Kocjan <eko...@gmail.com <javascript:>> > wrote: > > Agreed, I see goroutines in general as a big win. But what I intend to > talk about in the presentation: > - we have two unidirectional flows of data resembling something like a TCP > socket, easy to do with two goroutines with a for loop > - let's add caching, so some requests do not go to the server > - it would be tempting to just combine two goroutines into one and handle > caching in a single loop without using locks (I see developers avoid > atomics and locks if they don't have a lot of previous experience with > traditional MT primitives) > - this is surprisingly difficult to do properly with Go channels, see my > attempts: https://github.com/egonk/chandemo/blob/master/2_3.go and > https://github.com/egonk/chandemo/blob/master/2_4.go > <https://github.com/egonk/chandemo/blob/master/2_3.go> > - it is easy to do in actor systems, just move the code for both actors > into a single actor! > > The lesson here is that select is not a nice and safe compose statement > even if it appears so at the first glance, do not be afraid to use locks. > > Of course, if somebody comes up with a better implementation than 2_3.go > and 2_4.go, I would be very happy to include it in the talk. > > On Saturday, December 7, 2019 at 4:17:04 AM UTC+1, robert engels wrote: >> >> To clarify, with Go’s very lightweight threads it is “doing the >> multiplexing for you” - often only a single CPU is consumed if the producer >> and consumer work cannot be parallelized, otherwise you get this >> concurrency “for free”. >> >> You are trying to manually perform the multiplexing - you need async >> structures to do this well - Go doesn’t really support async by design - >> and it’s a much simpler programming model as a result. >> >> On Dec 6, 2019, at 12:02 PM, Robert Engels <ren...@ix.netcom.com> wrote: >> >> A channel is much closer to a pipe. There are producers and consumers and >> these are typically different threads of execution unless you have an event >> based (async) system - that is not Go. >> >> On Dec 6, 2019, at 9:30 AM, Egon Kocjan <eko...@gmail.com> wrote: >> >> >> There are goroutines in the examples of course, just a single goroutine >> per bidi channel seems hard. By contrast, I've worked with actor systems >> before and they are perfectly fine with a single fiber. >> >> On Friday, December 6, 2019 at 3:38:20 PM UTC+1, Robert Engels wrote: >>> >>> Channels are designed to be used with multiple go routines - if you’re >>> not you are doing something wrong. >>> >>> On Dec 6, 2019, at 8:32 AM, Egon Kocjan <eko...@gmail.com> wrote: >>> >>> >>> Hello >>> >>> I'm preparing a short talk about Go channels and select. More >>> specifically, I want to show what not to do. I chose a bidirectional >>> communication channel implementation, because it seems to be a common base >>> for a lot of problems but hard to implement correctly without using any >>> extra goroutines. All the code is here: >>> https://github.com/egonk/chandemo >>> >>> 1_1.go: easy with en extra goroutine (takes 1.2s for million ints) >>> 2_1.go: nice but completely wrong >>> 2_2.go: better but still deadlocks >>> 2_3.go: correct but ugly and slow (takes more than 2s for million ints) >>> 2_4.go: correct and a bit faster but still ugly (1.8s for million ints) >>> >>> So my question: is there a better way of doing it with just nested for >>> and select and no goroutines? Basically, what would 2_5.go look like? >>> >>> Thank you >>> Egon >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golan...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golan...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/bdc57eb0-b26f-4364-87fb-241b0807e8ae%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/bdc57eb0-b26f-4364-87fb-241b0807e8ae%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golan...@googlegroups.com <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/75d69b4e-4fb7-4f62-8011-f21e2a4c294a%40googlegroups.com > > <https://groups.google.com/d/msgid/golang-nuts/75d69b4e-4fb7-4f62-8011-f21e2a4c294a%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8b87adcc-2249-402c-b34c-20df5013860a%40googlegroups.com.
