I understand what you are saying but I’ll still suggest that your premise/design is not correct. There are plenty of useful lock free structures in Go (see github.com/robaho/go-concurrency-test) but that is not what you are attempting here... you are using async processing - these are completely different things. Using async in Go is an anti-pattern IMO.
> On Dec 8, 2019, at 12:11 AM, Egon Kocjan <ekoc...@gmail.com> wrote: > > > I'll cite myself: > "I'm preparing a short talk about Go channels and select. More specifically, > I want to show what not to do." > and > "it would be tempting to just combine two goroutines into one and handle > caching in a single loop without using locks (I see developers avoid atomics > and locks if they don't have a lot of previous experience with traditional MT > primitives)" > > Before I say one can't do something in Go, I wanted to ask here to make sure > I'm not missing something obvious. Basically, I intend to show how difficult > lock-free programming can be so don't force it - just use goroutines and > locks. > >> On Saturday, December 7, 2019 at 3:46:43 PM UTC+1, Robert Engels wrote: >> Probably not. Go is designed for 1:1 and there is no reason to do it >> differently. You could probably try to write an async event driven layer >> (which it looks like you’ve tried) but why??? >> >> It’s like saying I’d really like my plane to float - you can do that -but >> most likely you want a boat instead of a plane. >> >>>> On Dec 7, 2019, at 2:38 AM, Egon Kocjan <eko...@gmail.com> wrote: >>>> >>> >>> I'll try to clarify as best as I can, thanks again to anyone looking at >>> this. >>> >>> The simple server implementation of "output <- input+1" is here and it is >>> not "under our control" - it's what we have to work with: >>> https://github.com/egonk/chandemo/blob/master/server.go >>> >>> The test runner or client is here: >>> https://github.com/egonk/chandemo/blob/master/demo.go (it just pushes in >>> ints and gets server replies back through a connection layer) >>> >>> The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong >>> implementation of bidi-comm, which is what I'll be illustrating. I have >>> three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can >>> we remove the extra goroutine from 1_1.go and make the code nicer to read >>> than 2_3.go and 2_4.go. The extra goroutine that I'd like to be removed is >>> started here: >>> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14) >>> >>> What I mean by removed - no go statement, replaced presumably by some kind >>> of for/select combination. >>> >>>> On Saturday, December 7, 2019 at 7:02:50 AM UTC+1, robert engels wrote: >>>> I’m sorry but your design is not comprehendible by me, and I’ve done lots >>>> of TCP based services. >>>> >>>> i think you only need to emulate classic TCP processing - a reader thread >>>> (Go routine) on each side of the connection using range to read until >>>> closed. The connection is represented by 2 channels - one for each >>>> direction. >>>> >>>> I think you might be encountering a deadlock because the producer on one >>>> end is not also reading the incoming - so either restructure, or use 2 >>>> more threads for the producers. >>>> >>>> >>>> >>>>> On Dec 6, 2019, at 10:38 PM, Egon Kocjan <eko...@gmail.com> wrote: >>>>> >>>>> Agreed, I see goroutines in general as a big win. But what I intend to >>>>> talk about in the presentation: >>>>> - we have two unidirectional flows of data resembling something like a >>>>> TCP socket, easy to do with two goroutines with a for loop >>>>> - let's add caching, so some requests do not go to the server >>>>> - it would be tempting to just combine two goroutines into one and handle >>>>> caching in a single loop without using locks (I see developers avoid >>>>> atomics and locks if they don't have a lot of previous experience with >>>>> traditional MT primitives) >>>>> - this is surprisingly difficult to do properly with Go channels, see my >>>>> attempts: https://github.com/egonk/chandemo/blob/master/2_3.go and >>>>> https://github.com/egonk/chandemo/blob/master/2_4.go >>>>> - it is easy to do in actor systems, just move the code for both actors >>>>> into a single actor! >>>>> >>>>> The lesson here is that select is not a nice and safe compose statement >>>>> even if it appears so at the first glance, do not be afraid to use locks. >>>>> >>>>> Of course, if somebody comes up with a better implementation than 2_3.go >>>>> and 2_4.go, I would be very happy to include it in the talk. >>>>> >>>>>> On Saturday, December 7, 2019 at 4:17:04 AM UTC+1, robert engels wrote: >>>>>> To clarify, with Go’s very lightweight threads it is “doing the >>>>>> multiplexing for you” - often only a single CPU is consumed if the >>>>>> producer and consumer work cannot be parallelized, otherwise you get >>>>>> this concurrency “for free”. >>>>>> >>>>>> You are trying to manually perform the multiplexing - you need async >>>>>> structures to do this well - Go doesn’t really support async by design - >>>>>> and it’s a much simpler programming model as a result. >>>>>> >>>>>>> On Dec 6, 2019, at 12:02 PM, Robert Engels <ren...@ix.netcom.com> wrote: >>>>>>> >>>>>>> A channel is much closer to a pipe. There are producers and consumers >>>>>>> and these are typically different threads of execution unless you have >>>>>>> an event based (async) system - that is not Go. >>>>>>> >>>>>>>> On Dec 6, 2019, at 9:30 AM, Egon Kocjan <eko...@gmail.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> There are goroutines in the examples of course, just a single >>>>>>>> goroutine per bidi channel seems hard. By contrast, I've worked with >>>>>>>> actor systems before and they are perfectly fine with a single fiber. >>>>>>>> >>>>>>>>> On Friday, December 6, 2019 at 3:38:20 PM UTC+1, Robert Engels wrote: >>>>>>>>> Channels are designed to be used with multiple go routines - if >>>>>>>>> you’re not you are doing something wrong. >>>>>>>>> >>>>>>>>>>> On Dec 6, 2019, at 8:32 AM, Egon Kocjan <eko...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hello >>>>>>>>>> >>>>>>>>>> I'm preparing a short talk about Go channels and select. More >>>>>>>>>> specifically, I want to show what not to do. I chose a bidirectional >>>>>>>>>> communication channel implementation, because it seems to be a >>>>>>>>>> common base for a lot of problems but hard to implement correctly >>>>>>>>>> without using any extra goroutines. All the code is here: >>>>>>>>>> https://github.com/egonk/chandemo >>>>>>>>>> >>>>>>>>>> 1_1.go: easy with en extra goroutine (takes 1.2s for million ints) >>>>>>>>>> 2_1.go: nice but completely wrong >>>>>>>>>> 2_2.go: better but still deadlocks >>>>>>>>>> 2_3.go: correct but ugly and slow (takes more than 2s for million >>>>>>>>>> ints) >>>>>>>>>> 2_4.go: correct and a bit faster but still ugly (1.8s for million >>>>>>>>>> ints) >>>>>>>>>> >>>>>>>>>> So my question: is there a better way of doing it with just nested >>>>>>>>>> for and select and no goroutines? Basically, what would 2_5.go look >>>>>>>>>> like? >>>>>>>>>> >>>>>>>>>> Thank you >>>>>>>>>> Egon >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>> Groups "golang-nuts" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to golan...@googlegroups.com. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "golang-nuts" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>>> an email to golan...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/golang-nuts/bdc57eb0-b26f-4364-87fb-241b0807e8ae%40googlegroups.com. >>>>>> >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to golan...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/golang-nuts/75d69b4e-4fb7-4f62-8011-f21e2a4c294a%40googlegroups.com. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to golan...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/8b87adcc-2249-402c-b34c-20df5013860a%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/3b9bb722-d43f-4e70-8384-dc17cdec6090%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/F9F38669-D7EE-4A3F-BB58-96263F15DD95%40ix.netcom.com.
