Hi Caleb Your approach seems to me as a reasonable use case for channels and I'm sure it would be a lot complexer if you use a bunch of mutexes.
One part in the article looks a little bit weird to me: > // ... > s.mtx.Lock() > // ... > s.ch <- val // might block! > s.mtx.Unlock() > // ... > > My experience with go is limited but until now I have never seen that kind of code, I don't think it is something advanced go users would do or I'm wrong? Cheers snmed Am Mittwoch, 9. August 2017 14:30:38 UTC+2 schrieb Caleb Doxsey: > > Channels are very useful and necessary to really get your program using > all the resources at your disposal. Doing that with locks or callbacks is > error prone and makes solving some problems all but impossible. Let me give > you an example of a pattern we used a few days ago: > > We were processing message from a queue, writing some data to Cassandra, > and then committing offsets when the work was completed. It was really > important for the commits to come out in the same order they came in, so > that we never commit an offset when we haven't actually written all the > data. (on crash or restart its ok to process the same messages again > because the writing is idempotent) > > Modeled as a pipeline we had a few stages: > > intake -> decode -> encode -> write to cassandra -> commit > > The naive approach would be to spin up a bunch of writer goroutines, but > then the order of messages to commit would be unpredictable. Instead we > created two slices of channels: > > ins := make([]chan intakeMessage, options.workers) > outs := make([]chan commitMessage, options.workers) > > And spun up goroutines that read from one of the input channels and wrote > to the corresponding out channel: > > for i := 0; i < options.workers; i++ { > ins[i] = make(chan intakeMessage, 1) > outs[i] := make(chan commitMessage, 1) > go worker(ins[i], outs[i]) > } > > We then use these slices as a circular buffer and keep track of two > pointers, one for the next available in and another for the next remaining > out. There are 3 cases: > > 1. The slice is empty, in which case we insert at 0 and increment the > in counter > 2. The slice is full (all workers are busy), in which case we wait > until a result is pushed into the channel at the out counter > 3. We're somewhere in between, in which case we use a select on either > the next available in channel or the next remaining out counter > > So all the commits come out in the order they came in, we get nice > parallelism and there's no blocking or polling. > > There is overhead to channels but if you send messages of adequate size > you'll barely notice it in real programs. Batch and send slices. > > The thing I find remarkable about channels and goroutines is how often the > solutions for improving the performance of a Go program are almost exactly > the same as the solutions for distributing that program across multiple > machines. The patterns really map well to these kinds of problems. > > On Tuesday, August 8, 2017 at 2:01:12 AM UTC-4, snmed wrote: >> >> Hi Gophers >> >> I stumbled over a nice and very interesting Blog entry "Go channels are >> bad and you should feel bad >> <http://www.jtolds.com/writing/2016/03/go-channels-are-bad-and-you-should-feel-bad/>" >> >> , I would like to hear some opinions about that article >> from seasoned go developers. Because I just used go for a couple of >> months for my private web projects and rarely get in touch with channels. >> >> By the way, that article is not a rant and the author likes go very much >> as far as I can conclude from the article. >> >> Cheers snmed >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.