I will repeat this - and I meant no offense. You should read up on RWLocks because your criticism is incorrect. Multiple Go readers can hold the Read lock at the same time - so there are multiple writers adding to the channel concurrently (but the channel impl uses a lock, so it’s not strictly true, if they used a lock free queue then it would be)
> On Sep 4, 2019, at 4:55 AM, Leo Lara <l...@leopoldolara.com> wrote: > > > >> You should read up on how a RWLock works. > > I am not going to answer to that ;-) > > About this code: > > https://play.golang.org/p/YOwuYFiqtlf > > 1. I wouldn't say that there are several goroutines writing, as there are > several goroutines block in a mutex. Some might find this not important, > although there are some people that have looked into this and think it is > less efficient > https://opensource.com/article/18/7/locks-versus-channels-concurrent-go ; I > think it has to do on how the scheduler changes to another goroutine when the > goroutine is lock on a channel. > 2. More importantly IMHO, in your example the reader has to be aware that > there is a lock. Hence, it is not "transparent" in the sense that the reader > has to share something with the writer > >> >> The Sleep(1) in the producer is only to add some delay to demonstrate it >> gets terminated before the desired number of iterations. >> >>> On Aug 29, 2019, at 12:13 AM, Leo Lara <l...@leopoldolara.com> wrote: >>> >>> Hi Robert, >>> >>> To put you in context, it all started when I read >>> https://go101.org/article/channel-closing.html , that said that it is >>> impossible or at least you shouldn't close a channel that is being written >>> by several goroutines. Then I wrote this article with my solution >>> https://dev.to/leolara/closing-a-go-channel-written-by-several-goroutines-52j2 >>> also in >>> https://medium.com/@leolara/closing-a-go-channel-written-by-several-goroutines-eba3a6c9404b >>> I then created this issue https://github.com/go101/go101/issues/132 and >>> from there this topic was created by T L. >>> >>> Your example does not have several goruitnes writing so I think it is a >>> different problem. Perhaps that simple lock would work with several >>> goroutines, but I think there would be more contention with this lock. >>> >>> Anyway, I think I have already an optimisation to my code, I think using a >>> RW lock, if I put the "Add(1)" in a read lock and the wait in a Write lock >>> it might work better. The race condition that my lock prevents is only >>> related when an "Add" and a "Wait" run concurrently, several "Add" can run >>> concurrently. >>> >>>> On Thursday, August 29, 2019 at 4:05:06 AM UTC+2, robert engels wrote: >>>> Here is a version using RWLock https://play.golang.org/p/YOwuYFiqtlf >>>> >>>> It won’t run correctly in the playground because it terminates when all >>>> routines are asleep - which happens during the test (not sure why it does >>>> this, as sleeping is different than a deadlock). >>>> >>>> It is probably less efficient, and less orderly than the other example >>>> using WaitGroup but you get the idea I hope. It forcibly terminates the >>>> writers before they complete by design. >>>> >>>>> On Aug 28, 2019, at 4:09 PM, Michel Levieux <m.le...@capitaldata.fr> >>>>> wrote: >>>>> >>>>> One should also be careful regarding the conceptual demands he or she is >>>>> making. >>>>> Having a shared resource (that is complex enough that it cannot be >>>>> atomically accessed or modified) means essentially that "having multiple >>>>> writers being transparent to the readers", fundamentally, is not possible. >>>>> >>>>> From the moment itself when such a resource is shared, there must be some >>>>> sort of mecanism (that one using resources atomically usable) that >>>>> ensures the integrity of it. >>>>> Maybe what you're talking about is having it transparent in terms of >>>>> code, in which case we both agree, but if you're looking for something >>>>> transparent in essence, as in performance, logical construction and all >>>>> the rest, I think there is a misunderstanding here: even if it was added >>>>> in the language, there would be many many things going on under the hood, >>>>> as it is already (and cannot really be otherwise) for channel use alone. >>>>> >>>>> As for the priority using selects, I think it's more of something to be >>>>> dealt with on the "user-side". There are many kinds of priority in >>>>> general, and trying to implement something in the language itself would >>>>> IMO either be too specific compared to the nessecary time to do so or it >>>>> would probably have a huge overhead on the "classical' use case of the >>>>> select construct. >>>>> + the fact that it is apparently already possible using RWMutexes. >>>>> >>>>>> Le mer. 28 août 2019 à 22:37, Marcin Romaszewicz <mar...@gmail.com> a >>>>>> écrit : >>>>>> Think of a channel as existing for the lifetime of a particular data >>>>>> stream, and not have it be associated with either producer or consumer. >>>>>> Here's an example: >>>>>> >>>>>> https://play.golang.org/p/aEAXXtz2X1g >>>>>> >>>>>> The channel here is closed after all producers have exited, and all >>>>>> consumers continue to run until the channel is drained of data. >>>>>> >>>>>> The producers are managed by something somewhere in your code - and that >>>>>> is the scope at which it makes sense to create channel ownership. I've >>>>>> used a waitgroup to ensure that the channel is closed after all >>>>>> producers exit, but you can use whatever barrier construct you want. >>>>>> >>>>>> Even if you must have a channel per producer, you can safely close the >>>>>> producer side, without notifying the downstream about this. The example >>>>>> early in the thread uses multiple channels, with one channel being used >>>>>> to signal that the producers should exit. Channels aren't really the >>>>>> right model for this, you want a thread safe flag of some sort. For >>>>>> example: >>>>>> >>>>>> var exitFlag uint64 >>>>>> func producer(chan data int, wg *sync.WaitGroup) { >>>>>> defer wg.Done() >>>>>> for { >>>>>> shouldExit := atomic.LoadUint64(&exitFlag) >>>>>> if shouldExit == 1 { >>>>>> return >>>>>> } >>>>>> chan <- rand.Intn(100) >>>>>> } >>>>>> } >>>>>> >>>>>> Here's 10 producers and 3 consumers sharing a channel and closing it >>>>>> safely upon receiving an exit flag: >>>>>> https://play.golang.org/p/RiKi1PGVSvF >>>>>> >>>>>> -- Marcin >>>>>> >>>>>>> On Wed, Aug 28, 2019 at 11:29 AM Leo Lara <l...@leopoldolara.com> wrote: >>>>>>> I do not think priority select is *necessary*, it could be a nice >>>>>>> addition if the performance does not change. >>>>>>> >>>>>>>> On Wednesday, August 28, 2019 at 8:27:36 PM UTC+2, Leo Lara wrote: >>>>>>>> Hi Robert, >>>>>>>> >>>>>>>> From the article: """To bound more the problem, in my case, you >>>>>>>> control the writers but not the readers""" >>>>>>>> >>>>>>>> So what I was trying to do was to be able to close, with mutiple >>>>>>>> writers, while being transparent for the readers. The readers only >>>>>>>> need to read as usual form the channel. >>>>>>>> >>>>>>>> For example, if you want to write a library where the user just reads >>>>>>>> from a channel, this is an approach I found where the user of the >>>>>>>> lirbary deos nto have to do anything special. Of course, there might >>>>>>>> be another solution, but if you need to modify the reader we are >>>>>>>> talking about a different problem. >>>>>>>> >>>>>>>> Cheers!! >>>>>>>> >>>>>>>>> On Wednesday, August 28, 2019 at 7:17:24 PM UTC+2, Robert Engels >>>>>>>>> wrote: >>>>>>>>> A better solution is to wrap the writes using a RWLock, grab the read >>>>>>>>> lock for writing, and the Write lock for closing. Pretty simple. >>>>>>>>> >>>>>>>>> Just encapsulate it all in a MultiWriterChannel struct - generics >>>>>>>>> would help here :) >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Leo Lara >>>>>>>>> Sent: Aug 28, 2019 11:24 AM >>>>>>>>> To: golang-nuts >>>>>>>>> Subject: [go-nuts] Re: An old problem: lack of priority select cases >>>>>>>>> >>>>>>>>> This is connected with my article: >>>>>>>>> https://dev.to/leolara/closing-a-go-channel-written-by-several-goroutines-52j2 >>>>>>>>> >>>>>>>>> I think there I show it is possible to workaround that limitation >>>>>>>>> using standard Go tools. Of course, the code would be simple with >>>>>>>>> priority select, but also perhaps select would become less efficient. >>>>>>>>> >>>>>>>>>> On Wednesday, August 28, 2019 at 6:06:33 PM UTC+2, T L wrote: >>>>>>>>>> The old thread: >>>>>>>>>> https://groups.google.com/forum/#!topic/golang-nuts/ZrVIhHCrR9o >>>>>>>>>> >>>>>>>>>> Go channels are flexible, but in practice, I often encountered some >>>>>>>>>> situations in which channel are hard to use. >>>>>>>>>> Given an example: >>>>>>>>>> >>>>>>>>>> import "math/rand" >>>>>>>>>> >>>>>>>>>> type Producer struct { >>>>>>>>>> data chan int >>>>>>>>>> closed chan struct{} >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> func NewProducer() *Producer { >>>>>>>>>> p := &Producer { >>>>>>>>>> data: make(chan int), >>>>>>>>>> closed: make(chan struct{}), >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> go p.run() >>>>>>>>>> >>>>>>>>>> return p >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> func (p *Produce) Stream() chan int { >>>>>>>>>> return p.data >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> func (p *Producer) run() { >>>>>>>>>> for { >>>>>>>>>> // If non-blocking cases are selected by their appearance >>>>>>>>>> order, >>>>>>>>>> // then the following slect block is a perfect use. >>>>>>>>>> select { >>>>>>>>>> case(0) <-p.closed: return >>>>>>>>>> case p.data <- rand.Int(): >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> func (p *Produce) Clsoe() { >>>>>>>>>> close(p.closed) >>>>>>>>>> close(p.data) >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> func main() { >>>>>>>>>> p := NewProducer() >>>>>>>>>> for n := p.Stream() { >>>>>>>>>> // use n ... >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If the first case in the select block in the above example has a >>>>>>>>>> higher priority than the second one, >>>>>>>>>> then coding will be much happier for the use cases like the above >>>>>>>>>> one. >>>>>>>>>> >>>>>>>>>> In short, the above use case requires: >>>>>>>>>> * for receivers, data streaming end is notified by the close of a >>>>>>>>>> channel. >>>>>>>>>> * for senders, data will never be sent to closed channel. >>>>>>>>>> >>>>>>>>>> But, as Go 1 doesn't support priority select cases, it is much >>>>>>>>>> tedious to implement the code >>>>>>>>>> satisfying the above listed requirements. The final implementation >>>>>>>>>> is often very ugly and inefficient. >>>>>>>>>> >>>>>>>>>> Does anyone else also experience the pain? >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "golang-nuts" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to golan...@googlegroups.com. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/b284f880-034a-4721-8686-ef48d3e2c14c%40googlegroups.com. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "golang-nuts" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to golan...@googlegroups.com. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/golang-nuts/aeb38a0a-8268-42d7-a8eb-ce5ef01c5380%40googlegroups.com. >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "golang-nuts" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to golan...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/golang-nuts/CA%2Bv29LvcUhUvrZb_8AKYWj0A%2Bqd5LKBPmbz-RVBb%3DJn_gNZE6w%40mail.gmail.com. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to golan...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/golang-nuts/CANgi337s1Low95QvqJUAOTsqcVji7uMQ_jr%3DFftpt2uMz5_XSQ%40mail.gmail.com. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to golan...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/e5f37679-bdfb-4da3-854e-fea35cf82cb7%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/8893737c-148e-4d64-9a0c-cd72046cd0d1%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/34F5C212-9269-414E-87BC-8850382EA5BC%40ix.netcom.com.
