Re: [go-nuts] An old problem: lack of priority select cases

Robert Engels Fri, 30 Aug 2019 10:40:28 -0700

You changed the Read() method incorrectly - it should be using the Read lock, not the Write lock.

Still, as I pointed out when I posted it, Play has a problem where it aborts if all routines are sleeping (not just blocked), so you need to run it locally.

-----Original Message-----
From: T L
Sent: Aug 30, 2019 12:05 PM
To: golang-nuts
Subject: Re: [go-nuts] An old problem: lack of priority select cases

On Friday, August 30, 2019 at 12:39:41 PM UTC-4, Robert Engels wrote:

Makes no difference in the code I posted.... as long as they all use the same MultiWriterChannel. In fact, others can be late started, as they will fail fast if the channel is already closed.

https://play.golang.org/p/pcwIu2w8ZRb

All go routines are blocked in the modified version.

-----Original Message-----
From: T L
Sent: Aug 30, 2019 11:13 AM
To: golang-nuts
Subject: Re: [go-nuts] An old problem: lack of priority select cases

On Friday, August 30, 2019 at 10:35:29 AM UTC-4, Robert Engels wrote:
I don't think so. Why do you think that is the case? The RWLock is "fair" in the sense that once the 'closer' attempts to get the lock, it is guaranteed to get it (as the code is structured) - the subsequent readers will queue behind the "writer = closer".

How about unknown/random number of senders and readers?

-----Original Message-----
From: T L
Sent: Aug 30, 2019 8:50 AM
To: golang-nuts
Subject: Re: [go-nuts] An old problem: lack of priority select cases

@Robert
I think there is a difference between the code of @Leo and you.
In you code, the Wirte/Read/Close are all possible to block for ever.

On Thursday, August 29, 2019 at 8:59:10 PM UTC-4, Robert Engels wrote:

Oops. You are right. The original used two different methods Closed() and Read() and when I refactored I forgot to add the Read lock to the Read(). That's why you always have code reviews...
-----Original Message-----
From: T L
Sent: Aug 29, 2019 6:25 PM
To: golang-nuts
Subject: Re: [go-nuts] An old problem: lack of priority select cases

On Wednesday, August 28, 2019 at 10:05:06 PM UTC-4, robert engels wrote:
Here is a version using RWLock https://play.golang.org/p/YOwuYFiqtlf

Doesn't the Read method need to be guarded by the reader lock?

It won’t run correctly in the playground because it terminates when all routines are asleep - which happens during the test (not sure why it does this, as sleeping is different than a deadlock).

It is probably less efficient, and less orderly than the other example using WaitGroup but you get the idea I hope. It forcibly terminates the writers before they complete by design.

On Aug 28, 2019, at 4:09 PM, Michel Levieux <m.le...@capitaldata.fr> wrote:

One should also be careful regarding the conceptual demands he or she is making.
Having a shared resource (that is complex enough that it cannot be atomically accessed or modified) means essentially that "having multiple writers being transparent to the readers", fundamentally, is not possible.

From the moment itself when such a resource is shared, there must be some sort of mecanism (that one using resources atomically usable) that ensures the integrity of it.
Maybe what you're talking about is having it transparent in terms of code, in which case we both agree, but if you're looking for something transparent in essence, as in performance, logical construction and all the rest, I think there is a misunderstanding here: even if it was added in the language, there would be many many things going on under the hood, as it is already (and cannot really be otherwise) for channel use alone.

As for the priority using selects, I think it's more of something to be dealt with on the "user-side". There are many kinds of priority in general, and trying to implement something in the language itself would IMO either be too specific compared to the nessecary time to do so or it would probably have a huge overhead on the "classical' use case of the select construct.
+ the fact that it is apparently already possible using RWMutexes.

Le mer. 28 août 2019 à 22:37, Marcin Romaszewicz <mar...@gmail.com> a écrit :
Think of a channel as existing for the lifetime of a particular data stream, and not have it be associated with either producer or consumer. Here's an example:

https://play.golang.org/p/aEAXXtz2X1g

The channel here is closed after all producers have exited, and all consumers continue to run until the channel is drained of data.

The producers are managed by something somewhere in your code - and that is the scope at which it makes sense to create channel ownership. I've used a waitgroup to ensure that the channel is closed after all producers exit, but you can use whatever barrier construct you want.

Even if you must have a channel per producer, you can safely close the producer side, without notifying the downstream about this. The example early in the thread uses multiple channels, with one channel being used to signal that the producers should exit. Channels aren't really the right model for this, you want a thread safe flag of some sort. For example:

var exitFlag uint64
func producer(chan data int, wg *sync.WaitGroup) {
    defer wg.Done()
    for {
        shouldExit := atomic.LoadUint64(&exitFlag)
        if shouldExit == 1 {
             return
        }
        chan <- rand.Intn(100)
    }
}

Here's 10 producers and 3 consumers sharing a channel and closing it safely upon receiving an exit flag:
https://play.golang.org/p/RiKi1PGVSvF

-- Marcin

On Wed, Aug 28, 2019 at 11:29 AM Leo Lara <l...@leopoldolara.com> wrote:
I do not think priority select is *necessary*, it could be a nice addition if the performance does not change.

On Wednesday, August 28, 2019 at 8:27:36 PM UTC+2, Leo Lara wrote:
Hi Robert,

From the article: """To bound more the problem, in my case, you control the writers but not the readers"""

So what I was trying to do was to be able to close, with mutiple writers, while being transparent for the readers. The readers only need to read as usual form the channel.

For example, if you want to write a library where the user just reads from a channel, this is an approach I found where the user of the lirbary deos nto have to do anything special. Of course, there might be another solution, but if you need to modify the reader we are talking about a different problem.

Cheers!!

On Wednesday, August 28, 2019 at 7:17:24 PM UTC+2, Robert Engels wrote:
A better solution is to wrap the writes using a RWLock, grab the read lock for writing, and the Write lock for closing. Pretty simple.

Just encapsulate it all in a MultiWriterChannel struct - generics would help here :)

-----Original Message-----
From: Leo Lara
Sent: Aug 28, 2019 11:24 AM
To: golang-nuts
Subject: [go-nuts] Re: An old problem: lack of priority select cases

This is connected with my article: https://dev.to/leolara/closing-a-go-channel-written-by-several-goroutines-52j2

I think there I show it is possible to workaround that limitation using standard Go tools. Of course, the code would be simple with priority select, but also perhaps select would become less efficient.

On Wednesday, August 28, 2019 at 6:06:33 PM UTC+2, T L wrote:
The old thread: https://groups.google.com/forum/#!topic/golang-nuts/ZrVIhHCrR9o

Go channels are flexible, but in practice, I often encountered some situations in which channel are hard to use.
Given an example:

import "math/rand"

type Producer struct {
    data   chan int
    closed chan struct{}
}

func NewProducer() *Producer {
    p := &Producer {
        data:   make(chan int),
        closed: make(chan struct{}),
    }

    go p.run()

    return p
}

func (p *Produce) Stream() chan int {
    return p.data
}

func (p *Producer) run() {
    for {
        // If non-blocking cases are selected by their appearance order,
        // then the following slect block is a perfect use.
        select {
        case(0) <-p.closed: return
        case p.data <- rand.Int():
        }
    }
}

func (p *Produce) Clsoe() {
    close(p.closed)
    close(p.data)
}

func main() {
    p := NewProducer()
    for n := p.Stream() {
        // use n ...
    }
}

If the first case in the select block in the above example has a higher priority than the second one,
then coding will be much happier for the use cases like the above one.

In short, the above use case requires:
* for receivers, data streaming end is notified by the close of a channel.
* for senders, data will never be sent to closed channel.

But, as Go 1 doesn't support priority select cases, it is much tedious to implement the code
satisfying the above listed requirements. The final implementation is often very ugly and inefficient.

Does anyone else also experience the pain?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b284f880-034a-4721-8686-ef48d3e2c14c%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/aeb38a0a-8268-42d7-a8eb-ce5ef01c5380%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CA%2Bv29LvcUhUvrZb_8AKYWj0A%2Bqd5LKBPmbz-RVBb%3DJn_gNZE6w%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANgi337s1Low95QvqJUAOTsqcVji7uMQ_jr%3DFftpt2uMz5_XSQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a5cff3f8-cc1c-4719-9f2f-7b9c31086f6a%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4bf95bb2-33ed-46ed-9436-48df1072914f%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/d155dcf6-7c01-4f7e-b408-eef9903cd837%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/74653e49-f374-4ac8-998e-fd874cdf6bd4%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1995993326.5256.1567186808717%40wamui-bison.atl.sa.earthlink.net.

Re: [go-nuts] An old problem: lack of priority select cases

Reply via email to