Re: [go-nuts] Re: An old problem: lack of priority select cases

Robert Engels Fri, 04 Oct 2019 14:11:06 -0700

But even the two select blocks does not really make a difference. I have shared code that did this previously.

For another example, think of two channels A and B. A is the high priority channel where all events should be processed before B channel events - this is especially important considering that A events can be processed in 1 ms, and B events take 60 seconds.

So, as soon as you start processing a B event - you have delayed the processing of a "just after" A event by 60 seconds... probably not what you want. So you either need to be able to stop the processing of B, or you need to process them "concurrently" (this is a simplification of how to actually accomplish this).

This is why if closing the channel is independent of the processing of the channel, you need external synchronization and/or ordering.

Are there workloads where the A and B channels are always ready at the same time, probably... but if you ALWAYS processed A first - eventually B would run out of queue/memory space - and either deadlock or crash...

I will reiterate - your thinking on this problem is incorrect, which is why the solution you propose is of limited usefulness. Still, if you really want this, it is simple to write a function 'selectUsingPriority(A,B chan). But if you don't fully understand what I am saying, you are going to have obscure reliability problems in production if you use it.

-----Original Message-----
From: T L
Sent: Oct 4, 2019 3:52 PM
To: golang-nuts
Subject: Re: [go-nuts] Re: An old problem: lack of priority select cases

On Friday, October 4, 2019 at 4:38:36 PM UTC-4, Marcin Romaszewicz wrote:
What he's trying to say is that it is pointless to order select cases in this example, because it is impossible to guarantee ordering in an asynchronous system.

You have two asynchronous data streams, ctx.Done() and "v", in that example above.

Generally, those select cases will happen one by one, as those asynchronous channels deliver data. If "v" is ready first, shortly before ctx.Done(), then v will be selected first, followed by ctx.Done(). It doesn't matter that ctx.Done() wants to be higher priority, "v" arrived first, before ctx.Done() was posted, so it's handled first. If ctx.Done() happens first, the reverse happens. The only interesting case is when both "v" and ctx.Done() are ready at the same time, which will be unlikely in practice.

If ctx.Done() and "v" happen together, so that both cases of that select statement are available simultaneously, then sure, you can order them, but this will happen very infrequently, and you still MUST handle the case where "v" happens first, very close to ctx.Done(). So, if you MUST handle this case, you don't really need to bother with statement priority, since if your code is written well, this ordering won't matter. ctx.Done() might happen while you're in the middle of handling "v", for example.

This is not frequently, but not rare.

The desire to have priority to select statements presumes that you have several cases happen at the same time, which isn't how this will generally work. If you want to order things in a select, you have to change how it behaves, and order events within a particular time window. Say you write a poller which samples the event queue every second. If you do this, then sure, you can order things however you want, but the cost you pay is that second of buffering latency.

Yes, select case priority tries to always select one case of the two cases
when both the two cases are available. Without this feature, to achieve
this goal, we often need two select blocks (sorry, I mis-typed select block
as select case in my last comment).

One priority-order select block is not only cleaner than two random-order
select blocks, but also more efficient.

On Fri, Oct 4, 2019 at 1:19 PM T L <tapi...@gmail.com> wrote:

On Friday, October 4, 2019 at 4:09:09 PM UTC-4, Robert Engels wrote:
Because ctx.Done() and v being ready for read are independent events. You can not impose ordering on them unless there is an outer mutex that both events are subject to.

As an aside, this is why I think the best 'concurrent software' developers are those that have been exposed to at least some hardware design. Many programmers think in terms of 1 and 0 and everything being ordered. This is certainly not the case in hardware, nor concurrent software. (For example, in computer hardware, the clock/sync line is what is used as the outer controlling event, but still things like propagation times, etc. make even this simple statement not fully correct).

Still not understanding what you new saying. ;D

Again, select case priority enables use to deduce two select cases
to one select case in coding in many scenarios.
This often leads to cleaner code, avoid can avoid the harm caused
by missing a try-receive select case.

-----Original Message-----
From: T L
Sent: Oct 4, 2019 2:44 PM
To: golang-nuts
Subject: Re: [go-nuts] Re: An old problem: lack of priority select cases

On Friday, October 4, 2019 at 3:32:31 PM UTC-4, Robert Engels wrote:
You still are not understanding proper concurrent design. Priority select cases do not matter in the case of asynchronous external events.

It at least avoids code verbosity and improves code quantity..

BTW, I don't understand what you said. Could you elaborate more?

On Oct 4, 2019, at 1:46 PM, T L <tapi...@gmail.com> wrote:

I just found an example in the "context" package docs:

// // Stream generates values with DoSomething and sends them to out // // until DoSomething returns an error or ctx.Done is closed. // func Stream(ctx context.Context, out chan<- Value) error { // for { // v, err := DoSomething(ctx) // if err != nil { // return err // } // select { // case <-ctx.Done(): // return ctx.Err() // case out <- v: // } // } // }

It looks the send "out <- v" still has a possibility to be executed,
even if "ctx.Done()" is closed.
But if Go supports select case priority, then this will never happen.

On Wednesday, August 28, 2019 at 12:06:33 PM UTC-4, T L wrote:
The old thread: https://groups.google.com/forum/#!topic/golang-nuts/ZrVIhHCrR9o

Go channels are flexible, but in practice, I often encountered some situations in which channel are hard to use.
Given an example:

import "math/rand"

type Producer struct {
    data   chan int
    closed chan struct{}
}

func NewProducer() *Producer {
    p := &Producer {
        data:   make(chan int),
        closed: make(chan struct{}),
    }

    go p.run()

    return p
}

func (p *Produce) Stream() chan int {
    return p.data
}

func (p *Producer) run() {
    for {
        // If non-blocking cases are selected by their appearance order,
        // then the following slect block is a perfect use.
        select {
        case(0) <-p.closed: return
        case p.data <- rand.Int():
        }
    }
}

func (p *Produce) Clsoe() {
    close(p.closed)
    close(p.data)
}

func main() {
    p := NewProducer()
    for n := p.Stream() {
        // use n ...
    }
}

If the first case in the select block in the above example has a higher priority than the second one,
then coding will be much happier for the use cases like the above one.

In short, the above use case requires:
* for receivers, data streaming end is notified by the close of a channel.
* for senders, data will never be sent to closed channel.

But, as Go 1 doesn't support priority select cases, it is much tedious to implement the code
satisfying the above listed requirements. The final implementation is often very ugly and inefficient.

Does anyone else also experience the pain?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b7a14153-253e-43c4-bda5-96079601465f%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/77fca2e9-120d-45e0-8ae9-5d24b63827bd%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/cf695058-93ce-4b81-99d0-3c622a63202a%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/857fb242-ebcf-4e31-8f9f-b87738b58c1a%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/2008722885.11036.1570223449265%40wamui-aurora.atl.sa.earthlink.net.

Re: [go-nuts] Re: An old problem: lack of priority select cases

Reply via email to