On Fri, 03 Feb 2017 21:20:59 -0800, g...@google.com wrote: > See the following gist: > > https://gist.github.com/japhb/40772099ed24e20ec2c37c06f434594b > > (If you run that at the command line, you'll probably want to pipe it to > `head -30` or so; it will output a lot of lines very quickly!) > > Essentially it appears that unlike the friendly one-at-a-time behavior of > .act, react/whenever will try to exhaust all the emits from an unbounded > supply *before* delivering any of them to the whenever code -- which makes > it awfully hard to have the whenever tell the supply when to stop.
Firstly, the boring observations: there are two mistakes in the gist. 1) A role is not a closure, so: } does role { method done { $done = True } } Will not behave as you want. 2) In the react example, there is $s1.done, when I presume $s2.done was meant. Even with these corrected, the behavior under consideration still occurs. The deadlock we're seeing here is thanks to the intersection of two individually reasonable things. The first is the general principle that supplies are about taming, not introducing, concurrency. There are, of course, a number of Supply factory methods that will introduce concurrency (Supply.interval, for example), together with a number of supply operators that also will - typically, anything involving time, such as the delay method. Naturally, schedule-on also can. But these are all quite explicitly asking for the concurrency (and all are delegating to something else - a scheduler - to actually provide it). The second, which is in some ways a follow-on from the first, is the actor-like semantics of supply and react blocks. Only one thread may be inside of a given instance of a supply or react block at a time, including any of the whenever blocks inside of it. This has two important consequences: 1) You can be sure your setup logic inside of the supply or react block will complete before any messages are processed. 2) You can be sure that you'll never end up with data races on any of the variables declared inside of your supply or react block because only one message will be processed at a time. This all works out well if the supply being tapped truly *is* an asynchronous source of data - which is what supplies are primarily aimed at. In the case we're considering here, however, it is not. Thanks to the first principle, we don't introduce concurrency, so we tap the supply on the thread running the react block's body. It never hands back control due to the loop inside of it, running straight into the concurrency control mechanism. A one-word fix is to introduce a bit of concurrency explicitly: react { start whenever $s2 -> $n { say "Received $n"; $s2.done if $n >= 5; } } With this, the react block's setup can complete, and then it starts processing the messages. Longer term, a back-pressure model for supplies is something that wants looking in to, designing, and implementing. I put this off on the basis that Rx.Net is plenty useful without one, and RxJava introduced one after its initial release. Taken together, there was no incentive to rush one in. However, we might be able to find a solution in that space for this particular case. That said, back when I was teaching async programming, I always made a point to note that the places where synchrony and asynchrony meet are often sources of trouble. Here, a supply block whose body runs synchronously runs up against a construct (react) and data structure (Supply) whose designs are optimized for dealing with asynchronous data. Reduced to its essence, the code submitted here and the C# code I would show my students to illustrate the problem look strikingly similar: a blocking subscription prevents message processing, leading to a deadlock. It's worth noting that this general problem can *not* be solved through a back-pressure mechanism; it can only solve cases like the one in this ticket where when emit can serve as a preemption point in the case of back-pressure being applied. The consequences of making emit have such semantics, however, will probably run deep once we get into non-toy examples. (For example, will it end up with us declaring `emit` as being like `await` in 6.d where you may be on a different OS thread afterwards if you do it inside of the thread pool?) A perhaps simpler solution space to explore is providing an API that separates the obtaining of a Tap from the starting of processing. That would allow us to run the setup logic to completion. But...then what? Again, it's easy to make this toy example work because there's only one whenever block. But if there are more, then we're just moving the problem, and making it harder to diagnose, because instead of a "where are we deadlocked" backtrace showing the whenever line, it'd instead show...some other location in supply internals. So, a back-pressure model that allows us to round-robin is probably a bit better than this. tl;dr use "start whenever $supply { }" when $supply is going to work synchronously. We should also consider implementing a missing "tap-on" supply operator, so you can also write: whenever $supply.tap-on(ThreadPoolScheduler) { } Or do it at the source: supply { ...sync code here... }.tap-on(ThreadPoolScheduler) A simple implementation would likely be: method tap-on(Scheduler:D $scheduler) { supply { $scheduler.cue: { whenever self { .emit } } } } Making the code as originally submitted work is an interesting problem to ponder, but raises a bunch of non-trivial questions, and should be considered together with various other challenges. Hope this helps, /jnthn