On Wed, Aug 22, 2018 at 05:56:09PM +0100, Russel Winder via Digitalmars-d wrote: > On Thu, 2018-08-16 at 16:33 -0700, H. S. Teoh via Digitalmars-d wrote: > […] > > I read both articles, and am quite impressed by the revolutionary > > way of looking at concurrency. It provides a clean(er) abstraction > > that can be reasoned about much more easily than currently prevalent > > models of concurrency. Seems it would fit right in with D's > > message-based concurrency communication model. > > I found the assumptions about what goroutines were to be wrong. Yes > there is an interesting structure built using Python context managers > to manage tasks executed by time division multiplexing, but is that > really needed since the current systems work just fine if you have > threadpools and multiple executing threads – as Java, Go, etc. have > but Python does not.
I approached the article from a language-independent viewpoint. While I know a little bit of Python, I wasn't really very interested in the Python-specific aspects of the article, nor in the specific implementation the author had written. What caught my interest was the concept behind it -- the abstraction for concurrent/parallel computation that is easy to reason about, compared to other models. The main innovative idea, IMO, is the restriction of parallel/concurrent processing to the lifetime of an explicit object, in this case, a "nursery". (TBH a better term could have been chosen, but that doesn't change the underlying concept.) More specifically, the lifetime of this object can in turn be tied to a lexical scope, which gives you an explicit, powerful way to manage the lifetime of child processes (threads, coroutines, whatever), as opposed to the open-endedness of, say, spawning a thread that may run arbitrarily long relative to the parent thread. This restriction does not limit the expressive power of the abstraction -- it "gracefully degrades" to current open-ended models if, for example, you allocate a nursery on the heap and spawn child processes / threads / etc. into it. However, by restricting the open-endedness of child (process, thread, ...) lifetime, it gives you the ability to reason about control flow in a much more intuitive way. It restores the linearity of control flow in a given block of code (with the well-defined exception if a nursery was explicitly passed in), making it it much easier to reason about. Unless you're explicitly passing nurseries around, you no longer have to worry about whether some function you call in the block might spawn new processes that continue running after the block exits. You no longer need to explicitly manage shared resources and worry about whether resource X could be released at the end of the block. And so on. Even in the more complex case where nurseries are being passed around, you can still reason about the code with relative ease by examining the lifetime of the nursery objects. You no longer have to worry about the case where background processes continue running past the lifetime of the main program (function, block, etc.), or manually keeping track of child processes so that you can sync with them. Once you have this new way of thinking about concurrent processing, other possibilities open up, like returning values from child processes, propagating exceptions, cancellation, etc.. (Cancellation may require further consideration in non-Python implementations, but still, the model provides the basis for a cleaner approach to this than open-ended models allow.) […] > > Indeed. It certainly seems like a promising step toward addressing > > the nasty minefield that is today's concurrent programming models. > > I'd say processes and channels works just fine. What is this really > providing outside the Python sphere? (Also Javascript?) [...] Functionally, not very much. Readability and understandibility-wise, a lot. And that is the point. I personally couldn't care less what it contributes to Python, since I don't use Python very much outside of SCons, and within SCons concurrent processing is already taken care of for you and isn't an issue the user needs to worry about. So in that sense, Trio isn't really relevant to me. But what I do care about is the possibility of a model of concurrency that is much more easily understood and reasoned about, regardless of whether the underlying implementation uses explicit context-switching, fibres, threads, or full-blown processes. Basically, what we're talking about is the difference between a control flow graph that's an arbitrarily-branching tree (open-ended concurrency model with unrestricted child lifetimes: one entry point, arbitrary number of exits), vs. a single-entry single-exit graph where every branch eventually rejoins the parent (nursery model). Having an arbitrarily branching control flow means many concepts don't work, like return values, propagating exceptions back to the parent, managing child lifetimes, etc.. Having well-defined joining points for all children means that it's possible to have well-defined return values, exception propagation, manage child lifetimes, etc.. I don't claim this solves all the difficulties of comprehension in concurrent programming, but it does reduce the mental load by quite a bit. And that to me is a plus, because reduced mental load means the programmer is more likely to get it right, and can spend more effort actually focusing on the problem domain instead of wrestling with the nitty-gritty of concurrency. More productivity, less bugs. Like using a GC instead of manual memory management. Or writing in D instead of assembly language. :-D T -- Almost all proofs have bugs, but almost all theorems are true. -- Paul Pedersen