Hi Adam, thanks for trying out the new release (and for the blog update!) and also for your feedback; comments inline.
> 22 dec 2015 kl. 16:41 skrev Adam Warski <a...@warski.org>: > > Hello, > > I think that the new fusing support in akka-streams may have more > consequences than I initially suspected. It quite drastically changes how > concurrency is handled in akka-streams. > > So far the model was quite clear: every stage gets materialised into an > actor. If there's a split/broadcast, then all branches will be executed in > parallel. Yes, that is indeed very clear and simple, and you can get that back by setting akka.stream.materializer.auto-fusing=off in the configuration. > Now, with auto-fusing that changes. By default, if there's a split, the > branches won't be executed in parallel. I think that's quite a big semantical > change. This is a bit of an overstatement: the semantics of the stream processing do not change, the same rules apply to how elements flow and how back-pressure is handled, all branches of a broadcast will make progress concurrently—but not necessarily in parallel. This means that the management of resource allocation has changed, not the semantics. > A similar effect is when there are e.g. two computationally expensive, > consecutive .map stages (they will now be processed sequentially, not > concurrently), however with splits I think it was a very natural expectation > that things will be processed in parallel. Not necessarily. This is why the user gets to decide by declaring (some of) the branches as asynchronous. > Hence I have four questions: > > 1. if I have a split, where should the async boundary go? My first attempt > was adding the async boundary attribute to the split graph stage itself, but > that didn't work. By experimentation, I have to add it to the stages that are > connected to the split's outputs. I realize that in the final release frenzy we forgot to add documentation on how this actually works, I apologize. In short, adding attributes applies to the whole graph on which they are added, and adding an asynchronous boundary adds that boundary around the graph that has previously been constructed. This means that the right place is around the flow that gets attached to the split’s outputs. Currently that looks like someFlow.via(Flow[...].<your ops here>.withAttributes(Attributes.asyncBoundary)) which will run <your ops here> in one extra actor. > > 2. how to insert async boundaries in linear pipelines? E.g.: > > Source(List(1, 2, 3)) > .map(_ + 1) > .addAttributes(Attributes.asyncBoundary) This is an oversight, we’ll need to provide appropriate overrides of this method in all subclasses (ticket <https://github.com/akka/akka/issues/19261> created). > .map(_ * 2) > > doesn't compile > > 3. are you sure it's a good idea to make all processing stages fuseable by > default? E.g. conflate seems to be naturally concurrent, as it deals with > slower/faster components, which doesn't make much sense if everything is in > one thread :) This is a misconception: conflate is not concurrent, it is aware of back-pressure, and it works even better when fused because that removes additional (implicit) buffers that can lead to rather surprising behavior. All operations should be fusable, but I agree that the fusing algorithm should become more intelligent in selecting which parts of a graph to fuse—right now it is called “Fusing.aggressive” because it will fuse everything it can. > 4. which built-in stages are fuseable by default? The docs are quite vague > here saying only "linear" ones. Which ones are these? It is easier to name the exceptions: SslTlsStage, groupBy, and some sources and sinks (but for those the difference is typically not that large). > If akka-streams now has *explicit* concurrency control, instead of the > *implicit* one so far, maybe it would make sense to be explicit in the api > (different name?) which stages will run concurrently by default? The goal is to have everything fusable in order to have uniform expectations, and then users will place boundaries where needed. Another improvement I want to implement is that the fusing algorithm can be informed of a desired parallelism level and then it will try to split up a graph more or less intelligently to reach that. Regards, Roland > > Thanks, > Adam > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html> > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > >>>>>>>>>> <https://groups.google.com/group/akka-user> > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com > <mailto:akka-user+unsubscr...@googlegroups.com>. > To post to this group, send email to akka-user@googlegroups.com > <mailto:akka-user@googlegroups.com>. > Visit this group at https://groups.google.com/group/akka-user > <https://groups.google.com/group/akka-user>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. Dr. Roland Kuhn Akka Tech Lead Typesafe <http://typesafe.com/> – Reactive apps on the JVM. twitter: @rolandkuhn <http://twitter.com/#!/rolandkuhn> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
