Hi Adam, > 22 dec 2015 kl. 20:37 skrev Adam Warski <a...@warski.org>: > > >> Now, with auto-fusing that changes. By default, if there's a split, the >> branches won't be executed in parallel. I think that's quite a big >> semantical change. > > This is a bit of an overstatement: the semantics of the stream processing do > not change, the same rules apply to how elements flow and how back-pressure > is handled, all branches of a broadcast will make progress concurrently—but > not necessarily in parallel. This means that the management of resource > allocation has changed, not the semantics. > > Right, my wording was bad, though what the end-user observes could be quite > different. I couldn't find the right words, but changing the default stream > behaviour from "parallel" to "concurrent" describes it correctly I guess? > (which then, at some level, can be viewed as different semantics ;) ) > > Probably also the fact that in my oversimplified tests I use Thread.sleep for > simulating "work" plays a role ;)
:-) But you are right, even though “semantics” didn’t change—for some suitable definition—user expectations may still be violated. I’ll write up some docs right away, it would be great if you could review them—I’ll ping you (@adamw, right?). > >> A similar effect is when there are e.g. two computationally expensive, >> consecutive .map stages (they will now be processed sequentially, not >> concurrently), however with splits I think it was a very natural expectation >> that things will be processed in parallel. > > Not necessarily. This is why the user gets to decide by declaring (some of) > the branches as asynchronous. > > Sure, though that's how I thought about broadcast, or merge. Maybe it's a > common misconception, but probably it's just me :) What we’re aiming for is absolutely homogenous rules, it would be strange if some combinators worked differently than others. In this case that implies that we’ll need to teach people about this rule, but that cannot be avoided unless using magic—which is forbidden by rule #1 <http://doc.akka.io/docs/akka-stream-and-http-experimental/2.0/stream-design.html#What_shall_users_of_Akka_Streams_expect_>. > >> Hence I have four questions: >> >> 1. if I have a split, where should the async boundary go? My first attempt >> was adding the async boundary attribute to the split graph stage itself, but >> that didn't work. By experimentation, I have to add it to the stages that >> are connected to the split's outputs. > > I realize that in the final release frenzy we forgot to add documentation on > how this actually works, I apologize. In short, adding attributes applies to > the whole graph on which they are added, and adding an asynchronous boundary > adds that boundary around the graph that has previously been constructed. > This means that the right place is around the flow that gets attached to the > split’s outputs. Currently that looks like > > someFlow.via(Flow[...].<your ops > here>.withAttributes(Attributes.asyncBoundary)) > > which will run <your ops here> in one extra actor. > > so if I have a split component, and I want branches to be run in parallel > (different actors), I need to do: > > val branch1 = someFlow.withAttributes(asyncBoundary) > val branch2 = someFlow.withAttributes(asyncBoundary) > > split.out1 ~> branch1 > split.out2 ~> branch2 > > Correct? > Yes, correct. > For linear flows, e.g.: > > Source(List(1, 2, 3)) > .map(_ + 1) > .withAttributes(asyncBoundary) > .map(_ * 2) > .to(Sink.ignore) > > here two actors will be created? First wrapping the Source & map(_ + 1), > second the map(_ * 2) & sink? Again correct :-) Would you mind if I just used this example in the docs? >> >> 2. how to insert async boundaries in linear pipelines? E.g.: >> >> Source(List(1, 2, 3)) >> .map(_ + 1) >> .addAttributes(Attributes.asyncBoundary) > > This is an oversight, we’ll need to provide appropriate overrides of this > method in all subclasses (ticket > <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fakka%2Fakka%2Fissues%2F19261&sa=D&sntz=1&usg=AFQjCNEfY1IhROusDQU4j0f_TYmsIpCA_Q> > created). > > Thx :) >> 3. are you sure it's a good idea to make all processing stages fuseable by >> default? E.g. conflate seems to be naturally concurrent, as it deals with >> slower/faster components, which doesn't make much sense if everything is in >> one thread :) > > This is a misconception: conflate is not concurrent, it is aware of > back-pressure, and it works even better when fused because that removes > additional (implicit) buffers that can lead to rather surprising behavior. > All operations should be fusable, but I agree that the fusing algorithm > should become more intelligent in selecting which parts of a graph to > fuse—right now it is called “Fusing.aggressive” because it will fuse > everything it can. > > True about the buffers, took me a while to understand one example ;) Having > conflate only makes sense if there's some non-fused component out there I > guess, but you are correct of course that it can be fused with things > before/after it. >> 4. which built-in stages are fuseable by default? The docs are quite vague >> here saying only "linear" ones. Which ones are these? > > It is easier to name the exceptions: SslTlsStage, groupBy, and some sources > and sinks (but for those the difference is typically not that large). > > Ah, ok, well I still think it might be a good idea to explicitly mark them > as non-fuseable at least in the docs. Once you have to think about where to > put the async boundaries, such information might be useful. It does not hurt to consider all operators and stages to be fusable even when a very small number are not yet, that way you will not be surprised when we fix the remaining ones. > > Thanks for the answers! :) Thanks for the questions! Regards, Roland > > Adam > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html> > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > >>>>>>>>>> <https://groups.google.com/group/akka-user> > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com > <mailto:akka-user+unsubscr...@googlegroups.com>. > To post to this group, send email to akka-user@googlegroups.com > <mailto:akka-user@googlegroups.com>. > Visit this group at https://groups.google.com/group/akka-user > <https://groups.google.com/group/akka-user>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. Dr. Roland Kuhn Akka Tech Lead Typesafe <http://typesafe.com/> – Reactive apps on the JVM. twitter: @rolandkuhn <http://twitter.com/#!/rolandkuhn> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
