Why splitAfter/splitWhen/grouped also haven't maxSubstreams parameter by the same reason (bound resource consumption)?
On Tuesday, December 22, 2015 at 7:18:25 PM UTC-2, rkuhn wrote: > > We will make it fusable in the same fashion as > splitWhen/splitAfter/prefixAndTail are fusable. One minor correction: the > number of substreams is no longer unbounded, the API demands that you > specify a limit beyond which the groupBy logic will fail the stream. This > is necessary in order to place a bound on resource consumption, independent > of whether it is fused or not. > > Regards, > > Roland > > 22 dec 2015 kl. 21:44 skrev mathe...@sagaranatech.com <javascript:>: > > This topic clarified almost all questions I had on fusion. The only one > question remains is how groupBy stage will properly fused. If we are > modelling a stream of stream processing (for example, process telemetry > events for multiple IoT devices continuously is a stream of stream), maybe > makes no sense materialize this stream of stream into a single actor. The > point is the groupBy is the only one stage that can generate a possibly > infinite stream of stream (grouped, groupedWithin, splitWhen and splitAfter > generate stream of streams, but the substream generated are bounded, one > substream is completed before start another). This stage will or not > fused? > > Em terça-feira, 22 de dezembro de 2015 12:41:36 UTC-3, Adam Warski > escreveu: >> >> Hello, >> >> I think that the new fusing support in akka-streams may have more >> consequences than I initially suspected. It quite drastically changes how >> concurrency is handled in akka-streams. >> >> So far the model was quite clear: every stage gets materialised into an >> actor. If there's a split/broadcast, then all branches will be executed in >> parallel. >> >> Now, with auto-fusing that changes. By default, if there's a split, the >> branches won't be executed in parallel. I think that's quite a big >> semantical change. A similar effect is when there are e.g. two >> computationally expensive, consecutive .map stages (they will now be >> processed sequentially, not concurrently), however with splits I think it >> was a very natural expectation that things will be processed in parallel. >> >> Hence I have four questions: >> >> 1. if I have a split, where should the async boundary go? My first >> attempt was adding the async boundary attribute to the split graph stage >> itself, but that didn't work. By experimentation, I have to add it to the >> stages that are connected to the split's outputs. >> >> 2. how to insert async boundaries in linear pipelines? E.g.: >> >> Source(List(1, 2, 3)) >> .map(_ + 1) >> .addAttributes(Attributes.asyncBoundary) >> .map(_ * 2) >> >> doesn't compile >> >> 3. are you sure it's a good idea to make all processing stages fuseable >> by default? E.g. conflate seems to be naturally concurrent, as it deals >> with slower/faster components, which doesn't make much sense if everything >> is in one thread :) >> >> 4. which built-in stages are fuseable by default? The docs are quite >> vague here saying only "linear" ones. Which ones are these? >> >> If akka-streams now has *explicit* concurrency control, instead of the >> *implicit* one so far, maybe it would make sense to be explicit in the api >> (different name?) which stages will run concurrently by default? >> >> Thanks, >> Adam >> > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+...@googlegroups.com <javascript:>. > To post to this group, send email to akka...@googlegroups.com > <javascript:>. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > > > > *Dr. Roland Kuhn* > *Akka Tech Lead* > Typesafe <http://typesafe.com/> – Reactive apps on the JVM. > twitter: @rolandkuhn > <http://twitter.com/#!/rolandkuhn> > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
