There was some discussion about merging Seq in the docs. Anyway, it is a side point and I don't want to get derailed from my original questions so I will read your other responses later today. On 21 Jan 2015 08:50, "Endre Varga" <endre.va...@typesafe.com> wrote:
> > > On Wed, Jan 21, 2015 at 9:45 AM, Sam Halliday <sam.halli...@gmail.com> > wrote: > >> Err, Endre, yes. And when merging Seq[T] in a mapConcat we would all >> benefit greatly from faster merging, Seq[Seq[T]] => Seq[T] >> > What do you mean? I don't understand. What mapConcat does is that it takes > a Source[T] and a function T => Seq[U] and gives a Source[U] (similarly > with Flow). Source is not a Seq, it is a completely different beast. > > > >> On 21 Jan 2015 08:03, "Endre Varga" <endre.va...@typesafe.com> wrote: >> >>> Hi Sam, >>> >>> On Tue, Jan 20, 2015 at 9:56 PM, Sam Halliday <sam.halli...@gmail.com> >>> wrote: >>> >>>> One more comment on the streams API. It is really cool that you've >>>> thought about using mapConcat instead of flatMap to enable optimised merge >>>> operations. I just wanted to draw your attention to a clojure project that >>>> does super-fast merging of immutable Tree/Trie structures: >>>> https://github.com/JulesGosnell/seqspert >>>> >>> >>> I feel a misunderstanding here. Akka Streams is not about data >>> structures, it is about streams. I mean real streams, like live TCP >>> incoming bytes. Unfortunately Java also chose the name Stream for another >>> concept, but that is more targeted for collections. In Akka Streams on the >>> other hand streams backed by collections is the rare case, not the common >>> one (although many examples use collections as sources since they are easy >>> to show). >>> >>> -Endre >>> >>> >>>> The work is definitely portable to the Scala collection types, as they >>>> are based on the Clojure implementations. >>>> >>>> >>>> On Tuesday, 20 January 2015 19:04:45 UTC, Sam Halliday wrote: >>>>> >>>>> Dear Akka Team, >>>>> >>>>> In response to Bjorn's prompting[1], I have read all the Akka >>>>> Streams 1.0-M2 documentation[2]. >>>>> >>>>> I am very impressed, and excited! There has clearly been a lot of >>>>> thought put into this framework and I am looking forward to using >>>>> it in anger over the coming years. >>>>> >>>>> Bjorn asked if I felt any examples were missing, and sadly my >>>>> original request (that I've been going on about for years, >>>>> sorry!) is indeed missing. It is the case of a fast producer and >>>>> a slow consumer that is ideal for parallelisation. >>>>> >>>>> I believe it may be possible to use the current 1.0-M2 to address >>>>> my bugbear by using the Actor integration to write an actor that >>>>> has N instances behind a router, but it feels hacky to have to >>>>> use an Actor at all. What is really missing is a Junction that >>>>> multiplies the next Flow into N parallel parts that run on >>>>> separate threads. >>>>> >>>>> >>>>> In general, I felt that the documentation was missing any >>>>> commentary on concurrency and parallelisation. I was left >>>>> wondering what thread everything was happening on. Some initial >>>>> questions I have in this area: >>>>> >>>>> 1. Is everything actually executed in the same thread? What about >>>>> when you have a Junction? >>>>> >>>>> 2. Is it possible to be doing work to populate the Source's cache >>>>> while work is being executed in a Flow or Sink? >>>>> >>>>> It would be good to have a section in the documentation that >>>>> discusses this in more detail. >>>>> >>>>> >>>>> And, very importantly, it would be good to have the feature of >>>>> being able to split a Flow into N parallel parts! I recently >>>>> learnt how to do this in ScalazStreams but I'm much rather be >>>>> able to do it in Akka Streams as I find everything else about the >>>>> architecture to be so much easier to understand (plus integration >>>>> with Akka Actors is just tremendous). >>>>> >>>>> PS: I'm also very excited by Slick 3.0 which appears to be aiming >>>>> toward Reactive Streams and, I assume, integration with Akka Streams. e.g. >>>>> produce a Source[Entity] from a SELECT with pushback on extremely large >>>>> result sets. >>>>> >>>>> >>>>> [1] https://groups.google.com/d/msg/akka-user/1TlAy-oqOk8/xvJpyVMWytsJ >>>>> [2] http://doc.akka.io/docs/akka-stream-and-http-experimental/ >>>>> 1.0-M2/scala.html >>>>> [3] https://github.com/fommil/rx-playground/blob/master/src/ >>>>> main/scala/com/github/fommil/rx/scratch.scala#L204 >>>>> >>>>> Best regards, >>>>> Sam (fommil) >>>>> >>>>> -- >>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>> >>>>>>>>>> Check the FAQ: >>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>> >>>>>>>>>> Search the archives: >>>> https://groups.google.com/group/akka-user >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Akka User List" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to akka-user+unsubscr...@googlegroups.com. >>>> To post to this group, send email to akka-user@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/akka-user. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Akka User List" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/akka-user/SYmCYGQppKw/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> akka-user+unsubscr...@googlegroups.com. >>> To post to this group, send email to akka-user@googlegroups.com. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to a topic in the > Google Groups "Akka User List" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/akka-user/SYmCYGQppKw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
