[akka-user] How to run akka stream distributedly in a cluster?
How to run akka stream distributedly in a cluster? Suppose a Flow DSL will be translated to 100 actors. How to configure these actor in start in different machine of the cluster? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] How to run akka stream distributedly in a cluster?
That’s currently not supported. Streams are a very fresh module, and we’re still working to get the in-jvm semantics and APIs *right* before we go distributed. -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] How to run akka stream distributedly in a cluster?
Thanks, I come from bigdata background. In my opinion, the Flow DSL is not expressive for many cases, like data shuffle. Have you considered using DSL like the one used in cascading, or spark, or storm trident? How can I help in this process? Sean On Monday, July 14, 2014 4:26:58 PM UTC+8, Konrad Malawski wrote: That’s currently not supported. Streams are a very fresh module, and we’re still working to get the in-jvm semantics and APIs *right* before we go distributed. -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] How to run akka stream distributedly in a cluster?
HI Sean, I come from bigdata background. In my opinion, the Flow DSL is not expressive for many cases, like data shuffle. Have you considered using DSL like the one used in cascading, or spark, or storm trident? The primary goal for streams is to provide a unified way to handle backpressure across asynchronous and probably remote boundaries. So the primary goal is safe integration across multiple threads/hosts over network/IO etc. The shuffle you need is a fairly special one I am not sure it will be covered by akka-streams. Remember though that akka-streams is just an implementation of the reactive-streams standard -- in the future there will be probably adapters to Spark/Storm/etc. so you will be able to plug your akka streams together with big-data oriented frameworks. -Endre How can I help in this process? Sean On Monday, July 14, 2014 4:26:58 PM UTC+8, Konrad Malawski wrote: That’s currently not supported. Streams are a very fresh module, and we’re still working to get the in-jvm semantics and APIs *right* before we go distributed. -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] How to run akka stream distributedly in a cluster?
As Endre already highlighted, we have a different focus than the specialised-for-big-data frameworks. Our focus is to get the back-pressure semantics and common API among library implementors right (the reactive streams API). There’s a lot of discussions around these still happening at https://github.com/reactive-streams/reactive-streams if you’re interested. The spec is still evolving actively. APIs will continue to be improved of course and we are well aware of Spark/Storm/Scalding (with the last one I’ve spent a lot of time ;-)). If you think we’re missing any fundamental combinators please check if they’re already in our todo’s or open a ticket: https://github.com/akka/akka/issues?labels=t%3Astreampage=1state=open -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] How to run akka stream distributedly in a cluster?
Thank you both for your reply. I believe there lacks some combinators, I will comment later after I check the github todo list. On Monday, July 14, 2014 4:55:59 PM UTC+8, Konrad Malawski wrote: As Endre already highlighted, we have a different focus than the specialised-for-big-data frameworks. Our focus is to get the back-pressure semantics and common API among library implementors right (the reactive streams API). There’s a lot of discussions around these still happening at https://github.com/reactive-streams/reactive-streams if you’re interested. The spec is still evolving actively. APIs will continue to be improved of course and we are well aware of Spark/Storm/Scalding (with the last one I’ve spent a lot of time ;-)). If you think we’re missing any fundamental combinators please check if they’re already in our todo’s or open a ticket: https://github.com/akka/akka/issues?labels=t%3Astreampage=1state=open https://github.com/akka/akka/issues?labels=t%3Astreampage=1state=open -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
