Hi,
First of, there is no support for serializing parts of a graph and sending them
to other nodes for execution in akka-streams. That would be awesome to have,
but it's not there yet.
Second, your shape in the graph is no longer a UniformFanOutShape, since you
have connected a Source to the Balance, and thus removed the inlet on the graph
you are building. Then you return the balance which hasn't got the same shape
as the thing you are building.
Have you looked at the cookbook samples of how to balance out processing to a
number f workers?
B/
On 2 March 2015 at 23:06:52, rmarsch (rmarsc...@localytics.com) wrote:
Hi,
I have a use case that I believe Akka Streams may be well suited for, but I'm
not clearly seeing how to implement part of it from reading the documentation.
I have a single, finite input source from an iterator stream. I want to use
Balance to split this stream into N partitions and then be able to feed these
stream partitions each into a RunnableFlow that would be executed on its own
node in a distributed system.
in ~> balance.out(0) ~> sink0
balance.out(1) ~> sink1
...
balance.out(n) ~> sinkN
I have some rough code sketched together, but I think I'm missing some concepts
to tie everything together. Runtime complains about the inlet for the
UniformInOutShape which I'm assuming is what I have in the inputGraph. Any
nudge in the right direction would be appreciated:
val iteratorBuilder: () => Iterator[T] = ???
val inputGraph: Graph[UniformFanOutShape[T,T], Unit] = FlowGraph.partial() {
implicit builder : FlowGraph.Builder =>
import FlowGraph.Implicits._
val stream: SourceShape[T] = builder.add(Source(iteratorBuilder))
val balance = builder.add(Balance[T](numPartitions))
stream ~> balance
balance
}
partitions = (0 until numPartitions).map(i => {
val sink : Sink[T, Future[Stream[T]]] = Sink.fold(Stream.empty[T]) { (v, e)
=> v.+:(e) }
val outFlow: RunnableFlow[Future[Stream[T]]] = FlowGraph.closed(sink) {
implicit builder : FlowGraph.Builder => sink =>
import FlowGraph.Implicits._
val balance = builder.add(inputGraph)
balance.out(i) ~> sink
}
new Partition(i, outFlow)
}).toArray
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
--
Björn Antonsson
Typesafe Inc. – Reactive Apps on the JVM
twitter: bantonsson
JOIN US. REGISTER TODAY!
Scala
Days
March 16th-18th,
San Francisco
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
