I used `SourceQueue` to handle this case: first materialize a graph that
takes elements offered to a source queue, then create a sink that consumes
elements and publishes them to that source queue:
val kafkaSink: Sink[Event, Unit] = //...
val kafkaPublisherGraph: RunnableGraph[SourceQueue[Event]] =
Source.queue[Event](1024, OverflowStrategy.backpressure)
.to(kafkaSink)
val sourceQueue: SourceQueue[Event] = kafkaPublisherGraph.run
def queueWriter[T](queue: SourceQueue[T]): Sink[T, Unit] =
Flow[T]
.mapAsync(1)( elem => queue.offer(elem).map( notDropped => (notDropped,
elem) ) )
.to(Sink.foreach{
case (false, elem) => println(s"error: elem $elem rejected by queue")
case (true, elem) =>
})
full
example:
https://github.com/pkinsky/ws_to_kafka/blob/master/src/main/scala/com/pkinsky/StreamingUpload.scala
On Wednesday, January 27, 2016 at 9:28:36 PM UTC-8, Jakob Odersky wrote:
>
> What is the best approach to "connecting" streams at run-time?
>
> My use-case is a server that has an established connection to some
> backend service, modeled as a flow. Several clients can connect
> through websockets, also modeled as flows.
>
> If the number of clients was known at materialization, this scenario
> would translate smoothly into a fan-out stage, as illustrated in the
> following figure:
>
> <=> Client 1
> [I/O] <=> [Parser] <=> [FanOut] <=> Client 2
> ...
> <=> Client N
>
>
> The catch however is that during run-time, clients should be able
> connect and disconnect, without having to re-materialize (i.e.
> restart) the whole graph.
>
> I know that this kind of "mutable" configuration is not an appropriate
> situation to use graphs directly and have thought of another solution.
> My idea is to implement the above FanOut with a "multiplexer" actor. I
> would also implement custom GraphLogics that would interface with
> incoming connections (and also for the backend) and communicate with
> the multiplexer actor once materialized.
>
> Before I dive into that however, I was wondering if Akka already
> provides some similar solution out-of-the-box?
>
> thanks,
> --Jakob
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
