Re: [akka-user] akka-streams - How to define a Source from an arbitrary event stream?

2015-04-14 Thread Jeff
This is pretty clever. Can this same trick be applied to a Sink that feeds data into an external system? On Friday, February 20, 2015 at 2:20:42 AM UTC-8, drewhk wrote: > > Hi Simon, > > One trick I like to use is to define a Source in terms of a PushPullStage. > Now this sounds strange, since a

Re: [akka-user] akka-streams - How to define a Source from an arbitrary event stream?

2015-04-09 Thread Viktor Klang
pushRecursively() onComplete { case Failure(ex) => onError(ex) context.stop(self) case s => } that's not safe. On Thu, Apr 9, 2015 at 1:16 PM, Jakub Liska wrote: > I do it just via ActorPublisher, the "scroll" method is basically > asynchronously l

Re: [akka-user] akka-streams - How to define a Source from an arbitrary event stream?

2015-04-09 Thread Jakub Liska
I do it just via ActorPublisher, the "scroll" method is basically asynchronously loading elasticsearch records (classic cursor thingy). It's a combination of request demand and asynchronous source of events : def receive: Receive = { case Request(n) if totalDemand > 0 && n > 0 && isActiv

Re: [akka-user] akka-streams - How to define a Source from an arbitrary event stream?

2015-02-21 Thread Giovanni Alberto Caporaletti
but this approach forces you to call the thing that produces items and block in the same dispatcher as the consumer, doesn't it? What's the best option here? Having an Iterator[Future[T]] that return promises of something that's being executed in a different exec. context? Thank you G On Fri

Re: [akka-user] akka-streams - How to define a Source from an arbitrary event stream?

2015-02-20 Thread Luis Ángel Vicente Sánchez
That's quite a nice little trick Endre, way better that writing an ActorPublisher if you don't need to communicate with the Producer. I did something similar to create an infinite stream from Amazon SQS (using an infinite Iterator[Unit] and mapAsyncUnordered) but this seems a much better approach.

Re: [akka-user] akka-streams - How to define a Source from an arbitrary event stream?

2015-02-20 Thread Endre Varga
Hi Simon, One trick I like to use is to define a Source in terms of a PushPullStage. Now this sounds strange, since a PushPullStage is supposed to be someting that transforms incoming element into outgoing elements, how can that be a Source? Well, the trick is this: def mySource = Source.empty

[akka-user] akka-streams - How to define a Source from an arbitrary event stream?

2015-02-19 Thread Simon Schäfer
I struggle in nicely defining a Source that gets its elements from an arbitrary event stream. At the moment my code looks like this: def watchKey[A : Reads](key: SettingKey[A])(implicit ctx: ExecutionContext): Source[Out[A]] = { Source(new Publisher[Out[A]] { var requestedElems = 0L