Re: [akka-user] How can the sender know that a message has been delivered (confirmed) by a channel?
Hi Alex,
the main concern you express is that messages sent from an
EventsourcedProcessor are eventually delivered to their recipients, which is
what a durable message queue (formerly called PersistentChannel) is for. The
main use of the non-persistent Channel is to deduplicate messages, but it
fundamentally cannot do that reliably in any case (since a confirmation may
have been on its way when it crashed).
In the solution you describe, if your Earnings must make sure to send to the
Aggregator, then they will in either case need to resend the message
periodically until confirmed, not only during restart, otherwise the message
can be deferred indefinitely; this is the same regardless of which transport
helper is used. My concrete proposal would be to store unconfirmed outbound
messages in the Earnings’ state and resend based on a timer tick—this extends
naturally to recovery after a machine crash.
I think we’ll see this resending functionality factored out in a common trait
soon as more people start building software this way.
Regards,
Roland
21 maj 2014 kl. 15:30 skrev ahjohannessen ahjohannes...@gmail.com:
Hi Roland,
You state the following:
...talks about Channel since that was only needed to contain the
side-effecting replay nature of command sourced processors.
A channel is also needed when sending events from an eventsourced processor
to another actor when one needs at-least-once delivery rather
than at-most-once:
def receiveRecover: Receive = {
case event: String = handleEvent(event)
}
def handleEvent(event: String) = {
// update state
// ...
// reliably deliver events
channel ! Deliver(Persistent(event), destination.path)
// alternatively something that encapsulates this pair
}
We are dependent on this very functionality for reliable coordination in our
apps and removing Channel leaves us worried.
...Your description below is rather terse, so it is not fully clear to me
how you are using Channel in this case and what a replacement should be, can
you elaborate?
As stated earlier, we have many eventsourced processors of same type, e.g.
1 instances of Earnings, that are loaded on demand by a supervisor
when a command enters into the domain. When an instance is loaded the command
is forwarded to this guy and when the command
results in domain events we acknowledge to sender in the persist callback
*but* also forward the persistent message to another actor, let's call it
stream,
that needs to react to this. It ensures that a calculation is initiated by
delivering a message to a change reactor via a unique channel.
Furthermore stream also ensures that a message is delivered, via another
unique channel, to an EarningsAggregator that essentially replicates all of
those 1 instances' events, because we need a global view in order to
maintain read models.
In receiveRecover we *also* forward replayed events to stream in order to
ensure that messages get delivered, via channels, to the respective
destinations,
in the case of a JVM crash.
During system start up we start all recently active (say last 24 hours)
Earnings instances to ensure that all calculations are triggered and that our
EarningsAggregator
gets the event, again in the name of crash paranoia. These instances shut
themselves down after a reasonable receive timeout in order to keep memory
footprint low.
This roughly happens by sending a passivate me message to their supervisor
which in turn ensures a proper shutdown and all of the instance's messages
are
processed by using a combination of poison pill, become, stash and listening
to terminated.
In the above I mention Earnings eventsourced processor, however we have
around 10 similar types under same supervisor that are similarly structured
and have the same
need of what a channel gives us.
Other usages of channels that we have are communication among processes.
Often here it is perfectly fine for us to use channels because it ensures
that something eventually
will happen, some eventsourced processor emits an event to reactor (ordinary
actor) via a channel (also during receiveRecover) that transforms the event
to a command in the
language of another eventsourced processor. All of such processes are
idempotent and maintaining an queue on both sides is just a lot of useless
busywork in most of our
use cases.
I have no idea of what a replacement of Channel would be because I do not
have an issue with it and think it is a good primitive that solves many of
our use cases in ways
that are satisfying and saves us a lot of work trying to re-implement the
same functionality.
...My motivation here is to not remove needed functionality without improved
replacement.
That is good to know :)
On Tuesday, May 20, 2014 7:32:38 PM UTC+1, rkuhn wrote:
Hi Alex,
I have filed the ticket for Processor’s removal
Re: [akka-user] Does akka-persistence demand incorruptible actor state?
Hi Lawrence, 21 maj 2014 kl. 17:38 skrev Lawrence Wagerfield lawre...@dmz.wagerfield.com: Interesting, thanks for the heads-up regarding the deprecation of Processor :) While we're on the topic of event-sourcing: Is it legal to send messages on recovery? These are side-effecting, but will never manifest as a failure in the sender. If the only reason not to 'side-effect' is to avoid failure of the actor during recovery, then I guess the answer is 'yes', we can send messages. However, does the reasoning go further than that such that transmitting messages during recovery is a bad idea? No, sending messages is not a bad idea per se, but the recipient will have to be able to cope with duplicates since these messages might be sent multiple times. In general you will find that these messages should have an identifying property like a deterministic sequence number that allows recipients to keep track of what they have processed already. Regards, Roland Thanks, Lawrence On Tuesday, May 20, 2014 7:39:53 PM UTC+1, rkuhn wrote: Hi Lawrence, this is a very good point to bring up: systematic errors in the event application logic will indeed just be recreated during replay, so persistence does not lead to resilience in this case. However, it does allow you to fix the implementation of your actor and rerun the log in the correct fashion, fixing the bug retroactively. This works a lot better for event-sourcing due to explicit control over the side-effects, which is why we are going to remove the Processor trait and enhance EventsourcedProcessor’s throughput (the lack of which is currently the reason for having Processor at all), see https://github.com/akka/akka/issues/15230. Regards, Roland 14 maj 2014 kl. 22:21 skrev Lawrence Wagerfield lawr...@dmz.wagerfield.com: Akka documentation highlights that actor state is reset on failure in standard actor systems. However, an actor powered by Persistent messages will simply have these messages replayed and hence will rebuild the same state. Does akka-persistence therefore assume we have designed all processors/views to have incorruptible internal state? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com. To post to this group, send email to akka...@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. Dr. Roland Kuhn Akka Tech Lead Typesafe – Reactive apps on the JVM. twitter: @rolandkuhn -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. Dr. Roland Kuhn Akka Tech Lead Typesafe – Reactive apps on the JVM. twitter: @rolandkuhn -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Does akka-persistence demand incorruptible actor state?
In event sourcing principles you don't perform external side effects during replay/recovery. That doesn't prevent you from doing it, if you have a good reason for it. Sometimes it is more useful to send messages immediately after recovery, than during recovery. Later events in the recovery might change the state in such a way that you don't need to send out the messages. An anecdote, in the early implementation of the ShardCoordinator I used watch/unwatch from the recovery. That was functionally correct, but very inefficient use of remote death watch. A better solution was to only watch the remaining actors in the state after recovery. Regards, Patrik On Mon, May 26, 2014 at 7:33 AM, Roland Kuhn goo...@rkuhn.info wrote: Hi Lawrence, 21 maj 2014 kl. 17:38 skrev Lawrence Wagerfield lawre...@dmz.wagerfield.com: Interesting, thanks for the heads-up regarding the deprecation of Processor :) While we're on the topic of event-sourcing: Is it legal to send messages on recovery? These are side-effecting, but will never manifest as a failure in the sender. If the only reason not to 'side-effect' is to avoid failure of the actor during recovery, then I guess the answer is 'yes', we can send messages. However, does the reasoning go further than that such that transmitting messages during recovery is a bad idea? No, sending messages is not a bad idea per se, but the recipient will have to be able to cope with duplicates since these messages might be sent multiple times. In general you will find that these messages should have an identifying property like a deterministic sequence number that allows recipients to keep track of what they have processed already. Regards, Roland Thanks, Lawrence On Tuesday, May 20, 2014 7:39:53 PM UTC+1, rkuhn wrote: Hi Lawrence, this is a very good point to bring up: systematic errors in the event application logic will indeed just be recreated during replay, so persistence does not lead to resilience in this case. However, it does allow you to fix the implementation of your actor and rerun the log in the correct fashion, fixing the bug retroactively. This works a lot better for event-sourcing due to explicit control over the side-effects, which is why we are going to remove the Processor trait and enhance EventsourcedProcessor’s throughput (the lack of which is currently the reason for having Processor at all), see https://github.com/akka/ akka/issues/15230. Regards, Roland 14 maj 2014 kl. 22:21 skrev Lawrence Wagerfield lawr...@dmz.wagerfield.com: Akka documentation highlights that actor state is reset on failurehttp://doc.akka.io/docs/akka/snapshot/general/supervision.html#supervision-directives in standard actor systems. However, an actor powered by Persistent messages will simply have these messages replayed and hence will rebuild the same state. Does akka-persistence therefore assume we have designed all processors/views to have incorruptible internal state? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/ current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com. To post to this group, send email to akka...@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. *Dr. Roland Kuhn* *Akka Tech Lead* Typesafe http://typesafe.com/ – Reactive apps on the JVM. twitter: @rolandkuhn http://twitter.com/#!/rolandkuhn -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. *Dr. Roland Kuhn* *Akka Tech Lead* Typesafe http://typesafe.com/ – Reactive apps on the JVM. twitter: @rolandkuhn http://twitter.com/#!/rolandkuhn -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com.
Re: [akka-user] [akka-streams]: actor producers, load balancers
One thing unanswered from the previous email: Is it in the spirit of akka-stream/reactive streams to implement your own producers? Or should all producers (publishers) be created by the framework? In theory reactive streams aim to be usable between frameworks - so an Rx Producer would be consumable in akka-streams and vice-versa. So, yes it's ok to provide your own Producers (like from a database etc). Internally we have some more abstractions which make implementing these easier, but I'm not aware of plans of exposing them any time soon - for starters we need to update akka-streams to the updated reactive-streams interfaces. -- Cheers, Konrad 'ktoso' Malawski hAkker - Typesafe, Inc http://scaladays.org -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-streams]: actor producers, load balancers
On Monday, May 26, 2014 2:25:52 PM UTC+2, Konrad Malawski wrote: One thing unanswered from the previous email: Is it in the spirit of akka-stream/reactive streams to implement your own producers? Or should all producers (publishers) be created by the framework? In theory reactive streams aim to be usable between frameworks - so an Rx Producer would be consumable in akka-streams and vice-versa. So, yes it's ok to provide your own Producers (like from a database etc). Internally we have some more abstractions which make implementing these easier, but I'm not aware of plans of exposing them any time soon - for starters we need to update akka-streams to the updated reactive-streams interfaces. I see, but then I have to implement the subscription management etc myself - which I guess can be tricky ;) But why not try. Thanks for the help! Adam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-streams]: actor producers, load balancers
Hi Adam - is it reasonable (thinking about reactive streams in general) to have an actor which produces elements on-demand (instead of providing a collection/iterator/() = as is currently supported)? As far as I understand the current implementation, subscribers explicitly ask publishers for more elements (through Subscription.requestMore) - so it seems it would be possible to pass such a request to an actor and ask for the given amount of elements. Is there any chance to get actor producers in some future releases, or there are no such plans currently? There is already one bridge to build consumers based on actors: https://github.com/akka/akka/pull/15214 The producer side is still missing, but there will be something to support that as well. - another thing is if the streams are thought to be more local, or remote as well? There's currently the TCP stream implementation, which I guess would indicate remote as well (and in such scenarios the need for backpressure arises quite naturally, maybe even more than in locally), but do you plan to develop this somehow? Streaming through TCP is already remote, so in that sense it is already supported. Of course you need to open up the connections yourself now, and there is no registry or naming functionality to obtain remote stream endpoints. E.g. when there would be multiple consumers for a single producer, a useful component would be a load-balancer which takes into account the backpressure information. All stream components take into account backpressure signal independently of fan-in or fan-out behavior, so this is by default available. (we do not have balancing elements yet, but they will be there eventually. Workaround can be a groupBy assigning to a fixed number of output groups randomly, but this is a bit dangerous field). From the stream viewpoint it does not matter whether the output streams are local or TCP connections, the balancing should work exactly the same way. -Endre Thanks! -- Adam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-streams]: actor producers, load balancers
Hey Adam, Patrik has right now opened a ticket and started exposing the ActorProducer abstraction :-) https://github.com/akka/akka/issues/15288 This will help a lot in implementing external producers. :-) On Mon, May 26, 2014 at 2:52 PM, Endre Varga endre.va...@typesafe.comwrote: Hi Adam - is it reasonable (thinking about reactive streams in general) to have an actor which produces elements on-demand (instead of providing a collection/iterator/() = as is currently supported)? As far as I understand the current implementation, subscribers explicitly ask publishers for more elements (through Subscription.requestMore) - so it seems it would be possible to pass such a request to an actor and ask for the given amount of elements. Is there any chance to get actor producers in some future releases, or there are no such plans currently? There is already one bridge to build consumers based on actors: https://github.com/akka/akka/pull/15214 The producer side is still missing, but there will be something to support that as well. - another thing is if the streams are thought to be more local, or remote as well? There's currently the TCP stream implementation, which I guess would indicate remote as well (and in such scenarios the need for backpressure arises quite naturally, maybe even more than in locally), but do you plan to develop this somehow? Streaming through TCP is already remote, so in that sense it is already supported. Of course you need to open up the connections yourself now, and there is no registry or naming functionality to obtain remote stream endpoints. E.g. when there would be multiple consumers for a single producer, a useful component would be a load-balancer which takes into account the backpressure information. All stream components take into account backpressure signal independently of fan-in or fan-out behavior, so this is by default available. (we do not have balancing elements yet, but they will be there eventually. Workaround can be a groupBy assigning to a fixed number of output groups randomly, but this is a bit dangerous field). From the stream viewpoint it does not matter whether the output streams are local or TCP connections, the balancing should work exactly the same way. -Endre Thanks! -- Adam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Cheers, Konrad 'ktoso' Malawski hAkker - Typesafe, Inc http://scaladays.org -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: Inconsistent behavior of ClusterClient.Send (akka 2.3.2) ?
On Mon, May 26, 2014 at 07:41:53AM +0200, Patrik Nordwall wrote: Hi Eugene, I have not looked at the code yet, but in the config you have two consistent hashing routers. The envelope is unwrapped when the message pass the first consistent hashing envelope. Could that be the reason? Well, there are 2 different routers. So host A starts a router routerA, which, in turn, starts another routerB on the host B, once it joins the cluster. These 2 routers shouldn't interfere at all. -- Eugene N Dzhurinsky pgp5c90zZAboq.pgp Description: PGP signature
Re: [akka-user] Cluster Singleton duplicated if primary seed restarted
Hi, The docs also advice against auto-downing. However I do not really get the alternative. Manual downing would be unworkable, because it could render your application unavailable for to long. So should I implement some strategy in my akka solution, or in some external monitoring system? How are people using this in production? Cheers, Jeroen -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: Inconsistent behavior of ClusterClient.Send (akka 2.3.2) ?
It *seems* that I've fixed this weird behavior by making messages, being sent to the cluster, implementing the *ConsistentHashable* trait At least, the sample project and the components from my prototype - are both working well now. So it turns out that for the case if there are several *nested* routes of type *ConsistentHashingPool* - then it's not enough to just *wrap* a payload into an envelope - it's necessary to make it extending the aproppriate traits, so routers can decide what to do. If this is desired behavior - then it's better to put a note about it somewhere in the documentation. Thanks everybody for the patience :) -- Eugene N Dzhurinsky -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-streams]: actor producers, load balancers
- another thing is if the streams are thought to be more local, or remote as well? There's currently the TCP stream implementation, which I guess would indicate remote as well (and in such scenarios the need for backpressure arises quite naturally, maybe even more than in locally), but do you plan to develop this somehow? Streaming through TCP is already remote, so in that sense it is already supported. Of course you need to open up the connections yourself now, and there is no registry or naming functionality to obtain remote stream endpoints. Right, but I guess naming/lookup is out of scope for akka-stream? I guess I was asking whether your initial focus is going to be on local or remote, but I think Konrad already answered that :) E.g. when there would be multiple consumers for a single producer, a useful component would be a load-balancer which takes into account the backpressure information. All stream components take into account backpressure signal independently of fan-in or fan-out behavior, so this is by default available. (we do not have balancing elements yet, but they will be there eventually. Workaround can be a groupBy assigning to a fixed number of output groups randomly, but this is a bit dangerous field). From the stream viewpoint it does not matter whether the output streams are local or TCP connections, the balancing should work exactly the same way. So right now you have broadcast communication implemented when there are multiple consumers. The other pattern is point-to-point, where each element goes to one of the attached consumers - where load-balancing would make sense (as you have the demand information, you could do very good guesses :) ). Anyway, I'll be watching the developments here :) Also with Actor producers, I think it would be quite easy to implement such a load-balancer: for each consumer, you would have an actor-producer, which would report the demand to a parent actor, where the load-balancing would take place. Thanks for the info! Adam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-streams]: actor producers, load balancers
Awesome, subscribed :) Thanks, Adam On Monday, May 26, 2014 3:04:30 PM UTC+2, Konrad Malawski wrote: Hey Adam, Patrik has right now opened a ticket and started exposing the ActorProducer abstraction :-) https://github.com/akka/akka/issues/15288 This will help a lot in implementing external producers. :-) On Mon, May 26, 2014 at 2:52 PM, Endre Varga endre...@typesafe.comjavascript: wrote: Hi Adam - is it reasonable (thinking about reactive streams in general) to have an actor which produces elements on-demand (instead of providing a collection/iterator/() = as is currently supported)? As far as I understand the current implementation, subscribers explicitly ask publishers for more elements (through Subscription.requestMore) - so it seems it would be possible to pass such a request to an actor and ask for the given amount of elements. Is there any chance to get actor producers in some future releases, or there are no such plans currently? There is already one bridge to build consumers based on actors: https://github.com/akka/akka/pull/15214 The producer side is still missing, but there will be something to support that as well. - another thing is if the streams are thought to be more local, or remote as well? There's currently the TCP stream implementation, which I guess would indicate remote as well (and in such scenarios the need for backpressure arises quite naturally, maybe even more than in locally), but do you plan to develop this somehow? Streaming through TCP is already remote, so in that sense it is already supported. Of course you need to open up the connections yourself now, and there is no registry or naming functionality to obtain remote stream endpoints. E.g. when there would be multiple consumers for a single producer, a useful component would be a load-balancer which takes into account the backpressure information. All stream components take into account backpressure signal independently of fan-in or fan-out behavior, so this is by default available. (we do not have balancing elements yet, but they will be there eventually. Workaround can be a groupBy assigning to a fixed number of output groups randomly, but this is a bit dangerous field). From the stream viewpoint it does not matter whether the output streams are local or TCP connections, the balancing should work exactly the same way. -Endre Thanks! -- Adam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com javascript:. To post to this group, send email to akka...@googlegroups.comjavascript: . Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com javascript:. To post to this group, send email to akka...@googlegroups.comjavascript: . Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Cheers, Konrad 'ktoso' Malawski hAkker - Typesafe, Inc http://scaladays.org -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
