[akka-user][deprecated] Cluster Sharding: store ActorRefs or Identifiers in the actors?
I have an akka-http api that, upon receiving requests, sends messages to my Cluster via the cluster sharding region. The API layer doesn't know the ActorRefs of the actors in question, just the string identifiers. region ! AddTarget(sourceId, targetId) So Cluster Sharding works by looking up or creating the appropriate actor identified by "sourceId", passing "targetId" as an additional parameter. Each actor can have many targets - they'll all be talking to each other. >From within the source actor, I'm faced with the choice of storing those targets as List[String] or List[ActorRef]. If I stick with List[String], I can use context.parent to send the message to the targets, through the region. If I use List[ActorRef] instead (presumably by doing a one time Identify exchange so I can get the ActorRef from the response's sender()), I'd be able to skip the region lookup on all ensuing communications. Is one a best practice, or clearly better than the other? Thanks, Curt -- * ** New discussion forum: https://discuss.akka.io/ replacing akka-user google-group soon. ** This group will soon be put into read-only mode, and replaced by discuss.akka.io ** More details: https://akka.io/blog/news/2018/03/13/discuss.akka.io-announced * >> >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Akka Persistence and avoiding var in favor of context.become
Learning akka persistence, I was initially reluctant to use vars to capture state since we generally try to avoid vars when possible. But when experimenting with context.become(receiveBlock(stateParam)) inside a receiveRecover implementation, it appeared to interfere with its ability to stash other messages. (akka 2.4.20) It tried to handle normal messages that it didn't try to handle when I wasn't using context.become. Is it possible and reasonable to use context.become instead of var to capture state using Akka Persistence? At first glance, it seems like the interplay between receiveCommand and receiveRecover could get rather tricky, especially if your actor has initialization behavior that changes its receiveCommand behavior - for instance, initializing receiveCommand with an empty collection to start, i.e. def receiveCommand: Receive = initialReceive(List.empty) Thanks, Curt Siffert -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] beginner question on akka streams flow design.
Regarding that suggestion, here is a code sample on how you might accomplish the case of a graph stage that registers for updates from an actor: https://github.com/Keenworks/SampleActorStage Curt Siffert On Thursday, June 8, 2017 at 4:27:43 AM UTC-7, Akka Team wrote: > > One way would be mapAsync(1)(elem => > ask-actor-and-combine-elem-with-token). If asking for every element is too > expensive you could make a custom graph stage which asks the actor on > materialisation to get the first token, but then also registers for updates > from the actor, that can be done using the stage actor ref (see docs here: > http://doc.akka.io/docs/akka/current/scala/stream/stream-customize.html#integration-with-actors > ) > > -- > Johan > Akka Team > > On Thu, Jun 1, 2017 at 7:21 PM, Mohnish Kodnani > wrote: > >> Hi , >> I have the following use case that I want to model using akka streams and >> I am new to akka streams so would like to know how/what is the best way to >> model it. >> >> I have a kafka consumer which reads from one topic massages the message >> and sends it to a RESTful external service. >> This external restful service takes a token that needs to be sent with >> every message in the header. This token expires every roughly 24 hours. >> >> I was thinking of creating an FSM actor that will act as a Token Manager. >> It will be in Active state where it can send the current token on receiving >> say a GetToken message. Then every 24 hours it receives a scheduled event >> for FetchToken, then it will go into Fetching mode, where it will retrieve >> a new token from another service and in that state it should keep queuing >> the requests for GetToken. >> >> Now, how do i hook this actor (TokenManager) into my stream so that it >> can provide a token for every message from kafka but block the stream until >> it gets a new token when it expires. >> Or if there is another solution to what I am thinking instead of creating >> a separate actor. >> >> Thanks >> Mohnish. >> >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+...@googlegroups.com . >> To post to this group, send email to akka...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] ActorPublisher GraphStage alternative?
Here’s a code sample showing a possible usage of getStageActor. https://github.com/Keenworks/SampleActorStage <https://github.com/Keenworks/SampleActorStage> > On Jun 4, 2017, at 12:12 PM, Curt Siffert wrote: > > I believe I have figured this out thanks to finding a getStageActor example > in the reactive-kafka project. The insight I missed is that onPull() is only > called sometimes (I assume when consumer experiences back pressure and > signals readiness by pulling), and that push() should also be called from the > callback function. I will link to a code sample for others looking for a > smaller getStageActor example after I clean it up a bit. > > Curt > >> On Jun 2, 2017, at 9:15 PM, Curt Siffert > <mailto:c...@keenworks.com>> wrote: >> >> I made some progress figuring out how to use getStageActor (not >> getStageActorRef - the docs have an error) - the stage’s preStart() >> registers the stage’s actorRef with an outside actor. Messages forwarded to >> that actorRef (via the outside actor) will get sent to a callback function >> inside the stage. >> >> What I’m doing is using the callback to add items to a stage’s internal >> queue, and then using onPull to dequeue that queue if there are items in the >> queue. >> >> The part that is confusing to me is that when applied to a Source, I haven’t >> figured out how to get onPull to trigger from that callback. onPull only >> seems to fire right after materialization, when the queue is still empty, >> and then it doesn’t pull again after receiving messages and queuing, even >> though I have a stream wired up to process what what the Source emits. >> >> ActorPublisher was originally written to suggest that after creating the >> Source, you could then send messages to the actor in a periodic fashion and >> the stream would be able to process those messages. But so far I haven’t >> been able to duplicate that behavior. It’s as if the only way to use >> getStageActor is if its GraphStage is a FlowShape. >> >> In that case, I suppose I could use Source.queue but it seems like I’m >> missing something. Is it not possible to use getStageActor on a Source? If >> anyone has code samples of getStageActor on a Source, I’d love to see it. >> >> Thanks, >> Curt >> >> >>> On May 28, 2017, at 3:19 AM, Patrik Nordwall >> <mailto:patrik.nordw...@gmail.com>> wrote: >>> >>> Can't you use Source.queue? Backpressure can be maintained by piping the >>> result of the future back to an ordinary actor. >>> >>> /Patrik >>> lör 27 maj 2017 kl. 16:29 skrev Richard Rodseth >> <mailto:rrods...@gmail.com>>: >>> In case it helps: >>> >>> https://groups.google.com/d/topic/akka-user/AgVHHnl9ub4/discussion >>> <https://groups.google.com/d/topic/akka-user/AgVHHnl9ub4/discussion> >>> https://github.com/akka/akka/issues/22742 >>> <https://github.com/akka/akka/issues/22742> >>> >>> On Fri, May 26, 2017 at 1:34 PM, Curt Siffert >> <mailto:c...@keenworks.com>> wrote: >>> >>> Hi, I see in the docs for 2.5.2 that ActorPublisher/ActorSubscriber will be >>> deprecated. >>> >>> In my (still beginning) experiments with akka streams I used ActorPublisher >>> as a way to help create some back pressure controls while consuming >>> messages from an external queue. This worked just by consuming the queue >>> like normal and then for each message consumed, sending a message to >>> ActorPublisher. >>> >>> Without using ActorPublisher, I can use a Source.actorRef, but that doesn't >>> have back pressure controls. >>> >>> I know the recommended alternative to ActorPublisher is to use a custom >>> graph stage and I have started experimenting with that but so far I don't >>> see how to meet the ActorPublisher use case with it. So far it doesn't seem >>> like a custom Source has an ActorRef type signature like Source.actorRef >>> does. Once the custom stage is created, can I send a "tell" message to it >>> the way I did to ActorPublisher? Or am I supposed to use Source.actorRef >>> and then funnel it through the custom stage to get the back pressure >>> controls? >>> >>> Sorry if my question is muddled, I am still making my way through this. :-) >>> I recognize this is a bit weird since ideally the back pressure controls >>> would be applied to the queueing tech itself. &
Re: [akka-user] ActorPublisher GraphStage alternative?
I believe I have figured this out thanks to finding a getStageActor example in the reactive-kafka project. The insight I missed is that onPull() is only called sometimes (I assume when consumer experiences back pressure and signals readiness by pulling), and that push() should also be called from the callback function. I will link to a code sample for others looking for a smaller getStageActor example after I clean it up a bit. Curt > On Jun 2, 2017, at 9:15 PM, Curt Siffert wrote: > > I made some progress figuring out how to use getStageActor (not > getStageActorRef - the docs have an error) - the stage’s preStart() registers > the stage’s actorRef with an outside actor. Messages forwarded to that > actorRef (via the outside actor) will get sent to a callback function inside > the stage. > > What I’m doing is using the callback to add items to a stage’s internal > queue, and then using onPull to dequeue that queue if there are items in the > queue. > > The part that is confusing to me is that when applied to a Source, I haven’t > figured out how to get onPull to trigger from that callback. onPull only > seems to fire right after materialization, when the queue is still empty, and > then it doesn’t pull again after receiving messages and queuing, even though > I have a stream wired up to process what what the Source emits. > > ActorPublisher was originally written to suggest that after creating the > Source, you could then send messages to the actor in a periodic fashion and > the stream would be able to process those messages. But so far I haven’t been > able to duplicate that behavior. It’s as if the only way to use getStageActor > is if its GraphStage is a FlowShape. > > In that case, I suppose I could use Source.queue but it seems like I’m > missing something. Is it not possible to use getStageActor on a Source? If > anyone has code samples of getStageActor on a Source, I’d love to see it. > > Thanks, > Curt > > >> On May 28, 2017, at 3:19 AM, Patrik Nordwall > <mailto:patrik.nordw...@gmail.com>> wrote: >> >> Can't you use Source.queue? Backpressure can be maintained by piping the >> result of the future back to an ordinary actor. >> >> /Patrik >> lör 27 maj 2017 kl. 16:29 skrev Richard Rodseth > <mailto:rrods...@gmail.com>>: >> In case it helps: >> >> https://groups.google.com/d/topic/akka-user/AgVHHnl9ub4/discussion >> <https://groups.google.com/d/topic/akka-user/AgVHHnl9ub4/discussion> >> https://github.com/akka/akka/issues/22742 >> <https://github.com/akka/akka/issues/22742> >> >> On Fri, May 26, 2017 at 1:34 PM, Curt Siffert > <mailto:c...@keenworks.com>> wrote: >> >> Hi, I see in the docs for 2.5.2 that ActorPublisher/ActorSubscriber will be >> deprecated. >> >> In my (still beginning) experiments with akka streams I used ActorPublisher >> as a way to help create some back pressure controls while consuming messages >> from an external queue. This worked just by consuming the queue like normal >> and then for each message consumed, sending a message to ActorPublisher. >> >> Without using ActorPublisher, I can use a Source.actorRef, but that doesn't >> have back pressure controls. >> >> I know the recommended alternative to ActorPublisher is to use a custom >> graph stage and I have started experimenting with that but so far I don't >> see how to meet the ActorPublisher use case with it. So far it doesn't seem >> like a custom Source has an ActorRef type signature like Source.actorRef >> does. Once the custom stage is created, can I send a "tell" message to it >> the way I did to ActorPublisher? Or am I supposed to use Source.actorRef and >> then funnel it through the custom stage to get the back pressure controls? >> >> Sorry if my question is muddled, I am still making my way through this. :-) >> I recognize this is a bit weird since ideally the back pressure controls >> would be applied to the queueing tech itself. >> >> Curt >> >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> >> >>>>>>>>>> Check the FAQ: >> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html> >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> >>>>>>>>>> &l
Re: [akka-user] ActorPublisher GraphStage alternative?
I made some progress figuring out how to use getStageActor (not getStageActorRef - the docs have an error) - the stage’s preStart() registers the stage’s actorRef with an outside actor. Messages forwarded to that actorRef (via the outside actor) will get sent to a callback function inside the stage. What I’m doing is using the callback to add items to a stage’s internal queue, and then using onPull to dequeue that queue if there are items in the queue. The part that is confusing to me is that when applied to a Source, I haven’t figured out how to get onPull to trigger from that callback. onPull only seems to fire right after materialization, when the queue is still empty, and then it doesn’t pull again after receiving messages and queuing, even though I have a stream wired up to process what what the Source emits. ActorPublisher was originally written to suggest that after creating the Source, you could then send messages to the actor in a periodic fashion and the stream would be able to process those messages. But so far I haven’t been able to duplicate that behavior. It’s as if the only way to use getStageActor is if its GraphStage is a FlowShape. In that case, I suppose I could use Source.queue but it seems like I’m missing something. Is it not possible to use getStageActor on a Source? If anyone has code samples of getStageActor on a Source, I’d love to see it. Thanks, Curt > On May 28, 2017, at 3:19 AM, Patrik Nordwall > wrote: > > Can't you use Source.queue? Backpressure can be maintained by piping the > result of the future back to an ordinary actor. > > /Patrik > lör 27 maj 2017 kl. 16:29 skrev Richard Rodseth <mailto:rrods...@gmail.com>>: > In case it helps: > > https://groups.google.com/d/topic/akka-user/AgVHHnl9ub4/discussion > <https://groups.google.com/d/topic/akka-user/AgVHHnl9ub4/discussion> > https://github.com/akka/akka/issues/22742 > <https://github.com/akka/akka/issues/22742> > > On Fri, May 26, 2017 at 1:34 PM, Curt Siffert <mailto:c...@keenworks.com>> wrote: > > Hi, I see in the docs for 2.5.2 that ActorPublisher/ActorSubscriber will be > deprecated. > > In my (still beginning) experiments with akka streams I used ActorPublisher > as a way to help create some back pressure controls while consuming messages > from an external queue. This worked just by consuming the queue like normal > and then for each message consumed, sending a message to ActorPublisher. > > Without using ActorPublisher, I can use a Source.actorRef, but that doesn't > have back pressure controls. > > I know the recommended alternative to ActorPublisher is to use a custom graph > stage and I have started experimenting with that but so far I don't see how > to meet the ActorPublisher use case with it. So far it doesn't seem like a > custom Source has an ActorRef type signature like Source.actorRef does. Once > the custom stage is created, can I send a "tell" message to it the way I did > to ActorPublisher? Or am I supposed to use Source.actorRef and then funnel it > through the custom stage to get the back pressure controls? > > Sorry if my question is muddled, I am still making my way through this. :-) I > recognize this is a bit weird since ideally the back pressure controls would > be applied to the queueing tech itself. > > Curt > > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html> > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > >>>>>>>>>> <https://groups.google.com/group/akka-user> > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com > <mailto:akka-user+unsubscr...@googlegroups.com>. > To post to this group, send email to akka-user@googlegroups.com > <mailto:akka-user@googlegroups.com>. > Visit this group at https://groups.google.com/group/akka-user > <https://groups.google.com/group/akka-user>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. > > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> > >>>>>>>>>>
[akka-user] ActorPublisher GraphStage alternative?
Hi, I see in the docs for 2.5.2 that ActorPublisher/ActorSubscriber will be deprecated. In my (still beginning) experiments with akka streams I used ActorPublisher as a way to help create some back pressure controls while consuming messages from an external queue. This worked just by consuming the queue like normal and then for each message consumed, sending a message to ActorPublisher. Without using ActorPublisher, I can use a Source.actorRef, but that doesn't have back pressure controls. I know the recommended alternative to ActorPublisher is to use a custom graph stage and I have started experimenting with that but so far I don't see how to meet the ActorPublisher use case with it. So far it doesn't seem like a custom Source has an ActorRef type signature like Source.actorRef does. Once the custom stage is created, can I send a "tell" message to it the way I did to ActorPublisher? Or am I supposed to use Source.actorRef and then funnel it through the custom stage to get the back pressure controls? Sorry if my question is muddled, I am still making my way through this. :-) I recognize this is a bit weird since ideally the back pressure controls would be applied to the queueing tech itself. Curt -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] moving from redis cache to... ?
> On Apr 22, 2017, at 9:12 AM, Justin du coeur wrote: > Personally, I'd likely use Akka Cluster Sharding for this -- assuming each > request can be summarized in a way that fits as an entity identifier, it > seems like a good fit. In this model, the sharded entity would serve as a > read-through cache for the external service: your stream would go to the > sharded Actor -- if it's the first request, the Actor would make the external > request, otherwise it would already know about the value and return it. > Cluster Sharded Actors are a lot like an in-memory cache, and I use them for > exactly this in a number of cases. > > I'd probably only bother with Akka Persistence (I assume you're thinking of > building this on top of Redis) if the number of distinct request types is too > large to typically fit in memory (so the Sharded Actors would need to > sometimes get flushed), and too expensive/slow to want to ever re-run a > request externally. Might be worthwhile, might not; it depends on the > external constraints. That’s good to hear. I have some experience setting up an Akka Cluster in AWS before, but haven’t used cluster sharding. We might be able to get away without using Akka Persistence - we’re still working out those external constraints. Once “cached” (instantiated), the value should remain valid for quite a while (source data doesn’t change often), and the external lookup would be against another internal/nearby low-latency service anyway. If by distinct request types you mean number of values cached, we’re probably in the tens of millions, or perhaps hundreds of millions in time. It’s only one type of request though, two parameters that will return back a small data structure. We haven’t sized the data structure yet but it is roughly similar to an Address object - a flat structure with a handful of string fields. Since it’s a cluster that new instances can join, is memory effectively unbounded? That’s an actor instance for each value we’d otherwise cache, so tens or hundreds of millions of actor instances. I know akka scales out well but I have not seen much comparing it to the performance of something like Redis. I wonder at what point (if ever) an actor-per-cached-value doesn’t make sense anymore, if ever, compared to an external cache? I wonder if this also impacts what to pick for “state-store-mode”. Thanks again, Curt -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] moving from redis cache to... ?
Hi, I have a simple use case where a service: - reads from a queue - uses that information to make a call to an external web service - caches those results to Redis so it doesn't need to make that external call every time - uses those results to send modified information to another queue In the past we did it with Spring using synchronous requests for the external web service. We're getting more streamy now and there's a lot of interest in akka, which I've previously done some work with. So there's interest in rewriting this service in a reactive style. I've got a sample app that uses akka-streams to read from the queue, do an operation, and write back to another queue, and I can use akka-http to make the call to the service, but I'm stumped on how to appropriately cache. What's the best way in a case like this? The caching is simple - in the previous service it was just automatically caching a method call by its parameter values, via annotation. I've considered using scalacache since it has the familiar model of caching method calls by parameter. But I'm not sure it's really idiomatic for akka-streams (which I've just started using) or akka. Should I instead consider using akka persistent actors (one per service lookup I guess) or dumping Redis entirely and using akka distributed data? Thanks, Curt -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Why should nodes usually not include themselves in seed node list?
Hi, when discovering seed nodes dynamically, why is it important for every node (except for one) to not include itself in the seed node list? How could split brain happen if all nodes have the same list order of all seed nodes? >From what I understand, split-brain can only happen if multiple nodes put themselves first in the seed node list, is that correct? I ask because when doing static configuration, it's common for all nodes to have the same seed node list, which means all seed nodes would have themselves in the list. And that seems to be common, recommended practice. If doing dynamic discovery of seed nodes, is it okay for all nodes to have the same list of seed nodes, if we can guarantee they are all in the same order? From reading the docs, I gather that after sending a message to all seed nodes, if none respond, then it will try to join the first node in the list, which wouldn't work if the first node in the list isn't itself. We are considering using the AWS SDK by not querying the ASG for its instance list until the cloudformation stack says it is complete, which should guarantee that all nodes trying to join the cluster will get the same list of instances in the ASG. Then we can guarantee that all nodes would order the nodes the same - any reason this wouldn't work? Thanks, Curt -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Akka Cluster without Remoting?
To close the loop on this, here's what I ended up doing. Originally, a node would either start a Master actor system or a Worker actor system, depending on startup flags. (Again, Master would not communicate with Worker or vice versa through messages; each node was self-contained.) After changing it, all nodes start (join) the same Cluster actor system. I created a "Switchboard" Actor that can behave in either Master mode or Worker mode via context.become. I used ClusterSingleton to start an actor that does nothing other than send a message to SwitchboardActor to change its mode from Master to Worker. The Singleton actor does not receive any messages itself; it only serves to send that message to Switchboard upon startup. Seems to work fine, and now we don't need to use startup flags to specify which node is Master or Worker. The cluster figures itself out. Not sure if this is a classic pattern, but it seemed pretty straightforward, and didn't require any changes to actual workings of our Master/Worker systems. Curt On Friday, March 18, 2016 at 12:56:43 AM UTC-7, Curt Siffert wrote: > > Hi, I'm supporting a pre-existing Akka project that is currently set up to > run 1 Master instance and N Worker instances. > > It does not use routing/remoting. We start them in 'Master' or 'Worker' > modes via startup flags - each instance has its own actor system, and the > only information they communicate to each other is through Redis Pubsub. > Each of them does accept API requests so they are all behind a load > balancer. > > The Master actor system and the Worker actor system are slightly > different, in that Master polls some information and publishes to Redis, > while Worker subscribes to Redis so they all have the same information to > serve to the API requests. > > Now we have a desire to start up N instances without using the startup > flags, and have the instances just figure out for themselves which is > Master - and if it goes down, another would be master, etc. We don't want > more than one Master, since only one should poll the external information. > We don't need to send application messages across nodes since we'd keep > Redis. > > Looking at the Akka Cluster tutorials, I'm seeing a couple of options on > how to do this: > > 1) Just use leader election, and listen for LeaderChanged messages to make > Leader always be Master (stopping the Worker actor system and starting the > Master actor system if necessary). But this seems against the spirit of > keeping leader election decoupled from application functionality. Plus, in > experimenting, I'm seeing nodes switch Leader more often than I'd like > Master to switch (like if I take node #1 down and then bring it back up - > node #1 seems to end up Leader again even if node #2 was Leader for that > intervening time). > > 2) Use ClusterSingleton - but if I did this, what would be the way for the > Worker nodes to realize it is not the Master Singleton and start up the > different Worker actor system? With no routing/remoting, Master would not > be sending messages to them. > > It seems like this is a bit against the spirit of using Akka Cluster which > seems to assume routing. Is there a better way? > > Thanks, > Curt > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Akka Cluster without Remoting?
Hi, I'm supporting a pre-existing Akka project that is currently set up to run 1 Master instance and N Worker instances. It does not use routing/remoting. We start them in 'Master' or 'Worker' modes via startup flags - each instance has its own actor system, and the only information they communicate to each other is through Redis Pubsub. Each of them does accept API requests so they are all behind a load balancer. The Master actor system and the Worker actor system are slightly different, in that Master polls some information and publishes to Redis, while Worker subscribes to Redis so they all have the same information to serve to the API requests. Now we have a desire to start up N instances without using the startup flags, and have the instances just figure out for themselves which is Master - and if it goes down, another would be master, etc. We don't want more than one Master, since only one should poll the external information. We don't need to send application messages across nodes since we'd keep Redis. Looking at the Akka Cluster tutorials, I'm seeing a couple of options on how to do this: 1) Just use leader election, and listen for LeaderChanged messages to make Leader always be Master (stopping the Worker actor system and starting the Master actor system if necessary). But this seems against the spirit of keeping leader election decoupled from application functionality. Plus, in experimenting, I'm seeing nodes switch Leader more often than I'd like Master to switch (like if I take node #1 down and then bring it back up - node #1 seems to end up Leader again even if node #2 was Leader for that intervening time). 2) Use ClusterSingleton - but if I did this, what would be the way for the Worker nodes to realize it is not the Master Singleton and start up the different Worker actor system? With no routing/remoting, Master would not be sending messages to them. It seems like this is a bit against the spirit of using Akka Cluster which seems to assume routing. Is there a better way? Thanks, Curt -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Akka Cluster without Remoting?
On Friday, March 18, 2016 at 2:27:18 AM UTC-7, drewhk wrote: > > Hi Curt, > > On Fri, Mar 18, 2016 at 5:35 AM, Curt Siffert > wrote: > >> Hi, I'm supporting a pre-existing Akka project that is currently set up >> to run 1 Master instance and N Worker instances. >> >> It does not use routing/remoting. >> > > What do you mean by not using routing? How does that relate to remoting? I > sense some confusion here. > I mean that each instance's actors only talks to actors on its own instance; not to actors on other instances. It is quite likely I have some confusion, because so far we have not used either routing or remoting, and the tutorials I've seen for Akka Cluster use routing. > With no routing/remoting, Master would not be sending messages to them. >> > > Clustering works on top of remoting. I sense a confusion here, again. > In both of the tutorials I've seen, it appears that Master allocates work to the Workers by using routing to communicate with Workers on other nodes. Yes, I see that Clustering works on top of remoting, but beyond that we want to retain existing functionality. We just want an instance to know whether to start its own worker actor system, or its own master actor system. By the way, did you mistype? Did you mean it starts up the master subsystem if it sees the singleton locally? Is there another way that wouldn't require pinging/polling? -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
