Where in the documentation have you read that the leader is the oldest member? I would like to correct that, because it's not true.
ClusterSingleton run on the oldest node. The leader is something that is mostly used by Akka Cluster itself to manage the membership. It's the node with the "lowest" address, sorted by host and port, with a few additional rules related to member status. There are some use cases for applications to make use of the leader, but most applications should use singleton instead. Note that there is a ClusterSingletonProxy that keeps track of where the singleton lives. Cheers, Patrik tors 12 maj 2016 kl. 18:00 skrev Markus Fresh < [email protected]>: > Hello, > > I've now been working for a few weeks in a project that uses Akka combined > with Spring Boot 4.2.4. We have started setting up a (small) cluster and > ran into an odd situation; according to the documentation, I had expected > that the oldest cluster node will be used as the cluster leader, and also > (by default) as the node for ClusterSingleton actors. > > However, I have encountered behaviour that seems to indicate that Akka > remembers the address of earlier nodes that were removed and then re-added > to the cluster; such a node is then immediately set to be the cluster > leader when it re-joins the cluster after it has been re-started. On the > other hand, our ClusterSingleton actor still remains on the oldest node > (just as we expect). > > So the cluster leader, as reported by either of: > - Cluster.get(getContext().system()).state().leader() > - Cluster.get(getContext().system()).readView().isLeader() > and the actual cluster on which our ClusterSingleton actor resides are > different after the first node in a cluster was removed and then re-added > to the cluster (but not in the reverse situation, where the younger node is > removed an then re-added). This was a bit of a surprise to me, and I can't > find any explanation for this. > > The details: our application is a Spring Boot Java application. Framework > versions in use: > Java 8 (1.8.0_64) > Akka 2.4.2 (com.typesafe.akk:akka-cluster_2.11:2.4.2, > com.typesafe.akk:akka-cluster-tools_2.11:2.4.2) > Spring Boot 4.2.4 > > In my test, I run two copy of the application on my local machine; > combined, they represent a simple cluster with two nodes. The copies are > started with different Akka ports: > node 1: [email protected]:2551 > node 2: [email protected]:2552 > > I first start node 1: > > 15:31:41.288 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2551, status = Up) > 15:31:41.293 INFO curr leader: true, address: [email protected]:2551 * > [i.e. the current node is the current leader; the leader address is > [email protected]:2551 <http://[email protected]:2551>]* > 15:31:41.293 INFO LeaderChanged(Some(akka.tcp://[email protected]:2551)), > getLeader: akka.tcp://[email protected]:2551 > > Then cluster node 2 is started: > > 15:39:57.351 INFO curr leader: false, address: [email protected]:2551 *[so > node 2 is not the current leader; instead it's still [email protected]:2551 > <http://[email protected]:2551>, as expected]* > 15:39:57.351 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2551, status = Up), address: [email protected]:2551 > 15:39:57.352 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2552, status = Up), address: [email protected]:2552 > > > Node 1 notices this and adds the second node to the cluster; now when > asked who the current leader is, cluster 1 is returned, as expected: > cluster 1: > 15:39:37.296 INFO Cluster Node [akka.tcp://[email protected]:2551] - Node > [akka.tcp://[email protected]:2552] is JOINING, roles [] > 15:39:37.298 INFO curr leader: true, address: [email protected]:2551 *[node > 1 is still seen as the leader]* > 15:39:37.298 INFO Member event MemberJoined(Member(address = akka.tcp:// > [email protected]:2552, status = Joining)): > 15:39:37.755 INFO Cluster Node [akka.tcp://[email protected]:2551] - Leader > is moving node [akka.tcp://[email protected]:2552] to [Up] > 15:39:37.757 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2552, status = Up) > 15:39:37.757 INFO curr leader: true, address: [email protected]:2551 > > > I then stop node 1, and node 2 assumes the position of cluster leader: > node 2: > 15:43:18.949 WARN Marking node(s) as UNREACHABLE [Member(address = > akka.tcp://[email protected]:2551, status = Up)] > 15:43:18.951 INFO Member detected as unreachable: Member(address = > akka.tcp://[email protected]:2551, status = Up) > 15:43:18.951 INFO curr leader: true, address: [email protected]:2552 > 15:43:18.951 INFO LeaderChanged(Some(akka.tcp://[email protected]:2552)), > getLeader: akka.tcp://[email protected]:2552 > ... > 15:43:29.966 INFO Younger observed OldestChanged: [None -> myself] > 15:43:29.966 INFO Singleton manager starting singleton actor > [akka://blue/user/ourSupervisorActor/singleton] > 15:43:29.966 INFO ClusterSingletonManager state change *[Younger -> > Oldest]* > > Starting node 1 again should still keep the older node 2 as the cluster > leader. Here's what happens, as seen from node 1: > 15:46:15.776 INFO Cluster Node [akka.tcp://[email protected]:2551] - > Welcome from [akka.tcp://[email protected]:2552] > 15:46:17.224 INFO Singleton identified at [akka.tcp:// > [email protected]:2552/user/ourSupervisorActor/singleton] > 15:46:17.291 INFO ClusterSingletonManager state change* [Start -> > Younger]* > ... > 15:46:40.318 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2552, status = Up), address: [email protected]:2552 > 15:46:40.318 INFO curr leader: true, address: [email protected]:2551* [and > node 1 is already leader, although it's the Younger node]* > 15:46:40.319 INFO LeaderChanged(Some(akka.tcp://[email protected]:2551)), > getLeader: akka.tcp://[email protected]:2551 > > And from the point of view of node 2, this is seen as follows: > > 15:46:15.731 INFO Member event MemberJoined(Member(address = akka.tcp:// > [email protected]:2551, status = Joining)): > 15:46:16.001 INFO Cluster Node [akka.tcp://[email protected]:2552] - Leader > is moving node [akka.tcp://[email protected]:2551] to [Up] > 15:46:16.002 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2551, status = Up) > 15:46:16.003 INFO curr leader: false, address: [email protected]:2551 *[now > also sees node 1 as the new leader]* > 15:46:16.003 INFO LeaderChanged(Some(akka.tcp://[email protected]:2551)), > getLeader: akka.tcp://[email protected]:2551 > > So now node 1 is the leader again although it was just started up (and > node 2 is definitively older). > > I can then trigger a message to our cluster singleton actor on node 1, and > can then see that the message is sent to node 2 for processing, so our > ClusterSingleton actor is still on node 2 at this point. > > When I then stop and start node 2 again, node 1 will be kept as the > cluster leader. In other words, when node 2 is re-added it will not take > the leader position. > Node 1 says: > 16:05:11.862 INFO Member event MemberJoined(Member(address = akka.tcp:// > [email protected]:2552, status = Joining)): > 16:05:12.104 INFO Cluster Node [akka.tcp://[email protected]:2551] - Leader > is moving node [akka.tcp://[email protected]:2552] to [Up] > 16:05:12.106 INFO curr leader: true, address: [email protected]:2551 > 16:05:12.106 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2552, status = Up), address: [email protected]:2552 > > And node 2 reports: > 16:05:32.804 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2551, status = Up) > 16:05:32.804 INFO Member is Up: Member(address = akka.tcp:// > [email protected]:2552, status = Up) > 16:05:32.805 INFO curr leader: false, address: [email protected]:2551 > 16:05:32.807 INFO LeaderChanged: > LeaderChanged(Some(akka.tcp://[email protected]:2551)), getLeader: > akka.tcp://[email protected]:2551 > > So here the leader is still node 1, i.e. the node that was already alive > when node 2 was added. But when I re-added node 1 to the cluster that > consisted of just node 2, the position of leader was immediately switched > over to node 1. > > Now is this expected behaviour? I would have thought that the older, > existing node will always remain cluster leader when a new cluster node is > added (independently of the earlier history of such nodes, i.e. > independently of whether they have been the leader in the cluster before). > > [The reason I'm trying to trace the cluster leader is to prevent long > messages being sent to a ClusterSingleton actor from other cluster nodes: > in our application it is sufficient if only the actors inside the > ClusterSingleton node communicate with the ClusterSingleton actor, so we > want to prevent messages being sent by stopping them on the senders' side > already, not at the receiver's side.] > > Thanks for any insights and comments! > Markus > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
