Hi, I have a sample application where two nodes join an Akka cluster. On both nodes there is an actor running registered to DistributedPubSubMediator and both are sending a Ping once per second (that hits either itself or the other node).
Then I gracefully shut down Node B (leave cluster, wait for MemberRemoved event, wait some seconds, terminate actor system, wait for termination, exit the jvm). But even though both nodes see Node B leaving through gossip convergence at 22:16:58, Node A keeps the left Node B in the PubSub (you see this with the RECEIVED PONG(nr, ping sender, pong responder)). Why does PubSub not remove actors from exited nodes, but only actors from removed nodes (remove of Node B on Node A happens 4 seconds after exiting of Node B). In my case I only do not loose messages, because Node B is waiting another 10 seconds after it sees its one remove (which again, is 4 seconds earlier than Node A sees Node B removed). Thanks in advance. Christian Output Node A [info] Thu Jun 23 22:16:23 CEST 2016 RECEIVE Pong(0,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER MemberUp(Member(address = akka.tcp://airfocus@127.0.0.1:2551, status = Up)) [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER LeaderChanged(Some(akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER RoleLeaderChanged(api,Some(akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:24 CEST 2016 RECEIVE Pong(1,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:25 CEST 2016 RECEIVE Pong(2,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:26 CEST 2016 RECEIVE Pong(3,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:28 CEST 2016 RECEIVE Pong(4,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:29 CEST 2016 RECEIVE Pong(5,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:30 CEST 2016 RECEIVE Pong(6,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:31 CEST 2016 RECEIVE Pong(7,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:32 CEST 2016 RECEIVE Pong(8,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:33 CEST 2016 RECEIVE Pong(9,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:34 CEST 2016 RECEIVE Pong(10,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:35 CEST 2016 RECEIVE Pong(11,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:36 CEST 2016 RECEIVE Pong(12,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:37 CEST 2016 RECEIVE Pong(13,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:38 CEST 2016 RECEIVE Pong(14,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:39 CEST 2016 RECEIVE Pong(15,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:40 CEST 2016 RECEIVE Pong(16,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:41 CEST 2016 RECEIVE Pong(17,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:42 CEST 2016 RECEIVE Pong(18,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:43 CEST 2016 RECEIVE Pong(19,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:44 CEST 2016 RECEIVE Pong(20,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:45 CEST 2016 RECEIVE Pong(21,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:46 CEST 2016 RECEIVE Pong(22,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:47 CEST 2016 RECEIVE Pong(23,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:48 CEST 2016 RECEIVE Pong(24,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER MemberJoined(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Joining)) [info] Thu Jun 23 22:16:49 CEST 2016 RECEIVE Pong(25,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2551, akka.tcp://airfocus@127.0.0.1:2552)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER MemberUp(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Up)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER SeenChanged(false,Set(akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2551, akka.tcp://airfocus@127.0.0.1:2552)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:16:50 CEST 2016 RECEIVE Pong(26,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:16:51 CEST 2016 RECEIVE Pong(27,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:52 CEST 2016 RECEIVE Pong(28,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:53 CEST 2016 RECEIVE Pong(29,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:16:54 CEST 2016 RECEIVE Pong(30,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:16:55 CEST 2016 RECEIVE Pong(31,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:56 CEST 2016 RECEIVE Pong(32,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:57 CEST 2016 RECEIVE Pong(33,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:58 CEST 2016 RECEIVE Pong(34,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER MemberLeft(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Leaving)) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER MemberExited(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Exiting)) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:59 CEST 2016 RECEIVE Pong(35,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:17:00 CEST 2016 RECEIVE Pong(36,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:01 CEST 2016 RECEIVE Pong(37,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:17:02 CEST 2016 RECEIVE Pong(38,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:17:02 CEST 2016 CLUSTER UnreachableMember(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Exiting)) [info] Thu Jun 23 22:17:02 CEST 2016 CLUSTER ReachabilityChanged(akka.tcp://airfocus@127.0.0.1:2551 -> akka.tcp://airfocus@127.0.0.1:2552: Unreachable [Unreachable] (1)) [info] Thu Jun 23 22:17:02 CEST 2016 CLUSTER MemberRemoved(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Removed),Exiting) [info] Thu Jun 23 22:17:02 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:17:03 CEST 2016 RECEIVE Pong(39,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:04 CEST 2016 RECEIVE Pong(40,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:05 CEST 2016 RECEIVE Pong(41,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:06 CEST 2016 RECEIVE Pong(42,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:07 CEST 2016 RECEIVE Pong(43,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:08 CEST 2016 RECEIVE Pong(44,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:09 CEST 2016 RECEIVE Pong(45,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:10 CEST 2016 RECEIVE Pong(46,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:10 CEST 2016 CLUSTER MemberLeft(Member(address = akka.tcp://airfocus@127.0.0.1:2551, status = Leaving)) [info] Thu Jun 23 22:17:11 CEST 2016 RECEIVE Pong(47,akka.tcp://airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:17:11 CEST 2016 CLUSTER MemberExited(Member(address = akka.tcp://airfocus@127.0.0.1:2551, status = Exiting)) [info] Thu Jun 23 22:17:11 CEST 2016 CLUSTER ClusterShuttingDown [info] Thu Jun 23 22:17:11 CEST 2016 CLUSTER MemberRemoved(Member(address = akka.tcp://airfocus@127.0.0.1:2551, status = Removed),Exiting) [info] Thu Jun 23 22:17:11 CEST 2016 CLUSTER LeaderChanged(None) [info] Thu Jun 23 22:17:11 CEST 2016 CLUSTER RoleLeaderChanged(api,None) [info] Thu Jun 23 22:17:11 CEST 2016 CLUSTER SeenChanged(true,Set()) [info] Thu Jun 23 22:17:11 CEST 2016 CLUSTER ReachabilityChanged() Output Node B [info] Thu Jun 23 22:16:48 CEST 2016 RECEIVE Pong(0,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER MemberUp(Member(address = akka.tcp://airfocus@127.0.0.1:2551, status = Up)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER MemberJoined(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Joining)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER LeaderChanged(Some(akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER RoleLeaderChanged(api,Some(akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2551, akka.tcp://airfocus@127.0.0.1:2552)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:16:49 CEST 2016 RECEIVE Pong(1,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER MemberUp(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Up)) [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:16:50 CEST 2016 RECEIVE Pong(2,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:51 CEST 2016 RECEIVE Pong(3,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:52 CEST 2016 RECEIVE Pong(4,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:53 CEST 2016 RECEIVE Pong(5,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:54 CEST 2016 RECEIVE Pong(6,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:55 CEST 2016 RECEIVE Pong(7,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:56 CEST 2016 RECEIVE Pong(8,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2551) [info] Thu Jun 23 22:16:57 CEST 2016 RECEIVE Pong(9,akka.tcp://airfocus@127.0.0.1:2552,akka.tcp://airfocus@127.0.0.1:2552) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER MemberLeft(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Leaving)) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER SeenChanged(false,Set(akka.tcp://airfocus@127.0.0.1:2552)) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2552, akka.tcp://airfocus@127.0.0.1:2551)) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER MemberExited(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Exiting)) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER ReachabilityChanged() [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER ClusterShuttingDown [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER MemberRemoved(Member(address = akka.tcp://airfocus@127.0.0.1:2551, status = Removed),Up) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER MemberRemoved(Member(address = akka.tcp://airfocus@127.0.0.1:2552, status = Removed),Exiting) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER LeaderChanged(None) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER RoleLeaderChanged(api,None) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER SeenChanged(true,Set()) [info] Thu Jun 23 22:16:58 CEST 2016 CLUSTER ReachabilityChanged() -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
