[akka-user] Re: DistributedPubSub keeps sending to exited nodes
Ok, my assumption is right (see https://github.com/akka/akka/blob/master/akka-cluster-tools/src/main/scala/akka/cluster/pubsub/DistributedPubSubMediator.scala#L669). Is there a way to work around this? I don't want to wait some random time interval after leaving before actually shutting down. If not just the removing but already the exiting would remove the node, then one would have a more robust way (without random waits) to gracefully leave a cluster/pubsub. Also another question arises here: Wouldn't it be good, to temporarily remove nodes from pubsub that are unreachable and readd them when they are reachable again? Am Donnerstag, 23. Juni 2016 22:27:45 UTC+2 schrieb Christian Hoffmeister: > > Hi, > > I have a sample application where two nodes join an Akka cluster. On both > nodes there is an actor running registered to DistributedPubSubMediator and > both are sending a Ping once per second (that hits either itself or the > other node). > > Then I gracefully shut down Node B (leave cluster, wait for MemberRemoved > event, wait some seconds, terminate actor system, wait for termination, > exit the jvm). > > But even though both nodes see Node B leaving through gossip convergence > at 22:16:58, Node A keeps the left Node B in the PubSub (you see this with > the RECEIVED PONG(nr, ping sender, pong responder)). > > Why does PubSub not remove actors from exited nodes, but only actors from > removed nodes (remove of Node B on Node A happens 4 seconds after exiting > of Node B). In my case I only do not loose messages, because Node B is > waiting another 10 seconds after it sees its one remove (which again, is 4 > seconds earlier than Node A sees Node B removed). > > Thanks in advance. > Christian > > > Output Node A > > [info] Thu Jun 23 22:16:23 CEST 2016 RECEIVE Pong(0,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER MemberUp(Member(address = > akka.tcp://airfocus@127.0.0.1:2551, status = Up)) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER LeaderChanged(Some(akka.tcp:// > airfocus@127.0.0.1:2551)) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER > RoleLeaderChanged(api,Some(akka.tcp://airfocus@127.0.0.1:2551)) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER > SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2551)) > [info] Thu Jun 23 22:16:24 CEST 2016 RECEIVE Pong(1,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:25 CEST 2016 RECEIVE Pong(2,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:26 CEST 2016 RECEIVE Pong(3,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:28 CEST 2016 RECEIVE Pong(4,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:29 CEST 2016 RECEIVE Pong(5,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:30 CEST 2016 RECEIVE Pong(6,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:31 CEST 2016 RECEIVE Pong(7,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:32 CEST 2016 RECEIVE Pong(8,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:33 CEST 2016 RECEIVE Pong(9,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:34 CEST 2016 RECEIVE Pong(10,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:35 CEST 2016 RECEIVE Pong(11,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:36 CEST 2016 RECEIVE Pong(12,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:37 CEST 2016 RECEIVE Pong(13,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:38 CEST 2016 RECEIVE Pong(14,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:39 CEST 2016 RECEIVE Pong(15,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:40 CEST 2016 RECEIVE Pong(16,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:41 CEST 2016 RECEIVE Pong(17,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:42 CEST 2016 RECEIVE Pong(18,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:43 CEST 2016 RECEIVE Pong(19,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:44 CEST 2016 RECEIVE Pong(20,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:45 CEST 2016 RECEIVE Pong(21,akka.tcp:// > airfocu
[akka-user] Re: DistributedPubSub keeps sending to exited nodes
Have chatted with @ktoso about this and seems, that my thoughts are not totally wrong. Will create a ticket on GitHub for that. Am Donnerstag, 23. Juni 2016 22:27:45 UTC+2 schrieb Christian Hoffmeister: > > Hi, > > I have a sample application where two nodes join an Akka cluster. On both > nodes there is an actor running registered to DistributedPubSubMediator and > both are sending a Ping once per second (that hits either itself or the > other node). > > Then I gracefully shut down Node B (leave cluster, wait for MemberRemoved > event, wait some seconds, terminate actor system, wait for termination, > exit the jvm). > > But even though both nodes see Node B leaving through gossip convergence > at 22:16:58, Node A keeps the left Node B in the PubSub (you see this with > the RECEIVED PONG(nr, ping sender, pong responder)). > > Why does PubSub not remove actors from exited nodes, but only actors from > removed nodes (remove of Node B on Node A happens 4 seconds after exiting > of Node B). In my case I only do not loose messages, because Node B is > waiting another 10 seconds after it sees its one remove (which again, is 4 > seconds earlier than Node A sees Node B removed). > > Thanks in advance. > Christian > > > Output Node A > > [info] Thu Jun 23 22:16:23 CEST 2016 RECEIVE Pong(0,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER MemberUp(Member(address = > akka.tcp://airfocus@127.0.0.1:2551, status = Up)) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER LeaderChanged(Some(akka.tcp:// > airfocus@127.0.0.1:2551)) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER > RoleLeaderChanged(api,Some(akka.tcp://airfocus@127.0.0.1:2551)) > [info] Thu Jun 23 22:16:23 CEST 2016 CLUSTER > SeenChanged(true,Set(akka.tcp://airfocus@127.0.0.1:2551)) > [info] Thu Jun 23 22:16:24 CEST 2016 RECEIVE Pong(1,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:25 CEST 2016 RECEIVE Pong(2,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:26 CEST 2016 RECEIVE Pong(3,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:28 CEST 2016 RECEIVE Pong(4,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:29 CEST 2016 RECEIVE Pong(5,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:30 CEST 2016 RECEIVE Pong(6,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:31 CEST 2016 RECEIVE Pong(7,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:32 CEST 2016 RECEIVE Pong(8,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:33 CEST 2016 RECEIVE Pong(9,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:34 CEST 2016 RECEIVE Pong(10,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:35 CEST 2016 RECEIVE Pong(11,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:36 CEST 2016 RECEIVE Pong(12,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:37 CEST 2016 RECEIVE Pong(13,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:38 CEST 2016 RECEIVE Pong(14,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:39 CEST 2016 RECEIVE Pong(15,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:40 CEST 2016 RECEIVE Pong(16,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:41 CEST 2016 RECEIVE Pong(17,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:42 CEST 2016 RECEIVE Pong(18,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:43 CEST 2016 RECEIVE Pong(19,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:44 CEST 2016 RECEIVE Pong(20,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:45 CEST 2016 RECEIVE Pong(21,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:46 CEST 2016 RECEIVE Pong(22,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:47 CEST 2016 RECEIVE Pong(23,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:48 CEST 2016 RECEIVE Pong(24,akka.tcp:// > airfocus@127.0.0.1:2551,akka.tcp://airfocus@127.0.0.1:2551) > [info] Thu Jun 23 22:16:49 CEST 2016 CLUSTER MemberJoined(Member(add
