The log output looks sensible to me given that you are using auto downing. I read it as follows:-
1. Node 102 is genuinely not reachable from node 240. This may be due to network partition, or too much GC, IO, CPU, etc 2. Node 240's failure detector fails to receive sufficient heartbeats from 102 and marks it as unreachable and then auto-downs it 3. Node 102 recovers (e.g. network partition resolves itself) 4. Node 240 detects 102 as reachable again, but since it is marked down it is unable to rejoin the cluster In this scenario if you disabled auto-downing or configured it to be less aggressive, the 102 node could have successfully rejoined. Relevant quote from the docs: "unreachable is not a real member states but more of a flag in addition to > the state signaling that the cluster is unable to talk to this node" > Hope that helps Mark On Tuesday, 31 May 2016 09:53:48 UTC+1, [email protected] wrote: > > the log is follow: > 2016-05-31 07:40:54,053 | WARN | lt-dispatcher-16 | ClusterCoreDaemon > | 167 - com.typesafe.akka.slf4j - 2.3.10 | Cluster Node > [akka.tcp://[email protected]:2550] - Marking > node(s) as UNREACHABLE [Member(address = akka.tcp:// > [email protected]:2550, status = Up)] > 2016-05-31 07:41:08,785 | INFO | lt-dispatcher-14 | > kka://opendaylight-cluster-data) | 167 - com.typesafe.akka.slf4j - 2.3.10 | > Cluster Node [akka.tcp://[email protected]:2550] - > Leader is auto-downing unreachable node [akka.tcp:// > [email protected]:2550] > 2016-05-31 07:41:11,267 | INFO | lt-dispatcher-14 | > kka://opendaylight-cluster-data) | 167 - com.typesafe.akka.slf4j - 2.3.10 | > Cluster Node [akka.tcp://[email protected]:2550] - > Marking unreachable node [akka.tcp:// > [email protected]:2550] as [Down] > 2016-05-31 07:41:12,243 | INFO | lt-dispatcher-14 | > kka://opendaylight-cluster-data) | 167 - com.typesafe.akka.slf4j - 2.3.10 | > Cluster Node [akka.tcp://[email protected]:2550] - > Marking node(s) as REACHABLE [Member(address = akka.tcp:// > [email protected]:2550, status = Down)] > 2016-05-31 07:41:12,243 | INFO | lt-dispatcher-14 | > kka://opendaylight-cluster-data) | 167 - com.typesafe.akka.slf4j - 2.3.10 | > Cluster Node [akka.tcp://[email protected]:2550] - > Marking node(s) as REACHABLE [Member(address = akka.tcp:// > [email protected]:2550, status = Down)] > > And then any log, leader remove the node 192.168.23.102:2550 > > the Akka version is 2.3.10 > > thanks > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
