Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-21 Thread Guido Medina
I have a supervisor actor per node which is already subscribed to cluster events, I avoided using ask when the actor system is going down anyway and it is driven by the JVM shutdown hook, so instead I came up with this hack, maybe a bit ugly but it works: For supervisor actor which once it recei

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-21 Thread Akka Team
Just create a terminator actor, then ask it from the shutdown hook. The actor then initiates the leaving process from the cluster, subscribes for the relevant event and once done, replies to the ask. From the caller side it is just Await.result((shutdownActor ? PleaseLeave).andThen(system.terminat

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-21 Thread Guido Medina
You could if system.terminate() could do that for you as part of the cluster extension then I could simply Await for Future to be ready. On Monday, March 21, 2016 at 9:36:43 AM UTC, Akka Team wrote: > > But then you cannot expect a graceful removal from the cluster. > > -Endre > > On Fri, Mar 1

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-21 Thread Akka Team
But then you cannot expect a graceful removal from the cluster. -Endre On Fri, Mar 18, 2016 at 6:47 PM, Guido Medina wrote: > Won't happen, the micro-services are being sent Linux SIGTERM, which is > why I'm hooking on the JVM shutdown. > > On Friday, March 18, 2016 at 5:26:01 PM UTC, Konrad Ma

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Guido Medina
How long is enough time? I tried sleep 1 sec between leave and terminate calls and it didn't work. On Friday, March 18, 2016 at 5:13:50 PM UTC, Konrad Malawski wrote: > > Also, please don't do that: >cluster.leave(cluster.selfAddress()); > system.terminate(); > It is wrong. > > Yo

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Konrad Malawski
Also, please don't do that: cluster.leave(cluster.selfAddress()); system.terminate(); It is wrong. You need to give the cluster gossip enough time to complete the leaving "nicely". Proper graceful shutdown means you need to wait for _your own_ address to come back as MemberRemoved

[akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Guido Medina
Hi Benjamin, I also rely on cluster events and AFAIK you can expect (and trust) *MemberUp* and *MemberRemoved*, these IMHO are the only two consistent states you can trust. In other words, I register some actors only when their nodes reach *MemberUp* and unregister only when their nodes reach *

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Patrik Nordwall
Hi Ben There are some subtle exceptions to the convergence rule. Unreachable members with status Down or Exiting are not counted. /Patrik On Thu, Mar 17, 2016 at 11:25 PM, Guido Medina wrote: > In fact in the other nodes you get the following: > > INFO 22:10:20,383 ster) - Cluster Node [akka.

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Guido Medina
Won't happen, the micro-services are being sent Linux SIGTERM, which is why I'm hooking on the JVM shutdown. On Friday, March 18, 2016 at 5:26:01 PM UTC, Konrad Malawski wrote: > > I precisely explained for what event you need to wait :-) > > Proper graceful shutdown means you need to wait for *_

[akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Benjamin Black
Hi Guido, I think in your case you are shutting down before the node has communicated to the leader that it wants to leave. I wait to get the MemberExited message before shutting down the node. Maybe I should wait for the MemberRemoved? Either way the ultimate aim is to not have the unreachable

[akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Guido Medina
I just tried this: final Cluster cluster = Cluster.get(system); cluster.leave(cluster.selfAddress()); try { Thread.sleep(1000); } catch (InterruptedException ignored) { } system.terminate(); and it didn't work, still the dead node has to timeout in the others, in other words I don't see any clu

[akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Guido Medina
In fact in the other nodes you get the following: INFO 22:10:20,383 ster) - Cluster Node [akka.tcp://DevCluster@127.0.0.1:2552] - Leader is moving node [akka.tcp://DevCluster@127.0.0.1:38845] to [Exiting] INFO 22:10:20,384 orRef - Message [akka.cluster.GossipEnvelope] from Actor[ akka://DevClus

[akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Guido Medina
As for cluster.leave(cluster.selfAddress) my micro-services use the following to leave: Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { final Cluster cluster = Cluster.get(system); cluster.leave(cluster.selfAddress()); syst

Re: [akka-user] Re: Clarification on unreachable nodes in cluster

2016-03-19 Thread Konrad Malawski
I precisely explained for what event you need to wait :-) Proper graceful shutdown means you need to wait for _your own_ address to come back as MemberRemoved event, and THEN you can shut down the actor system and JVM. --  Cheers, Konrad 'ktoso’ Malawski Akka @ Lightbend On 18 March 2016 at 18: