I use version 2.2.3. I can see the failing node being flagged as unreachable and then as quarantined by the surviving nodes, I think that means it is correctly downed? I have akka.loglevel and akka,stdout-loglevel at DEBUG, but it is not very clear what to look for. For example, it is not clear to me how to see if the Send() that suffers a timeout actually leaves the client at all, or if it arrives at any receptionist, or if that receptionist then succeeds in contacting a destination actor. How do I diagnose where it fails?
T. On Wednesday, February 5, 2014 9:40:41 PM UTC+1, Björn Antonsson wrote: > > Hi, > > On 5 February 2014 at 16:42:48, Tycho Lamerigts > (tycho.l...@gmail.com<javascript:>) > wrote: > > I have a client that fires many requests to my cluster using > ClusterClient's Send(), one request at a time, with a 1 second timeout > waiting for a response. While all the cluster nodes are up, the requests > are correctly (randomly) distributed across the nodes and promptly receive > a response, no timeouts. If I kill a cluster node, then I expect one or two > of the requests to timeout because they end up being sent to the now-dead > node. After that, I expect ClusterClient to realize that the node has died > and I expect to no longer get timeouts (the workload can easily be handled > by the remaining nodes). Sometimes it works. Unfortunately, more often than > not it doesn't work and I continue getting timed out requests until I > restart every node in the cluster and the client. > > Any idea what causes this behavior? > > > Which version of akka are you using? Is the failing node corrcetly downed? > Have you enabled any debug logging to diagnose the behavior? > > > B/ > > > My ClusterClient is initialized with two receptionist addresses. My > cluster actually has more than 2 nodes and each node has a receptionist > with a registered destination actor. I tried playing with > contrib.cluster.receptionist.number-of-contacts but it did not seem to make > any difference. > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://akka.io/faq/ > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+...@googlegroups.com <javascript:>. > To post to this group, send email to akka...@googlegroups.com<javascript:> > . > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- > Björn Antonsson > Typesafe <http://typesafe.com/> – Reactive Apps on the JVM > twitter: @bantonsson <http://twitter.com/#!/bantonsson> > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/groups/opt_out.
