Problem resolved. I was running two clusters (dev and prod) and somehow (the mystery remains) the dev cluster was interacting with this one box in the prod cluster. My advice to anyone else who sees this issue is to check the IP addresses of the "from" nodes.
On Wednesday, July 2, 2014 4:31:32 AM UTC-4, Patrik Nordwall wrote: > > > > > On Wed, Jul 2, 2014 at 12:21 AM, Benjamin Black <benbl...@gmail.com > <javascript:>> wrote: > >> I upgraded to Akka 2.3.4 (scala 2.10), but still seeing the same issue. >> When I log Cluster(system).selfUniqueAddress I get something >> like UniqueAddress(akka.tcp://streaming@172.17.102.128:7000,630715883), >> with the last number changing everytime I restart. >> > > Then it would be great if you could describe how to reproduce the problem. > Preferably using a minimal sample, such as the SimpleClusterListener in the > cluster > sample <https://typesafe.com/activator/template/akka-sample-cluster-scala> > . > > >> >> The gossip message error always has the same number -1482656725. For >> example, >> >> 18:12:27.472 INFO [streaming-akka.actor.default-dispatcher-15] >> Cluster(akka://streaming) - Cluster Node [akka.tcp:// >> streaming@172.17.102.128:7000] - Ignoring received gossip intended for >> someone else, from [akka.tcp://streaming@172.17.110.143:7000] to >> [UniqueAddress(akka.tcp://streaming@172.17.102.128:7000,-1482656725)] >> >> > It is not an error to see a few of these while joining (especially when > joining several nodes at the same time), but if this logging continues > (never ends) something is wrong. > > Negative uid should not be an issue. It is just a random integer that is > generated when the ActorSystem is started. The reason for using the uid is > to be able to differentiate new and old actor system on the same host:port > from each other when it is restarted. > > Regards, > Patrik > > >> >> >> On Tuesday, July 1, 2014 2:33:52 PM UTC-4, Patrik Nordwall wrote: >> >>> Please use latest version, i.e. 2.3.4 >>> >>> There you find Cluster(system).selfUniqueAddress that includes the uid. >>> >>> I will look into this in more detail tomorrow. >>> >>> /Patrik >>> >>> 1 jul 2014 kl. 19:57 skrev Benjamin Black <benbl...@gmail.com>: >>> >>> I have a cluster of 15 nodes/boxes. I start the nodes roughly at the >>> same time. One of the nodes is behaving oddly and continually logging >>> "Ignoring received gossip intended for someone else". However, the node >>> does seem to work for a while before being being dropped from the cluster. >>> Basically this one node seems to think it is someone else, whilst also >>> behaviouring as itself. The code and config is exactly the same on all 15 >>> nodes so I don't understand why I'm getting this issue on only one node. >>> Maybe this is a hardware issue? >>> >>> Some logging: >>> >>> 11:27:45.412 INFO [main] Remoting - Starting remoting >>> 11:27:45.638 INFO [main] Remoting - Remoting started; listening on >>> addresses :[akka.tcp://streaming@172.17.102.128:7000] >>> 11:27:45.660 INFO [main] Cluster(akka://streaming) - Cluster Node >>> [akka.tcp://streaming@172.17.102.128:7000] - Starting up... >>> 11:27:45.714 INFO [main] Cluster(akka://streaming) - Cluster Node >>> [akka.tcp://streaming@172.17.102.128:7000] - Registered cluster JMX >>> MBean [akka:type=Cluster] >>> 11:27:45.715 INFO [main] Cluster(akka://streaming) - Cluster Node >>> [akka.tcp://streaming@172.17.102.128:7000] - Started up successfully >>> 11:27:45.830 INFO [streaming-akka.actor.default-dispatcher-3] >>> a.a.LocalActorRef - Message >>> [akka.cluster.InternalClusterAction$InitJoinAck] >>> from Actor[akka.tcp://streaming@172.17.100.98:7000/system/ >>> cluster/core/daemon#1997515880] to Actor[akka://streaming/system/ >>> cluster/core/daemon/joinSeedNodeProcess-1#1132911] was not delivered. >>> [1] dead letters encountered. This logging can be turned off or adjusted >>> with configuration settings 'akka.log-dead-letters' and >>> 'akka.log-dead-letters-during-shutdown'. >>> 11:27:45.872 INFO [streaming-akka.actor.default-dispatcher-5] >>> Cluster(akka://streaming) - Cluster Node [akka.tcp://streaming@172.17. >>> 102.128:7000] - Welcome from [akka.tcp://streaming@172.17.102.125:7000] >>> 11:27:45.911 INFO [streaming-akka.actor.default-dispatcher-2] >>> Cluster(akka://streaming) - Cluster Node [akka.tcp://streaming@172.17. >>> 102.128:7000] - Ignoring received gossip intended for someone else, >>> from [akka.tcp://streaming@172.17.102.68:7000] to >>> [UniqueAddress(akka.tcp://streaming@172.17.102.128:7000,-1482656725)] >>> 11:27:45.943 INFO [streaming-akka.actor.default-dispatcher-16] >>> Cluster(akka://streaming) - Cluster Node [akka.tcp://streaming@172.17. >>> 102.128:7000] - Ignoring received gossip intended for someone else, >>> from [akka.tcp://streaming@172.17.102.70:7000] to >>> [UniqueAddress(akka.tcp://streaming@172.17.102.128:7000,-1482656725)] >>> 11:27:46.122 INFO [streaming-akka.actor.default-dispatcher-16] >>> Cluster(akka://streaming) - Cluster Node [akka.tcp://streaming@172.17. >>> 102.128:7000] - Ignoring received gossip intended for someone else, >>> from [akka.tcp://streaming@172.17.102.69:7000] to >>> [UniqueAddress(akka.tcp://streaming@172.17.102.128:7000,-1482656725)] >>> >>> Config: >>> >>> akka { >>> cluster { >>> seed-nodes = [ >>> "akka.tcp://streaming@172.17.102.125:7000" >>> "akka.tcp://streaming@172.17.100.98:7000" >>> ] >>> } >>> remote.netty.tcp.hostname = 172.17.102.128 >>> } >>> >>> I thought it was weird that the unique address in the gossip messages >>> referred to a negative number. I added log.info(s"my unique ID: >>> ${AddressUidExtension(actorSystem).addressUid}") to the confused node >>> (I hope this is the correct code) and it gave me the answer "1549799231", >>> whilst continuing to give "-1482656725" in the gossip messages. I'm >>> guessing the problem is that the gossip messages have a corrupted address, >>> which is why the confused node believes these messages are not for itself. >>> I'm using Akka 2.3.2. >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>> current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>> group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+...@googlegroups.com <javascript:>. >> To post to this group, send email to akka...@googlegroups.com >> <javascript:>. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > Patrik Nordwall > Typesafe <http://typesafe.com/> - Reactive apps on the JVM > Twitter: @patriknw > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
