Was there a github issue created for this? I am seeing something that looks very similar and I don't want to create a duplicate ticket if one already exists.
On Wednesday, June 10, 2015 at 1:27:55 AM UTC-7, Patrik Nordwall wrote: > > We need logs (debug level) and description of the scenario. Perhaps it is > best that you create a github issue and we can discuss over there. > > /Patrik > > tis 9 jun 2015 kl. 23:58 skrev Brandon Arp <brand...@gmail.com > <javascript:>>: > >> I am seeing this as well with Akka 2.3.10. >> >> >> On Tuesday, June 9, 2015 at 2:27:18 AM UTC-7, Anders Båtstrand wrote: >> >>> Is there any news about this? I have experienced the same, using Akka >>> 2.3.10 and Cassandra (latest version of the plugin). >>> >>> Best regards, >>> >>> Anders >>> >>> tirsdag 11. november 2014 13.34.03 UTC+1 skrev Richard Bowker følgende: >>> >>> Hi Patrik, I have managed to repro it twice again. We have typesafe >>> support so I will get in touch with them to discuss how best to send the >>> repro setup, as it's not a simple attachment! >>> >>> thanks >>> >>> Rich >>> >>> On Monday, November 10, 2014 4:35:24 PM UTC, Patrik Nordwall wrote: >>> >>> >>> >>> On Mon, Nov 10, 2014 at 5:18 PM, Richard Bowker < >>> mechajoh...@googlemail.com> wrote: >>> >>> Hi Patrik, unfortunately not. In fact its only happened once to me so >>> far so may be a difficult one to reproduce. >>> >>> I will of course get back to you if I can find a trigger. >>> >>> >>> Thanks. That is problematic with these bugs. I don't know much about >>> Cassandra. Is it possible to export the data from cassandra if this happens >>> again so it could be analyzed (replayed) by us? Given that you don't have >>> any sensitive information in it. >>> >>> /Patrik >>> >>> >>> >>> Tackar! >>> >>> On Monday, November 10, 2014 4:04:18 PM UTC, Patrik Nordwall wrote: >>> >>> Hi Richard, >>> >>> That is not good. We have seen similar issue a few times and tracked it >>> down to bugs in the journal implementations. It will happen when events are >>> replayed in the wrong order. >>> >>> Is there a way we can reproduce this? >>> >>> Regards, >>> Patrik >>> >>> On Mon, Nov 10, 2014 at 2:42 PM, Richard Bowker <mechajoh...@googlemail. >>> com> wrote: >>> >>> I have had seen a similar problem when restarting nodes in a cluster >>> using sharding. >>> >>> after restarting, the node with the shard coordinator went into an >>> infinite error loop. >>> >>> I was using akka 2.3.6 and "com.github.krasserm" %% >>> "akka-persistence-cassandra" % "0.3.4" as the journal/persistence store. >>> >>> section of the error log below, I didn not know what to do to recover >>> this other than just manually delete all the akka keystores from the >>> database which obviously isn't ideal! >>> >>> any thoughts? >>> >>> thanks >>> >>> [ERROR] [11/10/2014 13:16:21.969] >>> [ClusterSystem-akka.actor.default-dispatcher-17] >>> [akka://ClusterSystem/user/sharding/PollServiceCoordinator/ >>> singleton/coordinator] >>> >>> requirement failed: Region Actor[akka.tcp://ClusterSystem >>> @172.31.18.169:2552/user/sharding/PollService#546005322] not >>> registered: State(Map(test47 -> Actor >>> >>> [akka://ClusterSystem/user/sharding/PollService#1625036981], test6 -> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test30 -> Actor >>> >>> [akka://ClusterSystem/user/sharding/PollService#1625036981], test42 -> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test29 -> >>> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test18 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], >>> >>> test14 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test36 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test25 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test28 -> >>> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test43 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], >>> >>> test32 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test20 -> Actor >>> >>> [akka://ClusterSystem/user/sharding/PollService#1625036981], test15 -> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test33 -> >>> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test22 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test0 >>> >>> -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test44 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], >>> >>> test11 -> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test41 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test37 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test16 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test9 -> >>> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test23 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test34 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test45 -> >>> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test38 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test8 >>> >>> -> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test19 -> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test35 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test5 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test24 -> >>> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], test2 >>> -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828] >>> <http://ClusterSystem@172.31.21.9:2552/user/sharding/PollService#1716980828%5D> >>> ),Map >>> >>> (Actor[akka://ClusterSystem/user/sharding/PollService#1625036981] -> >>> Vector(test6, test18, test28, test20, test33, test44, test16, test34, >>> test8, test24, test5, test11, >>> >>> test19, test25, test30, test38, test47), Actor[akka.tcp://ClusterSystem >>> @172.31.21.9:2552/user/sharding/PollService#1716980828] -> >>> Vector(test42, test36, test32, test15, >>> >>> test0, test41, test23, test45, test35, test2, test9, test14, test22, >>> test29, test37, test43)),Set(Actor >>> >>> [akka.tcp://ClusterSystem@172.31.24.129:49813/user/sharding/ >>> PollService#1785638107] >>> <http://ClusterSystem@172.31.24.129:49813/user/sharding/PollService#1785638107%5D> >>> )) >>> java.lang.IllegalArgumentException: requirement failed: Region >>> Actor[akka.tcp://ClusterSystem@172.31.18.169:2552/user/ >>> sharding/PollService#546005322] not registered: >>> >>> State(Map(test47 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test6 -> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test30 >>> >>> -> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test42 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], >>> >>> test29 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test18 -> Actor >>> >>> [akka://ClusterSystem/user/sharding/PollService#1625036981], test14 -> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test36 -> >>> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test25 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], >>> >>> test28 -> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test43 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test32 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test20 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test15 -> >>> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test33 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], >>> >>> test22 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test0 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828], test44 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], test11 -> >>> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test41 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], >>> >>> test37 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test16 -> Actor >>> >>> [akka://ClusterSystem/user/sharding/PollService#1625036981], test9 -> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test23 -> >>> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test34 -> Actor[akka://ClusterSystem/ >>> user/sharding/PollService#1625036981], >>> >>> test45 -> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test38 -> Actor >>> >>> [akka://ClusterSystem/user/sharding/PollService#1625036981], test8 -> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test19 -> Actor >>> >>> [akka://ClusterSystem/user/sharding/PollService#1625036981], test35 -> >>> Actor[akka.tcp://ClusterSystem@172.31.21.9:2552/user/ >>> sharding/PollService#1716980828], test5 -> >>> >>> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test24 -> Actor[akka://ClusterSystem/user/sharding/PollService#1625036981], >>> test2 -> Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828] >>> <http://ClusterSystem@172.31.21.9:2552/user/sharding/PollService#1716980828%5D> >>> ),Map(Actor[akka://ClusterSystem/user/sharding/PollService#1625036981] >>> -> Vector(test6, >>> >>> test18, test28, test20, test33, test44, test16, test34, test8, test24, >>> test5, test11, test19, test25, test30, test38, test47), Actor >>> >>> [akka.tcp://ClusterSystem@172.31.21.9:2552/user/sharding/ >>> PollService#1716980828] -> Vector(test42, test36, test32, test15, >>> test0, test41, test23, test45, test35, test2, >>> >>> test9, test14, test22, test29, test37, test43)),Set(Actor[akka.tcp:// >>> ClusterSystem@172.31.24.129:49813/user/sharding/PollService#1785638107] >>> <http://ClusterSystem@172.31.24.129:49813/user/sharding/PollService#1785638107%5D> >>> )) >>> at scala.Predef$.require(Predef.scala:219) >>> at akka.contrib.pattern.ShardCoordinator$Internal$ >>> State.updated(ClusterSharding.scala:1115) >>> at akka.contrib.pattern.ShardCoordinator$$anonfun$ >>> receiveRecover$1.applyOrElse(ClusterSharding.scala:1236) >>> at scala.runtime.AbstractPartialFunction.apply( >>> AbstractPartialFunction.scala:36) >>> at akka.persistence.Eventsourced$$anonfun$akka$persistence$ >>> Eventsourced$$recoveryBehavior$1.applyOrElse(Eventsourced.scala:168) >>> at akka.persistence.Recovery$class.runReceive(Recovery.scala:48) >>> at akka.contrib.pattern.ShardCoordinator.runReceive( >>> ClusterSharding.scala:1192) >>> at akka.persistence.Recovery$State$$anonfun$ >>> processPersistent$1.apply(Recovery.scala:33) >>> at akka.persistence.Recovery$State$$anonfun$ >>> processPersistent$1.apply(Recovery.scala:33) >>> at akka.persistence.Recovery$class.withCurrentPersistent( >>> Recovery.scala:185) >>> at akka.contrib.pattern.ShardCoordinator.withCurrentPersistent( >>> ClusterSharding.scala:1192) >>> at akka.persistence.Recovery$State$class.processPersistent( >>> Recovery.scala:33) >>> at akka.persistence.Recovery$$anon$1.processPersistent( >>> Recovery.scala:104) >>> at akka.persistence.Recovery$$anon$1.aroundReceive(Recovery. >>> scala:110) >>> at akka.persistence.Recovery$class.aroundReceive(Recovery. >>> scala:265) >>> at akka.contrib.pattern.ShardCoordinator.akka$ >>> persistence$Eventsourced$$super$aroundReceive( >>> ClusterSharding.scala:1192) >>> at akka.persistence.Eventsourced$$anon$1.aroundReceive( >>> Eventsourced.scala:35) >>> at akka.persistence.Eventsourced$class.aroundReceive( >>> Eventsourced.scala:369) >>> at akka.contrib.pattern.ShardCoordinator. >>> aroundReceive(ClusterSharding.scala:1192) >>> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) >>> at akka.actor.ActorCell.invoke(ActorCell.scala:487) >>> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) >>> at akka.dispatch.Mailbox.run(Mailbox.scala:220) >>> at akka.dispatch.ForkJoinExecutorConfigurator$ >>> AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) >>> at scala.concurrent.forkjoin.ForkJoinTask.doExec( >>> ForkJoinTask.java:260) >>> at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue. >>> runTask(ForkJoinPool.java:1339) >>> at scala.concurrent.forkjoin.ForkJoinPool.runWorker( >>> ForkJoinPool.java:1979) >>> at scala.concurrent.forkjoin.ForkJoinWorkerThread.run( >>> ForkJoinWorkerThread.java:107) >>> >>> [ERROR] [11/10/2014 13:16:21.978] >>> [ClusterSystem-akka.actor.default-dispatcher-15] >>> [akka://ClusterSystem/user/sharding/PollServiceCoordinator/ >>> singleton/coordinator] >>> >>> requirement failed: Region Actor[akka.tcp:// >>> >>> ... >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+...@googlegroups.com <javascript:>. >> To post to this group, send email to akka...@googlegroups.com >> <javascript:>. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > -- > /Patrik -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
