First a thank you to people answering questions.
Against first the environment
Akka 2.4.11
Scala 2.11.8
Cassandra 3.9
I've been chasing an issue for days where one of my persistent actors
appears to stop processing message to the point where the JVM memory grows
to 8GB before it gets an out of memory crash. Upon analyzing the dump I'm
finding the internal stash is overflowing with billions of messages queued.
I've done several 'jcmd <pid> Thread.print' commands to try and see if the
threads are blocking in my code. The dumps are revealing nothing and I
can't find a single one that's actually executing my code. I examined all
the akka.actor.default-dispatcher threads which appear to be in a waiting
state:
"manhattan-akka.actor.default-dispatcher-43" #89 prio=5 os_prio=0
tid=0x00007fa4f0007800 nid=0x97 waiting on condition [0x00007fa5a435b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000005c0fbdf80> (a
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
at scala.concurrent.forkjoin.ForkJoinPool.scan(ForkJoinPool.java:2075)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
And the Cassandra plug-in threads are also waiting:
"manhattan-cassandra-plugin-default-dispatcher-49" #95 prio=5 os_prio=0
tid=0x00007fa504049000 nid=0x9d waiting on condition [0x00007fa52f5fa000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000005c1b65640> (a
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
at scala.concurrent.forkjoin.ForkJoinPool.scan(ForkJoinPool.java:2075)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Trying to figure out if this a bug where a journal write is never returning
which is all I can think of at this time. I've now set a limit on the
internal stash but that will only prevent the OOM exception but my actor
will still stop processing data.
Any pearls of wisdom are appreciated.
-Richard
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
