FWIW, I'm seeing similar issues on a cluster. Three nodes, Cassandra 0.6.1, SUN JDK 1.6.0_b20. I will try to get some heap dumps to see what's building up.
I've seen this sort of issue in systems that make heavy use of java.util.concurrent queues/executors, e.g.: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6236036 That bug is long fixed, but it is an instance of how it can be harder to do nothing than something. -- Paul On May 26, 2010, at 11:32 PM, James Golick wrote: > We're seeing RAM usage continually climb until eventually, cassandra becomes > unresponsive. > > The JVM isn't OOM'ing. It has only committed 14/24GB of memory. So, I am > assuming that the memory usage is related to mmap'd IO. Fair assumption? > > I tried setting the IO mode to standard, but it seemed to be a little slower > and couldn't get the machine to come back online with adequate read > performance, so I set it back. I'll have to write a solid cache warming > script if I'm going to try that again. > > Any other ideas for what might be causing the issue? Is there something I > should monitor or look at next time it happens? > > Thanks
