I'm getting a Timeout Exception every now and again (currently every couple
of minutes or so).
Using revision 833288. Quorum set to ONE. My cassandra instance has been
running for two days and the data directory is around 16GB. I'm not sure
what the problem is, but let me know of any tests I can do to help reduce
the problem further. There are two variations on the exception, I have
pasted them both below.
ERROR [pool-1-thread-63] 2009-11-09 20:17:27,579 Cassandra.java (line
org.apache.cassandra.service.Cassandra$Processor) Internal error processing
get_slice
java.lang.RuntimeException: java.util.concurrent.TimeoutException: Operation
timed out - received only 0 responses from .
at
org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:103)
at
org.apache.cassandra.service.CassandraServer.getSlice(CassandraServer.java:177)
at
org.apache.cassandra.service.CassandraServer.multigetSliceInternal(CassandraServer.java:252)
at
org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:215)
at
org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:668)
at
org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:624)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.util.concurrent.TimeoutException: Operation timed out -
received only 0 responses from .
at
org.apache.cassandra.service.QuorumResponseHandler.get(QuorumResponseHandler.java:79)
at
org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:408)
at
org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:333)
at
org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:95)
... 9 more
ERROR [pool-1-thread-19] 2009-11-09 11:29:18,731 Cassandra.java (line
org.apache.cassandra.service.Cassandra$Processor) Internal error processing
get_slice
java.lang.RuntimeException: java.util.concurrent.TimeoutException: Operation
timed out - received only 1 responses from /10.121.217.5 .
at
org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:103)
at
org.apache.cassandra.service.CassandraServer.getSlice(CassandraServer.java:177)
at
org.apache.cassandra.service.CassandraServer.multigetSliceInternal(CassandraServer.java:252)
at
org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:215)
at
org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:668)
at
org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:624)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.util.concurrent.TimeoutException: Operation timed out -
received only 1 responses from /10.121.217.5 .
at
org.apache.cassandra.service.QuorumResponseHandler.get(QuorumResponseHandler.java:79)
at
org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:408)
at
org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:333)
at
org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:95)
... 9 more
Cheers,
Chris
