TimedOutException means the cluster could not perform the request in rpc_timeout time. The client should retry as the problem may be transitory.
In this case read performance may have slowed down due to the number of sstables 286. It hard to tell without knowing what the workload is. Aaron On 12 Apr 2011, at 09:56, mcasandra wrote: > I see this occurring often when all cassandra nodes all of a sudden show CPU > spike. All reads fail for about 2 mts. GC.log and system.log doesn't reveal > much. > > Only think I notice is that when I restart nodes there are tons of files > that gets deleted. cfstats from one of the nodes looks like this: > > nodetool -h `hostname` tpstats > Pool Name Active Pending Completed > ReadStage 27 27 21491 > RequestResponseStage 0 0 201641 > MutationStage 0 0 236513 > ReadRepairStage 0 0 7222 > GossipStage 0 0 31498 > AntiEntropyStage 0 0 0 > MigrationStage 0 0 0 > MemtablePostFlusher 0 0 324 > StreamStage 0 0 0 > FlushWriter 0 0 324 > FILEUTILS-DELETE-POOL 0 0 1220 > MiscStage 0 0 0 > FlushSorter 0 0 0 > InternalResponseStage 0 0 0 > HintedHandoff 1 3 9 > > -- > > ---------------- > Keyspace: StressKeyspace > Read Count: 21957 > Read Latency: 46.91765058978913 ms. > Write Count: 222104 > Write Latency: 0.008302124230090408 ms. > Pending Tasks: 0 > Column Family: StressStandard > SSTable count: 286 > Space used (live): 377916657941 > Space used (total): 377916657941 > Memtable Columns Count: 362 > Memtable Data Size: 164403613 > Memtable Switch Count: 326 > Read Count: 21958 > Read Latency: 631.464 ms. > Write Count: 222104 > Write Latency: 0.007 ms. > Pending Tasks: 0 > Key cache capacity: 1000000 > Key cache size: 22007 > Key cache hit rate: 0.002453626459907744 > Row cache: disabled > Compacted row minimum size: 87 > Compacted row maximum size: 5839588 > Compacted row mean size: 552698 > > ---------------- > > > -- > View this message in context: > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Timeout-during-stress-test-tp6262430p6263087.html > Sent from the cassandra-u...@incubator.apache.org mailing list archive at > Nabble.com.
