Indeed, I thought something odd could be happening to your cluster, but it seems it's working fine but the request is taking too long to complete.
I noticed from your cfstats the read count was about 10 in the first CF and in the second one it was about 1000... Would you be doing much more reads in the second one? If the load is higher, it could justify the timeout. Do both CFs have the same data model? Are you running exactly the same queries? Best regards, Marcelo. From: user@cassandra.apache.org Subject: Re: Cassandra Read Timeout Hello TP STATS Before Request: Pool Name Active Pending Completed Blocked All time blocked ReadStage 0 0 7592835 0 0 RequestResponseStage 0 0 0 0 0 MutationStage 0 0 215980736 0 0 ReadRepairStage 0 0 0 0 0 ReplicateOnWriteStage 0 0 0 0 0 GossipStage 0 0 0 0 0 AntiEntropyStage 0 0 0 0 0 MigrationStage 0 0 28 0 0 MemoryMeter 0 0 474 0 0 MemtablePostFlusher 0 0 32845 0 0 FlushWriter 0 0 4013 0 2239 MiscStage 0 0 0 0 0 PendingRangeCalculator 0 0 1 0 0 commitlog_archiver 0 0 0 0 0 InternalResponseStage 0 0 0 0 0 HintedHandoff 0 0 0 0 0 Message type Dropped RANGE_SLICE 0 READ_REPAIR 0 BINARY 0 READ 0 MUTATION 0 _TRACE 0 REQUEST_RESPONSE 0 COUNTER_MUTATION 0 TP STATS After Request: Pool Name Active Pending Completed Blocked All time blocked ReadStage 1 1 7592942 0 0 RequestResponseStage 0 0 0 0 0 MutationStage 0 0 215983339 0 0 ReadRepairStage 0 0 0 0 0 ReplicateOnWriteStage 0 0 0 0 0 GossipStage 0 0 0 0 0 AntiEntropyStage 0 0 0 0 0 MigrationStage 0 0 28 0 0 MemoryMeter 0 0 474 0 0 MemtablePostFlusher 0 0 32845 0 0 FlushWriter 0 0 4013 0 2239 MiscStage 0 0 0 0 0 PendingRangeCalculator 0 0 1 0 0 commitlog_archiver 0 0 0 0 0 InternalResponseStage 0 0 0 0 0 HintedHandoff 0 0 0 0 0 Message type Dropped RANGE_SLICE 0 READ_REPAIR 0 BINARY 0 READ 0 MUTATION 0 _TRACE 0 REQUEST_RESPONSE 0 COUNTER_MUTATION 0 The only items changed are : ReadStage increased by 107 + 1 Active/Pending and MutationStage which changed by 2603. Please note that system is writing all the time in batches ( each second 2 servers write one batch each ) so i dont see anything special in this numbers. Best regards Yulian Oifa