Indeed, I thought something odd could be happening to your cluster, but it seems it's working fine but the request is taking too long to complete.
I noticed from your cfstats the read count was about 10 in the first CF and in
the second one it was about 1000... Would you be doing much more reads in the
second one? If the load is higher, it could justify the timeout.
Do both CFs have the same data model? Are you running exactly the same queries?
Best regards,
Marcelo.
From: user@cassandra.apache.org
Subject: Re: Cassandra Read Timeout
Hello
TP STATS Before Request:
Pool Name Active Pending Completed Blocked All
time blocked
ReadStage 0 0 7592835 0
0
RequestResponseStage 0 0 0 0
0
MutationStage 0 0 215980736 0
0
ReadRepairStage 0 0 0 0
0
ReplicateOnWriteStage 0 0 0 0
0
GossipStage 0 0 0 0
0
AntiEntropyStage 0 0 0 0
0
MigrationStage 0 0 28 0
0
MemoryMeter 0 0 474 0
0
MemtablePostFlusher 0 0 32845 0
0
FlushWriter 0 0 4013 0
2239
MiscStage 0 0 0 0
0
PendingRangeCalculator 0 0 1 0
0
commitlog_archiver 0 0 0 0
0
InternalResponseStage 0 0 0 0
0
HintedHandoff 0 0 0 0
0
Message type Dropped
RANGE_SLICE 0
READ_REPAIR 0
BINARY 0
READ 0
MUTATION 0
_TRACE 0
REQUEST_RESPONSE 0
COUNTER_MUTATION 0
TP STATS After Request:
Pool Name Active Pending Completed Blocked All
time blocked
ReadStage 1 1 7592942 0
0
RequestResponseStage 0 0 0 0
0
MutationStage 0 0 215983339 0
0
ReadRepairStage 0 0 0 0
0
ReplicateOnWriteStage 0 0 0 0
0
GossipStage 0 0 0 0
0
AntiEntropyStage 0 0 0 0
0
MigrationStage 0 0 28 0
0
MemoryMeter 0 0 474 0
0
MemtablePostFlusher 0 0 32845 0
0
FlushWriter 0 0 4013 0
2239
MiscStage 0 0 0 0
0
PendingRangeCalculator 0 0 1 0
0
commitlog_archiver 0 0 0 0
0
InternalResponseStage 0 0 0 0
0
HintedHandoff 0 0 0 0
0
Message type Dropped
RANGE_SLICE 0
READ_REPAIR 0
BINARY 0
READ 0
MUTATION 0
_TRACE 0
REQUEST_RESPONSE 0
COUNTER_MUTATION 0
The only items changed are : ReadStage increased by 107 + 1 Active/Pending and
MutationStage which changed by 2603.
Please note that system is writing all the time in batches ( each second 2
servers write one batch each ) so i dont see anything special in this numbers.
Best regards
Yulian Oifa
