Secondary indexes can generate a lot of random i/o. iostat -x can confirm if that's your problem.
On Thu, Mar 29, 2012 at 5:52 PM, Chris Hart <ch...@remilon.com> wrote: > Hi, > > I have the following cluster: > > 136112946768375385385349842972707284580 > <ip address> MountainViewRAC1 Up Normal 1.86 GB 20.00% 0 > <ip address> MountainViewRAC1 Up Normal 2.17 GB 33.33% > 56713727820156410577229101238628035242 > <ip address> MountainViewRAC1 Up Normal 2.41 GB 33.33% > 113427455640312821154458202477256070485 > <ip address> Rackspace RAC1 Up Normal 3.9 GB > 13.33% 136112946768375385385349842972707284580 > > The following query runs quickly on all nodes except 1 MountainView node: > > select * from Access_Log where row_loaded = 0 limit 1; > > There is a secondary index on row_loaded. The query usually doesn't complete > (but sometimes does) on the bad node and returns very quickly on all other > nodes. I've upping the rpc timeout to a full minute (rpc_timeout_in_ms: > 60000) in the yaml, but it still often doesn't complete in a minute. It > seems just as likely to complete and takes about the same amount of time > whether the limit is 1, 100 or 1000. > > > Thanks for any help, > Chris -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
