Lots of replicate on write tasks pending, want to investigate

2013-07-03 Thread Andrew Bialecki
In one of our load tests, we're incrementing a single counter column as well as appending columns to a single row (essentially a timeline). You can think of it as counting the instances of an event and then keeping a timeline of those events. The ratio is of increments to "appends" is 1:1. When we

Re: Lots of replicate on write tasks pending, want to investigate

2013-07-03 Thread Robert Coli
On Wed, Jul 3, 2013 at 9:59 AM, Andrew Bialecki wrote: > 2. I'm assuming in our case the cause is incrementing counters because > disk reads are part of the write path for counters and are not for > appending columns to a row. Does that logic make sense? > That's a pretty reasonable assumption if

Re: Lots of replicate on write tasks pending, want to investigate

2013-07-03 Thread Andrew Bialecki
Can someone remind me why replicate on write tasks might be related to the high disk I/O? My understanding is the replicate on write involves sending the update to other nodes, so it shouldn't involve any disk activity -- disk activity would be during the mutation/write phase. The write path (not

Re: Lots of replicate on write tasks pending, want to investigate

2013-07-03 Thread Sylvain Lebresne
> The write path (not replicate on write) for counters involves a read, > I'm afraid you got it wrong. The read done during counter writes *is* done by the replicate on write taks. Though really, the replicate on write taks are just one part of the counter write path (they are not "not the write p