Re: do I need to add more nodes? minor compaction eat all IO

2011-07-26 Thread Jim Ancona
On Mon, Jul 25, 2011 at 6:41 PM, aaron morton wrote: > There are no hard and fast rules to add new nodes, but here are two > guidelines: > > 1) Single node load is getting too high, rule of thumb is 300GB is probably > too high. What is that rule of thumb based on? I would guess that working se

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-25 Thread aaron morton
There are no hard and fast rules to add new nodes, but here are two guidelines: 1) Single node load is getting too high, rule of thumb is 300GB is probably too high. 2) There are times when the cluster cannot keep up with throughout, for example the client is getting TimedOutExceptions or TPSta

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-25 Thread Yan Chunlu
as the wiki suggested: http://wiki.apache.org/cassandra/LargeDataSetConsiderations Adding nodes is a slow process if each node is responsible for a large amount of data. Plan for this; do not try to throw additional hardware at a cluster at the last minute. I really would like to know what's the

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-25 Thread Yan Chunlu
I am using normal SATA disk, actually I was worrying about whether it is okay if every time cassandra using all the io resources? further more when is the good time to add more nodes when I was just using normal SATA disk and with 100r/s it could reach 100 %util how large the data size it sho

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-24 Thread Jonathan Ellis
t; From: Jonathan Ellis [mailto:jbel...@gmail.com] > Sent: Saturday, July 23, 2011 5:05 PM > To: user@cassandra.apache.org > Subject: Re: do I need to add more nodes? minor compaction eat all IO > > On Sat, Jul 23, 2011 at 4:16 PM, Francois Richard wrote: >> My understanding is

RE: do I need to add more nodes? minor compaction eat all IO

2011-07-24 Thread Francois Richard
-Original Message- From: Jonathan Ellis [mailto:jbel...@gmail.com] Sent: Saturday, July 23, 2011 5:05 PM To: user@cassandra.apache.org Subject: Re: do I need to add more nodes? minor compaction eat all IO On Sat, Jul 23, 2011 at 4:16 PM, Francois Richard wrote: > My understanding is that dur

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-23 Thread Jonathan Ellis
On Sat, Jul 23, 2011 at 4:16 PM, Francois Richard wrote: > My understanding is that during compaction cassandra does a lot of non > sequential readsa then dumps the results with a big sequential write. Compaction reads and writes are both sequential, and 0.8 allows setting a MB/s to cap compacti

RE: do I need to add more nodes? minor compaction eat all IO

2011-07-23 Thread Francois Richard
From: Yan Chunlu [mailto:springri...@gmail.com] Sent: Saturday, July 23, 2011 9:16 AM To: cassandra-u...@incubator.apache.org Subject: do I need to add more nodes? minor compaction eat all IO I have three nodes and RF=3, every time it is do minor compaction, the cpu load(8 core) get to >30, and

do I need to add more nodes? minor compaction eat all IO

2011-07-23 Thread Yan Chunlu
I have three nodes and RF=3, every time it is do minor compaction, the cpu load(8 core) get to >30, and iostat -x 2 shows utils is 100%, is that means I need more nodes? the total data size is <60G thanks! --