Re: Overwhelming a cluster with writes?

2010-04-06 Thread Ilya Maykov
I'm running the nodes with a JVM heap size of 6GB, and here are the related options from my storage-conf.xml. As mentioned in the first email, I left everything at the default value. I briefly googled around for Cassandra performance tuning etc but haven't found a definitive guide ... any help

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Ran Tavory
Do you see one of the disks used by cassandra filled up when a node crashes? On Tue, Apr 6, 2010 at 9:39 AM, Ilya Maykov ivmay...@gmail.com wrote: I'm running the nodes with a JVM heap size of 6GB, and here are the related options from my storage-conf.xml. As mentioned in the first email, I

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Ilya Maykov
No, the disks on all nodes have about 750GB free space. Also as mentioned in my follow-up email, writing with ConsistencyLevel.ALL makes the slowdowns / crashes go away. -- Ilya On Mon, Apr 5, 2010 at 11:46 PM, Ran Tavory ran...@gmail.com wrote: Do you see one of the disks used by cassandra

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Benjamin Black
You are blowing away the mostly saner JVM_OPTS running it that way. Edit cassandra.in.sh (or wherever config is on your system) to increase mx to 4G (not 6G, for now) and leave everything else untouched and do not specify JVM_OPTS on the command line. See if you get the same behavior. b On

Re: Memcached protocol?

2010-04-06 Thread Tatu Saloranta
On Mon, Apr 5, 2010 at 5:10 PM, Paul Prescod p...@ayogo.com wrote: On Mon, Apr 5, 2010 at 4:48 PM, Tatu Saloranta tsalora...@gmail.com wrote: ... I would think that there is also possibility of losing some increments, or perhaps getting duplicate increments? I believe that with vector

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Rob Coli
On 4/5/10 11:48 PM, Ilya Maykov wrote: No, the disks on all nodes have about 750GB free space. Also as mentioned in my follow-up email, writing with ConsistencyLevel.ALL makes the slowdowns / crashes go away. I am not sure if the above is consistent with the cause of #896, but the other

Re: Flush Commit Log

2010-04-06 Thread JKnight JKnight
Yes, no problem with my live Cassandra server. Thanks, Jonathan. On Mon, Apr 5, 2010 at 11:19 PM, Jonathan Ellis jbel...@gmail.com wrote: On Mon, Apr 5, 2010 at 9:11 PM, JKnight JKnight beukni...@gmail.com wrote: Thanks Jonathan, When I run nodeprobe flush with parameter -host is

How do vector clocks and conflicts work?

2010-04-06 Thread Paul Prescod
This may be the blind leading the blind... On Mon, Apr 5, 2010 at 11:54 PM, Tatu Saloranta tsalora...@gmail.comwrote: ... I think the key is that this is not automatic -- there is no general mechanism for aggregating distinct modifications. Point being that you could choose one amongst right

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Ilya Maykov
That does sound similar. It's possible that the difference I'm seeing between ConsistencyLevel.ZERO and ConsistencyLevel.ALL is simply due to the fact that using ALL slows down the writers enough that the GC can keep up. I could do a test with multiple clients writing at ALL in parallel tomorrow.

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Ilya Maykov
Right, I meant 4GB heap vs. the standard 1GB. And all other options in cassandra.in.sh at their defaults. Sorry I am a bit new to JVM tuning, and very new to Cassandra :) -- Ilya On Tue, Apr 6, 2010 at 12:16 AM, Benjamin Black b...@b3k.us wrote: I am specifically suggesting you NOT use a heap

Re: Memcached protocol?

2010-04-06 Thread gabriele renzi
On Tue, Apr 6, 2010 at 2:10 AM, Paul Prescod p...@ayogo.com wrote: On Mon, Apr 5, 2010 at 4:48 PM, Tatu Saloranta tsalora...@gmail.com wrote: ... I would think that there is also possibility of losing some increments, or perhaps getting duplicate increments? I believe that with vector

Re: how to store list data in Apache Cassndra ?

2010-04-06 Thread David Strauss
Another option is to use a SuperColumnFamily, but that extends the depth of all such values to be arrays. The name and age columns would therefore also need to be SuperColumns -- just with a single sub-column each. Like many things in Cassandra, the preferred storage method depends on your

i have one mistake in Cassandra.java when i build it

2010-04-06 Thread 叶江
hi: i want to take some experiments on cassandra by java, but when i write client,a mistake can not convert int to ConsistencyLevel appear, so how can i solve ? thanks very much.

Re: i have one mistake in Cassandra.java when i build it

2010-04-06 Thread Jonathan Ellis
This means you rebuilt the Thrift code with an old compiler. If you look in lib/ the thrift jar is tagged with the svn revision we built with. Thrift has frequent regressions, so using that same revision is the best way to avoid unpleasant surprises. On Tue, Apr 6, 2010 at 4:34 AM, 叶江

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Jonathan Ellis
On Tue, Apr 6, 2010 at 2:13 AM, Ilya Maykov ivmay...@gmail.com wrote: That does sound similar. It's possible that the difference I'm seeing between ConsistencyLevel.ZERO and ConsistencyLevel.ALL is simply due to the fact that using ALL slows down the writers enough that the GC can keep up.

Re: Memcached protocol?

2010-04-06 Thread Jonathan Ellis
On Mon, Apr 5, 2010 at 6:48 PM, Tatu Saloranta tsalora...@gmail.com wrote: I would think that there is also possibility of losing some increments, or perhaps getting duplicate increments? It is not just isolation but also correctness that is hard to maintain but correctness also. This can be

if cassandra isn't ideal for keep track of counts, how does digg count diggs?

2010-04-06 Thread S Ahmed
From what I read in another thread, Cassandra isn't used for isn't 'ideal' for keeping track of counts. For example, I would undertand this to mean keeping track of which stories were dugg. If this is true, how would a site like digg keep track of the 'dugg' counter? Also, I am assuming with

odd problem retrieving binary values using C++

2010-04-06 Thread Chris Beaumont
Hi all... I am having a pretty tough time retrieving binary values out of my DB... I am using cassandra 0.5.1 on Centos 5.4 with java 1.6.0-19 Here is the simple test I am trying to run in C++ /* snip initialization */ { transport-open(); ColumnPath new_col;

Re: if cassandra isn't ideal for keep track of counts, how does digg count diggs?

2010-04-06 Thread S Ahmed
Chris, When you so patch, does that mean for Cassandra or your own internal codebase? Sounds interesting thanks! On Tue, Apr 6, 2010 at 12:54 PM, Chris Goffinet goffi...@digg.com wrote: That's not true. We have been using the Zookeper work we posted on jira. That's what we are using

Re: if cassandra isn't ideal for keep track of counts, how does digg count diggs?

2010-04-06 Thread Chris Goffinet
http://issues.apache.org/jira/browse/CASSANDRA-704 http://issues.apache.org/jira/browse/CASSANDRA-721 We have our own internal codebase of Cassandra at Digg. But we are using those above patches until we have the vector clock work cleaned up, that patch will also goto jira. Most likely the

Re: how to store list data in Apache Cassndra ?

2010-04-06 Thread Tatu Saloranta
On Tue, Apr 6, 2010 at 8:06 AM, Shuge Lee shuge@gmail.com wrote:      'girls': pickle.dumps(['java', 'actionscript', 'python']) I think this is a really bad idea, I can't do any search if using Pickle. Just to be sure: are you thinking of traditional queries, lookups by values (find entries

Re: A question of 'referential integrity'...

2010-04-06 Thread Tatu Saloranta
On Tue, Apr 6, 2010 at 10:12 AM, Steve sjh_cassan...@shic.co.uk wrote: On 06/04/2010 15:26, Eric Evans wrote: ... I've read all about QUORUM, and it is generally useful, but as far as I can tell, it can't give me a transaction... Correct. Only individual operations are atomic, and ordering of

Re: Overwhelming a cluster with writes?

2010-04-06 Thread Tatu Saloranta
On Tue, Apr 6, 2010 at 8:17 AM, Jonathan Ellis jbel...@gmail.com wrote: On Tue, Apr 6, 2010 at 2:13 AM, Ilya Maykov ivmay...@gmail.com wrote: That does sound similar. It's possible that the difference I'm seeing between ConsistencyLevel.ZERO and ConsistencyLevel.ALL is simply due to the fact

Re: How do vector clocks and conflicts work?

2010-04-06 Thread Tatu Saloranta
On Tue, Apr 6, 2010 at 8:45 AM, Mike Malone m...@simplegeo.com wrote: As long as the conflict resolver knows that two writers each tried to increment, then it can increment twice. The conflict resolver must know about the semantics of increment or decrement or string append or binary patch or

Net::Cassandra::Easy deletion failed

2010-04-06 Thread Mike Gallamore
Seems to be internal to java/cassandra itself. I have some tests and I want to make sure that I have a clean slate each time I run the test. Clean as far as my code cares is that value is not defined. I'm running bin/cassandra -f with the default install/options. So at the beginning of my

Re: A question of 'referential integrity'...

2010-04-06 Thread Steve
On 06/04/2010 18:50, Benjamin Black wrote: I'm finding this exchange very confusing. What exactly about Cassandra 'looks absolutely ideal' to you for your project? The write performance, the symmetric, peer to peer architecture, etc? Reasons I like Cassandra for this project: *

Re: A question of 'referential integrity'...

2010-04-06 Thread Steve
On 06/04/2010 18:53, Tatu Saloranta wrote: I've read all about QUORUM, and it is generally useful, but as far as I can tell, it can't give me a transaction... Correct. Only individual operations are atomic, and ordering of insertions is not guaranteed. As I thought. I think there

Re: How do vector clocks and conflicts work?

2010-04-06 Thread gabriele renzi
On Tue, Apr 6, 2010 at 9:11 AM, Paul Prescod pres...@gmail.com wrote: This may be the blind leading the blind... On Mon, Apr 5, 2010 at 11:54 PM, Tatu Saloranta tsalora...@gmail.com wrote: ... I think the key is that this is not automatic -- there is no general mechanism for aggregating

problem with Net::Cassanda::Easy deleting columns

2010-04-06 Thread Mike Gallamore
Hello I tried to post this earlier but something seems to have gone wrong with sending the message. I have a test perl script that I'm using to test the behaviour of some of my existing code. It is important that the values start in a clean state at the beginning of the tests, as I'm

Re: Net::Cassandra::Easy deletion failed

2010-04-06 Thread Ted Zlatanov
On Tue, 06 Apr 2010 11:07:03 -0700 Mike Gallamore mike.e.gallam...@googlemail.com wrote: MG Seems to be internal to java/cassandra itself. MG I have some tests and I want to make sure that I have a clean slate MG each time I run the test. Clean as far as my code cares is that MG value is not

Re: Net::Cassandra::Easy deletion failed

2010-04-06 Thread Ted Zlatanov
On Tue, 06 Apr 2010 13:24:45 -0700 Mike Gallamore mike.e.gallam...@googlemail.com wrote: MG Thanks for the reply. The newest version of the module I see on CPAN MG is 0.08b. I actually had 0.07 installed and am using 0.6beta3 for MG cassandra. Is there somewhere else I should look for the 0.09

Re: How do vector clocks and conflicts work?

2010-04-06 Thread Mike Malone
On Tue, Apr 6, 2010 at 11:03 AM, Tatu Saloranta tsalora...@gmail.comwrote: On Tue, Apr 6, 2010 at 8:45 AM, Mike Malone m...@simplegeo.com wrote: As long as the conflict resolver knows that two writers each tried to increment, then it can increment twice. The conflict resolver must know

Re: Net::Cassandra::Easy deletion failed

2010-04-06 Thread Mike Gallamore
On 04/06/2010 01:36 PM, Ted Zlatanov wrote: On Tue, 06 Apr 2010 13:24:45 -0700 Mike Gallamoremike.e.gallam...@googlemail.com wrote: MG Thanks for the reply. The newest version of the module I see on CPAN MG is 0.08b. I actually had 0.07 installed and am using 0.6beta3 for MG cassandra. Is