Re: GC Storm

2010-07-18 Thread Schubert Zhang
Benjamin, It is not difficult to stack thousands of SSTables. In a heavy inserting (many client threads), the memtable flush (generate new sstable) is fren On Mon, Jun 14, 2010 at 2:03 AM, Benjamin Black b...@b3k.us wrote: On Sat, Jun 12, 2010 at 7:46 PM, Anty anty@gmail.com wrote:

Re: GC Storm

2010-07-18 Thread Schubert Zhang
Benjamin and Jonathan, It is not difficult to stack thousands of small SSTables. In a heavy inserting (many client threads), the memtable flush (generate new sstable) is frequent (e.g. one in 30s). The compaction only run in a single thread and is CPU bound. Consider the compactionManager is

Re: GC Storm

2010-07-18 Thread Peter Schuller
(adding dev@) (2) Can we implement multi-thread compaction? I think this is the only way to scale. Or at least to implement concurrent compaction (whether it is by division into threads or not) of multiple size classes. As long as the worst-case compactions are significantly slower than

Re: GC Storm

2010-07-18 Thread Jonathan Ellis
On Sun, Jul 18, 2010 at 2:45 AM, Schubert Zhang zson...@gmail.com wrote: In a heavy inserting (many client threads), the memtable flush (generate new sstable) is frequent (e.g. one in 30s). This is a sign you should increase your memtable thresholds, btw. If you wrote out larger sstables,

Re: StorageProxy.mutate

2010-07-18 Thread Jonathan Ellis
UnavailableException means the node(s) with the requested data are down. A sub-case of this is when no nodes at all are known, which may be what you are seeing. On Sat, Jul 17, 2010 at 10:00 PM, Mubarak Seyed mubarak.se...@gmail.com wrote: I am trying to load data using

Re: GC Storm

2010-07-18 Thread Schubert Zhang
Agree to Peter Schuller. On Sun, Jul 18, 2010 at 8:40 PM, Jonathan Ellis jbel...@gmail.com wrote: On Sun, Jul 18, 2010 at 2:45 AM, Schubert Zhang zson...@gmail.com wrote: In a heavy inserting (many client threads), the memtable flush (generate new sstable) is frequent (e.g. one in 30s).

Log configuration

2010-07-18 Thread osishkin osishkin
I didn't find in the documentation a way to configure message logging that I'm looking for, so I appologize if this is a trivial question. Is there a simple guide to configuring logging options in Cassandra? I saw references to output.log, transactions log. I deduced from some other post here that

Re: Log configuration

2010-07-18 Thread gabriele renzi
On Sun, Jul 18, 2010 at 5:28 PM, osishkin osishkin osish...@gmail.com wrote: I didn't find in the documentation a way to configure message logging that I'm looking for, so I appologize if this is a trivial question. Is there a simple guide to configuring logging options in Cassandra? I saw

Re: PhantomReference in Cassandra

2010-07-18 Thread Schubert Zhang
Up On Sat, Jun 5, 2010 at 4:30 PM, Anty anty@gmail.com wrote: Hi:All in the code of SSTableReader.java private static final ReferenceQueueSSTableReader finalizerQueue = new ReferenceQueueSSTableReader() {{ Runnable runnable = new Runnable() { public

Re: StorageProxy.mutate

2010-07-18 Thread Mubarak Seyed
I was using single node and was running ClientOnlyExample from the same machines, heard from word_count README that i can use its storage-conf.xml to alias localhost to 127.0.0.2 and ListenAddress can be 127.0.0.2 and seed can be 127.0.0.1. Looks like ClientOnlyExample starts itself up as

Re: Bootstrap question

2010-07-18 Thread shimi
If I have problems with never ending bootstraping I do the following. I try each one if it doesn't help I try the next. It might not be the right thing to do but it worked for me. 1. Restart the bootstraping node 2. If I see streaming 0/ I restart the node and all the streaming nodes 3.

Re: When will major compaction be triggered

2010-07-18 Thread Schubert Zhang
Hi Jonathan, Could you provide info about the special case where a minor compaction, also happens to be a major one? On Wed, May 19, 2010 at 2:29 PM, Jonathan Ellis jbel...@gmail.com wrote: No. (Except in the special case where a minor compaction, also happens to be a major one.) On Tue,

Newbie to cassandra

2010-07-18 Thread sonia gehlot
Hi everyone, I am new to Cassandra and wanted to try and start learning Cassandra. I have database background. I am fully exposed and have full command on Netezza, Oracle, MySQL, Sybase, SQL etc basically all the relational databases. As Cassandra is gaining popularity day by day by its amazing

Re: Bootstrap question

2010-07-18 Thread Anthony Molinaro
Yeah, I tried all that already and it didn't seem to work, no new nodes will bootstrap, which makes me think there's some saved state somewhere, preventing a new node from bootstrapping. I think maybe the Location sstables? Is it safe to nuke those on all hosts and restart everything? (I just

Re: Newbie to cassandra

2010-07-18 Thread S Ahmed
read the wiki, read about nosql in general. download and install it, play with it. browse the source code. read the bigdata paper by google, dynamo by amazon. On Sun, Jul 18, 2010 at 2:46 PM, sonia gehlot sonia.geh...@gmail.comwrote: Hi everyone, I am new to Cassandra and wanted to try and

Re: StorageProxy.mutate

2010-07-18 Thread Jonathan Ellis
On Sun, Jul 18, 2010 at 11:29 AM, Mubarak Seyed mubarak.se...@gmail.com wrote: Looks like ClientOnlyExample starts itself up as Read-Only node and hence can't write, is it true? Yes, as the name implies. I tried starting node from bin/cassandra (it did with bind to Thrift Address

Re: When will major compaction be triggered

2010-07-18 Thread Jonathan Ellis
There isn't much to provide; if you only have 3 sstables on your system, for instance, and they get compacted together, it will still be a major compaction and result in tombstones being removed. On Sun, Jul 18, 2010 at 1:27 PM, Schubert Zhang zson...@gmail.com wrote: Hi Jonathan, Could you

Re: Newbie to cassandra

2010-07-18 Thread Jonathan Ellis
Start with the recommended articles on http://wiki.apache.org/cassandra/ArticlesAndPresentations On Sun, Jul 18, 2010 at 1:46 PM, sonia gehlot sonia.geh...@gmail.com wrote: Hi everyone, I am new to Cassandra and wanted to try and start learning Cassandra. I have database background. I am fully

Re: Newbie to cassandra

2010-07-18 Thread Bill Hastings
Or perhaps this one. This is the Cassandra paper from the guys at FB. http://www.cs.cornell.edu/projects/ladis2009/program.htm#session3 On Sun, Jul 18, 2010 at 1:21 PM, Jonathan Ellis jbel...@gmail.com wrote: Start with the recommended articles on

Re: Newbie to cassandra

2010-07-18 Thread Jonathan Ellis
Which is bullet #4 on the list I linked. :) On Sun, Jul 18, 2010 at 5:31 PM, Bill Hastings bllhasti...@gmail.com wrote: Or perhaps this one. This is the Cassandra paper from the guys at FB. http://www.cs.cornell.edu/projects/ladis2009/program.htm#session3 On Sun, Jul 18, 2010 at 1:21 PM,

RE: Newbie to cassandra

2010-07-18 Thread Dwight Smith
Is there a link available for the Cassandra Summit in SF?? From: Eben Hewitt [mailto:eben.hew...@gmail.com] Sent: Sunday, July 18, 2010 5:02 PM To: user@cassandra.apache.org Subject: Re: Newbie to cassandra Hi Sonia If you're still interested after the reading and want something more

Re: Newbie to cassandra

2010-07-18 Thread Jonathan Ellis
Summit: http://cassandrasummit2010.eventbrite.com/ Trainings in SF (day after Summit), NYC, Denver, Seattle: http://www.eventbrite.com/org/474011012?s=1848277 On Sun, Jul 18, 2010 at 7:07 PM, Dwight Smith dwight.sm...@alcatel-lucent.com wrote: Is there a link available for the Cassandra Summit

Re: Cassandra benchmarking on Rackspace Cloud

2010-07-18 Thread Oren Benjamin
Thanks for the info. Very helpful in validating what I've been seeing. As for the scaling limit... The above was single node testing. I'd expect to be able to add nodes and scale throughput. Unfortunately, I seem to be running into a cap of 21,000 reads/s regardless of the number of

goods search with cassandra

2010-07-18 Thread Chen Xinli
Hi, I want to implement goods search with cassandra; and I have some confusings. Can someone help me out? The case is that: There are about 1 million shops, every shop with about 10,000 goods, every goods with property like title, price etc.. The search is like give me 10 goods in a specific

Re: Cassandra benchmarking on Rackspace Cloud

2010-07-18 Thread Brandon Williams
On Sun, Jul 18, 2010 at 8:45 PM, Oren Benjamin o...@clearspring.com wrote: Thanks for the info. Very helpful in validating what I've been seeing. As for the scaling limit... The above was single node testing. I'd expect to be able to add nodes and scale throughput. Unfortunately, I seem

hooking into openstack/nova with cassandra

2010-07-18 Thread Jeremy Hanna
Currently nova, the openstack's (http://openstack.org) software for an open cloud server implementation uses redis. In the #openstack IRC channel, there was talk of abstracting the data storage model out. They are thinking that perhaps they will consider other options as it is going to be