RE: unique key generation

2011-02-09 Thread Brendan Poole
Are you sure about those odds? Winning the UK national lottery has a chance of 13 983 816 to 1 so for just 2 days the odds are 13 983 816^2 = 1.9554711 x 10^14 Brendan Poole Systems Developer NewLaw Solicitors Helmont House Churchill Way Cardiff brendan.p

Re: How do secondary indices work

2011-02-09 Thread Timo Nentwig
On Feb 8, 2011, at 21:23, Aaron Morton wrote: >>> 1) Is data stored in some external data structure, or is it stored in an >>> actual Cassandra table, as columns within column families? Yes. Own files next to the CF files and own node IndexColumnFamilies in JMX. And they are built asynchronousl

Re: regarding space taken by different column families in Cassandra

2011-02-09 Thread abhinav prakash rai
After 1 hour ,from the application was done, the size of data folder become 14 GB and the result of cfstats is matching with this number (and Space used (live) become equal to Space used (total) ). CF1-Space used (live) :7196278850 Space used (total): 7196278850 CF2-

Re: How do secondary indices work

2011-02-09 Thread altanis
Thank you for the reply, although I didn't quite understand you. All I got was that Index data is stored in some kind of external data structure. Alexander > > On Feb 8, 2011, at 21:23, Aaron Morton wrote: > 1) Is data stored in some external data structure, or is it stored in an a

Re: How do secondary indices work

2011-02-09 Thread altanis
Thank you for the links, I did read a bit in the comments of the ticket, but I couldn't get much out of it. I am mainly interested in how the index is stored and partitioned, not how it is used. I think the people in the dev list will probably be better qualified to answer that. My questions alway

Implemeting a LRU in Cassandra

2011-02-09 Thread Utku Can Topçu
Hi All, I'm sure people here have tried to solve similar questions. Say I'm tracking pages, I want to access the least recently used 1000 unique pages (i.e. columnnames). How can I achieve this? Using a row with say, ttl=60 seconds would solve the problem of accessing the least recently used uniq

Re: How do secondary indices work

2011-02-09 Thread Stu Hood
Alexander: The secondary indexes in 0.7.0 (type KEYS) are stored internally in a column family, and are kept synchronized with the base data via locking on a local node, meaning they are always consistent on the local node. Eventual consistency still applies between nodes, but a returned result wi

Anyone want to help out with http://wiki.apache.org/cassandra/MavenPlugin

2011-02-09 Thread Stephen Connolly
Until the release vote passes at mojo, you will need to do the following to follow the example: svn co https://svn.codehaus.org/mojo/trunk/sandbox/cassandra-maven-plugin cd cassandra-maven-plugin mvn install cd .. Otherwise the example should be fine. It's a wiki page, so I'm hoping that people

Re: How do secondary indices work

2011-02-09 Thread altanis
Thank you very much, this is the information I was looking for. I started adding secondary index functionality to Cassandra myself, and it turns out I am doing almost exactly the same thing. I will try to change my code to use your implementation as well to compare results. Alexander > Alexander:

Re: unique key generation

2011-02-09 Thread Victor Kabdebon
Yes i have done a mistake I know ! But I hoped nobody would notice :). It is the odds of winning 3 days in a row (standard probability fail). Still it is totally unlikely Sorry about this mistake, Best regards, Victor K.

Re: ApplicationState Schema has drifted from DatabaseDescriptor

2011-02-09 Thread Gary Dusbabek
Aaron, It looks like you're experiencing a side-effect of CASSANDRA-2083. There was at least one place (when node B received updated schema from node A) where gossip was not being updated with the correct schema even though DatabaseDescriptor had the right version. I'm pretty sure this is what yo

Re: How do secondary indices work

2011-02-09 Thread altanis
One more question: does each node keep an index of their own values, or is the index global? Alexander > Thank you very much, this is the information I was looking for. I started > adding secondary index functionality to Cassandra myself, and it turns out > I am doing almost exactly the same thin

[no subject]

2011-02-09 Thread Onur AKTAS
unsubscribe

unsubscribe

2011-02-09 Thread Onur AKTAS
unsubscribe

Out of control memory consumption

2011-02-09 Thread Huy Le
Hi, There is already an email thread on memory issue on this email list, but I creating a new thread as we are experiencing a different memory consumption issue. We are 12-server cluster. We use random partitioner with manually generated server tokens. Memory usage on one server keeps growing o

Re: How do secondary indices work

2011-02-09 Thread Jonathan Ellis
"Iterating through all of the rows matching an index clause on your cluster is guaranteed to touch N/RF of the nodes in your cluster, because each node only knows about data that is indexed locally." On Wed, Feb 9, 2011 at 9:13 AM, wrote: > One more question: does each node keep an index of thei

Re: Out of control memory consumption

2011-02-09 Thread Chris Burroughs
On 02/09/2011 11:15 AM, Huy Le wrote: > There is already an email thread on memory issue on this email list, but I > creating a new thread as we are experiencing a different memory consumption > issue. > > We are 12-server cluster. We use random partitioner with manually generated > server tokens

Re: Anyone want to help out with http://wiki.apache.org/cassandra/MavenPlugin

2011-02-09 Thread Stephen Connolly
oh you might have to check out and install mojo-sandbox-parent (a sibling svn url) sandbox projects are not allowed to deploy releases... the vote on dev@mojo will promote from sandbox and release in one vote 32 h to go - Stephen --- Sent from my Android phone, so random spelling mistakes, random

Re: Out of control memory consumption

2011-02-09 Thread Peter Schuller
> We are 12-server cluster.  We use random partitioner with manually generated > server tokens.  Memory usage on one server keeps growing out of control.  We > ran flush and cleared key and row caches but and ran GC but heap memory > usage won't go down.  The only way to heap memory usage to go dow

Re: Out of control memory consumption

2011-02-09 Thread Peter Schuller
(If you're looking at e.g. jconsole graphs a screenshot of the graph would not hurt.) -- / Peter Schuller

Specifying row caching on per query basis ?

2011-02-09 Thread Ertio Lew
Is there any way to specify on per query basis(like we specify the Consistency level), what rows be cached while you're reading them, from a row_cache enabled CF. I believe, this could lead to much more efficient use of the cache space!!( if you use same data for different features/ parts in your a

Re: Do supercolumns have a purpose?

2011-02-09 Thread Mike Malone
On Tue, Feb 8, 2011 at 2:03 AM, David Boxenhorn wrote: > Shaun, I agree with you, but marking them as deprecated is not good enough > for me. I can't easily stop using supercolumns. I need an upgrade path. > David, Cassandra is open source and community developed. The right thing to do is what'

Re: Out of control memory consumption

2011-02-09 Thread Huy Le
> > If the heap usages continues to grow an OOM will eventually be thrown. > Are you experiencing OOMs on these boxes? If you are not OOMing, then > what problem are you experiencing (excessive CPU use garbage collection > for one example)? > > > No OOM. The JVM just too busy doing GC when the us

Re: Do supercolumns have a purpose?

2011-02-09 Thread Norman Maurer
I still think super-columns are useful you just need to be aware of the limitations... Bye, Norman 2011/2/9 Mike Malone : > On Tue, Feb 8, 2011 at 2:03 AM, David Boxenhorn wrote: >> >> Shaun, I agree with you, but marking them as deprecated is not good enough >> for me. I can't easily stop usin

Re: Out of control memory consumption

2011-02-09 Thread Huy Le
> > To be clear: You are not talking about the size of the Java process in > top, but the actual amount of heap used as reported by the JVM via > jmx/jconsole/etc? > > This is memory usage shows in JMX that we are talking about. > Is the memory amount of memory that you consider high, the heap

Re: Using Cassandra-cli

2011-02-09 Thread Jonathan Ellis
"help update column family"? On Wed, Feb 9, 2011 at 1:15 PM, Eranda Sooriyabandara <0704...@gmail.com> wrote: > Hi Vishan, Aron and all, > > Thanks for the help. I tried it and successfully worked for me. > But I could not find a place where mention about the attributes of some > commands. > > e.g

Re: Specifying row caching on per query basis ?

2011-02-09 Thread Jonathan Ellis
Currently there is not. On Wed, Feb 9, 2011 at 12:04 PM, Ertio Lew wrote: > Is there any way to specify on per query basis(like we specify the > Consistency level), what rows be cached while you're reading them, > from a row_cache enabled CF. I believe, this could lead to much more > efficient us

Re: Out of control memory consumption

2011-02-09 Thread Robert Coli
On Wed, Feb 9, 2011 at 11:04 AM, Huy Le wrote: > Memory usage grows overtime. It is relatively typical for caches to exert memory pressure over time as they fill. What are your cache settings, for how many columnfamilies, and with what sized memtables? What version of Cassandra? =Rob

Re: Specifying row caching on per query basis ?

2011-02-09 Thread Ertio Lew
Is this under consideration for future releases ? or being thought about!? On Thu, Feb 10, 2011 at 12:56 AM, Jonathan Ellis wrote: > Currently there is not. > > On Wed, Feb 9, 2011 at 12:04 PM, Ertio Lew wrote: >> Is there any way to specify on per query basis(like we specify the >> Consistenc

Re: Specifying row caching on per query basis ?

2011-02-09 Thread Jonathan Ellis
Not really, no. If you can't trust LRU to cache the hottest rows perhaps you should split the data into different ColumnFamilies. On Wed, Feb 9, 2011 at 1:43 PM, Ertio Lew wrote: > Is this under consideration for future releases ? or being thought about!? > > > > On Thu, Feb 10, 2011 at 12:56 AM

Re: Specifying row caching on per query basis ?

2011-02-09 Thread Edward Capriolo
On Wed, Feb 9, 2011 at 2:43 PM, Ertio Lew wrote: > Is this under consideration for future releases ? or being thought about!? > > > > On Thu, Feb 10, 2011 at 12:56 AM, Jonathan Ellis wrote: >> Currently there is not. >> >> On Wed, Feb 9, 2011 at 12:04 PM, Ertio Lew wrote: >>> Is there any way to

Exceptions on 0.7.0

2011-02-09 Thread shimi
I have a 4 node test cluster were I test the port to 0.7.0 from 0.6.X On 3 out of the 4 nodes I get exceptions in the log. I am using RP. Changes that I did: 1. changed the replication factor from 3 to 4 2. configured the nodes to use Dynamic Snitch 3. RR of 0.33 I run repair on 2 nodes before I

Re: Specifying row caching on per query basis ?

2011-02-09 Thread buddhasystem
Jonathan, what if the data is really homogeneous, but over a long period of time. I decided that the users who hit the database for recent past should have a better ride. Splitting into a separate CF also has costs, right? In fact, if I were to go this way, do you think I can crank down the key c

Re: read latency in cassandra

2011-02-09 Thread Robert Coli
On Fri, Feb 4, 2011 at 11:13 AM, Dan Kuebrich wrote: > Is 2 seconds the normal "I went to disk" latency for cassandra? Cassandra exposes metrics on a per-CF basis which indicate latency. This includes both cache hits and misses, as well as requests for rows which do not exist. It does NOT include

Default Listen Port

2011-02-09 Thread Jeremy.Truelove
What's the easiest way to change the port nodes listen for comm on from other nodes? It appears that the default is 8080 which collides with my tomcat server on one of our dev boxes. I tried doing something in cassandra.yaml like listen_address: 192.1.fake.2: but that doesn't work it throws

Re: Default Listen Port

2011-02-09 Thread Chris Burroughs
On 02/09/2011 04:00 PM, jeremy.truel...@barclayscapital.com wrote: > What's the easiest way to change the port nodes listen for comm on > from other nodes? It appears that the default is 8080 which collides > with my tomcat server on one of our dev boxes. I tried doing > something in cassandra.yaml

RE: Default Listen Port

2011-02-09 Thread Jeremy.Truelove
Thanks for the heads up that worked. -Original Message- From: Chris Burroughs [mailto:chris.burrou...@gmail.com] Sent: Wednesday, February 09, 2011 4:04 PM To: user@cassandra.apache.org Cc: Truelove, Jeremy: IT (NYK) Subject: Re: Default Listen Port On 02/09/2011 04:00 PM, jeremy.truel..

Re: Default Listen Port

2011-02-09 Thread Edward Capriolo
On Wed, Feb 9, 2011 at 4:00 PM, wrote: > What’s the easiest way to change the port nodes listen for comm on from > other nodes? It appears that the default is 8080 which collides with my > tomcat server on one of our dev boxes. I tried doing something in > cassandra.yaml like > > > > listen_addre

Re: Do supercolumns have a purpose?

2011-02-09 Thread Bill de hÓra
On Thu, 2011-02-03 at 15:35 -0800, Mike Malone wrote: > In my dealings with the Cassandra code, super columns end up making a > mess all over the place when algorithms need to be special cased and > branch based on the column/supercolumn distinction. > > > I won't even mention what it does to t

What will happen if I try to compact with insufficient headroom?

2011-02-09 Thread buddhasystem
One of my nodes is 76% full. I know that one of CFs represents 90% of the data, others are really minor. Can I still compact under these conditions? Will it crash and lose the data? Will it try to create one very large file out of fragments, for that dominating CF? TIA -- View this message in c

Re: ApplicationState Schema has drifted from DatabaseDescriptor

2011-02-09 Thread Aaron Morton
Thanks Gary. I'll keep an eye on things and see if it happens again. From reading the code I'm wondering if there is a small chance of a race condition in HintedHandoffManager.waitForSchemaAgreement() .Could the following happen? I'm a little unsure on exactly how the endpoint state is removed from

Re: ApplicationState Schema has drifted from DatabaseDescriptor

2011-02-09 Thread Brandon Williams
On Wed, Feb 9, 2011 at 4:31 PM, Aaron Morton wrote: > Thanks Gary. I'll keep an eye on things and see if it happens again. > > From reading the code I'm wondering if there is a small chance of a race > condition in HintedHandoffManager.waitForSchemaAgreement() . > > Could the following happen? I'm

RE: Exceptions on 0.7.0

2011-02-09 Thread Dan Hendry
Out of curiosity, do you really have on the order of 1,986,622,313 elements (I believe elements=keys) in the cf? Dan From: shimi [mailto:shim...@gmail.com] Sent: February-09-11 15:06 To: user@cassandra.apache.org Subject: Exceptions on 0.7.0 I have a 4 node test cluster were I test the

Re: unsubscribe

2011-02-09 Thread Chance Li
unsubscribe

Re: unsubscribe

2011-02-09 Thread Aaron Morton
instructions are herehttp://wiki.apache.org/cassandra/FAQ#unsubscribeOn 10 Feb, 2011,at 02:38 PM, Chance Li wrote:unsubscribe

Re: Using Cassandra-cli

2011-02-09 Thread Eranda Sooriyabandara
Hi all, Thanks Jonathan and Eric, you both describes what I want. Now I am looking forward to play with them. thanks Eranda

Re: time to live rows

2011-02-09 Thread Wangpei (Peter)
AFAIK 2nd index only works for operator EQ. -邮件原件- 发件人: Kallin Nagelberg [mailto:kallin.nagelb...@gmail.com] 发送时间: 2011年2月9日 3:36 收件人: user@cassandra.apache.org 主题: Re: time to live rows I'm thinking if this row expiry notion doesn't pan out then I might create a 'lastAccessed' column wi

Re: Row Key Types

2011-02-09 Thread Wangpei (Peter)
Did you set compare_with attribute of your ColumnFamily to TimeUUIDType? -邮件原件- 发件人: Bill Speirs [mailto:bill.spe...@gmail.com] 发送时间: 2011年2月2日 0:47 收件人: Cassandra Usergroup 主题: Row Key Types What is the type of a Row Key? Can you define how they are compared? I ask because I'm using Ti

RE: Do supercolumns have a purpose?

2011-02-09 Thread Viktor Jevdokimov
SCFs are very useful and I hope lives forever. We need them! Best regards/ Pagarbiai Viktor Jevdokimov Senior Developer Email: viktor.jevdoki...@adform.com Phone: +370 5 212 3063 Fax: +370 5 261 0453 Konstitucijos pr. 23, LT-08105 Vilnius, Lithuania Disclaimer: The information contained in

Re: Do supercolumns have a purpose?

2011-02-09 Thread David Boxenhorn
Mike, my problem is that I have an database and codebase that already uses supercolumns. If I had to do it over, it wouldn't use them, for the reasons you point out. In fact, I have a feeling that over time supercolumns will become deprecated de facto, if not de jure. That's why I would like to see