Re: Decorator Algorithm
a quick followup on this : when using Byte ordered partitioner. how does a short key get mapped to the 128bit token ? what about keys longer than 128 ? does Cassandra just pad and truncate ? thanks Le 24 juin 2011 04:53, Maki Watanabe watanabe.m...@gmail.com a écrit : A little addendum Key := Your data to identify a row Token := Index on the ring calculated from Key. The calculation is defined in replication strategy. You can lookup responsible nodes (endpoints) for a specific key with JMX getNaturalEndpoints interface. maki 2011/6/24 aaron morton aa...@thelastpickle.com: Various places in the code call IPartitioner.decorateKey() which returns a DecoratedKeyT which contains both the original key and the TokenT The RandomPartitioner md5 to hash the key ByteBuffer and create a BigInteger. OPP converts the key into utf8 encoded String. Using the token to find which endpoints contain replicas is done by the AbstractReplicationStrategy.calculateNaturalEndpoints() implementations. Does that help? - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 23 Jun 2011, at 19:58, Jonathan Colby wrote: Hi - I'd like to understand more how the token is hashed with the key to determine on which node the data is stored - called decorating in cassandra speak. Can anyone share any documentation on this or describe this more in detail? Yes, I could look at the code, but I was hoping to be able to read more about how it works first. thanks. -- w3m
Re: Counter Column
On 27/06/2011 17:04, Artem Orobets wrote: Hi! As I know, we use counter column only with replication factor ALL, so is it mean that we can't read data while any replica will fail? you can use any consistency level, using replicate_on_write=true when create the counter column family. -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
Re : Re : get_range_slices result
i used TimeUUIDType as type in storage-conf.xml file ColumnFamily Name=table CompareWith=TimeUUIDType / and i used it as comparator in my java code, but in the execution i get exception : Erreur --java.io.UnsupportedEncodingException: TimeUUIDType how can i write it? BR De : David Boxenhorn da...@citypath.com À : user@cassandra.apache.org Cc : karim abbouh karim_...@yahoo.fr Envoyé le : Vendredi 24 Juin 2011 11h25 Objet : Re: Re : get_range_slices result You can get the best of both worlds by repeating the key in a column, and creating a secondary index on that column. On Fri, Jun 24, 2011 at 1:16 PM, Sylvain Lebresne sylv...@datastax.com wrote: On Fri, Jun 24, 2011 at 10:21 AM, karim abbouh karim_...@yahoo.fr wrote: i want get_range_slices() function returns records sorted(orded) by the key(rowId) used during the insertion. is it possible? You will have to use the OrderPreservingPartitioner. This is no without inconvenience however. See for instance http://wiki.apache.org/cassandra/StorageConfiguration#line-100 or http://ria101.wordpress.com/2010/02/22/cassandra-randompartitioner-vs-orderpreservingpartitioner/ that give more details on the pros and cons (the short version being that the main advantage of OrderPreservingPartitioner is what you're asking for, but it's main drawback is that load-balancing the cluster will likely be very very hard). In general the advice is to stick with RandomPartitioner and design a data model that avoids needing range slices (or at least needing that the result is sorted). This is very often not too hard and more efficient, and much more simpler than to deal with the load balancing problems of OrderPreservingPartitioner. -- Sylvain De : aaron morton aa...@thelastpickle.com À : user@cassandra.apache.org Envoyé le : Jeudi 23 Juin 2011 20h30 Objet : Re: get_range_slices result Not sure what your question is. Does this help ? http://wiki.apache.org/cassandra/FAQ#range_rp Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 23 Jun 2011, at 21:59, karim abbouh wrote: how can get_range_slices() function returns sorting key ? BR
Re: Counter Column
if i write at ALL and read at ONE,is that setting required ? thanks Le 27 juin 2011 17:22, Donal Zang zan...@ihep.ac.cn a écrit : On 27/06/2011 17:04, Artem Orobets wrote: Hi! As I know, we use counter column only with replication factor ALL, so is it mean that we can't read data while any replica will fail? you can use any consistency level, using replicate_on_write=true when create the counter column family. -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
listen_address and rpc_address
Hi Everyone, Quick question regarding the listen_address and rpc_address parameters. We're trying to run some map-reduce jobs on our ring and my goal is to split the Cassandra intra-cluster traffic from the Thrift calls. I've populated the listen_address with the IP address of one interface and rpc_address with the IP addres of another interface. Let's call them IP A (listen) and IP B (rpc). Now our jobs are setup so the connect to IP B on the seed node and use the describe ring method to get the ring information and then know where to connect on each node (Forgive my very summary explanations but I don't want to dive in the code just yet). The issue is Cassandra returns the IP A for the ring nodes but I'd like to force the clients to connect to IP B. Is there anything we can do to make Cassandra return the RPC addresses when issuing that method or should I simply make RPC listen on all addresses? Thanks, Sebastien Coutu Hopper Inc.
Re: Counter Column
Let me make that simpler. Don't ever use replicate_on_write=false (even if you think that it is what you want, there is a good chance it's not). Obviously, the default is replicate_on_write=true. -- Sylvain On Mon, Jun 27, 2011 at 7:03 PM, Philippe watche...@gmail.com wrote: if i write at ALL and read at ONE,is that setting required ? thanks Le 27 juin 2011 17:22, Donal Zang zan...@ihep.ac.cn a écrit : On 27/06/2011 17:04, Artem Orobets wrote: Hi! As I know, we use counter column only with replication factor ALL, so is it mean that we can't read data while any replica will fail? you can use any consistency level, using replicate_on_write=true when create the counter column family. -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
Re: listen_address and rpc_address
I believe you are describing the issure here: https://issues.apache.org/jira/browse/CASSANDRA-1777 Until that issue is fixed though, the solution is to use the same interface for both. On Mon, Jun 27, 2011 at 12:09 PM, Sebastien Coutu sco...@openplaces.org wrote: Hi Everyone, Quick question regarding the listen_address and rpc_address parameters. We're trying to run some map-reduce jobs on our ring and my goal is to split the Cassandra intra-cluster traffic from the Thrift calls. I've populated the listen_address with the IP address of one interface and rpc_address with the IP addres of another interface. Let's call them IP A (listen) and IP B (rpc). Now our jobs are setup so the connect to IP B on the seed node and use the describe ring method to get the ring information and then know where to connect on each node (Forgive my very summary explanations but I don't want to dive in the code just yet). The issue is Cassandra returns the IP A for the ring nodes but I'd like to force the clients to connect to IP B. Is there anything we can do to make Cassandra return the RPC addresses when issuing that method or should I simply make RPC listen on all addresses? Thanks, Sebastien Coutu Hopper Inc.
Truncate introspection
Is there any straightforward means of seeing what's going on after issuing a truncate (on 0.7.5)? I'm not seeing evidence that anything actually happened. I've disabled read repair on the column family in question and don't have anything actively reading/writing at present, apart from my one-off tests to see if rows have disappeared. Thanks in advance.
Re: Truncate introspection
There's a JMX method to get the number of sstables in a CF, is that what you're looking for? On Mon, Jun 27, 2011 at 1:04 PM, Ethan Rowe et...@the-rowes.com wrote: Is there any straightforward means of seeing what's going on after issuing a truncate (on 0.7.5)? I'm not seeing evidence that anything actually happened. I've disabled read repair on the column family in question and don't have anything actively reading/writing at present, apart from my one-off tests to see if rows have disappeared. Thanks in advance. -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Truncate introspection
If those went to zero, it would certainly tell me something happened. :) I guess watching that would be a way of seeing something was going on. Is the truncate itself propagating a ring-wide marker or anything so the CF is logically empty before being physically removed? That's the impression I got from the docs but it wasn't totally clear to me. On Mon, Jun 27, 2011 at 3:33 PM, Jonathan Ellis jbel...@gmail.com wrote: There's a JMX method to get the number of sstables in a CF, is that what you're looking for? On Mon, Jun 27, 2011 at 1:04 PM, Ethan Rowe et...@the-rowes.com wrote: Is there any straightforward means of seeing what's going on after issuing a truncate (on 0.7.5)? I'm not seeing evidence that anything actually happened. I've disabled read repair on the column family in question and don't have anything actively reading/writing at present, apart from my one-off tests to see if rows have disappeared. Thanks in advance. -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Sharing Cassandra with Solandra
Hi everyone, Would anyone care to talk about their experiences with using Solandra along side another application that uses Cassandra (also on the same node)? I'm curious about any resource contention issues or compatibility between C* versions and Sol. Also, I read the developer somewhere say that you have to run Solandra on every C* node in the ring. I'm not sure if I interpreted that correctly. Also, what's the index size to data size ratio to expect (ballpark)? How does it perform? Any caveats? Thanks! aj
Re: Sharing Cassandra with Solandra
On Mon, 2011-06-27 at 15:06 -0600, AJ wrote: Would anyone care to talk about their experiences with using Solandra along side another application that uses Cassandra (also on the same node)? I'm curious about any resource contention issues or compatibility between C* versions and Sol. Also, I read the developer somewhere say that you have to run Solandra on every C* node in the ring. I'm not sure if I interpreted that correctly. Also, what's the index size to data size ratio to expect (ballpark)? How does it perform? Any caveats? We're currently keeping the clusters separate at Pantheon Systems because our core API (which runs on standard Cassandra) is often ready for the next Cassandra version at a different time than Solandra. Solandra recently gained dual 0.7/0.8 support, but we're still opting to use the version on Cassandra that Solandra is primarily being built and tested on (which is currently 0.8). signature.asc Description: This is a digitally signed message part
Re: Truncate introspection
Each node (independently) has logic that guarantees that any writes processed before the truncate, will be wiped out. This does not mean that each node will wipe out the same data, or even that each node will process the truncate (which would result in a timedoutexception). It also does not mean you can't have writes immediately after the truncate that would race w/ a truncate, check for zero sstables procedure. On Mon, Jun 27, 2011 at 3:35 PM, Ethan Rowe et...@the-rowes.com wrote: If those went to zero, it would certainly tell me something happened. :) I guess watching that would be a way of seeing something was going on. Is the truncate itself propagating a ring-wide marker or anything so the CF is logically empty before being physically removed? That's the impression I got from the docs but it wasn't totally clear to me. On Mon, Jun 27, 2011 at 3:33 PM, Jonathan Ellis jbel...@gmail.com wrote: There's a JMX method to get the number of sstables in a CF, is that what you're looking for? On Mon, Jun 27, 2011 at 1:04 PM, Ethan Rowe et...@the-rowes.com wrote: Is there any straightforward means of seeing what's going on after issuing a truncate (on 0.7.5)? I'm not seeing evidence that anything actually happened. I've disabled read repair on the column family in question and don't have anything actively reading/writing at present, apart from my one-off tests to see if rows have disappeared. Thanks in advance. -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Auto compaction to be staggered ?
Is there an enhancement on the roadmap to stagger the auto compactions on different nodes, to avoid more than one node compacting at any given time (or as few nodes as possible to compact at any given time). If not, any workarounds ? Thanks.
Clock skew
During writes, the timestamp field in the column is the system-time of that node (correct me if that is not the case and the system-time of the co-ordinator is what gets applied to all the replicas). During reads, the latest write wins. What if there is a clock skew ? It could lead to a stale write over-riding the actual latest write, just because the clock of that node is ahead of the other node. Right ?
Re: Auto compaction to be staggered ?
On 6/27/2011 4:01 PM, A J wrote: Is there an enhancement on the roadmap to stagger the auto compactions on different nodes, to avoid more than one node compacting at any given time (or as few nodes as possible to compact at any given time). If not, any workarounds ? Thanks. +1. I proposed the same in my *Ideas for Big Data Support* thread, 5.) Postponed Major Compactions: The option to postpone auto-triggered major compactions until a pre-defined time of day or week or until staff can do it manually. aj
Re: listen_address and rpc_address
Hi Nick, Thanks, it looks a lot like what I'm experiencing. I've voted on the issue. Regards, Sebastien Coutu On Mon, Jun 27, 2011 at 1:31 PM, Nick Bailey n...@datastax.com wrote: I believe you are describing the issure here: https://issues.apache.org/jira/browse/CASSANDRA-1777 Until that issue is fixed though, the solution is to use the same interface for both. On Mon, Jun 27, 2011 at 12:09 PM, Sebastien Coutu sco...@openplaces.org wrote: Hi Everyone, Quick question regarding the listen_address and rpc_address parameters. We're trying to run some map-reduce jobs on our ring and my goal is to split the Cassandra intra-cluster traffic from the Thrift calls. I've populated the listen_address with the IP address of one interface and rpc_address with the IP addres of another interface. Let's call them IP A (listen) and IP B (rpc). Now our jobs are setup so the connect to IP B on the seed node and use the describe ring method to get the ring information and then know where to connect on each node (Forgive my very summary explanations but I don't want to dive in the code just yet). The issue is Cassandra returns the IP A for the ring nodes but I'd like to force the clients to connect to IP B. Is there anything we can do to make Cassandra return the RPC addresses when issuing that method or should I simply make RPC listen on all addresses? Thanks, Sebastien Coutu Hopper Inc.
Re: RAID or no RAID
RAID0 so you have one big volume. For performance (cassandra does not stripe sstables across the data dirs) and otherwise you'll have fragmentation and wont be able to utilise all your space. Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 28 Jun 2011, at 11:46, mcasandra wrote: Which one is preferred RAID0 or spreading data files accross various disks on the same node? I like RAID0 but what would be the most convincing argument to put additional RAID controller card in the machine? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RAID-or-no-RAID-tp6522904p6522904.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: Clock skew
Without exception the timestamp is set by the client, not the server. The one exception to the without exception rule is CounterColumnType operations. If you are in a situation where you need better timing than you can get with ntp you should try to design around it. Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 28 Jun 2011, at 10:03, A J wrote: During writes, the timestamp field in the column is the system-time of that node (correct me if that is not the case and the system-time of the co-ordinator is what gets applied to all the replicas). During reads, the latest write wins. What if there is a clock skew ? It could lead to a stale write over-riding the actual latest write, just because the clock of that node is ahead of the other node. Right ?
Re: RAID or no RAID
If you have a quality HW raid controller with proper performance (and far from all have good performance) you cam definitely benefit from a battery backed up write cache on it, although the benefits will not be huge on raid 0. Unless you get a really good price on that high performance HW raid with battery backup, it is probably not worth it for raid 0. When that is said, raid 5 is pretty speedy as well with a good controller with battery cache so don't rule that out if you have the controller anyway and may save you from some manual recover operations.. Regards, Terje On 28 Jun 2011, at 08:46, mcasandra mohitanch...@gmail.com wrote: Which one is preferred RAID0 or spreading data files accross various disks on the same node? I like RAID0 but what would be the most convincing argument to put additional RAID controller card in the machine? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RAID-or-no-RAID-tp6522904p6522904.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
regarding CASSANDRA-2653
Hi all, I just stumbled on to what looks like issue CASSANDRA-2653. Here's my stack trace: ERROR [ReadStage:10] 2011-06-27 15:22:36,087 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[ReadStage:10,5,main] java.lang.AssertionError: No data found for NamesQueryFilter(columns=java.nio.HeapByteBuffer[pos=12 lim=21 cap=22]) in DecoratedKey(102020034675873608148572353603207993225, c295):QueryPath(columnFamilyName='XXX', superColumnName='null', columnName='null') (original filter NamesQueryFilter(columns=java.nio.HeapByteBuffer[pos=12 lim=21 cap=22])) from expression ' EQ ' at org.apache.cassandra.db.ColumnFamilyStore.scan(ColumnFamilyStore.java:1603) at org.apache.cassandra.service.IndexScanVerbHandler.doVerb(IndexScanVerbHandler.java:42) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) I am on version 0.7.6. My question is, why is that ticket marked as minor? Does that mean most people are not encountering this problem on index queries? If that's the case I'd really like to understand the conditions under which this problem occurs so I can at least work around it. I am doing an indexslicequery, specifying a column family, a single column name and an equals expression. For my client, I'm using hector version 0.7.0-29. I'd also like to know when this bug is expected to be fixed and when 0.7.7 might be released. I have 0.7.5 running in production right now and I really need an upgrade to resolve different bug. Thanks, Shu
Re: Clock skew
oftentimes people use time actually subconsciously to express causal relations (before/after), as long as you have some other means to establish causal relations, you don't really need to have an exactly clock. On Mon, Jun 27, 2011 at 4:54 PM, aaron morton aa...@thelastpickle.comwrote: Without exception the timestamp is set by the client, not the server. The one exception to the without exception rule is CounterColumnType operations. If you are in a situation where you need better timing than you can get with ntp you should try to design around it. Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 28 Jun 2011, at 10:03, A J wrote: During writes, the timestamp field in the column is the system-time of that node (correct me if that is not the case and the system-time of the co-ordinator is what gets applied to all the replicas). During reads, the latest write wins. What if there is a clock skew ? It could lead to a stale write over-riding the actual latest write, just because the clock of that node is ahead of the other node. Right ?
remove all the columns of a key in a column family
All: Can I remove all the columns of a key in a column family under the condition that not know what columns the column family has? Best Regards Donna li
Re: RAID or no RAID
I thought there is an option to give multiple data dirs in cassandra.yaml. What's the purpose of that? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RAID-or-no-RAID-tp6522904p6523523.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
remove all the columns of a key in a column family
Cassandra version is 0.7.2, when I use batch_mutate, the following exception throw “TException:Deletion does not yet support SliceRange predicates”, which version support delete the whole row of a key? Best Regards Donna li 发件人: Donna Li 发送时间: 2011年6月28日 10:59 收件人: user@cassandra.apache.org 主题: remove all the columns of a key in a column family All: Can I remove all the columns of a key in a column family under the condition that not know what columns the column family has? Best Regards Donna li
Re: RAID or no RAID
Not sure what the intended purpose is, but we've mostly used it as an emergency disk-capacity-increase option. It's not as good as raid because each disk size is counted individually (a compacted sstable can only be on one disk) so compaction size limits aren't expanded as one might expect. On Mon, Jun 27, 2011 at 11:30 PM, mcasandra mohitanch...@gmail.com wrote: I thought there is an option to give multiple data dirs in cassandra.yaml. What's the purpose of that? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RAID-or-no-RAID-tp6522904p6523523.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: RAID or no RAID
Not sure what the intended purpose is, but we've mostly used it as an emergency disk-capacity-increase option Thats what I've used it for. Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 28 Jun 2011, at 15:55, Dan Kuebrich wrote: Not sure what the intended purpose is, but we've mostly used it as an emergency disk-capacity-increase option. It's not as good as raid because each disk size is counted individually (a compacted sstable can only be on one disk) so compaction size limits aren't expanded as one might expect. On Mon, Jun 27, 2011 at 11:30 PM, mcasandra mohitanch...@gmail.com wrote: I thought there is an option to give multiple data dirs in cassandra.yaml. What's the purpose of that? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RAID-or-no-RAID-tp6522904p6523523.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
