If I always send the schema change requests to one particular node in the cassandra cluster.
Hi, Can the schema disagreement problem be avoided? Thanks. -- Dikang Gu 0086 - 18611140205
Re: HOW TO select a column or all columns that start with X
Thank you! Will this situation work only for UTF8Type comparator? On Wed, Aug 3, 2011 at 4:50 AM, Tyler Hobbs ty...@datastax.com wrote: A minor correction: To get all columns starting with ABC_, you would set column_start=ABC_ and column_finish=ABC` (the '`' character comes after '_'), and ignore the last column in your results if it happened to be ABC`. column_finish, or the slice end in other clients, is inclusive. You could of course use ABC_~ as column_finish and avoid the check if you know that you don't have column names like ABC_~FOO that you want to include. On Tue, Aug 2, 2011 at 7:17 PM, aaron morton aa...@thelastpickle.comwrote: Yup, thats a pretty common pattern. How exactly depends on the client you are using. Say you were using pycassam, you would do a get() http://pycassa.github.com/pycassa/api/pycassa/columnfamily.html#pycassa.columnfamily.ColumnFamily.get with column_start=ABC_ , count to whatever, and column_finish not provided. You can also provide a finish and use the highest encoded character, e.g. ascii 126 is ~ so if you used column_finish = ABC_~ you would get everything that starts with ABC_ Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 09:28, Eldad Yamin wrote: Hello, I wonder if I can select a column or all columns that start with X. E.g I have columns ABC_1, ABC_2, ZZZ_1 and I want to select all columns that start with ABC_ - is that possible? Thanks! -- Tyler Hobbs Software Engineer, DataStax http://datastax.com/ Maintainer of the pycassa http://github.com/pycassa/pycassa Cassandra Python client library
Re: Nodetool ring not showing all nodes in cluster
I finally solved this by using nodetool move and specifying token number explicitly. Thanks for all your help !! Cheers, Aishwarya On Tue, Aug 2, 2011 at 5:31 PM, aaron morton aa...@thelastpickle.com wrote: initial_token is read from the yaml file once only, during bootstrap. It is then stored in the LocationInfo system CF and used from there. It sounds like when you did the move you deleted these files, but then started the nodes each with their own seed. So you created 3 separate clusters, when each one bootstrapped it auto allocated itself an initial token and stored it in LocationInfo. You have 3 clusters, each with one node and each node has the same token as it was the first node into a new empty cluster. Try this: 1 - Shut down the two (lets call them B and C) you want to join the first one (called A). 2 - Delete their LocationInfo CF 3 - ensure the seed list for ALL nodes points to node A. 4 - ensure the initial token is set correctly for B and C 5 - start B and C one at a time and make sure nodetool ring and describe cluster; in the CLI agree before starting the next *IF* node A has the incorrect token I would fix this after you get B and C back into the ring. Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 09:43, Aishwarya Venkataraman wrote: I corrected the seed list and checked the cluster name. They are all good now. But still nodetool -ring shows only one node. INFO 21:36:59,735 Starting Messaging Service on port 7000 INFO 21:36:59,748 Using saved token 113427455640312814857969558651062452224 Nodes a_ipadrr and b_ipaddr have the same token 113427455640312814857969558651062452224. a_ipadrr is the new owner. All the nodes seem to be using the same initial token, despite me specifying an initial_token in the config file. Is this an issue ? How do I force cassandra to use the token in the cassandra.yaml file ? Thanks, Aishwarya On Tue, Aug 2, 2011 at 2:34 PM, Jonathan Ellis jbel...@gmail.com wrote: Yes. Different cluster names could also cause this. On Tue, Aug 2, 2011 at 4:21 PM, Jeremiah Jordan jeremiah.jor...@morningstar.com wrote: All of the nodes should have the same seedlist. Don't use localhost as one of the items in it if you have multiple nodes. On Tue, 2011-08-02 at 10:10 -0700, Aishwarya Venkataraman wrote: Nodetool does not show me all the nodes. Assuming I have three nodes A, B and C. The seedlist of A is localhost. Seedlist of B is localhost, A_ipaddr and seedlist of C is localhost,B_ipaddr,A_ipaddr. I have autobootstrap set to false for all 3 nodes since they all have the correct data and do not hav to migrate data from any particular node. My problem here is why does n't nodetool ring show me all nodes in the ring ? I agree that the cluster thinks that only one node is present. How do I fix this ? Thanks, Aishwarya On Tue, Aug 2, 2011 at 9:56 AM, samal sa...@wakya.in wrote: ERROR 08:53:47,678 Internal error processing batch_mutate java.lang.IllegalStateException: replication factor (3) exceeds number of endpoints (1) You already answered It always keeps showing only one node and mentions that it is handling 100% of the load. Cluster think only one node is present in ring, it doesn't agree RF=3 it is expecting RF=1. Original Q: I m not exactly sure what is the problem. But Does nodetool ring show all the host? What is your seed list? Is bootstrapped node has seed ip of its own? AFAIK gossip work even without actively joining a ring. On Tue, Aug 2, 2011 at 7:21 AM, Aishwarya Venkataraman cyberai...@gmail.com wrote: Replies inline. Thanks, Aishwarya On Tue, Aug 2, 2011 at 7:12 AM, Sorin Julean sorin.jul...@gmail.com wrote: Hi, Until someone answers with more details, few questions: 1. did you moved the system keyspace as well ? Yes. But I deleted the LocationInfo* files under the system folder. Shall I go ahead and delete the entire system folder ? 2. the gossip IP of the new nodes are the same as the old ones ? No. The Ip is different. 3. which cassandra version are you running ? I am using 0.8.1 If 1. is yes and 2. is no, for a quick fix: take down the cluster, remove system keyspace, bring the cluster up and bootstrap the nodes. Kind regards, Sorin On Tue, Aug 2, 2011 at 2:53 PM, Aishwarya Venkataraman cyberai...@gmail.com wrote: Hello, I recently migrated 400 GB of data that was on a different cassandra cluster (3 node with RF= 3) to a new cluster. I have a 3 node cluster with replication factor set to three. When I run nodetool ring, it does not show me all the nodes in the cluster. It always keeps showing only one node and mentions that it is handling 100% of the load. But when I look at the logs, the nodes are able to talk to each other via the gossip protocol. Why does this happen ? Can you tell me what I am doing
Re: cassandra consistency level
So what you're saying is that no matter what consistency level I'm using, the data will be written to all CF nodes right away, the consistency level is just for making sure that all CF nodes are UP and all data is written. In other words, if one of the nodes is down - the write (or read) will fail. I'm asking that because I'm a bit worried with consistency, for example: Every action that my client is doing is stored in a CF.x in a specific column by his user_id. I'm doing that by de-serializing the data that already found in the column, adding new data (the action), serializing and storing the data. so I'm worrying that some of the user actions will drop due low-consistency when there are lots of changes to a specific column in a sort period of time. I know that I can solve this situation in a different way by storing each action in a new column etc... but this is just an example that explain my question in a simple way. Thanks! On Wed, Aug 3, 2011 at 3:21 AM, aaron morton aa...@thelastpickle.comwrote: Not sure I understand your question exactly, but will take a shot… Writes are sent to every UP node, the consistency level is how many nodes we require to complete before we say the request completed successfully. So we also make sure that CL nodes are UP before we start the request. If you run CL ALL then Replication Factor nodes must be up for each key you are writing. With the exception of CL ONE reads are also sent to all UP replicas. Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 09:32, Eldad Yamin wrote: Is consistency level All for write actually grenty that my data is updated in all of my node? is it apply to read actions as-well? I've read it on the wiki, I just want to make sure. Thanks!
Re: Question about eventually consistent in Cassandra
A slight correction/clarification: 1) DeletedColumns always win. If there are two deletes the highest timestamp wins DeletedColumns only take precedence over a normal value if they have a greater *(or equal)* timestamp. *DeletedColumns always win* is only true if you're talking about two columns with identical timestamps. -- Sam Overton Acunu | http://www.acunu.com | @acunu
Sample Cassandra project in Tomcat
Hiii, Can any one pleaze send me any sample application which is (.war) implemented in java/jsp and cassandra db (Tomcat)
Killing cassandra is not working
H,, I was running cassandra in my mac and after some time the machine got to sleep mode. Now after the machine is On. I tried to kill the process of cassandra. But i could not able to do that as i could not able to find out the process id. theres no process there when i pinged PS command in mac os. I though its already killed and i have tried to start cassandra by typing bin/cassandra. I got the below error (in red color) due to the same. ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690] FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) Can you please resolve this issue. Thnkx in Adv[?] 330.gif
Re: Sample Cassandra project in Tomcat
I don't know much about this, may help you.. http://www.codefreun.de/apolloUI/ http://www.codefreun.de/apollo/ On Wed, Aug 3, 2011 at 3:36 PM, CASSANDRA learner cassandralear...@gmail.com wrote: Hiii, Can any one pleaze send me any sample application which is (.war) implemented in java/jsp and cassandra db (Tomcat)
Re: Killing cassandra is not working
try to use *grep* command to check the port where your cassandra was running. On 3 August 2011 16:01, CASSANDRA learner cassandralear...@gmail.comwrote: H,, I was running cassandra in my mac and after some time the machine got to sleep mode. Now after the machine is On. I tried to kill the process of cassandra. But i could not able to do that as i could not able to find out the process id. theres no process there when i pinged PS command in mac os. I though its already killed and i have tried to start cassandra by typing bin/cassandra. I got the below error (in red color) due to the same. ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690] FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) Can you please resolve this issue. Thnkx in Adv[?] 330.gif
Re: Sample Cassandra project in Tomcat
I suppose what you are looking for is an example of interacting with a java app. You should have a look at the high(er) level client hector https://github.com/rantav/hector/ You should find what you are looking for there. If you are looking for a tomcat (and .war) example, you should send an email to the tomcat mailing list. 2011/8/3 CASSANDRA learner cassandralear...@gmail.com: Hiii, Can any one pleaze send me any sample application which is (.war) implemented in java/jsp and cassandra db (Tomcat)
Re: Killing cassandra is not working
Seems like you have already a Cassandra instance running, so the second instance cannot open the same port twice. I would suggest you to kill all instances of Cassandra and start it again. 2011/8/3 Nilabja Banerjee nilabja.baner...@gmail.com try to use *grep* command to check the port where your cassandra was running. On 3 August 2011 16:01, CASSANDRA learner cassandralear...@gmail.comwrote: H,, I was running cassandra in my mac and after some time the machine got to sleep mode. Now after the machine is On. I tried to kill the process of cassandra. But i could not able to do that as i could not able to find out the process id. theres no process there when i pinged PS command in mac os. I though its already killed and i have tried to start cassandra by typing bin/cassandra. I got the below error (in red color) due to the same. ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690] FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) Can you please resolve this issue. Thnkx in Adv[?] 330.gif
Re: Killing cassandra is not working
Thnks for the reply Nila When i did PS command, I could not able to find any process related to cassandra. Thts the problem.. On Wed, Aug 3, 2011 at 4:12 PM, Benoit Perroud ben...@noisette.ch wrote: Seems like you have already a Cassandra instance running, so the second instance cannot open the same port twice. I would suggest you to kill all instances of Cassandra and start it again. 2011/8/3 Nilabja Banerjee nilabja.baner...@gmail.com try to use *grep* command to check the port where your cassandra was running. On 3 August 2011 16:01, CASSANDRA learner cassandralear...@gmail.comwrote: H,, I was running cassandra in my mac and after some time the machine got to sleep mode. Now after the machine is On. I tried to kill the process of cassandra. But i could not able to do that as i could not able to find out the process id. theres no process there when i pinged PS command in mac os. I though its already killed and i have tried to start cassandra by typing bin/cassandra. I got the below error (in red color) due to the same. ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690] FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) Can you please resolve this issue. Thnkx in Adv[?] 330.gif
Re: Sample Cassandra project in Tomcat
Hi, Thnks for the info On Wed, Aug 3, 2011 at 4:03 PM, samal sa...@wakya.in wrote: I don't know much about this, may help you.. http://www.codefreun.de/apolloUI/ http://www.codefreun.de/apollo/ On Wed, Aug 3, 2011 at 3:36 PM, CASSANDRA learner cassandralear...@gmail.com wrote: Hiii, Can any one pleaze send me any sample application which is (.war) implemented in java/jsp and cassandra db (Tomcat)
Exception encountered during startup - org.apache.cassandra.db.marshal.MarshalException: UUIDs must be exactly 16 bytes
Hello, I would be thankfull for any advice how to recover from the following problem: 1.) When I Start Cassandra, then I get the following exception: INFO 12:51:08,964 Heap size: 1573912576/1574961152 INFO 12:51:08,966 JNA not found. Native methods will be disabled. INFO 12:51:08,973 Loading settings from file:/usr/local/cassandra/0.8.1/conf/cassandra.yaml INFO 12:51:09,102 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap INFO 12:51:09,205 Global memtable threshold is enabled at 500MB INFO 12:51:09,206 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,211 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,212 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,214 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,215 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,216 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,251 Opening /usr/local/cassandra/0.8.1/data/data/system/NodeIdInfo-g-1 INFO 12:51:09,332 Opening /usr/local/cassandra/0.8.1/data/data/system/IndexInfo-g-1 INFO 12:51:09,343 Opening /usr/local/cassandra/0.8.1/data/data/system/Schema-g-61 INFO 12:51:09,348 Opening /usr/local/cassandra/0.8.1/data/data/system/Migrations-g-61 INFO 12:51:09,364 Opening /usr/local/cassandra/0.8.1/data/data/system/LocationInfo-g-25 INFO 12:51:09,365 Opening /usr/local/cassandra/0.8.1/data/data/system/LocationInfo-g-26 INFO 12:51:09,401 Loading schema version d4d71860-bd04-11e0--242d50cf1fd7 ERROR 12:51:09,579 Exception encountered during startup. org.apache.cassandra.db.marshal.MarshalException: UUIDs must be exactly 16 bytes at org.apache.cassandra.db.marshal.TimeUUIDType.getString(TimeUUIDType.java:117) at org.apache.cassandra.config.CFMetaData.getDefaultIndexName(CFMetaData.java:971) at org.apache.cassandra.config.CFMetaData.inflate(CFMetaData.java:381) at org.apache.cassandra.config.KSMetaData.inflate(KSMetaData.java:172) at org.apache.cassandra.db.DefsTable.loadFromStorage(DefsTable.java:99) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:479) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:139) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:315) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:80) Exception encountered during startup. org.apache.cassandra.db.marshal.MarshalException: UUIDs must be exactly 16 bytes at org.apache.cassandra.db.marshal.TimeUUIDType.getString(TimeUUIDType.java:117) at org.apache.cassandra.config.CFMetaData.getDefaultIndexName(CFMetaData.java:971) at org.apache.cassandra.config.CFMetaData.inflate(CFMetaData.java:381) at org.apache.cassandra.config.KSMetaData.inflate(KSMetaData.java:172) at org.apache.cassandra.db.DefsTable.loadFromStorage(DefsTable.java:99) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:479) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:139) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:315) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:80) I have to say, that my Linux Box crashed with an Out Of Memory Excetpion (Kernel Panic) before.. Might this be the cause? Otherwise I am really wondering, why the system is left in such an unstable state, that I event can't start cassandra anymore. 2.) What can I do to recover from this the bst way ? I want to start cassandra again, without losing data. How to do this / what to do next? Thank you very much!! Jens
Re: Killing cassandra is not working
so use netstat to find out which process had opened the port. 2011/8/3 CASSANDRA learner cassandralear...@gmail.com Thnks for the reply Nila When i did PS command, I could not able to find any process related to cassandra. Thts the problem.. On Wed, Aug 3, 2011 at 4:12 PM, Benoit Perroud ben...@noisette.ch wrote: Seems like you have already a Cassandra instance running, so the second instance cannot open the same port twice. I would suggest you to kill all instances of Cassandra and start it again. 2011/8/3 Nilabja Banerjee nilabja.baner...@gmail.com try to use *grep* command to check the port where your cassandra was running. On 3 August 2011 16:01, CASSANDRA learner cassandralear...@gmail.comwrote: H,, I was running cassandra in my mac and after some time the machine got to sleep mode. Now after the machine is On. I tried to kill the process of cassandra. But i could not able to do that as i could not able to find out the process id. theres no process there when i pinged PS command in mac os. I though its already killed and i have tried to start cassandra by typing bin/cassandra. I got the below error (in red color) due to the same. ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690] FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) Can you please resolve this issue. Thnkx in Adv[?] 330.gif
Significance of java_pidxxx.hprof
As per subject, Please explain me what is the significance of java_pidxxx.hprof
Re: Sample Cassandra project in Tomcat
2011/8/3 CASSANDRA learner cassandralear...@gmail.com: Hi, can you please send me the mailing list address of tomcat http://tomcat.apache.org/lists.html On Wed, Aug 3, 2011 at 4:07 PM, Benoit Perroud ben...@noisette.ch wrote: I suppose what you are looking for is an example of interacting with a java app. You should have a look at the high(er) level client hector https://github.com/rantav/hector/ You should find what you are looking for there. If you are looking for a tomcat (and .war) example, you should send an email to the tomcat mailing list. 2011/8/3 CASSANDRA learner cassandralear...@gmail.com: Hiii, Can any one pleaze send me any sample application which is (.war) implemented in java/jsp and cassandra db (Tomcat)
Re: Schema Disagreement
It means the node you ran the command against could not contact node 192.168.1.25 it's probably down. Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 14:03, Dikang Gu wrote: I followed the instructions in the FAQ, but got the following when describe cluster; Snitch: org.apache.cassandra.locator.SimpleSnitch Partitioner: org.apache.cassandra.dht.RandomPartitioner Schema versions: dd73c740-bd84-11e0--98dab94442fb: [192.168.1.28, 192.168.1.9, 192.168.1.27] UNREACHABLE: [192.168.1.25] What's the UNREACHABLE? Thanks. -- Dikang Gu 0086 - 18611140205 On Wednesday, August 3, 2011 at 11:28 AM, Jonathan Ellis wrote: Have you seen http://wiki.apache.org/cassandra/FAQ#schema_disagreement ? On Tue, Aug 2, 2011 at 10:25 PM, Dikang Gu dikan...@gmail.com wrote: I also encounter the schema disagreement in my 0.8.1 cluster today… The disagreement occurs when I create a column family using the hector api, and I found the following errors in my cassandra/system.log ERROR [pool-2-thread-99] 2011-08-03 11:21:18,051 Cassandra.java (line 3378) Internal error processing remove java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:73) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337) at org.apache.cassandra.service.StorageProxy.insertLocal(StorageProxy.java:360) at org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:241) at org.apache.cassandra.service.StorageProxy.access$000(StorageProxy.java:62) at org.apache.cassandra.service.StorageProxy$1.apply(StorageProxy.java:99) at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:210) at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:154) at org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:560) at org.apache.cassandra.thrift.CassandraServer.internal_remove(CassandraServer.java:539) at org.apache.cassandra.thrift.CassandraServer.remove(CassandraServer.java:547) at org.apache.cassandra.thrift.Cassandra$Processor$remove.process(Cassandra.java:3370) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) And when I try to decommission, I got this: ERROR [pool-2-thread-90] 2011-08-03 11:24:35,611 Cassandra.java (line 3462) Internal error processing batch_mutate java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:73) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337) at org.apache.cassandra.service.StorageProxy.insertLocal(StorageProxy.java:360) at org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:241) at org.apache.cassandra.service.StorageProxy.access$000(StorageProxy.java:62) at org.apache.cassandra.service.StorageProxy$1.apply(StorageProxy.java:99) at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:210) at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:154) at org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:560) at org.apache.cassandra.thrift.CassandraServer.internal_batch_mutate(CassandraServer.java:511) at org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:519) at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.process(Cassandra.java:3454) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) What does this mean? Thanks. -- Dikang Gu 0086 - 18611140205 On Tuesday, August 2, 2011 at 6:04 PM, aaron morton wrote: Hang on, using brain now. That is triggering a small bug in the code see https://issues.apache.org/jira/browse/CASSANDRA-2984 For not just remove the column meta data. Cheers - Aaron Morton Freelance Cassandra Developer
Re: Cassandra encountered an internal error processing this request: TApplicationError type: 6 message:Internal error
There really should be something logged at the ERROR level in the server side log, that error is raised when an unhanded exception bubbles out to the thrift layer on the server. Double check the logging is configured correctly. Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 14:19, Dikang Gu wrote: I got this error when processing a lot operations…. 2011-08-03 11:26:35,786 ERROR [com.iw.nebula.dao.simpledb.SimpleDBAdapter] - Cassandra encountered an internal error processing this request: TApplicationError type: 6 message:Internal error processing batch_mutate 2011-08-03 11:48:21,998 ERROR [com.iw.nebula.dao.simpledb.SimpleDBAdapter] - Cassandra encountered an internal error processing this request: TApplicationError type: 6 message:Internal error processing get_range_slices I did not see anything wrong in the cassandra/system.log What's your suggestions? -- Dikang Gu 0086 - 18611140205
Re: If I always send the schema change requests to one particular node in the cassandra cluster.
Yes. Only run one schema change at a time, and only do it when the cluster has converged to a single schema. The CLI checks for this. What process are you following now ? Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 16:19, Dikang Gu wrote: Hi, Can the schema disagreement problem be avoided? Thanks. -- Dikang Gu 0086 - 18611140205
Re: HOW TO select a column or all columns that start with X
and AsciiType - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 16:35, eldad87 wrote: Thank you! Will this situation work only for UTF8Type comparator? On Wed, Aug 3, 2011 at 4:50 AM, Tyler Hobbs ty...@datastax.com wrote: A minor correction: To get all columns starting with ABC_, you would set column_start=ABC_ and column_finish=ABC` (the '`' character comes after '_'), and ignore the last column in your results if it happened to be ABC`. column_finish, or the slice end in other clients, is inclusive. You could of course use ABC_~ as column_finish and avoid the check if you know that you don't have column names like ABC_~FOO that you want to include. On Tue, Aug 2, 2011 at 7:17 PM, aaron morton aa...@thelastpickle.com wrote: Yup, thats a pretty common pattern. How exactly depends on the client you are using. Say you were using pycassam, you would do a get() http://pycassa.github.com/pycassa/api/pycassa/columnfamily.html#pycassa.columnfamily.ColumnFamily.get with column_start=ABC_ , count to whatever, and column_finish not provided. You can also provide a finish and use the highest encoded character, e.g. ascii 126 is ~ so if you used column_finish = ABC_~ you would get everything that starts with ABC_ Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 09:28, Eldad Yamin wrote: Hello, I wonder if I can select a column or all columns that start with X. E.g I have columns ABC_1, ABC_2, ZZZ_1 and I want to select all columns that start with ABC_ - is that possible? Thanks! -- Tyler Hobbs Software Engineer, DataStax Maintainer of the pycassa Cassandra Python client library
Re: cassandra consistency level
In other words, if one of the nodes is down - the write (or read) will fail. No. Cassandra will always try to get a write distributed to RF nodes. The Consistency Level is how many replicas you want to be written before you accept the request was a success. e.g. with RF 3 and QUORUM you are saying wait until at least 2 replicas have completed. So you can have one node down. If the write is not replicated during the request it may eventually get there via the Read Repair, Hinted Handoff or Repair processes. If you always use QUORUM for read and write you will get consistent behaviour, see http://wiki.apache.org/cassandra/ArchitectureOverview I would recommend using QUORUM until you can find a good reason to use something else (performance is not a good reason). For you model, it sounds like you should handle each user action as a separate column. This has to do with concurrency around the update, not the consistency model. Have a look at the data modelling slides here http://www.datastax.com/events/cassandrasf2011/presentations Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 16:59, Eldad Yamin wrote: So what you're saying is that no matter what consistency level I'm using, the data will be written to all CF nodes right away, the consistency level is just for making sure that all CF nodes are UP and all data is written. In other words, if one of the nodes is down - the write (or read) will fail. I'm asking that because I'm a bit worried with consistency, for example: Every action that my client is doing is stored in a CF.x in a specific column by his user_id. I'm doing that by de-serializing the data that already found in the column, adding new data (the action), serializing and storing the data. so I'm worrying that some of the user actions will drop due low-consistency when there are lots of changes to a specific column in a sort period of time. I know that I can solve this situation in a different way by storing each action in a new column etc... but this is just an example that explain my question in a simple way. Thanks! On Wed, Aug 3, 2011 at 3:21 AM, aaron morton aa...@thelastpickle.com wrote: Not sure I understand your question exactly, but will take a shot… Writes are sent to every UP node, the consistency level is how many nodes we require to complete before we say the request completed successfully. So we also make sure that CL nodes are UP before we start the request. If you run CL ALL then Replication Factor nodes must be up for each key you are writing. With the exception of CL ONE reads are also sent to all UP replicas. Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 09:32, Eldad Yamin wrote: Is consistency level All for write actually grenty that my data is updated in all of my node? is it apply to read actions as-well? I've read it on the wiki, I just want to make sure. Thanks!
Re: Do we have any restrictions on the number of column families in a keyspace?
There were never any restrictions other than common sense. The more you make, even under 0.8, the more frequently you have to flush to disk. 0.8 automates the memory management, which used to involve random guesses, and basically makes it's usable. So I'm just saying there is a down side. If you create CF's like you would create tables in a RDBMS you will probably have some unhappy days ahead. cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 18:14, samal wrote: yes for 0.7 no for 0.8
Re: Question about eventually consistent in Cassandra
yup, my bad. A - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 19:45, Sam Overton wrote: A slight correction/clarification: 1) DeletedColumns always win. If there are two deletes the highest timestamp wins DeletedColumns only take precedence over a normal value if they have a greater (or equal) timestamp. DeletedColumns always win is only true if you're talking about two columns with identical timestamps. -- Sam Overton Acunu | http://www.acunu.com | @acunu
Re: HOW TO select a column or all columns that start with X
Thanks! On Wed, Aug 3, 2011 at 3:03 PM, aaron morton aa...@thelastpickle.comwrote: and AsciiType - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 16:35, eldad87 wrote: Thank you! Will this situation work only for UTF8Type comparator? On Wed, Aug 3, 2011 at 4:50 AM, Tyler Hobbs ty...@datastax.com wrote: A minor correction: To get all columns starting with ABC_, you would set column_start=ABC_ and column_finish=ABC` (the '`' character comes after '_'), and ignore the last column in your results if it happened to be ABC`. column_finish, or the slice end in other clients, is inclusive. You could of course use ABC_~ as column_finish and avoid the check if you know that you don't have column names like ABC_~FOO that you want to include. On Tue, Aug 2, 2011 at 7:17 PM, aaron morton aa...@thelastpickle.comwrote: Yup, thats a pretty common pattern. How exactly depends on the client you are using. Say you were using pycassam, you would do a get() http://pycassa.github.com/pycassa/api/pycassa/columnfamily.html#pycassa.columnfamily.ColumnFamily.get with column_start=ABC_ , count to whatever, and column_finish not provided. You can also provide a finish and use the highest encoded character, e.g. ascii 126 is ~ so if you used column_finish = ABC_~ you would get everything that starts with ABC_ Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 09:28, Eldad Yamin wrote: Hello, I wonder if I can select a column or all columns that start with X. E.g I have columns ABC_1, ABC_2, ZZZ_1 and I want to select all columns that start with ABC_ - is that possible? Thanks! -- Tyler Hobbs Software Engineer, DataStax http://datastax.com/ Maintainer of the pycassa http://github.com/pycassa/pycassa Cassandra Python client library
Install Cassandra on EC2
Hi, Is there any manual or important notes I should know before I try to install Cassandra on EC2? Thanks!
Re: Exception encountered during startup - org.apache.cassandra.db.marshal.MarshalException: UUIDs must be exactly 16 bytes
This is a known bug: https://issues.apache.org/jira/browse/CASSANDRA-2867 (note that the issue title is not exact, in that this is not a problem with an upgrade from 0.7 to 0.8). This will be corrected in 0.8.3 that is under a vote right now to be released. -- Sylvain On Wed, Aug 3, 2011 at 12:57 PM, jens müller jens.mueller.li...@googlemail.com wrote: Hello, I would be thankfull for any advice how to recover from the following problem: 1.) When I Start Cassandra, then I get the following exception: INFO 12:51:08,964 Heap size: 1573912576/1574961152 INFO 12:51:08,966 JNA not found. Native methods will be disabled. INFO 12:51:08,973 Loading settings from file:/usr/local/cassandra/0.8.1/conf/cassandra.yaml INFO 12:51:09,102 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap INFO 12:51:09,205 Global memtable threshold is enabled at 500MB INFO 12:51:09,206 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,211 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,212 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,214 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,215 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,216 Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable) INFO 12:51:09,251 Opening /usr/local/cassandra/0.8.1/data/data/system/NodeIdInfo-g-1 INFO 12:51:09,332 Opening /usr/local/cassandra/0.8.1/data/data/system/IndexInfo-g-1 INFO 12:51:09,343 Opening /usr/local/cassandra/0.8.1/data/data/system/Schema-g-61 INFO 12:51:09,348 Opening /usr/local/cassandra/0.8.1/data/data/system/Migrations-g-61 INFO 12:51:09,364 Opening /usr/local/cassandra/0.8.1/data/data/system/LocationInfo-g-25 INFO 12:51:09,365 Opening /usr/local/cassandra/0.8.1/data/data/system/LocationInfo-g-26 INFO 12:51:09,401 Loading schema version d4d71860-bd04-11e0--242d50cf1fd7 ERROR 12:51:09,579 Exception encountered during startup. org.apache.cassandra.db.marshal.MarshalException: UUIDs must be exactly 16 bytes at org.apache.cassandra.db.marshal.TimeUUIDType.getString(TimeUUIDType.java:117) at org.apache.cassandra.config.CFMetaData.getDefaultIndexName(CFMetaData.java:971) at org.apache.cassandra.config.CFMetaData.inflate(CFMetaData.java:381) at org.apache.cassandra.config.KSMetaData.inflate(KSMetaData.java:172) at org.apache.cassandra.db.DefsTable.loadFromStorage(DefsTable.java:99) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:479) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:139) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:315) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:80) Exception encountered during startup. org.apache.cassandra.db.marshal.MarshalException: UUIDs must be exactly 16 bytes at org.apache.cassandra.db.marshal.TimeUUIDType.getString(TimeUUIDType.java:117) at org.apache.cassandra.config.CFMetaData.getDefaultIndexName(CFMetaData.java:971) at org.apache.cassandra.config.CFMetaData.inflate(CFMetaData.java:381) at org.apache.cassandra.config.KSMetaData.inflate(KSMetaData.java:172) at org.apache.cassandra.db.DefsTable.loadFromStorage(DefsTable.java:99) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:479) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:139) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:315) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:80) I have to say, that my Linux Box crashed with an Out Of Memory Excetpion (Kernel Panic) before.. Might this be the cause? Otherwise I am really wondering, why the system is left in such an unstable state, that I event can't start cassandra anymore. 2.) What can I do to recover from this the bst way ? I want to start cassandra again, without losing data. How to do this / what to do next? Thank you very much!! Jens
Re: Install Cassandra on EC2
One thing you might want to look at is the Apache Whirr project (which is awesome by the way!). It automagically handles spinning up a cluster of resources on EC2 (or rackspace for that matter), installing and configuring cassandra, and starting it. One thing to be aware of if you go this route. By default in the yaml file all data is written under the /var folder. But on a server started by Whirr, this folder only has something like 4gb. Most of the hard disk space is under the /mnt folder. So you'll either need to change what folders are pointed to what drives (not sure if you can or not...I'm sure you could), or change the yaml file to point the /mnt folder. On Wed, Aug 3, 2011 at 6:28 AM, Eldad Yamin elda...@gmail.com wrote: Hi, Is there any manual or important notes I should know before I try to install Cassandra on EC2? Thanks! -- Thanks, John C
Re: Install Cassandra on EC2
Thanks! But I prefer to learn how to Install first - if you have any good references (I didn't find any, even general installation for a EC2/regular machine) I'm also going to try and install Solandra, I hope that Whirr will support it in the near future. On Wed, Aug 3, 2011 at 5:43 PM, John Conwell j...@iamjohn.me wrote: One thing you might want to look at is the Apache Whirr project (which is awesome by the way!). It automagically handles spinning up a cluster of resources on EC2 (or rackspace for that matter), installing and configuring cassandra, and starting it. One thing to be aware of if you go this route. By default in the yaml file all data is written under the /var folder. But on a server started by Whirr, this folder only has something like 4gb. Most of the hard disk space is under the /mnt folder. So you'll either need to change what folders are pointed to what drives (not sure if you can or not...I'm sure you could), or change the yaml file to point the /mnt folder. On Wed, Aug 3, 2011 at 6:28 AM, Eldad Yamin elda...@gmail.com wrote: Hi, Is there any manual or important notes I should know before I try to install Cassandra on EC2? Thanks! -- Thanks, John C
Cassandra and Solandra Installation guid
Hi, I'd like to get tutorials on how to install Cassandra and Solandra - I couldn't find anything helpful. In addition, how to use (index/search) Solandra tutorials will be great. Thanks!
Re: CQL: How to escape single quotes?
Hi,
thank's for your help. This does work.
Odd, when you generally escape special chars with an backslash.
Original-Nachricht
Datum: Tue, 02 Aug 2011 21:42:44 -0500
Von: Eric Evans eev...@rackspace.com
An: user@cassandra.apache.org
Betreff: Re: CQL: How to escape single quotes?
On Wed, 2011-08-03 at 03:58 +0200, Jens Hartung wrote:
Is there a way, to insert Strings with single quotes to Cassandra via
CQL?
Yes, escape them with another single quote ('aa''bb').
--
Eric Evans
eev...@rackspace.com
--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
Re: HOW TO select a column or all columns that start with X
I believe you can set start to be ABC_ and finish to be ABC_\ (for UTF8) to get everything that contains exactly ABC_ and set finish to ABC_\ to get everything that starts with ABC_. You probably want to do a simple string comparison test to verify. On Tue, Aug 2, 2011 at 6:50 PM, Tyler Hobbs ty...@datastax.com wrote: A minor correction: To get all columns starting with ABC_, you would set column_start=ABC_ and column_finish=ABC` (the '`' character comes after '_'), and ignore the last column in your results if it happened to be ABC`. column_finish, or the slice end in other clients, is inclusive. You could of course use ABC_~ as column_finish and avoid the check if you know that you don't have column names like ABC_~FOO that you want to include. On Tue, Aug 2, 2011 at 7:17 PM, aaron morton aa...@thelastpickle.comwrote: Yup, thats a pretty common pattern. How exactly depends on the client you are using. Say you were using pycassam, you would do a get() http://pycassa.github.com/pycassa/api/pycassa/columnfamily.html#pycassa.columnfamily.ColumnFamily.get with column_start=ABC_ , count to whatever, and column_finish not provided. You can also provide a finish and use the highest encoded character, e.g. ascii 126 is ~ so if you used column_finish = ABC_~ you would get everything that starts with ABC_ Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 3 Aug 2011, at 09:28, Eldad Yamin wrote: Hello, I wonder if I can select a column or all columns that start with X. E.g I have columns ABC_1, ABC_2, ZZZ_1 and I want to select all columns that start with ABC_ - is that possible? Thanks! -- Tyler Hobbs Software Engineer, DataStax http://datastax.com/ Maintainer of the pycassa http://github.com/pycassa/pycassa Cassandra Python client library
Installation Exception
Hi, I'm trying to install Cassandra on Amazon EC2 without success, this is what I did: 1. Created new Small EC2 instance (this is just for testing), running Ubuntu OS - custom AIM (ami-596f3c1c) from: http://uec-images.ubuntu.com/releases/11.04/release/ 2. Installed Java: # sudo add-apt-repository deb http://archive.canonical.com/ lucid partner # sudo apt-get update # sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts openjdk-6-jre 3. Upgraded: # sudo apt-get upgrade 4. Downloaded Cassandra: # cd /usr/src/ # sudo wget http://apache.mivzakim.net//cassandra/0.8.2/apache-cassandra-0.8.2-src.tar.gz # sudo tar xvfz apache-cassandra-* # cd apache-cassandra-* 5. Config (according to README.txt) # sudo mkdir -p /var/log/cassandra # sudo chown -R `whoami` /var/log/cassandra # sudo mkdir -p /var/lib/cassandra # sudo chown -R `whoami` /var/lib/cassandra 6. RUN CASSANDRA # bin/cassandra -f The I got Exception: ubuntu@ip-10-170-31-128:/usr/src/apache-cassandra-0.8.2-src$ bin/cassandra -f Exception in thread main java.lang.NoClassDefFoundError: org/apache/cassandra/thrift/CassandraDaemon Caused by: java.lang.ClassNotFoundException: org.apache.cassandra.thrift.CassandraDaemon at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: org.apache.cassandra.thrift.CassandraDaemon. Program will exit. Any idea what is wrong? Thanks!
Re: Installation Exception
did u compile source code? :) you have downloaded source code not binary. try with binary. On Wed, Aug 3, 2011 at 9:14 PM, Eldad Yamin elda...@gmail.com wrote: Hi, I'm trying to install Cassandra on Amazon EC2 without success, this is what I did: 1. Created new Small EC2 instance (this is just for testing), running Ubuntu OS - custom AIM (ami-596f3c1c) from: http://uec-images.ubuntu.com/releases/11.04/release/ 2. Installed Java: # sudo add-apt-repository deb http://archive.canonical.com/ lucid partner # sudo apt-get update # sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts openjdk-6-jre 3. Upgraded: # sudo apt-get upgrade 4. Downloaded Cassandra: # cd /usr/src/ # sudo wget http://apache.mivzakim.net//cassandra/0.8.2/apache-cassandra-0.8.2-src.tar.gz # sudo tar xvfz apache-cassandra-* # cd apache-cassandra-* 5. Config (according to README.txt) # sudo mkdir -p /var/log/cassandra # sudo chown -R `whoami` /var/log/cassandra # sudo mkdir -p /var/lib/cassandra # sudo chown -R `whoami` /var/lib/cassandra 6. RUN CASSANDRA # bin/cassandra -f The I got Exception: ubuntu@ip-10-170-31-128:/usr/src/apache-cassandra-0.8.2-src$ bin/cassandra -f Exception in thread main java.lang.NoClassDefFoundError: org/apache/cassandra/thrift/CassandraDaemon Caused by: java.lang.ClassNotFoundException: org.apache.cassandra.thrift.CassandraDaemon at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: org.apache.cassandra.thrift.CassandraDaemon. Program will exit. Any idea what is wrong? Thanks!
Re: Installation Exception
http://www.datastax.com/dev/blog/setting-up-a-cassandra-cluster-with-the-datastax-ami On Wed, Aug 3, 2011 at 10:44 AM, Eldad Yamin elda...@gmail.com wrote: Hi, I'm trying to install Cassandra on Amazon EC2 without success, this is what I did: Created new Small EC2 instance (this is just for testing), running Ubuntu OS - custom AIM (ami-596f3c1c) from: http://uec-images.ubuntu.com/releases/11.04/release/ Installed Java: # sudo add-apt-repository deb http://archive.canonical.com/ lucid partner # sudo apt-get update # sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts openjdk-6-jre Upgraded: # sudo apt-get upgrade Downloaded Cassandra: # cd /usr/src/ # sudo wget http://apache.mivzakim.net//cassandra/0.8.2/apache-cassandra-0.8.2-src.tar.gz # sudo tar xvfz apache-cassandra-* # cd apache-cassandra-* Config (according to README.txt) # sudo mkdir -p /var/log/cassandra # sudo chown -R `whoami` /var/log/cassandra # sudo mkdir -p /var/lib/cassandra # sudo chown -R `whoami` /var/lib/cassandra RUN CASSANDRA # bin/cassandra -f The I got Exception: ubuntu@ip-10-170-31-128:/usr/src/apache-cassandra-0.8.2-src$ bin/cassandra -f Exception in thread main java.lang.NoClassDefFoundError: org/apache/cassandra/thrift/CassandraDaemon Caused by: java.lang.ClassNotFoundException: org.apache.cassandra.thrift.CassandraDaemon at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: org.apache.cassandra.thrift.CassandraDaemon. Program will exit. Any idea what is wrong? Thanks! -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Install Cassandra on EC2
Hi Eldad, Check out http://wiki.apache.org/cassandra/CloudConfig There are a few ways listed there including a step-by-step guide. Dave Viner On Wed, Aug 3, 2011 at 7:49 AM, Eldad Yamin elda...@gmail.com wrote: Thanks! But I prefer to learn how to Install first - if you have any good references (I didn't find any, even general installation for a EC2/regular machine) I'm also going to try and install Solandra, I hope that Whirr will support it in the near future. On Wed, Aug 3, 2011 at 5:43 PM, John Conwell j...@iamjohn.me wrote: One thing you might want to look at is the Apache Whirr project (which is awesome by the way!). It automagically handles spinning up a cluster of resources on EC2 (or rackspace for that matter), installing and configuring cassandra, and starting it. One thing to be aware of if you go this route. By default in the yaml file all data is written under the /var folder. But on a server started by Whirr, this folder only has something like 4gb. Most of the hard disk space is under the /mnt folder. So you'll either need to change what folders are pointed to what drives (not sure if you can or not...I'm sure you could), or change the yaml file to point the /mnt folder. On Wed, Aug 3, 2011 at 6:28 AM, Eldad Yamin elda...@gmail.com wrote: Hi, Is there any manual or important notes I should know before I try to install Cassandra on EC2? Thanks! -- Thanks, John C
Re: Question about eventually consistent in Cassandra
What happens when DC is in different time zone so 9:00 pacific vs 11:00 Central -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Question-about-eventually-consistent-in-Cassandra-tp6646430p6649490.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: Install Cassandra on EC2
Some quick thoughts that might be helpful: - use ephemeral instances and RAID0 over the local volumes for both cassandra's data as well as the log directory. The log directory because if you crash due to heap size, the heap dump will be stored in the log directory. you don't want that to go in your root/OS partition. - probably want to stripe across AZs so that a single AZ failure doesn't affect you as much. - for seeds, it's nice to use elastic ips so that your seed configuration doesn't have to change if a node is replaced. - the ec2snitch makes it so each AZ appears as a rack wrt topology - simpler as it inspects the ec2 metadata. if you need more than one DC in your cluster (we need a second virtual DC for analytics), you'll probably want to use the property file snitch. there's a cross region ec2snitch that's coming in 1.0. would probably be good to add some ec2 specific tips in the wiki. the page that dave mentioned is a good step-by-step, but there's been a lot of community knowledge accumulated about best practices in the year since that was done. On Aug 3, 2011, at 8:28 AM, Eldad Yamin wrote: Hi, Is there any manual or important notes I should know before I try to install Cassandra on EC2? Thanks!
How to monitor progress of nodetool repair?
Running v0.8.2, I can't see how to monitor the status/progress of a nodetool repair. Any advice? The nodetool repair command from the command line is not returning, so I assume it's still running. But there's little CPU or disk activity. Using jconsole to look at the AntiEntropyStage attributes, the CompletedTasks has a non-zero value, but all the other attributes like ActiveCount, PendingTasks, CurrentlyBlockedTasks are at 0. nodetool tpstats reports 0 for active, pending, blocked for all attributes nodetool netstats has a few pending tasks occasionally, and the completed tasks are going up. system.log at INFO level has the initial Sending AEService tree message, but nothing else. -gary
Re: Installation Exception
Thanks Jonathan, I saw the EC2 AMI that was made by datastax - I prefer not to use it becuse I want to learn how to install Cassandra first. On Wed, Aug 3, 2011 at 8:03 PM, Jonathan Ellis jbel...@gmail.com wrote: http://www.datastax.com/dev/blog/setting-up-a-cassandra-cluster-with-the-datastax-ami On Wed, Aug 3, 2011 at 10:44 AM, Eldad Yamin elda...@gmail.com wrote: Hi, I'm trying to install Cassandra on Amazon EC2 without success, this is what I did: Created new Small EC2 instance (this is just for testing), running Ubuntu OS - custom AIM (ami-596f3c1c) from: http://uec-images.ubuntu.com/releases/11.04/release/ Installed Java: # sudo add-apt-repository deb http://archive.canonical.com/ lucid partner # sudo apt-get update # sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts openjdk-6-jre Upgraded: # sudo apt-get upgrade Downloaded Cassandra: # cd /usr/src/ # sudo wget http://apache.mivzakim.net//cassandra/0.8.2/apache-cassandra-0.8.2-src.tar.gz # sudo tar xvfz apache-cassandra-* # cd apache-cassandra-* Config (according to README.txt) # sudo mkdir -p /var/log/cassandra # sudo chown -R `whoami` /var/log/cassandra # sudo mkdir -p /var/lib/cassandra # sudo chown -R `whoami` /var/lib/cassandra RUN CASSANDRA # bin/cassandra -f The I got Exception: ubuntu@ip-10-170-31-128:/usr/src/apache-cassandra-0.8.2-src$ bin/cassandra -f Exception in thread main java.lang.NoClassDefFoundError: org/apache/cassandra/thrift/CassandraDaemon Caused by: java.lang.ClassNotFoundException: org.apache.cassandra.thrift.CassandraDaemon at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: org.apache.cassandra.thrift.CassandraDaemon. Program will exit. Any idea what is wrong? Thanks! -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Installation Exception
Thanks! I missed that lol! BTW, how do I compile it? Thanks! On Wed, Aug 3, 2011 at 6:51 PM, samal sa...@wakya.in wrote: did u compile source code? :) you have downloaded source code not binary. try with binary. On Wed, Aug 3, 2011 at 9:14 PM, Eldad Yamin elda...@gmail.com wrote: Hi, I'm trying to install Cassandra on Amazon EC2 without success, this is what I did: 1. Created new Small EC2 instance (this is just for testing), running Ubuntu OS - custom AIM (ami-596f3c1c) from: http://uec-images.ubuntu.com/releases/11.04/release/ 2. Installed Java: # sudo add-apt-repository deb http://archive.canonical.com/ lucid partner # sudo apt-get update # sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts openjdk-6-jre 3. Upgraded: # sudo apt-get upgrade 4. Downloaded Cassandra: # cd /usr/src/ # sudo wget http://apache.mivzakim.net//cassandra/0.8.2/apache-cassandra-0.8.2-src.tar.gz # sudo tar xvfz apache-cassandra-* # cd apache-cassandra-* 5. Config (according to README.txt) # sudo mkdir -p /var/log/cassandra # sudo chown -R `whoami` /var/log/cassandra # sudo mkdir -p /var/lib/cassandra # sudo chown -R `whoami` /var/lib/cassandra 6. RUN CASSANDRA # bin/cassandra -f The I got Exception: ubuntu@ip-10-170-31-128:/usr/src/apache-cassandra-0.8.2-src$ bin/cassandra -f Exception in thread main java.lang.NoClassDefFoundError: org/apache/cassandra/thrift/CassandraDaemon Caused by: java.lang.ClassNotFoundException: org.apache.cassandra.thrift.CassandraDaemon at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: org.apache.cassandra.thrift.CassandraDaemon. Program will exit. Any idea what is wrong? Thanks!
Planet Cassandra (an aggregation site for Cassandra News)
Greetings all, I just wanted to send a note out to let everyone know about Planet Cassandra -- an aggregation site for Cassandra news and blogs. Andrew Llavore from DataStax and I built the site. We are currently waiting for approval from the Apache Software Foundation before we publicly launch. However, in the meantime, we'd love to hear from you. If you have any favorite Cassandra-related blogs, or blogs that frequently contain quality Cassandra content, please send us the URL, so that we can contact the author about including a site feed. If you have any questions or comments, please send them to pla...@geekaustin.org. -Lynn Bender -- -Lynn Bender http://geekaustin.org http://linuxagainstpoverty.org http://twitter.com/linearb http://twitter.com/geekaustin
Re: Planet Cassandra (an aggregation site for Cassandra News)
Awesome, great news! On Wed, Aug 3, 2011 at 11:53 AM, Lynn Bender line...@gmail.com wrote: Greetings all, I just wanted to send a note out to let everyone know about Planet Cassandra -- an aggregation site for Cassandra news and blogs. Andrew Llavore from DataStax and I built the site. We are currently waiting for approval from the Apache Software Foundation before we publicly launch. However, in the meantime, we'd love to hear from you. If you have any favorite Cassandra-related blogs, or blogs that frequently contain quality Cassandra content, please send us the URL, so that we can contact the author about including a site feed. If you have any questions or comments, please send them to pla...@geekaustin.org. -Lynn Bender -- -Lynn Bender http://geekaustin.org http://linuxagainstpoverty.org http://twitter.com/linearb http://twitter.com/geekaustin
Re: cassandra server disk full
The last patch on that ticket is what we're running in prod. Its working well for us with disk_failure_mode: readwrite. In the case of filesystem errors the node shuts off thrift and gossip. While the gossip is propagating we can continue to serve some reads out of the caches. -ryan On Tue, Aug 2, 2011 at 9:27 AM, Jim Ancona j...@anconafamily.com wrote: On Mon, Aug 1, 2011 at 6:12 PM, Ryan King r...@twitter.com wrote: On Fri, Jul 29, 2011 at 12:02 PM, Chris Burroughs chris.burrou...@gmail.com wrote: On 07/25/2011 01:53 PM, Ryan King wrote: Actually I was wrong– our patch will disable gosisp and thrift but leave the process running: https://issues.apache.org/jira/browse/CASSANDRA-2118 If people are interested in that I can make sure its up to date with our latest version. Thanks Ryan. /me expresses interest. /me too! Zombie nodes when the file system does something interesting are not fun. In our experience this only gets triggered on hardware failures that would otherwise seriously degrade the performance or cause lots of errors. After the nodes traffic coalesces we get an alert which we can then deal with. -ryan
Solandra
Hello, I have a cluster of 3 Cassandra nodes and I would like to start using Solandra. 1. How can I install Solandra and make use the existing nodes? 2. Will it be better to install Solandra on a new node and add it to the existing cluster? 3. How Solandra index, does it operate automatically or I need to tell Solandra to index CF.keys every time a new key is create or update? Thanks!
Re: Install Cassandra on EC2
Pre build AMI here http://www.datastax.com/dev/blog/setting-up-a-cassandra-cluster-with-the-datastax-ami Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 4 Aug 2011, at 03:24, Jeremy Hanna wrote: Some quick thoughts that might be helpful: - use ephemeral instances and RAID0 over the local volumes for both cassandra's data as well as the log directory. The log directory because if you crash due to heap size, the heap dump will be stored in the log directory. you don't want that to go in your root/OS partition. - probably want to stripe across AZs so that a single AZ failure doesn't affect you as much. - for seeds, it's nice to use elastic ips so that your seed configuration doesn't have to change if a node is replaced. - the ec2snitch makes it so each AZ appears as a rack wrt topology - simpler as it inspects the ec2 metadata. if you need more than one DC in your cluster (we need a second virtual DC for analytics), you'll probably want to use the property file snitch. there's a cross region ec2snitch that's coming in 1.0. would probably be good to add some ec2 specific tips in the wiki. the page that dave mentioned is a good step-by-step, but there's been a lot of community knowledge accumulated about best practices in the year since that was done. On Aug 3, 2011, at 8:28 AM, Eldad Yamin wrote: Hi, Is there any manual or important notes I should know before I try to install Cassandra on EC2? Thanks!
Re: Write everywhere, read anywhere
On Wed, Aug 3, 2011 at 4:00 PM, Philippe watche...@gmail.com wrote: Hello, I have a 3-node, RF=3, cluster configured to write at CL.ALL and read at CL.ONE. When I take one of the nodes down, writes fail which is what I expect. When I run a repair, I see data being streamed from those column families... that I didn't expect. How can the nodes diverge ? Does this mean that reading at CL.ONE may return inconsistent data ? we abort the mutation before hand when there are enough replicas alive. If a mutation went through and in the middle of it a replica goes down, in that case you can write to some nodes and the request will Timeout. In that case the CL.ONE may return inconsistence data. Question 2 : I've doing this rather than CL.QUORUM because I've been expecting CL.ONE to return data faster than CL.QUORUM. Is that a good assumption ? Yes, it's ok for writes to be down for a while. When you hit a node that own the piece of data, CL.ONE will be faster as you don't have to wait for a read across the network to reach another node. For CL.QUORUM we fire reads in parallel to all the replicas and wait until completing quorum. If I'm not wrong, in some cases the difference may be negligible for CL.ONE and CL.QUORUM when you hit a coordinator that doesn't own the data since you are going over the network anyway (assuming all nodes take the same time to reply) Thanks
Re: HOW TO select a column or all columns that start with X
It seems to me that your column name consists of two components. If you have the luxury to upgrade your cassandra to 0.8.1+, I think you can think about using the composite type/column. Conceptually, it might suit your use case better. On Wed, Aug 3, 2011 at 5:28 AM, Eldad Yamin elda...@gmail.com wrote: Hello, I wonder if I can select a column or all columns that start with X. E.g I have columns ABC_1, ABC_2, ZZZ_1 and I want to select all columns that start with ABC_ - is that possible? Thanks!
Re: Input on difficult migration issue from 0.7.4 to 0.8.2
Hi Johnathan, One more question. I'm experimenting with the chef recipes for brisk. Upon the initial installation, the brisk recipe removes the existing system Keyspace if it's present. The new cluster will not have the Spidertracks keyspace in it's meta data, and will be a completely vanilla installation. Can I leave the SSTable files in place, and create the keyspace and column families via the CLI without needing to restart the whole cluster? I.E, when a CF is added to the schema, if an SSTable for that CF exists, is it merged with the commitlog in the next compaction, and used for reads or is it ignored? Thanks, Todd On Tue, 2011-08-02 at 22:17 -0500, Jonathan Ellis wrote: Yes, that should work. For better/worse, CF metadata is stored in the system schema, not individual sstables. On Tue, Aug 2, 2011 at 10:00 PM, Todd Nine t...@spidertracks.com wrote: Hi Johnathan. Are these the correct steps for the upgrade? 1. Copy SSTable from 0.7.4 snapshot 2. Drop CFS in all nodes in 0.7.4 cluster 3. Upgrade to 0.8.2 and start cluster (with no clients connected) 4. Create CFs using cli in cluster 5. Shutdown all nodes 6. Move the SSTables into place on all the nodes 7. Restart the nodes This would definitely be my preferred approach, however I wasn't sure if SSTables stored the CF meta data. Thanks, Todd On Tue, 2011-08-02 at 21:34 -0500, Jonathan Ellis wrote: If the bytes are compatible why not just use the existing sstable files after redefining the schema? On Tue, Aug 2, 2011 at 8:06 PM, Todd Nine t...@spidertracks.com wrote: Hi all, I seem to have backed myself into a corner and I cannot easily upgrade from a custom 0.7.4 installation to the default 0.8.2. Any help I could get would be greatly appreciated. Below is an outline of the problem. Current installation: 0.7.4 with Ed Anuff's custom composite comparators. https://github.com/riptano/hector-composite/commit/74407df3ef97c3fa90eae9f68d84b50be3d9938e CF Comprator Type : DynamicComposite New installation: 0.8.2 with Casasndra's native DynamicComposite column time. CF Comparator Type: DynamicComposite(a=AsciiType,b=BytesType,i=IntegerType,x=LexicalUUIDType,l=LongType,t=TimeUUIDType,s=UTF8Type,u=UUIDType,A=AsciiType(reversed=true),B=BytesType(reversed=true),I=IntegerType(reversed=true),X=LexicalUUIDType(reversed=true),L=LongType(reversed=true),T=TimeUUIDType(reversed=true),S=UTF8Type(reversed=true),U=UUIDType(reversed=true)) I cannot simply upgrade Cassandra, this fails because the comparator is incorrectly defined for version 0.8.1 and on. My issue is that the column family definition has changed, the bytes that are stored are compatible, I simply need to re-define the CF and migrate the column data over. Initially I was thinking I could perform a read from the rows and columns from a single 0.7 node, then insert them into the 0.8 cluster, however I cannot have 2 different versions of the thrift API running in the same java JVM due to namespace conflicts. Idea is to perform the following steps, if anyone has any better suggestions, they would be greatly appreciated. 1. Back up all my CFs that use the dynamic composite, and copy the SSTable out of my Keyspace's data directory to tmp 2. Drop all CFs that use dynamic composite 3. Re-create the CFs with the new comparator definition 4. Using an external program directly read the 0.7 SSTables (without starting the cassandra daemon) and insert the rows and columns into the 0.8 cluster via thrift. Can anyone point me at a good example for reading rows and columns directly from the SSTables without using the thrift api? Thanks, Todd
Re: Input on difficult migration issue from 0.7.4 to 0.8.2
You'd have to check ColumnFamilyStore.open to see for sure. It might work that way, but if it does it's an accident -- seems totally fair for it to assume that there are no SSTables for a newly created CF. On Wed, Aug 3, 2011 at 9:25 PM, Todd Nine t...@spidertracks.com wrote: ** Hi Johnathan, One more question. I'm experimenting with the chef recipes for brisk. Upon the initial installation, the brisk recipe removes the existing system Keyspace if it's present. The new cluster will not have the Spidertracks keyspace in it's meta data, and will be a completely vanilla installation. Can I leave the SSTable files in place, and create the keyspace and column families via the CLI without needing to restart the whole cluster? I.E, when a CF is added to the schema, if an SSTable for that CF exists, is it merged with the commitlog in the next compaction, and used for reads or is it ignored? Thanks, Todd On Tue, 2011-08-02 at 22:17 -0500, Jonathan Ellis wrote: Yes, that should work. For better/worse, CF metadata is stored in the system schema, not individual sstables. On Tue, Aug 2, 2011 at 10:00 PM, Todd Nine t...@spidertracks.com wrote: Hi Johnathan. Are these the correct steps for the upgrade? 1. Copy SSTable from 0.7.4 snapshot 2. Drop CFS in all nodes in 0.7.4 cluster 3. Upgrade to 0.8.2 and start cluster (with no clients connected) 4. Create CFs using cli in cluster 5. Shutdown all nodes 6. Move the SSTables into place on all the nodes 7. Restart the nodes This would definitely be my preferred approach, however I wasn't sure if SSTables stored the CF meta data. Thanks, Todd On Tue, 2011-08-02 at 21:34 -0500, Jonathan Ellis wrote: If the bytes are compatible why not just use the existing sstable files after redefining the schema? On Tue, Aug 2, 2011 at 8:06 PM, Todd Nine t...@spidertracks.com wrote: Hi all, I seem to have backed myself into a corner and I cannot easily upgrade from a custom 0.7.4 installation to the default 0.8.2. Any help I could get would be greatly appreciated. Below is an outline of the problem. Current installation: 0.7.4 with Ed Anuff's custom composite comparators. https://github.com/riptano/hector-composite/commit/74407df3ef97c3fa90eae9f68d84b50be3d9938e CF Comprator Type : DynamicComposite New installation: 0.8.2 with Casasndra's native DynamicComposite column time. CF Comparator Type: DynamicComposite(a=AsciiType,b=BytesType,i=IntegerType,x=LexicalUUIDType,l=LongType,t=TimeUUIDType,s=UTF8Type,u=UUIDType,A=AsciiType(reversed=true),B=BytesType(reversed=true),I=IntegerType(reversed=true),X=LexicalUUIDType(reversed=true),L=LongType(reversed=true),T=TimeUUIDType(reversed=true),S=UTF8Type(reversed=true),U=UUIDType(reversed=true)) I cannot simply upgrade Cassandra, this fails because the comparator is incorrectly defined for version 0.8.1 and on. My issue is that the column family definition has changed, the bytes that are stored are compatible, I simply need to re-define the CF and migrate the column data over. Initially I was thinking I could perform a read from the rows and columns from a single 0.7 node, then insert them into the 0.8 cluster, however I cannot have 2 different versions of the thrift API running in the same java JVM due to namespace conflicts. Idea is to perform the following steps, if anyone has any better suggestions, they would be greatly appreciated. 1. Back up all my CFs that use the dynamic composite, and copy the SSTable out of my Keyspace's data directory to tmp 2. Drop all CFs that use dynamic composite 3. Re-create the CFs with the new comparator definition 4. Using an external program directly read the 0.7 SSTables (without starting the cassandra daemon) and insert the rows and columns into the 0.8 cluster via thrift. Can anyone point me at a good example for reading rows and columns directly from the SSTables without using the thrift api? Thanks, Todd -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
