unsubscribe
Unsubscribe This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
Unsibscribe
Unsubscribe This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
unsubscribe
Leonid Ilyevsky Moon Capital Management, LP 499 Park Avenue New York, NY 10022 P: (212) 652-4586 F: (212) 652-4501 E: lilyev...@mooncapital.com [cid:image001.png@01CDF3EE.E9EA60F0] This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. inline: image001.png
Cannot create CF in Cassandra 1.1.2
My cassandra used to work fine, now it does not create a table. No error message, but the table is not there. I execute create table statement many times, the same thing. When I do it, I see in the system.log the the lines below. What can be the problem? Pre-history: before I noticed it is broken, I went through some issues related to the lack of space, so I was adding more storage and restarting both servers (I have my thing partitioned between 2 nodes). Is there any workaround to bring it back to normal, besides cleaning it all up and starting from scratch? = INFO [MigrationStage:1] 2012-07-31 13:51:20,090 ColumnFamilyStore.java (line 643) Enqueuing flush of Memtable-schema_columnfamilies@766316124(1520/1900 serialized/live bytes, 20 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,091 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@766316124(1520/1900 serialized/live bytes, 20 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,155 Memtable.java (line 307) Completed flushing /Data_Store1/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hd-54-Data.db (1575 bytes) for commitlog position ReplayPosition(segmentId=3346642617210212, position=12976) INFO [MigrationStage:1] 2012-07-31 13:51:20,156 ColumnFamilyStore.java (line 643) Enqueuing flush of Memtable-schema_columns@2106973490(2048/2560 serialized/live bytes, 30 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,157 Memtable.java (line 266) Writing Memtable-schema_columns@2106973490(2048/2560 serialized/live bytes, 30 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,211 Memtable.java (line 307) Completed flushing /Data_Store1/cassandra/data/system/schema_columns/system-schema_columns-hd-54-Data.db (2111 bytes) for commitlog position ReplayPosition(segmentId=3346642617210212, position=12976) This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Cannot create CF in Cassandra 1.1.2
No, I am using cql 3, but it should not matter. The point is, it used to work, and I am sure that if I just clean up the whole thing and start from scratch, it will work again. Something got corrupted. But I will give a shot to the cli, just to rule out any cql 3 issues. -Original Message- From: rohit bhatia [mailto:rohit2...@gmail.com] Sent: Tuesday, July 31, 2012 2:40 PM To: user@cassandra.apache.org Subject: Re: Cannot create CF in Cassandra 1.1.2 I believe u r using cassandra-cli. Please use help create for the proper syntax.. See http://wiki.apache.org/cassandra/DataModel/ for cassandra datamodel. Also, the flushing events are expected. For example run create keyspace name1 , use name1 , create column family name2 This should work with default settings.. On Tue, Jul 31, 2012 at 11:34 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: My cassandra used to work fine, now it does not create a table. No error message, but the table is not there. I execute create table statement many times, the same thing. When I do it, I see in the system.log the the lines below. What can be the problem? Pre-history: before I noticed it is broken, I went through some issues related to the lack of space, so I was adding more storage and restarting both servers (I have my thing partitioned between 2 nodes). Is there any workaround to bring it back to normal, besides cleaning it all up and starting from scratch? = INFO [MigrationStage:1] 2012-07-31 13:51:20,090 ColumnFamilyStore.java (line 643) Enqueuing flush of Memtable-schema_columnfamilies@766316124(1520/1900 serialized/live bytes, 20 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,091 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@766316124(1520/1900 serialized/live bytes, 20 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,155 Memtable.java (line 307) Completed flushing /Data_Store1/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hd-54-Data.db (1575 bytes) for commitlog position ReplayPosition(segmentId=3346642617210212, position=12976) INFO [MigrationStage:1] 2012-07-31 13:51:20,156 ColumnFamilyStore.java (line 643) Enqueuing flush of Memtable-schema_columns@2106973490(2048/2560 serialized/live bytes, 30 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,157 Memtable.java (line 266) Writing Memtable-schema_columns@2106973490(2048/2560 serialized/live bytes, 30 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,211 Memtable.java (line 307) Completed flushing /Data_Store1/cassandra/data/system/schema_columns/system-schema_columns-hd-54-Data.db (2111 bytes) for commitlog position ReplayPosition(segmentId=3346642617210212, position=12976) This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Cannot create CF in Cassandra 1.1.2
Now I got tired of this, so I cleaned up everything and started from the empty keyspace. Everything is ok so far. How can I prevent this from happening in the future? -Original Message- From: Leonid Ilyevsky [mailto:lilyev...@mooncapital.com] Sent: Tuesday, July 31, 2012 3:11 PM To: 'user@cassandra.apache.org' Subject: RE: Cannot create CF in Cassandra 1.1.2 Yes, I tried it now with CLI, the same thing. No error, it says that schema agrees across the cluster, but when I call describe, the new column family is not there. Another proof is, I can repeat the same create statement multiple times, and it does not complain that it exists already. I am positive that my data is corrupted in some way, so the question is, how do I repair it. -Original Message- From: Leonid Ilyevsky [mailto:lilyev...@mooncapital.com] Sent: Tuesday, July 31, 2012 2:55 PM To: 'user@cassandra.apache.org' Subject: RE: Cannot create CF in Cassandra 1.1.2 No, I am using cql 3, but it should not matter. The point is, it used to work, and I am sure that if I just clean up the whole thing and start from scratch, it will work again. Something got corrupted. But I will give a shot to the cli, just to rule out any cql 3 issues. -Original Message- From: rohit bhatia [mailto:rohit2...@gmail.com] Sent: Tuesday, July 31, 2012 2:40 PM To: user@cassandra.apache.org Subject: Re: Cannot create CF in Cassandra 1.1.2 I believe u r using cassandra-cli. Please use help create for the proper syntax.. See http://wiki.apache.org/cassandra/DataModel/ for cassandra datamodel. Also, the flushing events are expected. For example run create keyspace name1 , use name1 , create column family name2 This should work with default settings.. On Tue, Jul 31, 2012 at 11:34 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: My cassandra used to work fine, now it does not create a table. No error message, but the table is not there. I execute create table statement many times, the same thing. When I do it, I see in the system.log the the lines below. What can be the problem? Pre-history: before I noticed it is broken, I went through some issues related to the lack of space, so I was adding more storage and restarting both servers (I have my thing partitioned between 2 nodes). Is there any workaround to bring it back to normal, besides cleaning it all up and starting from scratch? = INFO [MigrationStage:1] 2012-07-31 13:51:20,090 ColumnFamilyStore.java (line 643) Enqueuing flush of Memtable-schema_columnfamilies@766316124(1520/1900 serialized/live bytes, 20 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,091 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@766316124(1520/1900 serialized/live bytes, 20 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,155 Memtable.java (line 307) Completed flushing /Data_Store1/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hd-54-Data.db (1575 bytes) for commitlog position ReplayPosition(segmentId=3346642617210212, position=12976) INFO [MigrationStage:1] 2012-07-31 13:51:20,156 ColumnFamilyStore.java (line 643) Enqueuing flush of Memtable-schema_columns@2106973490(2048/2560 serialized/live bytes, 30 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,157 Memtable.java (line 266) Writing Memtable-schema_columns@2106973490(2048/2560 serialized/live bytes, 30 ops) INFO [FlushWriter:5] 2012-07-31 13:51:20,211 Memtable.java (line 307) Completed flushing /Data_Store1/cassandra/data/system/schema_columns/system-schema_columns-hd-54-Data.db (2111 bytes) for commitlog position ReplayPosition(segmentId=3346642617210212, position=12976) This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any
Batch update efficiency with composite key
I have a question about efficiency of updates to a CF with composite key. Let say I have 100 of logical rows to update, and they all belong to the same physical wide row. In my naïve understanding (correct me if I am wrong), in order to update a logical row, Cassandra has to retrieve the whole physical row, add columns to it, and put it back. So I put all my 100 updates in a batch and send it over. Would Cassandra be smart enough to recognize that they all belong to one physical row, retrieve it once, do all the updates and put it back once? Is my batch thing even relevant in this case? What happens if I just send updates one by one? I want to understand why I should use batches. I don't really care about one timestamp for all records, I only care about efficiency. So I thought, I want to at least save on the number of remote calls, but I also wonder what happens on Cassandra side. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
How to speed up data loading
I am loading a large set of data into a CF with composite key. The load is going pretty slow, hundreds or even thousands times slower than it would do in RDBMS. I have a choice of how granular my physical key (the first component of the primary key) is, this way I can balance between smaller rows and too many keys vs. wide rows and fewer keys. What are the guidelines about this? How the width of the physical row affects the speed of load? I see that Cassandra is doing a lot of processing behind the scene, even when I kill the client, the server is still consuming a lot of CPU for a long time. What else should I look at ? Anything in configuration? This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: help using org.apache.cassandra.cql3
I see. The reason I looked at that package was, I need to use the batch feature, and I could not make it work using thrift with the CF having composite key. It worked fine with the simple key, but not composite, I was getting an error while trying to do the update. Sylvain suggested (in reply to my other posting) that I use cql3 batch statement, but I am not sure how to do it efficiently from Java. Can batch statement be prepared? Is it OK to put 1 of update statements in one batch, with 5 question marks in it? The set that many variables? Maybe I can try small example first, just to see if it works at all. From: Derek Williams [mailto:de...@fyrie.net] Sent: Tuesday, July 10, 2012 7:19 PM To: user@cassandra.apache.org Subject: Re: help using org.apache.cassandra.cql3 On Tue, Jul 10, 2012 at 3:04 PM, Leonid Ilyevsky lilyev...@mooncapital.commailto:lilyev...@mooncapital.com wrote: I am trying to use the org.apache.cassandra.cql3 package. Having problem connecting to the server using ClientState. I was not sure what to put in the credentials map (I did not set any users/passwords on my server), so I tried setting empty strings for “username” and “password”, setting them to bogus values, passing null to the login method – there was no difference. It does not complain at the login(), but then it complains about setKeyspace(my keyspace), saying that the specified keyspace does not exist (it obviously does exist). The configuration was loaded from cassandra.yaml used by the server. I did not have any problem like this when I used org.apache.cassandra.thrift.Cassandra.Client . What am I doing wrong? I think that package just contains server classes. Everything you need should be in org.apache.cassandra.thrift. To use cql3 I just use the client methods 'execute_cql_query', 'prepare_cql_query' and 'execute_prepared_cql_query', after setting cql version to '3.0.0'. -- Derek Williams This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (“Moon Capital”). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: help using org.apache.cassandra.cql3
Thanks Sylvain, I actually tried the prepared batch, works fine. I did the 1000 rows in one batch, 20 columns each, and it was good. Then I tried 1, and it still works, I am going to measure which way it is faster overall. -Original Message- From: Sylvain Lebresne [mailto:sylv...@datastax.com] Sent: Wednesday, July 11, 2012 9:32 AM To: user@cassandra.apache.org Subject: Re: help using org.apache.cassandra.cql3 When I said to use the BATCH statement I mean't using a query that is a BATCH statement, so something like: BEGIN BATCH INSERT ...; INSERT ...; ... APPLY BATCH; If you want to that from java, you will want to look at the jdbc driver (http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/), though I don't know what is the status of the support for CQL3. On Wed, Jul 11, 2012 at 2:18 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: Is it OK to put 1 of update statements in one batch, with 5 question marks in it? The set that many variables? Yes batch statement can be prepared and in theory there isn't much limit on the number of update statement (nor question marks) you can put in one batch. However, the way C* work best is if you do reasonably sized batches. It's even more true for CQL in the sense that by using a huge batch statement you'll pay the parsing. So you probably want to prepare one batch statement with a reasonable number of statement in it (you'll have to test to find number that give the best performances, but I would typically start with say 50-100 and see if the performance are good enough) and reuse that to insert the data. The other reason why breaking the insert into smallish batches is a good idea is that it allows you to parallelize the insert using multiple threads. And you need to parallelize if you want to get the best out of C*. -- Sylvain Maybe I can try small example first, just to see if it works at all. From: Derek Williams [mailto:de...@fyrie.net] Sent: Tuesday, July 10, 2012 7:19 PM To: user@cassandra.apache.org Subject: Re: help using org.apache.cassandra.cql3 On Tue, Jul 10, 2012 at 3:04 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: I am trying to use the org.apache.cassandra.cql3 package. Having problem connecting to the server using ClientState. I was not sure what to put in the credentials map (I did not set any users/passwords on my server), so I tried setting empty strings for username and password, setting them to bogus values, passing null to the login method - there was no difference. It does not complain at the login(), but then it complains about setKeyspace(my keyspace), saying that the specified keyspace does not exist (it obviously does exist). The configuration was loaded from cassandra.yaml used by the server. I did not have any problem like this when I used org.apache.cassandra.thrift.Cassandra.Client . What am I doing wrong? I think that package just contains server classes. Everything you need should be in org.apache.cassandra.thrift. To use cql3 I just use the client methods 'execute_cql_query', 'prepare_cql_query' and 'execute_prepared_cql_query', after setting cql version to '3.0.0'. -- Derek Williams This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax
RE: Dynamic CF
Thanks Sylvain, this is useful. So I guess, in the batch_mutate call, in the map that I pass to it, only the first element of the composite key should be used as a key (because it is the real key), and the other parts of the key should be passed as regular columns? Is this correct? While I am waiting for your confirmation, I am going to try it. -Original Message- From: Sylvain Lebresne [mailto:sylv...@datastax.com] Sent: Tuesday, July 10, 2012 8:24 AM To: user@cassandra.apache.org Subject: Re: Dynamic CF On Fri, Jul 6, 2012 at 10:49 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: At this point I am really confused about what direction Cassandra is going. CQL 3 has the benefit of composite keys, but no dynamic columns. I thought, the whole point of Cassandra was to provide dynamic tables. CQL3 absolutely provide dynamic tables/wide rows, the syntax is just different. The typical example for wide rows is a time serie, for instance keeping all the events for a given event_kind in the same C* row ordered by time. You declare that in CQL3 using: CREATE TABLE events ( event_kind text, time timestamp, event_name text, event_details text, PRIMARY KEY (event_kind, time) ) The important part in such definition is that one CQL row (i.e a given event_kind, time, event_name, even_details) does not map to an internal Cassandra row. More precisely, all events sharing the same event_kind will be in the same internal row. This is a wide row/dynamic table in the sense of thrift. I need to have a huge table to store market quotes, and be able to query it by name and timestamp (t1 = t = t2), therefore I wanted the composite key. Loading data to such table using prepared statements (CQL 3-based) was very slow, because it makes a server call for each row. You should use a BATCH statement which is the equivalent to batch_mutate. -- Sylvain This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Dynamic CF
(); m.setColumn_or_supercolumn(cc); columnsMutations.add(m); // Inner mutation map mutations.put(testtable2, columnsMutations); // outer map : use the partition key mutationMap.put(keyBuffer, mutations); // Execute client.batch_mutate(mutationMap, ConsistencyLevel.ANY); } } -Original Message- From: Sylvain Lebresne [mailto:sylv...@datastax.com] Sent: Tuesday, July 10, 2012 10:37 AM To: user@cassandra.apache.org Subject: Re: Dynamic CF On Tue, Jul 10, 2012 at 4:17 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: So I guess, in the batch_mutate call, in the map that I pass to it, only the first element of the composite key should be used as a key (because it is the real key), and the other parts of the key should be passed as regular columns? Is this correct? While I am waiting for your confirmation, I am going to try it. I would really advise you to use the BATCH statement of CQL3 rather than the thrift batch_mutate call. If only because until https://issues.apache.org/jira/browse/CASSANDRA-4377 is resolved it won't work at all, but also because the whole point of CQL3 is to hide that kind of complexity. -- Sylvain -Original Message- From: Sylvain Lebresne [mailto:sylv...@datastax.com] Sent: Tuesday, July 10, 2012 8:24 AM To: user@cassandra.apache.org Subject: Re: Dynamic CF On Fri, Jul 6, 2012 at 10:49 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: At this point I am really confused about what direction Cassandra is going. CQL 3 has the benefit of composite keys, but no dynamic columns. I thought, the whole point of Cassandra was to provide dynamic tables. CQL3 absolutely provide dynamic tables/wide rows, the syntax is just different. The typical example for wide rows is a time serie, for instance keeping all the events for a given event_kind in the same C* row ordered by time. You declare that in CQL3 using: CREATE TABLE events ( event_kind text, time timestamp, event_name text, event_details text, PRIMARY KEY (event_kind, time) ) The important part in such definition is that one CQL row (i.e a given event_kind, time, event_name, even_details) does not map to an internal Cassandra row. More precisely, all events sharing the same event_kind will be in the same internal row. This is a wide row/dynamic table in the sense of thrift. I need to have a huge table to store market quotes, and be able to query it by name and timestamp (t1 = t = t2), therefore I wanted the composite key. Loading data to such table using prepared statements (CQL 3-based) was very slow, because it makes a server call for each row. You should use a BATCH statement which is the equivalent to batch_mutate. -- Sylvain This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Dynamic CF
I see now there is a package org.apache.cassandra.cql3.statements, with
BatchStatement class. Is this what I should use?
-Original Message-
From: Leonid Ilyevsky [mailto:lilyev...@mooncapital.com]
Sent: Tuesday, July 10, 2012 11:45 AM
To: user@cassandra.apache.org
Subject: RE: Dynamic CF
I see. I actually tried it, and it consistently throws an exception. Below is
my test code. I have two tests; test1 is for the composite key case, and test2
is for the simple key. The test2 works fine, while test1 gives me:
Exception in thread main InvalidRequestException(why:Not enough bytes to read
value of component 0)
at
org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20253)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:922)
at
org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:908)
at com.moon.cql.BatchTest.test1(BatchTest.java:99)
at com.moon.cql.BatchTest.main(BatchTest.java:45)
So you suggest to use BATCH statement. Since I do it from Java, it means
creating a huge string (I may need to update thousands records at once), and
executing it. Does it even make sense? Why is this going to be any better than
simply execute prepared statement multiple times? The only thing it does is
reduce number of calls to the server, but I have to figure out if this is the
bottle neck I need to optimize.
Or maybe I need to break all my updates in a number of batches.
By the way, can a batch statement be prepared? With thousands of question marks
in it?
public class BatchTest {
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws TTransportException,
InvalidRequestException, TException, UnavailableException,
TimedOutException {
String host = args[0];
int port = Integer.parseInt(args[1]);
test1(host, port);
//test2(host, port);
}
private static void test1(String host, int port) throws TTransportException,
InvalidRequestException, TException, UnavailableException,
TimedOutException {
TTransport transport =
new TFramedTransport(new org.apache.thrift.transport.TSocket(
host, port));
transport.open();
TProtocol protocol = new TBinaryProtocol(transport);
Cassandra.Client client = new Cassandra.Client(protocol);
client.set_cql_version(3.0.0);
client.set_keyspace(test);
MapByteBuffer, MapString, ListMutation mutationMap =
new HashMap();
MapString, ListMutation mutations = new HashMap();
ListMutation columnsMutations = new ArrayList();
// key
ByteBuffer keyBuffer = AsciiType.instance.decompose(KEY1);
// key1 as column
Column key1 = new Column();
key1.setName(key1.getBytes());
key1.setValue(LongType.instance.decompose(System.nanoTime()));
key1.setTimestamp(System.currentTimeMillis());
ColumnOrSuperColumn cc = new ColumnOrSuperColumn();
cc.setColumn(key1);
Mutation m = new Mutation();
m.setColumn_or_supercolumn(cc);
columnsMutations.add(m);
// value column
Column value = new Column();
value.setName(value.getBytes());
value.setValue(DoubleType.instance.decompose(5.3));
value.setTimestamp(System.currentTimeMillis());
cc = new ColumnOrSuperColumn();
cc.setColumn(value);
m = new Mutation();
m.setColumn_or_supercolumn(cc);
columnsMutations.add(m);
// Inner mutation map
mutations.put(testtable1, columnsMutations);
// outer map : use the partition key
mutationMap.put(keyBuffer, mutations);
// Execute
client.batch_mutate(mutationMap, ConsistencyLevel.ANY);
}
private static void test2(String host, int port) throws
TTransportException,
InvalidRequestException, TException, UnavailableException,
TimedOutException {
TTransport transport =
new TFramedTransport(new org.apache.thrift.transport.TSocket(
host, port));
transport.open();
TProtocol protocol = new TBinaryProtocol(transport);
Cassandra.Client client = new Cassandra.Client(protocol);
client.set_cql_version(3.0.0);
client.set_keyspace(test);
MapByteBuffer, MapString, ListMutation mutationMap =
new HashMap();
MapString, ListMutation mutations = new HashMap();
ListMutation columnsMutations = new ArrayList();
// key
ByteBuffer keyBuffer = AsciiType.instance.decompose(KEY1);
// value column
Column value = new Column
help using org.apache.cassandra.cql3
I am trying to use the org.apache.cassandra.cql3 package. Having problem connecting to the server using ClientState. I was not sure what to put in the credentials map (I did not set any users/passwords on my server), so I tried setting empty strings for username and password, setting them to bogus values, passing null to the login method - there was no difference. It does not complain at the login(), but then it complains about setKeyspace(my keyspace), saying that the specified keyspace does not exist (it obviously does exist). The configuration was loaded from cassandra.yaml used by the server. I did not have any problem like this when I used org.apache.cassandra.thrift.Cassandra.Client . What am I doing wrong? Appreciate your help, Leonid This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Composite key in thrift java api
Thanks Aaron, I guess, you suggest I just borrow the logic from that CompositeType.java, the decompose method, to comply with the protocol. I thought I can just call it, but I don't see the decompose(Object... objects) in the current 1.1.2 version, so probably your snippet is from the older version. On the other hand, in the new version I see CompositeType.Builder inner class, looks like I should use that. Could you please confirm? Thanks, Leonid From: aaron morton [mailto:aa...@thelastpickle.com] Sent: Thursday, July 05, 2012 7:10 PM To: user@cassandra.apache.org Subject: Re: Composite key in thrift java api I would really prefer to do it in Cassandra itself, See https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/marshal/CompositeType.java Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at 10:40 AM, Leonid Ilyevsky wrote: I need to create a ByteBuffer instance containing the proper composite key, based on the values of the components of the key. I am going to use it for update operation. I tried to simply concatenate the buffers corresponding to the components, but I am not sure this is correct, because I am getting exception that comes from the server : InvalidRequestException(why:Not enough bytes to read value of component 0) In the server log I see this: org.apache.thrift.transport.TTransportException: Cannot read. Remote side has closed. Tried to read 4 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.) (I believe here when it says server side it actually means client, because it is the server's log). Seems like the buffer that my client sends is too short. I suspect there is a way in thrift to do it properly, but I don't know how. Looks like Hector has a Composite class that maybe can help, but at this point I would really prefer to do it in Cassandra itself, without Hector. Thanks! Leonid This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
Dynamic CF
Can I create a dynamic Cf using cql3? What is the syntax? Here is what I tried: cqlsh:test create table mytest ( ... asset ascii, datetime int, count int, primary key(asset, datetime) ) ... with default_validation_class = double; Bad Request: default_validation_class is not a valid keyword argument for CREATE TABLE This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Dynamic CF
Thanks Jeremy, but this doesn't work for me. I am using cql3, because I need new features like composite keys. The manual you pointed to is for 2.0. I have suspicion that cql3 does not support dynamic tables at all. Is there a manual for cql3? -Original Message- From: Jeremy Hanna [mailto:jeremy.hanna1...@gmail.com] Sent: Friday, July 06, 2012 4:06 PM To: user@cassandra.apache.org Subject: Re: Dynamic CF you can use the cqlsh help but it will eventually refer you to a cql reference such as this one that says what the options are. Looks like you need just 'default_validation'. http://www.datastax.com/docs/1.0/references/cql/index#cql-column-family-storage-parameters On Jul 6, 2012, at 2:13 PM, Leonid Ilyevsky wrote: Can I create a dynamic Cf using cql3? What is the syntax? Here is what I tried: cqlsh:test create table mytest ( ... asset ascii, datetime int, count int, primary key(asset, datetime) ) ... with default_validation_class = double; Bad Request: default_validation_class is not a valid keyword argument for CREATE TABLE This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Dynamic CF
Indeed the cql 3.0 reference in cassandra11.pdf does not mention the dynamic tables whatsoever. Now I found the following posting: http://www.datastax.com/support-forums/topic/flexible-schema-in-cql-3 where the moderator said Yes, CQL 3 expects all columns to be defined in the metadata. At this point I am really confused about what direction Cassandra is going. CQL 3 has the benefit of composite keys, but no dynamic columns. I thought, the whole point of Cassandra was to provide dynamic tables. I need to have a huge table to store market quotes, and be able to query it by name and timestamp (t1 = t = t2), therefore I wanted the composite key. Loading data to such table using prepared statements (CQL 3-based) was very slow, because it makes a server call for each row. So I looked at the API way of doing it, using batch_mutate. Unfortunately, it seems that batch_mutate would not support composite key. Then I thought of making wide rows, so I store many quotes in one row (few thousand). For this I really need dynamic table. I would appreciate any useful suggestion. -Original Message- From: Leonid Ilyevsky [mailto:lilyev...@mooncapital.com] Sent: Friday, July 06, 2012 4:16 PM To: user@cassandra.apache.org Subject: RE: Dynamic CF Thanks Jeremy, but this doesn't work for me. I am using cql3, because I need new features like composite keys. The manual you pointed to is for 2.0. I have suspicion that cql3 does not support dynamic tables at all. Is there a manual for cql3? -Original Message- From: Jeremy Hanna [mailto:jeremy.hanna1...@gmail.com] Sent: Friday, July 06, 2012 4:06 PM To: user@cassandra.apache.org Subject: Re: Dynamic CF you can use the cqlsh help but it will eventually refer you to a cql reference such as this one that says what the options are. Looks like you need just 'default_validation'. http://www.datastax.com/docs/1.0/references/cql/index#cql-column-family-storage-parameters On Jul 6, 2012, at 2:13 PM, Leonid Ilyevsky wrote: Can I create a dynamic Cf using cql3? What is the syntax? Here is what I tried: cqlsh:test create table mytest ( ... asset ascii, datetime int, count int, primary key(asset, datetime) ) ... with default_validation_class = double; Bad Request: default_validation_class is not a valid keyword argument for CREATE TABLE This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any
RE: Dynamic CF
Thanks Derek, I actually saw your post, but this is not a good substitution for a real dynamic table. Your trick will make a table with even more rows; I wanted wide rows so the reads and writes would be more efficient. From: Derek Williams [mailto:de...@fyrie.net] Sent: Friday, July 06, 2012 4:58 PM To: user@cassandra.apache.org Subject: Re: Dynamic CF On Fri, Jul 6, 2012 at 2:49 PM, Leonid Ilyevsky lilyev...@mooncapital.commailto:lilyev...@mooncapital.com wrote: At this point I am really confused about what direction Cassandra is going. CQL 3 has the benefit of composite keys, but no dynamic columns. I thought, the whole point of Cassandra was to provide dynamic tables. See my earlier post on the subject: http://www.mail-archive.com/user@cassandra.apache.org/msg23160.html You can have a dynamic table with CQL3, you just can't have a table with a mix of dynamic/nondynamic columns. -- Derek Williams This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (“Moon Capital”). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
batch_mutate
My current way of inserting rows one by one is too slow (I use cql3 prepared statements) , so I want to try batch_mutate. Could anybody give me more details about the interface? In the javadoc it says: public void batch_mutate(java.util.Mapjava.nio.ByteBuffer,java.util.Mapjava.lang.String,java.util.ListMutationfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\Mutation.html mutation_map, ConsistencyLevelfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\ConsistencyLevel.html consistency_level) throws InvalidRequestExceptionfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\InvalidRequestException.html, UnavailableExceptionfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\UnavailableException.html, TimedOutExceptionfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\TimedOutException.html, org.apache.thrift.TException Description copied from interface: Cassandra.Ifacefile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\Cassandra.Iface.html#batch_mutate%28java.util.Map,%20org.apache.cassandra.thrift.ConsistencyLevel%29 Mutate many columns or super columns for many row keys. See also: Mutation. mutation_map maps key to column family to a list of Mutation objects to take place at that scope. * I need to understand the meaning of the elements of mutation_map parameter. My guess is, the key in the outer map is columnfamily name, is this correct? The key in the inner map is, probably, a key to the columnfamily (it is somewhat confusing that it is String while the outer key is ByteBuffer, I wonder what is the rational). If this is correct, how should I do it if my key is a composite one. Does anybody have an example? Thanks, Leonid This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: batch_mutate
I actually found an answer to my first question at http://wiki.apache.org/cassandra/API. So I got it wrong: actually the outer key is the key in the table, and the inner key is the table name (this was somewhat counter-intuitive). Does it mean that the popular use case is when we need to update multiple column families using the same key? Shouldn't we design our space in such a way that those columns live in the same column family? From: Leonid Ilyevsky [mailto:lilyev...@mooncapital.com] Sent: Thursday, July 05, 2012 10:39 AM To: 'user@cassandra.apache.org' Subject: batch_mutate My current way of inserting rows one by one is too slow (I use cql3 prepared statements) , so I want to try batch_mutate. Could anybody give me more details about the interface? In the javadoc it says: public void batch_mutate(java.util.Mapjava.nio.ByteBuffer,java.util.Mapjava.lang.String,java.util.ListMutationfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\Mutation.html mutation_map, ConsistencyLevelfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\ConsistencyLevel.html consistency_level) throws InvalidRequestExceptionfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\InvalidRequestException.html, UnavailableExceptionfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\UnavailableException.html, TimedOutExceptionfile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\TimedOutException.html, org.apache.thrift.TException Description copied from interface: Cassandra.Ifacefile:///C:\Tools\apache-cassandra-1.1.1\javadoc\org\apache\cassandra\thrift\Cassandra.Iface.html#batch_mutate%28java.util.Map,%20org.apache.cassandra.thrift.ConsistencyLevel%29 Mutate many columns or super columns for many row keys. See also: Mutation. mutation_map maps key to column family to a list of Mutation objects to take place at that scope. * I need to understand the meaning of the elements of mutation_map parameter. My guess is, the key in the outer map is columnfamily name, is this correct? The key in the inner map is, probably, a key to the columnfamily (it is somewhat confusing that it is String while the outer key is ByteBuffer, I wonder what is the rational). If this is correct, how should I do it if my key is a composite one. Does anybody have an example? Thanks, Leonid This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
Composite key in thrift java api
I need to create a ByteBuffer instance containing the proper composite key, based on the values of the components of the key. I am going to use it for update operation. I tried to simply concatenate the buffers corresponding to the components, but I am not sure this is correct, because I am getting exception that comes from the server : InvalidRequestException(why:Not enough bytes to read value of component 0) In the server log I see this: org.apache.thrift.transport.TTransportException: Cannot read. Remote side has closed. Tried to read 4 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.) (I believe here when it says server side it actually means client, because it is the server's log). Seems like the buffer that my client sends is too short. I suspect there is a way in thrift to do it properly, but I don't know how. Looks like Hector has a Composite class that maybe can help, but at this point I would really prefer to do it in Cassandra itself, without Hector. Thanks! Leonid This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
Column names overhead
What is the penalty for using longer column names? Should I sacrifice longer self-explanatory names for shorter cryptic ones to save the disk space? On one hand, I understand that Cassandra row id a key-value map, but on another hand, it probably uses compression when storing them. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
Help with configuring replication
Before going into complex clustering topologies, I would like to try the most simple configuration: just set up two nodes that will completely replicate each other. Could somebody tell me how to configure it? Thanks! This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Setting column to null
Thanks Roshni, I actually looked at Hector, didn't like it that much. Too many methods, feels like there are ten ways to do the same thing, but very difficult to find the one you need. Many packages look like they are simply migrated from Cassandra and renamed, what is the point? In my opinion, it is not well architectured API. Anyway, I prefer simplicity of CQL, I want to just prepare and execute statements. By the way, the manual for CQL is decent compare to the jungle of Hector (and Cassandra API too). -Original Message- From: Roshni Rajagopal [mailto:roshni.rajago...@wal-mart.com] Sent: Tuesday, June 12, 2012 1:20 AM To: user@cassandra.apache.org Subject: Re: Setting column to null Leonid, Are you using some client for doing these operations..? Hector is a java client which provides APIs for adding/deleting columns to a column family in cassandra. I don¹t think you really need to write your wrapper in this format- you are restricting the number of columns it can use etc.I suggest your code can accept user input to get col family name, operation, keys , and operation, and accordingly call the appropriate hector API for adding/deleting data. Regards, Roshni On 11/06/12 7:20 PM, Leonid Ilyevsky lilyev...@mooncapital.com wrote: Thanks, I understand what you are telling me. Obviously deleting the column is the proper way to do this in Cassandra. What I was looking for, is some convenient wrapper on top of that which will do it for me. Here is my scenario. I have a function that takes a record to be saved in Cassandra (array of objects, or MapString, Object). Let say, it can have up to 8 columns. I prepare a statement like this: Insert into table values(?, ?, ?, ?, ?, ?, ?, ?) If I somehow could put null when I execute it, it would be enough to prepare that statement once and execute it multiple times. I would then expect that when some element is null, the corresponding column is not inserted (for the new key) or deleted (for the existing key). The way it is now, in my code I have to examine which columns are present and which are not, depending on that I have to generate customized statement, and it is going to be different for the case of existing key versus case of the new key. Isn't this too much hassle? Related question. I assumed that prepared statement in Cassandra is there for the same reason as in RDBMS, that is, for efficiency. In the above scenario, how expensive is it to execute specialized statement for every record compare to prepared statement executed multiple times? If I need to execute those specialized statements, should I still use prepared statement or should I just generate a string with everything in ascii format? -Original Message- From: Roshni Rajagopal [mailto:roshni.rajago...@wal-mart.com] Sent: Monday, June 11, 2012 12:58 AM To: user@cassandra.apache.org Subject: Re: Setting column to null Would you want to view data like this there was a key, which had this column , but now it does not have any value as of this time. Unless you specifically want this information, I believe you should just delete the column, rather than have an alternate value for NULL or create a composite column. Because in cassandra that¹s the way deletion is dealt with, putting NULLs is the way we deal with it in RDBMS because we have a fixed number of columns which always have to have some value, even if its NULL, and we have to have the same set of columns for every row. In Cassandara, we can delete the column, and in most scenarios that¹s what we should do, unless we specifically want to preserve some history that this column was turned null at this timeŠEach row can have different columns. Regards, Roshni From: Edward Capriolo edlinuxg...@gmail.commailto:edlinuxg...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Setting column to null Your best bet is to define the column as a composite column where one part represents is null and the other part is the data. On Friday, June 8, 2012, shashwat shriparv dwivedishash...@gmail.commailto:dwivedishash...@gmail.com wrote: What you can do is you can define some specific variable like NULLDATA some thing like that to update in columns that does have value On Fri, Jun 8, 2012 at 11:58 PM, aaron morton aa...@thelastpickle.commailto:aa...@thelastpickle.com wrote: You don't nee to set columns to null, delete the column instead. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 8/06/2012, at 9:34 AM, Leonid Ilyevsky wrote: Is it possible to explicitly set a column value to null? I see that if insert statement does not include a specific column, that column comes up as null (assuming we are creating a record with new unique key
Number format in cqlsh
Is there a way to control the floating numbers format in the cqlsh output? I need more digits than it gives by default (in my tests, I see only one digit after the point). This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
select count(*) returns 10000
The select count(*) ... query returns correct count only if it is = 1, otherwise it returns exactly 1. This happens in both Java API and cqlsh. Can somebody verify? This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Number format in cqlsh
Filed the ticket: Cassandra CASSANDRA-4336 From: paul cannon [mailto:p...@datastax.com] Sent: Tuesday, June 12, 2012 12:49 PM To: user@cassandra.apache.org Subject: Re: Number format in cqlsh No, but a way could be added pretty easily. File a ticket on Jira? p On Tuesday, June 12, 2012, Leonid Ilyevsky wrote: Is there a way to control the floating numbers format in the cqlsh output? I need more digits than it gives by default (in my tests, I see only one digit after the point). This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (“Moon Capital”). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
RE: Setting column to null
Thanks, I understand what you are telling me. Obviously deleting the column is the proper way to do this in Cassandra. What I was looking for, is some convenient wrapper on top of that which will do it for me. Here is my scenario. I have a function that takes a record to be saved in Cassandra (array of objects, or MapString, Object). Let say, it can have up to 8 columns. I prepare a statement like this: Insert into table values(?, ?, ?, ?, ?, ?, ?, ?) If I somehow could put null when I execute it, it would be enough to prepare that statement once and execute it multiple times. I would then expect that when some element is null, the corresponding column is not inserted (for the new key) or deleted (for the existing key). The way it is now, in my code I have to examine which columns are present and which are not, depending on that I have to generate customized statement, and it is going to be different for the case of existing key versus case of the new key. Isn't this too much hassle? Related question. I assumed that prepared statement in Cassandra is there for the same reason as in RDBMS, that is, for efficiency. In the above scenario, how expensive is it to execute specialized statement for every record compare to prepared statement executed multiple times? If I need to execute those specialized statements, should I still use prepared statement or should I just generate a string with everything in ascii format? -Original Message- From: Roshni Rajagopal [mailto:roshni.rajago...@wal-mart.com] Sent: Monday, June 11, 2012 12:58 AM To: user@cassandra.apache.org Subject: Re: Setting column to null Would you want to view data like this there was a key, which had this column , but now it does not have any value as of this time. Unless you specifically want this information, I believe you should just delete the column, rather than have an alternate value for NULL or create a composite column. Because in cassandra that’s the way deletion is dealt with, putting NULLs is the way we deal with it in RDBMS because we have a fixed number of columns which always have to have some value, even if its NULL, and we have to have the same set of columns for every row. In Cassandara, we can delete the column, and in most scenarios that’s what we should do, unless we specifically want to preserve some history that this column was turned null at this time…Each row can have different columns. Regards, Roshni From: Edward Capriolo edlinuxg...@gmail.commailto:edlinuxg...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Setting column to null Your best bet is to define the column as a composite column where one part represents is null and the other part is the data. On Friday, June 8, 2012, shashwat shriparv dwivedishash...@gmail.commailto:dwivedishash...@gmail.com wrote: What you can do is you can define some specific variable like NULLDATA some thing like that to update in columns that does have value On Fri, Jun 8, 2012 at 11:58 PM, aaron morton aa...@thelastpickle.commailto:aa...@thelastpickle.com wrote: You don't nee to set columns to null, delete the column instead. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 8/06/2012, at 9:34 AM, Leonid Ilyevsky wrote: Is it possible to explicitly set a column value to null? I see that if insert statement does not include a specific column, that column comes up as null (assuming we are creating a record with new unique key). But if we want to update a record, how we set it to null? Another situation is when I use prepared cql3 statement (in Java) and send parameters when I execute it. If I want to leave some column unassigned, I need a special statement without that column. What I would like is, prepare one statement including all columns, and then be able to set some of them to null. I tried to set corresponding ByteBuffer parameter to null, obviously got an exception. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (“Moon Capital”). Moon Capital does not provide legal, accounting or tax advice. Any
Setting column to null
Is it possible to explicitly set a column value to null? I see that if insert statement does not include a specific column, that column comes up as null (assuming we are creating a record with new unique key). But if we want to update a record, how we set it to null? Another situation is when I use prepared cql3 statement (in Java) and send parameters when I execute it. If I want to leave some column unassigned, I need a special statement without that column. What I would like is, prepare one statement including all columns, and then be able to set some of them to null. I tried to set corresponding ByteBuffer parameter to null, obviously got an exception. This email, along with any attachments, is confidential and may be legally privileged or otherwise protected from disclosure. Any unauthorized dissemination, copying or use of the contents of this email is strictly prohibited and may be in violation of law. If you are not the intended recipient, any disclosure, copying, forwarding or distribution of this email is strictly prohibited and this email and any attachments should be deleted immediately. This email and any attachments do not constitute an offer to sell or a solicitation of an offer to purchase any interest in any investment vehicle sponsored by Moon Capital Management LP (Moon Capital). Moon Capital does not provide legal, accounting or tax advice. Any statement regarding legal, accounting or tax matters was not intended or written to be relied upon by any person as advice. Moon Capital does not waive confidentiality or privilege as a result of this email.
