Issue after dropping a counter column from a table
Hi, Recently, I've added a new counter column (let's call it A) to a counter table, C*3.11.4 I've had to remove it thereafter, because of an application issue. Later, the recreation was needed. When I've executed the ADD command again, got an error for recreating counter column, so I've created it with a new name (let's call it B). However, now I've got this issue: when I try to restore a snapshot backup using sstableloader, the process fails because A doesn't exist in the data, but appears in the schema.cql generated by the snapshot command: CREATE TABLE IF NOT EXISTS keyspace.table( id uuid PRIMARY KEY, A counter, B counter ) . ALTER TABLE keyspace.table DROP A USING TIMESTAMP 1569582510859000; If I do a select in system_schema.dropped_columns, it shows the dropped counter column. How can I properly remove all references to it? Felipe Esteves Tecnologia felipe.este...@b2wdigital.com --
is static column thread safe or only counter column is thread safe?
Hi All, is static column thread safe or only counter column is thread safe? Thanks!
Re: What is the maximum value of Cassandra Counter Column?
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/serializers/CounterSerializer.java public class CounterSerializer extends LongSerializer On Sun, Oct 23, 2016 at 10:16 AM, Ben Slater <ben.sla...@instaclustr.com> wrote: > http://cassandra.apache.org/doc/latest/cql/types.html? > highlight=counter#counters > > On Sun, 23 Oct 2016 at 19:15 Kant Kodali <k...@peernova.com> wrote: > >> where does it say counter is implemented as long? >> >> On Sun, Oct 23, 2016 at 1:13 AM, Ali Akhtar <ali.rac...@gmail.com> wrote: >> >> Probably: https://docs.oracle.com/javase/8/docs/api/java/ >> lang/Long.html#MAX_VALUE >> >> On Sun, Oct 23, 2016 at 1:12 PM, Kant Kodali <k...@peernova.com> wrote: >> >> What is the maximum value of Cassandra Counter Column? >> >> >> >>
Re: What is the maximum value of Cassandra Counter Column?
http://cassandra.apache.org/doc/latest/cql/types.html?highlight=counter#counters On Sun, 23 Oct 2016 at 19:15 Kant Kodali <k...@peernova.com> wrote: > where does it say counter is implemented as long? > > On Sun, Oct 23, 2016 at 1:13 AM, Ali Akhtar <ali.rac...@gmail.com> wrote: > > Probably: > https://docs.oracle.com/javase/8/docs/api/java/lang/Long.html#MAX_VALUE > > On Sun, Oct 23, 2016 at 1:12 PM, Kant Kodali <k...@peernova.com> wrote: > > What is the maximum value of Cassandra Counter Column? > > > >
Re: What is the maximum value of Cassandra Counter Column?
It seems obvious. On Sun, Oct 23, 2016 at 1:15 PM, Kant Kodali <k...@peernova.com> wrote: > where does it say counter is implemented as long? > > On Sun, Oct 23, 2016 at 1:13 AM, Ali Akhtar <ali.rac...@gmail.com> wrote: > >> Probably: https://docs.oracle.com/javase/8/docs/api/java/lan >> g/Long.html#MAX_VALUE >> >> On Sun, Oct 23, 2016 at 1:12 PM, Kant Kodali <k...@peernova.com> wrote: >> >>> What is the maximum value of Cassandra Counter Column? >>> >> >> >
Re: What is the maximum value of Cassandra Counter Column?
where does it say counter is implemented as long? On Sun, Oct 23, 2016 at 1:13 AM, Ali Akhtar <ali.rac...@gmail.com> wrote: > Probably: https://docs.oracle.com/javase/8/docs/api/java/ > lang/Long.html#MAX_VALUE > > On Sun, Oct 23, 2016 at 1:12 PM, Kant Kodali <k...@peernova.com> wrote: > >> What is the maximum value of Cassandra Counter Column? >> > >
Re: What is the maximum value of Cassandra Counter Column?
Probably: https://docs.oracle.com/javase/8/docs/api/java/lang/Long.html#MAX_VALUE On Sun, Oct 23, 2016 at 1:12 PM, Kant Kodali <k...@peernova.com> wrote: > What is the maximum value of Cassandra Counter Column? >
What is the maximum value of Cassandra Counter Column?
What is the maximum value of Cassandra Counter Column?
Re: Can not delete record with counter column in a batch.
On 10/13/2016 08:11 PM, Alexei Levashov wrote:
> Previously I did it in a logged batch without any issues but upgrading
> from Cassandra 1.2.x to 2.1 brings the following Exception message:
> "Error executing batch. Message: Cannot include a counter statement in a
> logged batch"
>
> Question:
> It looks like exception thrown from this line:
>
> if (isLogged() && statement.isCounter()) throw new
> InvalidRequestException("Cannot include a counter statement in a logged
> batch");
>
> Why does DELETE statement for the table containing counter field return
> isCounter() == true?
> Is it a bug or expected behavior?
This appears to be expected behavior.
If you look at blame on the line you linked, it was commit ee401cf8
which you can follow to the commit:
https://github.com/apache/cassandra/commit/ee401cf8131a779069805cbe9ef4ab05d4a63b9a
Which notes: CASSANDRA-7351
https://issues.apache.org/jira/browse/CASSANDRA-7351
It looks like you may use UNLOGGED batch.
--
Kind regards,
Michael
Can not delete record with counter column in a batch.
Configuration : Cassandra 2.1
(if it matters Datastax Java driver 3.0.0 - driver users list couldn't help)
Platform Unix.
I have two tables:
CREATE TABLE unmapped_doc_type (
application_id uuid,
doc_type_id text,
metadata text,
PRIMARY KEY (application_id, doc_type_id)
)
CREATE TABLE unmapped_doc_type_counter (
application_id uuid,
doc_type_id text,
occurrences counter,
PRIMARY KEY (application_id, doc_type_id)
)
Table unmapped_doc_type has metadata for doc_type and
unmapped_doc_type_counter one has the number of occurences this doc type
was requested.
When I delete record in unmapped_doc_type I have to delete the same (same
values of application_id, doc_type_id) in unmapped_doc_type_counter table.
I am building DELETE statements using QueryBuilder, wrapping them into
com.datastax.driver.core.querybuilder.Batch as new Batch(statements, true)
and calling session.execute(batch) with this batch.
Previously I did it in a logged batch without any issues but upgrading from
Cassandra 1.2.x to 2.1 brings the following Exception message:
"Error executing batch. Message: Cannot include a counter statement in a
logged batch"
Sample of statement that fails:
"
BEGIN BATCH
DELETE FROM unmapped_doc_type
WHERE doc_type_id='567cfa05-aa41-4d4d-b65d-42dfd198ef2b' AND
application_id=41367e0c-5d6f-4b2f-897b-e25d6a52ee0e;
DELETE FROM unmapped_doc_type_counter
WHERE doc_type_id='567cfa05-aa41-4d4d-b65d-42dfd198ef2b' AND
application_id=41367e0c-5d6f-4b2f-897b-e25d6a52ee0e;
APPLY BATCH;
".
Question:
It looks like exception thrown from this line:
if (isLogged() && statement.isCounter()) throw new
InvalidRequestException("Cannot include a counter statement in a logged
batch");
Why does DELETE statement for the table containing counter field return
isCounter() == true?
Is it a bug or expected behavior?
Thank you,
Alexei.
Re: Counter Column
Thanks. I went through some articles which mentioned that the client to pass the timestamp for insert and update. Is that anyway we can avoid it and Cassandra assume the current time of the server? Thanks Ajay On Dec 26, 2014 10:50 PM, Eric Stevens migh...@gmail.com wrote: Timestamps are timezone independent. This is a property of timestamps, not a property of Cassandra. A given moment is the same timestamp everywhere in the world. To display this in a human readable form, you then need to know what timezone you're attempting to represent the timestamp as, this is the information necessary to convert it to local time. On Fri, Dec 26, 2014 at 2:05 AM, Ajay ajay.ga...@gmail.com wrote: Hi, If the nodes of Cassandra ring are in different timezone, could it affect the counter column as it depends on the timestamp? Thanks Ajay
Re: Counter Column
In java, http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#currentTimeMillis() return the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC. It means the timestamp which Cassandra uses is not independent on the timezone. 2014-12-27 21:08 GMT+08:00 Ajay ajay.ga...@gmail.com: Thanks. I went through some articles which mentioned that the client to pass the timestamp for insert and update. Is that anyway we can avoid it and Cassandra assume the current time of the server? Thanks Ajay On Dec 26, 2014 10:50 PM, Eric Stevens migh...@gmail.com wrote: Timestamps are timezone independent. This is a property of timestamps, not a property of Cassandra. A given moment is the same timestamp everywhere in the world. To display this in a human readable form, you then need to know what timezone you're attempting to represent the timestamp as, this is the information necessary to convert it to local time. On Fri, Dec 26, 2014 at 2:05 AM, Ajay ajay.ga...@gmail.com wrote: Hi, If the nodes of Cassandra ring are in different timezone, could it affect the counter column as it depends on the timestamp? Thanks Ajay -- Thanks, Phil Yang
Re: Counter Column
sorry for typo.. timestamp which Cassandra uses is independent on the timezone. Usually, it is recommended to use NTP to reduce the difference of timestamps in each nodes 2014-12-27 21:20 GMT+08:00 Phil Yang ud1...@gmail.com: In java, http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#currentTimeMillis() return the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC. It means the timestamp which Cassandra uses is not independent on the timezone. 2014-12-27 21:08 GMT+08:00 Ajay ajay.ga...@gmail.com: Thanks. I went through some articles which mentioned that the client to pass the timestamp for insert and update. Is that anyway we can avoid it and Cassandra assume the current time of the server? Thanks Ajay On Dec 26, 2014 10:50 PM, Eric Stevens migh...@gmail.com wrote: Timestamps are timezone independent. This is a property of timestamps, not a property of Cassandra. A given moment is the same timestamp everywhere in the world. To display this in a human readable form, you then need to know what timezone you're attempting to represent the timestamp as, this is the information necessary to convert it to local time. On Fri, Dec 26, 2014 at 2:05 AM, Ajay ajay.ga...@gmail.com wrote: Hi, If the nodes of Cassandra ring are in different timezone, could it affect the counter column as it depends on the timestamp? Thanks Ajay -- Thanks, Phil Yang -- Thanks, Phil Yang
Re: Counter Column
Having the client pass the timestamp is optional, if you do not provide one from the client, then it will use the server's timestamp. On Sat, Dec 27, 2014, 6:25 AM Phil Yang ud1...@gmail.com wrote: sorry for typo.. timestamp which Cassandra uses is independent on the timezone. Usually, it is recommended to use NTP to reduce the difference of timestamps in each nodes 2014-12-27 21:20 GMT+08:00 Phil Yang ud1...@gmail.com: In java, http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#currentTimeMillis() return the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC. It means the timestamp which Cassandra uses is not independent on the timezone. 2014-12-27 21:08 GMT+08:00 Ajay ajay.ga...@gmail.com: Thanks. I went through some articles which mentioned that the client to pass the timestamp for insert and update. Is that anyway we can avoid it and Cassandra assume the current time of the server? Thanks Ajay On Dec 26, 2014 10:50 PM, Eric Stevens migh...@gmail.com wrote: Timestamps are timezone independent. This is a property of timestamps, not a property of Cassandra. A given moment is the same timestamp everywhere in the world. To display this in a human readable form, you then need to know what timezone you're attempting to represent the timestamp as, this is the information necessary to convert it to local time. On Fri, Dec 26, 2014 at 2:05 AM, Ajay ajay.ga...@gmail.com wrote: Hi, If the nodes of Cassandra ring are in different timezone, could it affect the counter column as it depends on the timestamp? Thanks Ajay -- Thanks, Phil Yang -- Thanks, Phil Yang
Counter Column
Hi, If the nodes of Cassandra ring are in different timezone, could it affect the counter column as it depends on the timestamp? Thanks Ajay
Re: Counter Column
Timestamps are timezone independent. This is a property of timestamps, not a property of Cassandra. A given moment is the same timestamp everywhere in the world. To display this in a human readable form, you then need to know what timezone you're attempting to represent the timestamp as, this is the information necessary to convert it to local time. On Fri, Dec 26, 2014 at 2:05 AM, Ajay ajay.ga...@gmail.com wrote: Hi, If the nodes of Cassandra ring are in different timezone, could it affect the counter column as it depends on the timestamp? Thanks Ajay
Store counter with non-counter column in the same column family?
I was wondering if there is plan to allow creating counter column and standard column in the same table. Here is my use case: I want to use counter to count how many users like a given item in my application. The like count needs to be returned along with details of item in query. To support querying items in different ways, I use both application-maintained denormalized index tables and DSE search for indexing. (DSE search is also used for text searching) Since current counter implementation doesn't allow having counter columns and non-counter columns in the same table, I have to propagate the current count from counter table to the main item table and index tables, so that like counts can be returned by those index tables without sending extra requests to counter table and DSE search is able to build index on like count column in the main item table to support like count related queries (such as sorting by like count). IMHO, the only way to sync data between counter table and normal table within a reasonable time (sub-seconds) currently is to read the current value from counter table right after the update. However it suffers from several issues: 1. Read-after-write may not return the correct count when replication factor 1 unless consistency level ALL/LOCAL_ALL is used 2. There are two extra non-parallelizable round-trips between the application server and cassandra, which can have great impact on performance. If it is possible to store counter in standard column family, only one write will be needed to update like count in the main table. Counter value will also be eventually synced between replicas so that there is no need for application to use extra mechanism like scheduled task to get the correct counts. A related issue is lifting the limitation of not allowing updating counter columns and normal columns in one batch, since it is quite common to not only have a counter for statistics but also store the details, such as storing the relation of which user likes which items in my user case. Any idea?
Re: Store counter with non-counter column in the same column family?
It's not possible to mix counter and non counter columns because currently the semantic of counter is only increment/decrement (thus NOT idempotent) and requires some special handling compared to other C* columns. On Mon, Dec 22, 2014 at 11:33 AM, ziju feng pkdog...@gmail.com wrote: I was wondering if there is plan to allow creating counter column and standard column in the same table. Here is my use case: I want to use counter to count how many users like a given item in my application. The like count needs to be returned along with details of item in query. To support querying items in different ways, I use both application-maintained denormalized index tables and DSE search for indexing. (DSE search is also used for text searching) Since current counter implementation doesn't allow having counter columns and non-counter columns in the same table, I have to propagate the current count from counter table to the main item table and index tables, so that like counts can be returned by those index tables without sending extra requests to counter table and DSE search is able to build index on like count column in the main item table to support like count related queries (such as sorting by like count). IMHO, the only way to sync data between counter table and normal table within a reasonable time (sub-seconds) currently is to read the current value from counter table right after the update. However it suffers from several issues: 1. Read-after-write may not return the correct count when replication factor 1 unless consistency level ALL/LOCAL_ALL is used 2. There are two extra non-parallelizable round-trips between the application server and cassandra, which can have great impact on performance. If it is possible to store counter in standard column family, only one write will be needed to update like count in the main table. Counter value will also be eventually synced between replicas so that there is no need for application to use extra mechanism like scheduled task to get the correct counts. A related issue is lifting the limitation of not allowing updating counter columns and normal columns in one batch, since it is quite common to not only have a counter for statistics but also store the details, such as storing the relation of which user likes which items in my user case. Any idea?
Re: Store counter with non-counter column in the same column family?
You can cheat it by using the non counter column as part of your primary key (clustering column specifically) but the cases where this could work are limited and the places this is a good idea are even more rare. As for using counters in batches are already a not well regarded concept and counter batches have a number of troubling behaviors, as already stated increments aren't idempotent and batch implies retry. As for DSE search its doing something drastically different internally and the type of counting its doing is many orders of magnitude faster ( think bitmask style matching + proper async 2i to minimize fanout cost) Generally speaking counting accurately while being highly available creates an interesting set of logical tradeoffs. Example what do you do if you're not able to communicate between two data centers, but both are up and serving likes quite happily? Is your counting down? Do you keep counting but serve up different answers? More accurately since problems are rarely data center to data center but more frequently between replicas, how much availability are you willing to give up in exchange for a globally accurate count? On Dec 22, 2014 6:00 AM, DuyHai Doan doanduy...@gmail.com wrote: It's not possible to mix counter and non counter columns because currently the semantic of counter is only increment/decrement (thus NOT idempotent) and requires some special handling compared to other C* columns. On Mon, Dec 22, 2014 at 11:33 AM, ziju feng pkdog...@gmail.com wrote: I was wondering if there is plan to allow creating counter column and standard column in the same table. Here is my use case: I want to use counter to count how many users like a given item in my application. The like count needs to be returned along with details of item in query. To support querying items in different ways, I use both application-maintained denormalized index tables and DSE search for indexing. (DSE search is also used for text searching) Since current counter implementation doesn't allow having counter columns and non-counter columns in the same table, I have to propagate the current count from counter table to the main item table and index tables, so that like counts can be returned by those index tables without sending extra requests to counter table and DSE search is able to build index on like count column in the main item table to support like count related queries (such as sorting by like count). IMHO, the only way to sync data between counter table and normal table within a reasonable time (sub-seconds) currently is to read the current value from counter table right after the update. However it suffers from several issues: 1. Read-after-write may not return the correct count when replication factor 1 unless consistency level ALL/LOCAL_ALL is used 2. There are two extra non-parallelizable round-trips between the application server and cassandra, which can have great impact on performance. If it is possible to store counter in standard column family, only one write will be needed to update like count in the main table. Counter value will also be eventually synced between replicas so that there is no need for application to use extra mechanism like scheduled task to get the correct counts. A related issue is lifting the limitation of not allowing updating counter columns and normal columns in one batch, since it is quite common to not only have a counter for statistics but also store the details, such as storing the relation of which user likes which items in my user case. Any idea?
Re: Store counter with non-counter column in the same column family?
I just skimmed through JIRA https://issues.apache.org/jira/browse/CASSANDRA-4775 and it seems there has been some effort to make update idempotent. Perhaps the problem can be fixed in the near future? Anyway, what is the current best practice for such use case? (Counting and displaying counts in different queries) I don't need a 100% accurate count and strong consistency. Performance and application complexity is my main concern. Thanks On Mon, Dec 22, 2014 at 10:37 PM, Ryan Svihla rsvi...@datastax.com wrote: You can cheat it by using the non counter column as part of your primary key (clustering column specifically) but the cases where this could work are limited and the places this is a good idea are even more rare. As for using counters in batches are already a not well regarded concept and counter batches have a number of troubling behaviors, as already stated increments aren't idempotent and batch implies retry. As for DSE search its doing something drastically different internally and the type of counting its doing is many orders of magnitude faster ( think bitmask style matching + proper async 2i to minimize fanout cost) Generally speaking counting accurately while being highly available creates an interesting set of logical tradeoffs. Example what do you do if you're not able to communicate between two data centers, but both are up and serving likes quite happily? Is your counting down? Do you keep counting but serve up different answers? More accurately since problems are rarely data center to data center but more frequently between replicas, how much availability are you willing to give up in exchange for a globally accurate count? On Dec 22, 2014 6:00 AM, DuyHai Doan doanduy...@gmail.com wrote: It's not possible to mix counter and non counter columns because currently the semantic of counter is only increment/decrement (thus NOT idempotent) and requires some special handling compared to other C* columns. On Mon, Dec 22, 2014 at 11:33 AM, ziju feng pkdog...@gmail.com wrote: I was wondering if there is plan to allow creating counter column and standard column in the same table. Here is my use case: I want to use counter to count how many users like a given item in my application. The like count needs to be returned along with details of item in query. To support querying items in different ways, I use both application-maintained denormalized index tables and DSE search for indexing. (DSE search is also used for text searching) Since current counter implementation doesn't allow having counter columns and non-counter columns in the same table, I have to propagate the current count from counter table to the main item table and index tables, so that like counts can be returned by those index tables without sending extra requests to counter table and DSE search is able to build index on like count column in the main item table to support like count related queries (such as sorting by like count). IMHO, the only way to sync data between counter table and normal table within a reasonable time (sub-seconds) currently is to read the current value from counter table right after the update. However it suffers from several issues: 1. Read-after-write may not return the correct count when replication factor 1 unless consistency level ALL/LOCAL_ALL is used 2. There are two extra non-parallelizable round-trips between the application server and cassandra, which can have great impact on performance. If it is possible to store counter in standard column family, only one write will be needed to update like count in the main table. Counter value will also be eventually synced between replicas so that there is no need for application to use extra mechanism like scheduled task to get the correct counts. A related issue is lifting the limitation of not allowing updating counter columns and normal columns in one batch, since it is quite common to not only have a counter for statistics but also store the details, such as storing the relation of which user likes which items in my user case. Any idea?
Re: Store counter with non-counter column in the same column family?
increment wouldn't be idempotent from the client unless you knew the count at the time of the update (which you could do with LWT but that has pretty harsh performance), that particular jira is about how they're laid out and avoiding race conditions between nodes, which was resolved in 2.1 beta 1 (which is now in officially out in the 2.1.x branch) General improvements on counters in 2.1 are laid out here http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters As for best practice the answer is multiple tables for multiple query paths, or you can use something like solr or spark, take a look at the spark cassandra connector for a good way to count on lots of data from lots of different query paths https://github.com/datastax/spark-cassandra-connector. On Mon, Dec 22, 2014 at 9:22 PM, ziju feng pkdog...@gmail.com wrote: I just skimmed through JIRA https://issues.apache.org/jira/browse/CASSANDRA-4775 and it seems there has been some effort to make update idempotent. Perhaps the problem can be fixed in the near future? Anyway, what is the current best practice for such use case? (Counting and displaying counts in different queries) I don't need a 100% accurate count and strong consistency. Performance and application complexity is my main concern. Thanks On Mon, Dec 22, 2014 at 10:37 PM, Ryan Svihla rsvi...@datastax.com wrote: You can cheat it by using the non counter column as part of your primary key (clustering column specifically) but the cases where this could work are limited and the places this is a good idea are even more rare. As for using counters in batches are already a not well regarded concept and counter batches have a number of troubling behaviors, as already stated increments aren't idempotent and batch implies retry. As for DSE search its doing something drastically different internally and the type of counting its doing is many orders of magnitude faster ( think bitmask style matching + proper async 2i to minimize fanout cost) Generally speaking counting accurately while being highly available creates an interesting set of logical tradeoffs. Example what do you do if you're not able to communicate between two data centers, but both are up and serving likes quite happily? Is your counting down? Do you keep counting but serve up different answers? More accurately since problems are rarely data center to data center but more frequently between replicas, how much availability are you willing to give up in exchange for a globally accurate count? On Dec 22, 2014 6:00 AM, DuyHai Doan doanduy...@gmail.com wrote: It's not possible to mix counter and non counter columns because currently the semantic of counter is only increment/decrement (thus NOT idempotent) and requires some special handling compared to other C* columns. On Mon, Dec 22, 2014 at 11:33 AM, ziju feng pkdog...@gmail.com wrote: I was wondering if there is plan to allow creating counter column and standard column in the same table. Here is my use case: I want to use counter to count how many users like a given item in my application. The like count needs to be returned along with details of item in query. To support querying items in different ways, I use both application-maintained denormalized index tables and DSE search for indexing. (DSE search is also used for text searching) Since current counter implementation doesn't allow having counter columns and non-counter columns in the same table, I have to propagate the current count from counter table to the main item table and index tables, so that like counts can be returned by those index tables without sending extra requests to counter table and DSE search is able to build index on like count column in the main item table to support like count related queries (such as sorting by like count). IMHO, the only way to sync data between counter table and normal table within a reasonable time (sub-seconds) currently is to read the current value from counter table right after the update. However it suffers from several issues: 1. Read-after-write may not return the correct count when replication factor 1 unless consistency level ALL/LOCAL_ALL is used 2. There are two extra non-parallelizable round-trips between the application server and cassandra, which can have great impact on performance. If it is possible to store counter in standard column family, only one write will be needed to update like count in the main table. Counter value will also be eventually synced between replicas so that there is no need for application to use extra mechanism like scheduled task to get the correct counts. A related issue is lifting the limitation of not allowing updating counter columns and normal columns in one batch, since it is quite common to not only have a counter for statistics but also store the details, such as storing the relation of which user likes which items
Re: Store counter with non-counter column in the same column family?
Thanks for the advise, I'll definitely take a look at how Spark works and how it can help with counting. One last question: My current implementation of counting is 1) increment counter 2) read counter immediately after the write 3) write counts to multiple tables for different query paths and solr. If I switch to Spark, do I still needs to use counter or counting will be done by spark on regular table? On Tue, Dec 23, 2014 at 11:31 AM, Ryan Svihla rsvi...@datastax.com wrote: increment wouldn't be idempotent from the client unless you knew the count at the time of the update (which you could do with LWT but that has pretty harsh performance), that particular jira is about how they're laid out and avoiding race conditions between nodes, which was resolved in 2.1 beta 1 (which is now in officially out in the 2.1.x branch) General improvements on counters in 2.1 are laid out here http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters As for best practice the answer is multiple tables for multiple query paths, or you can use something like solr or spark, take a look at the spark cassandra connector for a good way to count on lots of data from lots of different query paths https://github.com/datastax/spark-cassandra-connector. On Mon, Dec 22, 2014 at 9:22 PM, ziju feng pkdog...@gmail.com wrote: I just skimmed through JIRA https://issues.apache.org/jira/browse/CASSANDRA-4775 and it seems there has been some effort to make update idempotent. Perhaps the problem can be fixed in the near future? Anyway, what is the current best practice for such use case? (Counting and displaying counts in different queries) I don't need a 100% accurate count and strong consistency. Performance and application complexity is my main concern. Thanks On Mon, Dec 22, 2014 at 10:37 PM, Ryan Svihla rsvi...@datastax.com wrote: You can cheat it by using the non counter column as part of your primary key (clustering column specifically) but the cases where this could work are limited and the places this is a good idea are even more rare. As for using counters in batches are already a not well regarded concept and counter batches have a number of troubling behaviors, as already stated increments aren't idempotent and batch implies retry. As for DSE search its doing something drastically different internally and the type of counting its doing is many orders of magnitude faster ( think bitmask style matching + proper async 2i to minimize fanout cost) Generally speaking counting accurately while being highly available creates an interesting set of logical tradeoffs. Example what do you do if you're not able to communicate between two data centers, but both are up and serving likes quite happily? Is your counting down? Do you keep counting but serve up different answers? More accurately since problems are rarely data center to data center but more frequently between replicas, how much availability are you willing to give up in exchange for a globally accurate count? On Dec 22, 2014 6:00 AM, DuyHai Doan doanduy...@gmail.com wrote: It's not possible to mix counter and non counter columns because currently the semantic of counter is only increment/decrement (thus NOT idempotent) and requires some special handling compared to other C* columns. On Mon, Dec 22, 2014 at 11:33 AM, ziju feng pkdog...@gmail.com wrote: I was wondering if there is plan to allow creating counter column and standard column in the same table. Here is my use case: I want to use counter to count how many users like a given item in my application. The like count needs to be returned along with details of item in query. To support querying items in different ways, I use both application-maintained denormalized index tables and DSE search for indexing. (DSE search is also used for text searching) Since current counter implementation doesn't allow having counter columns and non-counter columns in the same table, I have to propagate the current count from counter table to the main item table and index tables, so that like counts can be returned by those index tables without sending extra requests to counter table and DSE search is able to build index on like count column in the main item table to support like count related queries (such as sorting by like count). IMHO, the only way to sync data between counter table and normal table within a reasonable time (sub-seconds) currently is to read the current value from counter table right after the update. However it suffers from several issues: 1. Read-after-write may not return the correct count when replication factor 1 unless consistency level ALL/LOCAL_ALL is used 2. There are two extra non-parallelizable round-trips between the application server and cassandra, which can have great impact on performance. If it is possible to store counter in standard column family, only one write
Re: Store counter with non-counter column in the same column family?
Spark can count a regular table. Spark sql would be the easiest thing to get started with most likely. https://github.com/datastax/spark-cassandra-connector/blob/master/doc/2_loading.md Go down to the spark sql section to get some idea of the ease of use. On Dec 22, 2014 10:00 PM, ziju feng pkdog...@gmail.com wrote: Thanks for the advise, I'll definitely take a look at how Spark works and how it can help with counting. One last question: My current implementation of counting is 1) increment counter 2) read counter immediately after the write 3) write counts to multiple tables for different query paths and solr. If I switch to Spark, do I still needs to use counter or counting will be done by spark on regular table? On Tue, Dec 23, 2014 at 11:31 AM, Ryan Svihla rsvi...@datastax.com wrote: increment wouldn't be idempotent from the client unless you knew the count at the time of the update (which you could do with LWT but that has pretty harsh performance), that particular jira is about how they're laid out and avoiding race conditions between nodes, which was resolved in 2.1 beta 1 (which is now in officially out in the 2.1.x branch) General improvements on counters in 2.1 are laid out here http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters As for best practice the answer is multiple tables for multiple query paths, or you can use something like solr or spark, take a look at the spark cassandra connector for a good way to count on lots of data from lots of different query paths https://github.com/datastax/spark-cassandra-connector. On Mon, Dec 22, 2014 at 9:22 PM, ziju feng pkdog...@gmail.com wrote: I just skimmed through JIRA https://issues.apache.org/jira/browse/CASSANDRA-4775 and it seems there has been some effort to make update idempotent. Perhaps the problem can be fixed in the near future? Anyway, what is the current best practice for such use case? (Counting and displaying counts in different queries) I don't need a 100% accurate count and strong consistency. Performance and application complexity is my main concern. Thanks On Mon, Dec 22, 2014 at 10:37 PM, Ryan Svihla rsvi...@datastax.com wrote: You can cheat it by using the non counter column as part of your primary key (clustering column specifically) but the cases where this could work are limited and the places this is a good idea are even more rare. As for using counters in batches are already a not well regarded concept and counter batches have a number of troubling behaviors, as already stated increments aren't idempotent and batch implies retry. As for DSE search its doing something drastically different internally and the type of counting its doing is many orders of magnitude faster ( think bitmask style matching + proper async 2i to minimize fanout cost) Generally speaking counting accurately while being highly available creates an interesting set of logical tradeoffs. Example what do you do if you're not able to communicate between two data centers, but both are up and serving likes quite happily? Is your counting down? Do you keep counting but serve up different answers? More accurately since problems are rarely data center to data center but more frequently between replicas, how much availability are you willing to give up in exchange for a globally accurate count? On Dec 22, 2014 6:00 AM, DuyHai Doan doanduy...@gmail.com wrote: It's not possible to mix counter and non counter columns because currently the semantic of counter is only increment/decrement (thus NOT idempotent) and requires some special handling compared to other C* columns. On Mon, Dec 22, 2014 at 11:33 AM, ziju feng pkdog...@gmail.com wrote: I was wondering if there is plan to allow creating counter column and standard column in the same table. Here is my use case: I want to use counter to count how many users like a given item in my application. The like count needs to be returned along with details of item in query. To support querying items in different ways, I use both application-maintained denormalized index tables and DSE search for indexing. (DSE search is also used for text searching) Since current counter implementation doesn't allow having counter columns and non-counter columns in the same table, I have to propagate the current count from counter table to the main item table and index tables, so that like counts can be returned by those index tables without sending extra requests to counter table and DSE search is able to build index on like count column in the main item table to support like count related queries (such as sorting by like count). IMHO, the only way to sync data between counter table and normal table within a reasonable time (sub-seconds) currently is to read the current value from counter table right after the update. However it suffers from several issues: 1. Read-after-write may
Re: Counter column impossible to delete and re-insert
Hello Clément This is a known anti-pattern. You should never re-use a deleted counter column otherwise there will be unpredictable result for the counter value. Le 6 nov. 2014 08:45, Clément Fumey clement@gmail.com a écrit : Hi, I have a table with counter column . When I insert (update) a row, delete it and try to re-insert, it fail to re-insert the row. Here is the commands i use : CREATE TABLE test( testId int, year int, testCounter counter, PRIMARY KEY (testId, year) )WITH CLUSTERING ORDER BY (year DESC); UPDATE test SET testcounter = testcounter +5 WHERE testid = 2 AND year = 2014; DELETE FROM test WHERE testid = 2 AND year = 2014; UPDATE test SET testcounter = testcounter +5 WHERE testid = 2 AND year = 2014; The last command failed, there is no error message but the table is empty after it. Is that normal? Am I doing something wrong? Regards Clément
Counter column impossible to delete and re-insert
Hi, I have a table with counter column . When I insert (update) a row, delete it and try to re-insert, it fail to re-insert the row. Here is the commands i use : CREATE TABLE test( testId int, year int, testCounter counter, PRIMARY KEY (testId, year) )WITH CLUSTERING ORDER BY (year DESC); UPDATE test SET testcounter = testcounter +5 WHERE testid = 2 AND year = 2014; DELETE FROM test WHERE testid = 2 AND year = 2014; UPDATE test SET testcounter = testcounter +5 WHERE testid = 2 AND year = 2014; The last command failed, there is no error message but the table is empty after it. Is that normal? Am I doing something wrong? Regards Clément
Re: Cassandra counter column family performance
I get a lot of TExceptions What are the exceptions ? In general counters are slower than writes, but that does not lead them to fail like that. Check the logs for errors and/or messages from the GCInspector saying the garbage collection is going on. Cheers A - Aaron Morton New Zealand @aaronmorton Co-Founder Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 13/05/2014, at 9:51 pm, Batranut Bogdan batra...@yahoo.com wrote: Hello all, I have a counter CF defined as pk text PRIMARY KEY, a counter, b counter, c counter, d counter After inserting a few million keys... 55 mil, the performance goes down the drain, 2-3 nodes in the cluster are on medium load, and when inserting batches of same lengths writes take longer and longer until the whole cluster becomes loaded and I get a lot of TExceptions... and the cluster becomes unresponsive. Did anyone have the same problem? Feel free to comment and share experiences about counter CF performance.
Cassandra counter column family performance
Hello all, I have a counter CF defined as pk text PRIMARY KEY, a counter, b counter, c counter, d counter After inserting a few million keys... 55 mil, the performance goes down the drain, 2-3 nodes in the cluster are on medium load, and when inserting batches of same lengths writes take longer and longer until the whole cluster becomes loaded and I get a lot of TExceptions... and the cluster becomes unresponsive. Did anyone have the same problem? Feel free to comment and share experiences about counter CF performance.
Re: How to create counter column family via Pycassa?
What are the all possible values for cf_kwargs ??
SYSTEM_MANAGER.create_column_family('Narrative','Twitter_search_test',
comparator_type=UTF8Type, )
- Here I want to specify, Column data types and row key type. How can
I do that ?
On Thu, Aug 15, 2013 at 12:30 PM, Tyler Hobbs ty...@datastax.com wrote:
The column_validation_classes arg is just for defining individual column
types. Glad you got it figured out, though.
On Thu, Aug 15, 2013 at 11:23 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
Thanks for quick reply. Apparantly, I was trying this to get working
cf_kwargs = {'default_validation_class':COUNTER_COLUMN_TYPE}
sys.create_column_family('my_ks', 'vote_count',
column_validation_classes=cf_kwargs) #1
But this works:
sys.create_column_family('my_ks', 'vote_count', **cf_kwargs) #2
I thought #1 should work.
On Thu, Aug 15, 2013 at 9:15 PM, Tyler Hobbs ty...@datastax.com wrote:
The only thing that makes a CF a counter CF is that the default
validation class is CounterColumnType, which you can set through
SystemManager.create_column_family().
On Thu, Aug 15, 2013 at 10:38 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
I do not find a way to create a counter column family in Pycassa.
This[1] does not help.
Appreciate if someone can help me.
Thanks
1.
http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
Re: How to create counter column family via Pycassa?
What options are available depends on what version of Cassandra you're
using.
You can specify the row key type with 'key_validation_class'.
For column types, use 'column_validation_classes', which is a dict mapping
column names to types. For example:
sys.create_column_family('mykeyspace', 'users',
column_validation_classes={'username': UTF8Type, 'age': IntegerType})
On Wed, Dec 11, 2013 at 10:32 AM, Kumar Ranjan winnerd...@gmail.com wrote:
What are the all possible values for cf_kwargs ??
SYSTEM_MANAGER.create_column_family('Narrative','Twitter_search_test',
comparator_type=UTF8Type, )
- Here I want to specify, Column data types and row key type. How can
I do that ?
On Thu, Aug 15, 2013 at 12:30 PM, Tyler Hobbs ty...@datastax.com wrote:
The column_validation_classes arg is just for defining individual column
types. Glad you got it figured out, though.
On Thu, Aug 15, 2013 at 11:23 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
Thanks for quick reply. Apparantly, I was trying this to get working
cf_kwargs = {'default_validation_class':COUNTER_COLUMN_TYPE}
sys.create_column_family('my_ks', 'vote_count',
column_validation_classes=cf_kwargs) #1
But this works:
sys.create_column_family('my_ks', 'vote_count', **cf_kwargs) #2
I thought #1 should work.
On Thu, Aug 15, 2013 at 9:15 PM, Tyler Hobbs ty...@datastax.com wrote:
The only thing that makes a CF a counter CF is that the default
validation class is CounterColumnType, which you can set through
SystemManager.create_column_family().
On Thu, Aug 15, 2013 at 10:38 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
I do not find a way to create a counter column family in Pycassa.
This[1] does not help.
Appreciate if someone can help me.
Thanks
1.
http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
Re: How to create counter column family via Pycassa?
validators = {
'approved': 'BooleanType',
'text': 'UTF8Type',
'favorite_count':'IntegerType',
'retweet_count': 'IntegerType',
'expanded_url': 'UTF8Type',
'tuid': 'LongType',
'screen_name': 'UTF8Type',
'profile_image': 'UTF8Type',
'embedly_data': 'CompositeType',
'created_at':'UTF8Type',
}
SYSTEM_MANAGER.create_column_family('Narrative','Twitter_search_test',
comparator_type='CompositeType', default_validation_class='UTF8Type',
key_validation_class='UTF8Type', column_validation_classes=validators)
throws:
*InvalidRequestException*: InvalidRequestException(why='Invalid definition
for comparator org.apache.cassandra.db.marshal.CompositeType.
Can you please explain why?
On Wed, Dec 11, 2013 at 12:08 PM, Tyler Hobbs ty...@datastax.com wrote:
What options are available depends on what version of Cassandra you're
using.
You can specify the row key type with 'key_validation_class'.
For column types, use 'column_validation_classes', which is a dict mapping
column names to types. For example:
sys.create_column_family('mykeyspace', 'users',
column_validation_classes={'username': UTF8Type, 'age': IntegerType})
On Wed, Dec 11, 2013 at 10:32 AM, Kumar Ranjan winnerd...@gmail.comwrote:
What are the all possible values for cf_kwargs ??
SYSTEM_MANAGER.create_column_family('Narrative','Twitter_search_test',
comparator_type=UTF8Type, )
- Here I want to specify, Column data types and row key type. How
can I do that ?
On Thu, Aug 15, 2013 at 12:30 PM, Tyler Hobbs ty...@datastax.com wrote:
The column_validation_classes arg is just for defining individual column
types. Glad you got it figured out, though.
On Thu, Aug 15, 2013 at 11:23 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
Thanks for quick reply. Apparantly, I was trying this to get working
cf_kwargs = {'default_validation_class':COUNTER_COLUMN_TYPE}
sys.create_column_family('my_ks', 'vote_count',
column_validation_classes=cf_kwargs) #1
But this works:
sys.create_column_family('my_ks', 'vote_count', **cf_kwargs) #2
I thought #1 should work.
On Thu, Aug 15, 2013 at 9:15 PM, Tyler Hobbs ty...@datastax.comwrote:
The only thing that makes a CF a counter CF is that the default
validation class is CounterColumnType, which you can set through
SystemManager.create_column_family().
On Thu, Aug 15, 2013 at 10:38 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
I do not find a way to create a counter column family in Pycassa.
This[1] does not help.
Appreciate if someone can help me.
Thanks
1.
http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
Re: How to create counter column family via Pycassa?
This works, When I remove the comparator_type
validators = {
'tid': 'IntegerType',
'approved': 'BooleanType',
'text': 'UTF8Type',
'favorite_count':'IntegerType',
'retweet_count': 'IntegerType',
'expanded_url': 'UTF8Type',
'tuid': 'LongType',
'screen_name': 'UTF8Type',
'profile_image': 'UTF8Type',
'embedly_data': 'BytesType',
'created_at':'UTF8Type',
}
SYSTEM_MANAGER.create_column_family('Narrative','Twitter_search',
default_validation_class='UTF8Type', key_validation_class='UTF8Type',
column_validation_classes=validators)
On Wed, Dec 11, 2013 at 12:23 PM, Kumar Ranjan winnerd...@gmail.com wrote:
I am using ccm cassandra version
*1.2.11*
On Wed, Dec 11, 2013 at 12:19 PM, Kumar Ranjan winnerd...@gmail.comwrote:
validators = {
'approved': 'BooleanType',
'text': 'UTF8Type',
'favorite_count':'IntegerType',
'retweet_count': 'IntegerType',
'expanded_url': 'UTF8Type',
'tuid': 'LongType',
'screen_name': 'UTF8Type',
'profile_image': 'UTF8Type',
'embedly_data': 'CompositeType',
'created_at':'UTF8Type',
}
SYSTEM_MANAGER.create_column_family('Narrative','Twitter_search_test',
comparator_type='CompositeType', default_validation_class='UTF8Type',
key_validation_class='UTF8Type', column_validation_classes=validators)
throws:
*InvalidRequestException*: InvalidRequestException(why='Invalid
definition for comparator org.apache.cassandra.db.marshal.CompositeType.
Can you please explain why?
On Wed, Dec 11, 2013 at 12:08 PM, Tyler Hobbs ty...@datastax.com wrote:
What options are available depends on what version of Cassandra you're
using.
You can specify the row key type with 'key_validation_class'.
For column types, use 'column_validation_classes', which is a dict
mapping column names to types. For example:
sys.create_column_family('mykeyspace', 'users',
column_validation_classes={'username': UTF8Type, 'age': IntegerType})
On Wed, Dec 11, 2013 at 10:32 AM, Kumar Ranjan winnerd...@gmail.comwrote:
What are the all possible values for cf_kwargs ??
SYSTEM_MANAGER.create_column_family('Narrative','Twitter_search_test',
comparator_type=UTF8Type, )
- Here I want to specify, Column data types and row key type. How
can I do that ?
On Thu, Aug 15, 2013 at 12:30 PM, Tyler Hobbs ty...@datastax.comwrote:
The column_validation_classes arg is just for defining individual
column types. Glad you got it figured out, though.
On Thu, Aug 15, 2013 at 11:23 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
Thanks for quick reply. Apparantly, I was trying this to get working
cf_kwargs = {'default_validation_class':COUNTER_COLUMN_TYPE}
sys.create_column_family('my_ks', 'vote_count',
column_validation_classes=cf_kwargs) #1
But this works:
sys.create_column_family('my_ks', 'vote_count', **cf_kwargs) #2
I thought #1 should work.
On Thu, Aug 15, 2013 at 9:15 PM, Tyler Hobbs ty...@datastax.comwrote:
The only thing that makes a CF a counter CF is that the default
validation class is CounterColumnType, which you can set through
SystemManager.create_column_family().
On Thu, Aug 15, 2013 at 10:38 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
I do not find a way to create a counter column family in Pycassa.
This[1] does not help.
Appreciate if someone can help me.
Thanks
1.
http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
Re: Maintaining counter column consistency
Hi Ben, If you make sure R + W N you should be fine. Have a read of this http://www.slideshare.net/benjaminblack/introduction-to-cassandra-replication-and-consistency Thanks, H On 1 Oct 2013, at 18:29, Ben Hood 0x6e6...@gmail.comhttp://gmail.com wrote: Hi, We're maintaining a bunch of application specific counters that are incremented on a per event basis just after the event has been inserted. Given the fact that they can get of sync, we were wondering if there are any best practices or just plain real world experience for handling the consistency of these counters? The application could tolerate an inconsistency for a while, so I'm not sure that the cost of any full-on ACID semantics (should they actually be possible in Cassandra) would be justified. So the first inclination was to issue the increment after the insert and hope for the best. Then at some later point, we would run a reconciliation on the underlying data in the column family and compare this with the counter values. Obviously you can only do this once a counter column has gone cold - i.e. it wouldn't make sense to reconcile something that could still get incremented. Does it make sense to put the insert and increment in a CQL batch? Does anybody have any high level advice for this design deliberation? Cheers, Ben
Re: Maintaining counter column consistency
Hi Haithem, I might have phrased my question wrongly - I wasn't referring to the considerations of consistency level or replication factors - I was referring to fact that I want to insert a row and increment a counter in the same operation. I was concerned about the inconsistency that could arise if the counter increment failed, after the underlying record on which the increment was based succeeded. So I wasn't talking about the consistency between Cassandra nodes, rather the consistency between an idempotent base record and an non-idempotent summary counter. Cheers, Ben On October 2, 2013 at 10:09:40 AM, Haithem Jarraya (a-hjarr...@expedia.com) wrote: Hi Ben, If you make sure R + W N you should be fine. Have a read of this http://www.slideshare.net/benjaminblack/introduction-to-cassandra-replication-and-consistency Thanks, H On 1 Oct 2013, at 18:29, Ben Hood 0x6e6...@gmail.com wrote: Hi, We're maintaining a bunch of application specific counters that are incremented on a per event basis just after the event has been inserted. Given the fact that they can get of sync, we were wondering if there are any best practices or just plain real world experience for handling the consistency of these counters? The application could tolerate an inconsistency for a while, so I'm not sure that the cost of any full-on ACID semantics (should they actually be possible in Cassandra) would be justified. So the first inclination was to issue the increment after the insert and hope for the best. Then at some later point, we would run a reconciliation on the underlying data in the column family and compare this with the counter values. Obviously you can only do this once a counter column has gone cold - i.e. it wouldn't make sense to reconcile something that could still get incremented. Does it make sense to put the insert and increment in a CQL batch? Does anybody have any high level advice for this design deliberation? Cheers, Ben
Maintaining counter column consistency
Hi, We're maintaining a bunch of application specific counters that are incremented on a per event basis just after the event has been inserted. Given the fact that they can get of sync, we were wondering if there are any best practices or just plain real world experience for handling the consistency of these counters? The application could tolerate an inconsistency for a while, so I'm not sure that the cost of any full-on ACID semantics (should they actually be possible in Cassandra) would be justified. So the first inclination was to issue the increment after the insert and hope for the best. Then at some later point, we would run a reconciliation on the underlying data in the column family and compare this with the counter values. Obviously you can only do this once a counter column has gone cold - i.e. it wouldn't make sense to reconcile something that could still get incremented. Does it make sense to put the insert and increment in a CQL batch? Does anybody have any high level advice for this design deliberation? Cheers, Ben
How to create counter column family via Pycassa?
I do not find a way to create a counter column family in Pycassa. This[1] does not help. Appreciate if someone can help me. Thanks 1. http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family
Re: How to create counter column family via Pycassa?
The only thing that makes a CF a counter CF is that the default validation class is CounterColumnType, which you can set through SystemManager.create_column_family(). On Thu, Aug 15, 2013 at 10:38 AM, Pinak Pani nishant.has.a.quest...@gmail.com wrote: I do not find a way to create a counter column family in Pycassa. This[1] does not help. Appreciate if someone can help me. Thanks 1. http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family -- Tyler Hobbs DataStax http://datastax.com/
Re: How to create counter column family via Pycassa?
Thanks for quick reply. Apparantly, I was trying this to get working
cf_kwargs = {'default_validation_class':COUNTER_COLUMN_TYPE}
sys.create_column_family('my_ks', 'vote_count',
column_validation_classes=cf_kwargs) #1
But this works:
sys.create_column_family('my_ks', 'vote_count', **cf_kwargs) #2
I thought #1 should work.
On Thu, Aug 15, 2013 at 9:15 PM, Tyler Hobbs ty...@datastax.com wrote:
The only thing that makes a CF a counter CF is that the default validation
class is CounterColumnType, which you can set through
SystemManager.create_column_family().
On Thu, Aug 15, 2013 at 10:38 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
I do not find a way to create a counter column family in Pycassa.
This[1] does not help.
Appreciate if someone can help me.
Thanks
1.
http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family
--
Tyler Hobbs
DataStax http://datastax.com/
Re: How to create counter column family via Pycassa?
The column_validation_classes arg is just for defining individual column
types. Glad you got it figured out, though.
On Thu, Aug 15, 2013 at 11:23 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
Thanks for quick reply. Apparantly, I was trying this to get working
cf_kwargs = {'default_validation_class':COUNTER_COLUMN_TYPE}
sys.create_column_family('my_ks', 'vote_count',
column_validation_classes=cf_kwargs) #1
But this works:
sys.create_column_family('my_ks', 'vote_count', **cf_kwargs) #2
I thought #1 should work.
On Thu, Aug 15, 2013 at 9:15 PM, Tyler Hobbs ty...@datastax.com wrote:
The only thing that makes a CF a counter CF is that the default
validation class is CounterColumnType, which you can set through
SystemManager.create_column_family().
On Thu, Aug 15, 2013 at 10:38 AM, Pinak Pani
nishant.has.a.quest...@gmail.com wrote:
I do not find a way to create a counter column family in Pycassa.
This[1] does not help.
Appreciate if someone can help me.
Thanks
1.
http://pycassa.github.io/pycassa/api/pycassa/system_manager.html#pycassa.system_manager.SystemManager.create_column_family
--
Tyler Hobbs
DataStax http://datastax.com/
--
Tyler Hobbs
DataStax http://datastax.com/
Re: Added extra column as composite key while creation counter column family
On Tue, Apr 16, 2013 at 10:29 PM, Kuldeep Mishra kuld.cs.mis...@gmail.comwrote: cassandra 1.2.0 Is it a bug in 1.2.0 ? While I can't speak to this specific issue, 1.2.0 has meaningful known issues. I suggest upgrade to 1.2.3(/4) ASAP. =Rob
Re: Added extra column as composite key while creation counter column family
cassandra 1.2.0
Is it a bug in 1.2.0 ?
Thanks
KK
On Wed, Apr 17, 2013 at 2:56 AM, aaron morton aa...@thelastpickle.comwrote:
What version are you using ?
WIth 1.2.4 …
cqlsh:dev CREATE TABLE counters (
... key text,
... value counter,
... PRIMARY KEY (key)
... ) WITH COMPACT STORAGE;
cqlsh:dev describe table counters;
CREATE TABLE counters (
key text PRIMARY KEY,
value counter
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
Cheers
-
Aaron Morton
Freelance Cassandra Consultant
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 15/04/2013, at 3:58 PM, Kuldeep Mishra kuld.cs.mis...@gmail.com
wrote:
Hi,
While I creating counter column family a extra column is being added
what I do ?
Table creation script
CREATE TABLE counters (
key text,
value counter,
PRIMARY KEY (key)
) WITH COMPACT STORAGE
after describing column family I am getting following
CREATE TABLE counters (
key text,
column1 text,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
extra column column1 is added
Please help
--
Thanks and Regards
Kuldeep Kumar Mishra
+919540965199
--
Thanks and Regards
Kuldeep Kumar Mishra
+919540965199
Added extra column as composite key while creation counter column family
Hi,
While I creating counter column family a extra column is being added
what I do ?
Table creation script
CREATE TABLE counters (
key text,
value counter,
PRIMARY KEY (key)
) WITH COMPACT STORAGE
after describing column family I am getting following
CREATE TABLE counters (
key text,
* column1 text,*
value counter,
PRIMARY KEY (key,* column1*)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
extra column column1 is added
Please help
--
Thanks and Regards
Kuldeep Kumar Mishra
+919540965199
Re: Reading a counter column value
it's just a standard get_slice call, the ColumnOrSuperColumn result has either Column, SuperColumn, CounterColumn or SuperCounterColumn set. Cheers - Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 18/03/2013, at 2:09 AM, Chidambaran Subramanian chi...@gmail.com wrote: I created a CounterColumnType column family . create column family counter1 with default_validation_class=CounterColumnType and replicate_on_write=true; Unfortunately I could not figure out a way to read the column value. I am using the Thrift API , and I couldn't make out any sense. I can't use anything else right now, since I have a bunch of stuff already written using Thrift. Any help is really appreciated. Regards Chiddu
Reading a counter column value
I created a CounterColumnType column family . create column family counter1 with default_validation_class=CounterColumnType and replicate_on_write=true; Unfortunately I could not figure out a way to read the column value. I am using the Thrift API , and I couldn't make out any sense. I can't use anything else right now, since I have a bunch of stuff already written using Thrift. Any help is really appreciated. Regards Chiddu
Re: Can I create a counter column family with many rows in 1.1.10?
What would be the exact CQL3 syntax to create a counter CF with composite row key and not predefined column names ? Is the following supposed to work ? CREATE TABLE composite_counter ( aid text, key1 text, key2 text, key3 text, value counter, PRIMARY KEY (aid, key1, key2, key3) ) First, when I do so I have no error shown, but I *can't* see this CF appear in my OpsCenter. update composite_counter set value = value + 5 where aid = '1' and key1 = 'test1' and key2 = 'test2' and key3 = 'test3'; works as expected too. But how can I have multiple counter columns using the schemaless property of cassandra ? I mean before, when I created counter CF with cli, things like this used to work: update composite_counter set 'value2' = 'value2' + 5 where aid = '1' and key1 = 'test1' and key2 = 'test2' and key3 = 'test3'; = Bad Request: line 1:29 no viable alternative at input 'value2' I also tried: update composite_counter set value2 = value2 + 5 where aid = '1' and key1 = 'test1' and key2 = 'test2' and key3 = 'test3'; = Bad Request: Unknown identifier value2 (as expected I guess) I want to make a counter CF with composite keys and a lot of counters using this pattern 20130306#event or (20130306, event), not sure if I should use composite columns there. Is it mandatory to create the CF with at least one column with the counter type ? I mean I will probably never use a column named 'value', I defined it just to be sure the CF is defined as a counter CF. 2013/3/6 Abhijit Chanda abhijit.chan...@gmail.com Thanks @aaron for the rectification On Wed, Mar 6, 2013 at 1:17 PM, aaron morton aa...@thelastpickle.comwrote: Note that CQL 3 in 1.1 is compatible with CQL 3 in 1.2. Also you do not have to use CQL 3, you can still use the cassandra-cli to create CF's. The syntax you use to populate it depends on the client you are using. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 5/03/2013, at 9:16 PM, Abhijit Chanda abhijit.chan...@gmail.com wrote: Yes you can , you just have to use CQL3 and 1.1.10 onward cassandra supports CQL3. Just you have to aware of the fact that a column family that contains a counter column can only contain counters. In other other words either all the columns of the column family excluding KEY have the counter type or none of them can have it. Best Regards, -- Abhijit Chanda +91-974395 -- Abhijit Chanda +91-974395
RE: Can I create a counter column family with many rows in 1.1.10?
Ah, I'ts with many columns, not rows. I use this in cql 2-3 create table cnt (key text PRIMARY KEY, y2003 counter, y2004 counter);it says this is not a counter column family, and if I try to use default_validation_class=CounterType,it says this is not a valid keyword.What I'm supposed to type in order to create it? From: aa...@thelastpickle.com Subject: Re: Can I create a counter column family with many rows in 1.1.10? Date: Tue, 5 Mar 2013 23:47:38 -0800 To: user@cassandra.apache.org Note that CQL 3 in 1.1 is compatible with CQL 3 in 1.2. Also you do not have to use CQL 3, you can still use the cassandra-cli to create CF's. The syntax you use to populate it depends on the client you are using. Cheers -Aaron MortonFreelance Cassandra DeveloperNew Zealand @aaronmortonhttp://www.thelastpickle.com On 5/03/2013, at 9:16 PM, Abhijit Chanda abhijit.chan...@gmail.com wrote:Yes you can , you just have to use CQL3 and 1.1.10 onward cassandra supports CQL3. Just you have to aware of the fact that a column family that contains a counter column can only contain counters. In other other words either all the columns of the column family excluding KEY have the counter type or none of them can have it. Best Regards, -- Abhijit Chanda +91-974395
RE: Can I create a counter column family with many rows in 1.1.10?
I got it now. From: mateus.ffrei...@hotmail.com To: user@cassandra.apache.org Subject: RE: Can I create a counter column family with many rows in 1.1.10? Date: Wed, 6 Mar 2013 08:42:37 -0300 Ah, I'ts with many columns, not rows. I use this in cql 2-3 create table cnt (key text PRIMARY KEY, y2003 counter, y2004 counter);it says this is not a counter column family, and if I try to use default_validation_class=CounterType,it says this is not a valid keyword.What I'm supposed to type in order to create it? From: aa...@thelastpickle.com Subject: Re: Can I create a counter column family with many rows in 1.1.10? Date: Tue, 5 Mar 2013 23:47:38 -0800 To: user@cassandra.apache.org Note that CQL 3 in 1.1 is compatible with CQL 3 in 1.2. Also you do not have to use CQL 3, you can still use the cassandra-cli to create CF's. The syntax you use to populate it depends on the client you are using. Cheers -Aaron MortonFreelance Cassandra DeveloperNew Zealand @aaronmortonhttp://www.thelastpickle.com On 5/03/2013, at 9:16 PM, Abhijit Chanda abhijit.chan...@gmail.com wrote:Yes you can , you just have to use CQL3 and 1.1.10 onward cassandra supports CQL3. Just you have to aware of the fact that a column family that contains a counter column can only contain counters. In other other words either all the columns of the column family excluding KEY have the counter type or none of them can have it. Best Regards, -- Abhijit Chanda +91-974395
Re: Can I create a counter column family with many rows in 1.1.10?
If you have one column in the table that is not part of the primary key and is a counter, then all columns that are not part of the primary key must also be a counter. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 6/03/2013, at 2:56 AM, Alain RODRIGUEZ arodr...@gmail.com wrote: What would be the exact CQL3 syntax to create a counter CF with composite row key and not predefined column names ? Is the following supposed to work ? CREATE TABLE composite_counter ( aid text, key1 text, key2 text, key3 text, value counter, PRIMARY KEY (aid, key1, key2, key3) ) First, when I do so I have no error shown, but I *can't* see this CF appear in my OpsCenter. update composite_counter set value = value + 5 where aid = '1' and key1 = 'test1' and key2 = 'test2' and key3 = 'test3'; works as expected too. But how can I have multiple counter columns using the schemaless property of cassandra ? I mean before, when I created counter CF with cli, things like this used to work: update composite_counter set 'value2' = 'value2' + 5 where aid = '1' and key1 = 'test1' and key2 = 'test2' and key3 = 'test3'; = Bad Request: line 1:29 no viable alternative at input 'value2' I also tried: update composite_counter set value2 = value2 + 5 where aid = '1' and key1 = 'test1' and key2 = 'test2' and key3 = 'test3'; = Bad Request: Unknown identifier value2 (as expected I guess) I want to make a counter CF with composite keys and a lot of counters using this pattern 20130306#event or (20130306, event), not sure if I should use composite columns there. Is it mandatory to create the CF with at least one column with the counter type ? I mean I will probably never use a column named 'value', I defined it just to be sure the CF is defined as a counter CF. 2013/3/6 Abhijit Chanda abhijit.chan...@gmail.com Thanks @aaron for the rectification On Wed, Mar 6, 2013 at 1:17 PM, aaron morton aa...@thelastpickle.com wrote: Note that CQL 3 in 1.1 is compatible with CQL 3 in 1.2. Also you do not have to use CQL 3, you can still use the cassandra-cli to create CF's. The syntax you use to populate it depends on the client you are using. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 5/03/2013, at 9:16 PM, Abhijit Chanda abhijit.chan...@gmail.com wrote: Yes you can , you just have to use CQL3 and 1.1.10 onward cassandra supports CQL3. Just you have to aware of the fact that a column family that contains a counter column can only contain counters. In other other words either all the columns of the column family excluding KEY have the counter type or none of them can have it. Best Regards, -- Abhijit Chanda +91-974395 -- Abhijit Chanda +91-974395
Re: Can I create a counter column family with many rows in 1.1.10?
Note that CQL 3 in 1.1 is compatible with CQL 3 in 1.2. Also you do not have to use CQL 3, you can still use the cassandra-cli to create CF's. The syntax you use to populate it depends on the client you are using. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 5/03/2013, at 9:16 PM, Abhijit Chanda abhijit.chan...@gmail.com wrote: Yes you can , you just have to use CQL3 and 1.1.10 onward cassandra supports CQL3. Just you have to aware of the fact that a column family that contains a counter column can only contain counters. In other other words either all the columns of the column family excluding KEY have the counter type or none of them can have it. Best Regards, -- Abhijit Chanda +91-974395
Re: Can I create a counter column family with many rows in 1.1.10?
Thanks @aaron for the rectification On Wed, Mar 6, 2013 at 1:17 PM, aaron morton aa...@thelastpickle.comwrote: Note that CQL 3 in 1.1 is compatible with CQL 3 in 1.2. Also you do not have to use CQL 3, you can still use the cassandra-cli to create CF's. The syntax you use to populate it depends on the client you are using. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 5/03/2013, at 9:16 PM, Abhijit Chanda abhijit.chan...@gmail.com wrote: Yes you can , you just have to use CQL3 and 1.1.10 onward cassandra supports CQL3. Just you have to aware of the fact that a column family that contains a counter column can only contain counters. In other other words either all the columns of the column family excluding KEY have the counter type or none of them can have it. Best Regards, -- Abhijit Chanda +91-974395 -- Abhijit Chanda +91-974395
Re: Operation Consideration with Counter Column Families
Thanks Aaron, so will there only be one value for each counter column per sstable just like regular columns? Yes. For some reason I was under the impression that Cassandra keeps a log of all the increments not the actual value. Not as far as I understand. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 6/02/2013, at 11:15 AM, Drew Kutcharian d...@venarc.com wrote: Thanks Aaron, so will there only be one value for each counter column per sstable just like regular columns? For some reason I was under the impression that Cassandra keeps a log of all the increments not the actual value. On Feb 5, 2013, at 12:36 PM, aaron morton aa...@thelastpickle.com wrote: Are there any specific operational considerations one should make when using counter columns families? Performance, as they incur a read and a write. There were some issues with overcounts in log replay (see the changes.txt). How are counter column families stored on disk? Same as regular CF's. How do they effect compaction? None. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 6/02/2013, at 7:47 AM, Drew Kutcharian d...@venarc.com wrote: Hey Guys, Are there any specific operational considerations one should make when using counter columns families? How are counter column families stored on disk? How do they effect compaction? -- Drew
Operation Consideration with Counter Column Families
Hey Guys, Are there any specific operational considerations one should make when using counter columns families? How are counter column families stored on disk? How do they effect compaction? -- Drew
Re: Operation Consideration with Counter Column Families
Are there any specific operational considerations one should make when using counter columns families? Performance, as they incur a read and a write. There were some issues with overcounts in log replay (see the changes.txt). How are counter column families stored on disk? Same as regular CF's. How do they effect compaction? None. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 6/02/2013, at 7:47 AM, Drew Kutcharian d...@venarc.com wrote: Hey Guys, Are there any specific operational considerations one should make when using counter columns families? How are counter column families stored on disk? How do they effect compaction? -- Drew
Re: Operation Consideration with Counter Column Families
Thanks Aaron, so will there only be one value for each counter column per sstable just like regular columns? For some reason I was under the impression that Cassandra keeps a log of all the increments not the actual value. On Feb 5, 2013, at 12:36 PM, aaron morton aa...@thelastpickle.com wrote: Are there any specific operational considerations one should make when using counter columns families? Performance, as they incur a read and a write. There were some issues with overcounts in log replay (see the changes.txt). How are counter column families stored on disk? Same as regular CF's. How do they effect compaction? None. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 6/02/2013, at 7:47 AM, Drew Kutcharian d...@venarc.com wrote: Hey Guys, Are there any specific operational considerations one should make when using counter columns families? How are counter column families stored on disk? How do they effect compaction? -- Drew
Re: Counter column families (pending replicate on write stage tasks)
On Mon, Nov 12, 2012 at 3:35 PM, cem cayiro...@gmail.com wrote: We are currently facing a performance issue with counter column families. I see lots of pending ReplicateOnWriteStage tasks in tpstats. Then I disabled replicate_on_write. It helped a lot. I want to use like that but I am not sure how to use it. Quoting Sylvain, one of the primary authors/maintainers of the Counters support... https://issues.apache.org/jira/browse/CASSANDRA-3868 I don't disagree about the efficiency of the valve, but at what price? 'Bootstrapping a node will make you lose increments (you don't know which ones, you don't know how many and this even if nothing goes wrong)' is a pretty bad drawback. That is pretty much why that option makes me uncomfortable: it does give you better performance, so people may be tempted to use it. Now if it was only a matter of replicating writes only through read-repair/repair, then ok, it's pretty dangerous but it's rather easy to explain/understand the drawback (if you don't lose a disk, you don't lose increments, and you'd better use CL.ALL or have read_repair_chance to 1). But the fact that it doesn't work with bootstrap/move makes me wonder if having the option at all is not making a disservice to users. IOW, don't be tempted to turn off replicate_on_write. It breaks counters. If you are under capacity at a steady state, increase capacity. =Rob -- =Robert Coli AIMGTALK - rc...@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb
Re: Counter column family
My problem was the result of Hector bug, see http://groups.google.com/group/hector-users/browse_thread/thread/8359538ed387564e So please ignore question, Thanks, *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Tue, Apr 17, 2012 at 6:59 PM, Tamar Fraenkel ta...@tok-media.com wrote: Hi! I want to understand how incrementing of counter works. - I have a 3 node ring, - I use FailoverPolicy.FAIL_FAST, - RF is 2, I have the following counter column family ColumnFamily: tk_counters Key Validation Class: org.apache.cassandra.db.marshal.CompositeType( org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal. UUIDType) Default column value validator: org.apache.cassandra.db.marshal. CounterColumnType Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type Row cache size / save period in seconds / keys to save : 0.0/0/all Row Cache Provider: org.apache.cassandra.cache. SerializingCacheProvider Key cache size / save period in seconds: 0.0/14400 GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Bloom Filter FP chance: default Built indexes: [] Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy My CL for this column family is Write=2, Read=1. When I increment a counter (using hector mutator), and execute returns without errors, what is the status of the nodes at that stage. Can execute return before the nodes are really updated? So that if a read is done immediately after the increment it will still read the previous values? Thanks, *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 tokLogo.png
Counter column family
Hi! I want to understand how incrementing of counter works. - I have a 3 node ring, - I use FailoverPolicy.FAIL_FAST, - RF is 2, I have the following counter column family ColumnFamily: tk_counters Key Validation Class: org.apache.cassandra.db.marshal.CompositeType( org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal. UUIDType) Default column value validator: org.apache.cassandra.db.marshal. CounterColumnType Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type Row cache size / save period in seconds / keys to save : 0.0/0/all Row Cache Provider: org.apache.cassandra.cache. SerializingCacheProvider Key cache size / save period in seconds: 0.0/14400 GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Bloom Filter FP chance: default Built indexes: [] Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy My CL for this column family is Write=2, Read=1. When I increment a counter (using hector mutator), and execute returns without errors, what is the status of the nodes at that stage. Can execute return before the nodes are really updated? So that if a read is done immediately after the increment it will still read the previous values? Thanks, *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 tokLogo.png
Re: counter column family
You may have better luck with Hector specific questions on the Hector User Group https://groups.google.com/forum/?fromgroups#!forum/hector-users Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 4/04/2012, at 5:54 PM, Tamar Fraenkel wrote: Hi! So, if I am using Hector, I need to do: cassandraHostConfigurator.setRetryDownedHosts(false)? How will this affect my application generally? Thanks Tamar Fraenkel Senior Software Engineer, TOK Media tokLogo.png ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Tue, Mar 27, 2012 at 4:25 PM, R. Verlangen ro...@us2.nl wrote: You should use a connection pool without retries to prevent a single increment of +1 have a result of e.g. +3. 2012/3/27 Rishabh Agrawal rishabh.agra...@impetus.co.in You can even define how much increment you want. But let me just warn you, as far my knowledge, it has consistency issues. From: puneet loya [mailto:puneetl...@gmail.com] Sent: Tuesday, March 27, 2012 5:59 PM To: user@cassandra.apache.org Subject: Re: counter column family thanxx a ton :) :) the counter column family works synonymous as 'auto increment' in other databases rite? I mean we have a column of type integer which increments with every insert. Am i goin the rite way?? please reply :) On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl wrote: create column family MyCounterColumnFamily with default_validation_class=CounterColumnType and key_validation_class=UTF8Type and comparator=UTF8Type; There you go! Keys must be utf8, as well as the column names. Of course you can change those validators. Cheers! 2012/3/27 puneet loya puneetl...@gmail.com Can u give an example of create column family with counter column in it. Please reply Regards, Puneet Loya -- With kind regards, Robin Verlangen www.robinverlangen.nl Impetus to sponsor and exhibit at Structure Data 2012, NY; Mar 21-22. Know more about our Big Data quick-start program at the event. New Impetus webcast ‘Cloud-enabled Performance Testing vis-à-vis On-premise’ available at http://bit.ly/z6zT4L. NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference. -- With kind regards, Robin Verlangen www.robinverlangen.nl
Counter Column
I have encountered the following piece of information regarding the use of ‘Counter Column’ in Cassandra: “If a write fails unexpectedly (timeout or loss of connection to the coordinator node) the client will not know if the operation has been performed. A retry can result in an over count” (- quoted from http://wiki.apache.org/cassandra/Counters). is it still relevant? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Counter-Column-tp7432010p7432010.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: Counter Column
On Tue, Apr 3, 2012 at 9:11 AM, Avi-h avih...@gmail.com wrote: I have encountered the following piece of information regarding the use of ‘Counter Column’ in Cassandra: “If a write fails unexpectedly (timeout or loss of connection to the coordinator node) the client will not know if the operation has been performed. A retry can result in an over count” (- quoted from http://wiki.apache.org/cassandra/Counters). is it still relevant? It is (there is an open ticket to fix, https://issues.apache.org/jira/browse/CASSANDRA-2495, but to be honest we don't really have a good solution to fix it so far). -- Sylvain
Re: Counter Column
Again, it will be relevant until CASSANDRA-2495 is fixed. Until then (then being undefined so far), it affects all version that have counters (including 1.0.8). -- Sylvain On Tue, Apr 3, 2012 at 12:23 PM, Avi-h avih...@gmail.com wrote: this bug is for 0.8 beta 1, is it also relevant for 1.0.8? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Counter-Column-tp7432010p7432450.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: Counter Column
Sylvain explained a lot of things about counters at Cassandra SF 2011 : http://blip.tv/datastax/counters-in-cassandra-5497678 (video), http://www.datastax.com/wp-content/uploads/2011/07/cassandra_sf_counters.pdf(slides). I think it is always important knowing how the things work. Alain 2012/4/3 Sylvain Lebresne sylv...@datastax.com Again, it will be relevant until CASSANDRA-2495 is fixed. Until then (then being undefined so far), it affects all version that have counters (including 1.0.8). -- Sylvain On Tue, Apr 3, 2012 at 12:23 PM, Avi-h avih...@gmail.com wrote: this bug is for 0.8 beta 1, is it also relevant for 1.0.8? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Counter-Column-tp7432010p7432450.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
RE: Counter Column
Right, it affects every version of Cassandra from 0.8 beta 1 until the Fix Version, which right now is None, so it isn't fixed yet... From: Avi-h [avih...@gmail.com] Sent: Tuesday, April 03, 2012 5:23 AM To: cassandra-u...@incubator.apache.org Subject: Re: Counter Column this bug is for 0.8 beta 1, is it also relevant for 1.0.8? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Counter-Column-tp7432010p7432450.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: counter column family
Hi! So, if I am using Hector, I need to do: cassandraHostConfigurator.setRetryDownedHosts(false)? How will this affect my application generally? Thanks *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Tue, Mar 27, 2012 at 4:25 PM, R. Verlangen ro...@us2.nl wrote: You should use a connection pool without retries to prevent a single increment of +1 have a result of e.g. +3. 2012/3/27 Rishabh Agrawal rishabh.agra...@impetus.co.in You can even define how much increment you want. But let me just warn you, as far my knowledge, it has consistency issues. *From:* puneet loya [mailto:puneetl...@gmail.com] *Sent:* Tuesday, March 27, 2012 5:59 PM *To:* user@cassandra.apache.org *Subject:* Re: counter column family thanxx a ton :) :) the counter column family works synonymous as 'auto increment' in other databases rite? I mean we have a column of type integer which increments with every insert. Am i goin the rite way?? please reply :) On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl wrote: *create column family MyCounterColumnFamily with default_validation_class=CounterColumnType and key_validation_class=UTF8Type and comparator=UTF8Type;* There you go! Keys must be utf8, as well as the column names. Of course you can change those validators. Cheers! 2012/3/27 puneet loya puneetl...@gmail.com Can u give an example of create column family with counter column in it. Please reply Regards, Puneet Loya -- With kind regards, Robin Verlangen www.robinverlangen.nl -- Impetus to sponsor and exhibit at Structure Data 2012, NY; Mar 21-22. Know more about our Big Data quick-start program at the event. New Impetus webcast ‘Cloud-enabled Performance Testing vis-à-vis On-premise’ available at http://bit.ly/z6zT4L. NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference. -- With kind regards, Robin Verlangen www.robinverlangen.nl tokLogo.png
Re: counter column family
On Tue, Mar 27, 2012 at 9:35 AM, puneet loya puneetl...@gmail.com wrote: now i want to have a field incrementing with every row insertion. how do i do it in cassandra?? There's nothing that will do it automatically. You need to increment it yourself. -- Tyler Hobbs DataStax http://datastax.com/
counter column family
Can u give an example of create column family with counter column in it. Please reply Regards, Puneet Loya
Re: counter column family
*create column family MyCounterColumnFamily with default_validation_class=CounterColumnType and key_validation_class=UTF8Type and comparator=UTF8Type;* There you go! Keys must be utf8, as well as the column names. Of course you can change those validators. Cheers! 2012/3/27 puneet loya puneetl...@gmail.com Can u give an example of create column family with counter column in it. Please reply Regards, Puneet Loya -- With kind regards, Robin Verlangen www.robinverlangen.nl
Re: counter column family
thanxx a ton :) :) the counter column family works synonymous as 'auto increment' in other databases rite? I mean we have a column of type integer which increments with every insert. Am i goin the rite way?? please reply :) On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl wrote: *create column family MyCounterColumnFamily with default_validation_class=CounterColumnType and key_validation_class=UTF8Type and comparator=UTF8Type;* There you go! Keys must be utf8, as well as the column names. Of course you can change those validators. Cheers! 2012/3/27 puneet loya puneetl...@gmail.com Can u give an example of create column family with counter column in it. Please reply Regards, Puneet Loya -- With kind regards, Robin Verlangen www.robinverlangen.nl
Re: counter column family
You should use a connection pool without retries to prevent a single increment of +1 have a result of e.g. +3. 2012/3/27 Rishabh Agrawal rishabh.agra...@impetus.co.in You can even define how much increment you want. But let me just warn you, as far my knowledge, it has consistency issues. *From:* puneet loya [mailto:puneetl...@gmail.com] *Sent:* Tuesday, March 27, 2012 5:59 PM *To:* user@cassandra.apache.org *Subject:* Re: counter column family thanxx a ton :) :) the counter column family works synonymous as 'auto increment' in other databases rite? I mean we have a column of type integer which increments with every insert. Am i goin the rite way?? please reply :) On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl wrote: *create column family MyCounterColumnFamily with default_validation_class=CounterColumnType and key_validation_class=UTF8Type and comparator=UTF8Type;* There you go! Keys must be utf8, as well as the column names. Of course you can change those validators. Cheers! 2012/3/27 puneet loya puneetl...@gmail.com Can u give an example of create column family with counter column in it. Please reply Regards, Puneet Loya -- With kind regards, Robin Verlangen www.robinverlangen.nl -- Impetus to sponsor and exhibit at Structure Data 2012, NY; Mar 21-22. Know more about our Big Data quick-start program at the event. New Impetus webcast ‘Cloud-enabled Performance Testing vis-à-vis On-premise’ available at http://bit.ly/z6zT4L. NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference. -- With kind regards, Robin Verlangen www.robinverlangen.nl
Re: counter column family
wen i m using a counter column.. i m nt able to add columns of other type
to the column family.. is it so or it is just synactical error??
[default@CMDCv99] create column family status
... with comparator = AsciiType
... and column_metadata =
... [{
... column_name : Test,
... validation_class : IntegerType,
... index_type : 0,
... index_name : IdxName},
... {
... column_name : 'other name',
... validation_class : CounterColumnType
... }];
Cannot add a counter column (other name) in a non counter column family
On Tue, Mar 27, 2012 at 6:55 PM, R. Verlangen ro...@us2.nl wrote:
You should use a connection pool without retries to prevent a single
increment of +1 have a result of e.g. +3.
2012/3/27 Rishabh Agrawal rishabh.agra...@impetus.co.in
You can even define how much increment you want. But let me just warn
you, as far my knowledge, it has consistency issues.
*From:* puneet loya [mailto:puneetl...@gmail.com]
*Sent:* Tuesday, March 27, 2012 5:59 PM
*To:* user@cassandra.apache.org
*Subject:* Re: counter column family
thanxx a ton :) :)
the counter column family works synonymous as 'auto increment' in other
databases rite?
I mean we have a column of type integer which increments with every
insert.
Am i goin the rite way??
please reply :)
On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl wrote:
*create column family MyCounterColumnFamily with
default_validation_class=CounterColumnType and
key_validation_class=UTF8Type and comparator=UTF8Type;*
There you go! Keys must be utf8, as well as the column names. Of course
you can change those validators.
Cheers!
2012/3/27 puneet loya puneetl...@gmail.com
Can u give an example of create column family with counter column in it.
Please reply
Regards,
Puneet Loya
--
With kind regards,
Robin Verlangen
www.robinverlangen.nl
--
Impetus to sponsor and exhibit at Structure Data 2012, NY; Mar 21-22.
Know more about our Big Data quick-start program at the event.
New Impetus webcast ‘Cloud-enabled Performance Testing vis-à-vis
On-premise’ available at http://bit.ly/z6zT4L.
NOTE: This message may contain information that is confidential,
proprietary, privileged or otherwise protected by law. The message is
intended solely for the named addressee. If received in error, please
destroy and notify the sender. Any use of this email is prohibited when
received in error. Impetus does not represent, warrant and/or guarantee,
that the integrity of this communication has been maintained nor that the
communication is free of errors, virus, interception or interference.
--
With kind regards,
Robin Verlangen
www.robinverlangen.nl
Re: counter column family
Counter columns are special, they must be in a column family to themselves.
On 03/27/2012 09:32 AM, puneet loya wrote:
wen i m using a counter column.. i m nt able to add columns of other
type to the column family.. is it so or it is just synactical error??
[default@CMDCv99] create column family status
... with comparator = AsciiType
... and column_metadata =
... [{
... column_name : Test,
... validation_class : IntegerType,
... index_type : 0,
... index_name : IdxName},
... {
... column_name : 'other name',
... validation_class : CounterColumnType
... }];
Cannot add a counter column (other name) in a non counter column family
On Tue, Mar 27, 2012 at 6:55 PM, R. Verlangen ro...@us2.nl
mailto:ro...@us2.nl wrote:
You should use a connection pool without retries to prevent a
single increment of +1 have a result of e.g. +3.
2012/3/27 Rishabh Agrawal rishabh.agra...@impetus.co.in
mailto:rishabh.agra...@impetus.co.in
You can even define how much increment you want. But let me
just warn you, as far my knowledge, it has consistency issues.
*From:*puneet loya [mailto:puneetl...@gmail.com
mailto:puneetl...@gmail.com]
*Sent:* Tuesday, March 27, 2012 5:59 PM
*To:* user@cassandra.apache.org mailto:user@cassandra.apache.org
*Subject:* Re: counter column family
thanxx a ton :) :)
the counter column family works synonymous as 'auto increment'
in other databases rite?
I mean we have a column of type integer which increments with
every insert.
Am i goin the rite way??
please reply :)
On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl
mailto:ro...@us2.nl wrote:
*create column family MyCounterColumnFamily with
default_validation_class=CounterColumnType and
key_validation_class=UTF8Type and comparator=UTF8Type;*
There you go! Keys must be utf8, as well as the column names.
Of course you can change those validators.
Cheers!
2012/3/27 puneet loya puneetl...@gmail.com
mailto:puneetl...@gmail.com
Can u give an example of create column family with counter
column in it.
Please reply
Regards,
Puneet Loya
--
With kind regards,
Robin Verlangen
www.robinverlangen.nl http://www.robinverlangen.nl
Impetus to sponsor and exhibit at Structure Data 2012, NY; Mar
21-22. Know more about our Big Data quick-start program at the
event.
New Impetus webcast ‘Cloud-enabled Performance Testing
vis-à-vis On-premise’ available at http://bit.ly/z6zT4L.
NOTE: This message may contain information that is
confidential, proprietary, privileged or otherwise protected
by law. The message is intended solely for the named
addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in
error. Impetus does not represent, warrant and/or guarantee,
that the integrity of this communication has been maintained
nor that the communication is free of errors, virus,
interception or interference.
--
With kind regards,
Robin Verlangen
www.robinverlangen.nl http://www.robinverlangen.nl
Re: counter column family
now i want to have a field incrementing with every row insertion. how do i
do it in cassandra??
On Tue, Mar 27, 2012 at 7:34 PM, Dave Brosius dbros...@mebigfatguy.comwrote:
Counter columns are special, they must be in a column family to
themselves.
On 03/27/2012 09:32 AM, puneet loya wrote:
wen i m using a counter column.. i m nt able to add columns of other type
to the column family.. is it so or it is just synactical error??
[default@CMDCv99] create column family status
... with comparator = AsciiType
... and column_metadata =
... [{
... column_name : Test,
... validation_class : IntegerType,
... index_type : 0,
... index_name : IdxName},
... {
... column_name : 'other name',
... validation_class : CounterColumnType
... }];
Cannot add a counter column (other name) in a non counter column family
On Tue, Mar 27, 2012 at 6:55 PM, R. Verlangen ro...@us2.nl wrote:
You should use a connection pool without retries to prevent a single
increment of +1 have a result of e.g. +3.
2012/3/27 Rishabh Agrawal rishabh.agra...@impetus.co.in
You can even define how much increment you want. But let me just warn
you, as far my knowledge, it has consistency issues.
*From:* puneet loya [mailto:puneetl...@gmail.com]
*Sent:* Tuesday, March 27, 2012 5:59 PM
*To:* user@cassandra.apache.org
*Subject:* Re: counter column family
thanxx a ton :) :)
the counter column family works synonymous as 'auto increment' in other
databases rite?
I mean we have a column of type integer which increments with every
insert.
Am i goin the rite way??
please reply :)
On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl wrote:
*create column family MyCounterColumnFamily with
default_validation_class=CounterColumnType and
key_validation_class=UTF8Type and comparator=UTF8Type;*
There you go! Keys must be utf8, as well as the column names. Of course
you can change those validators.
Cheers!
2012/3/27 puneet loya puneetl...@gmail.com
Can u give an example of create column family with counter column in it.
Please reply
Regards,
Puneet Loya
--
With kind regards,
Robin Verlangen
www.robinverlangen.nl
--
Impetus to sponsor and exhibit at Structure Data 2012, NY; Mar 21-22.
Know more about our Big Data quick-start program at the event.
New Impetus webcast ‘Cloud-enabled Performance Testing vis-à-vis
On-premise’ available at http://bit.ly/z6zT4L.
NOTE: This message may contain information that is confidential,
proprietary, privileged or otherwise protected by law. The message is
intended solely for the named addressee. If received in error, please
destroy and notify the sender. Any use of this email is prohibited when
received in error. Impetus does not represent, warrant and/or guarantee,
that the integrity of this communication has been maintained nor that the
communication is free of errors, virus, interception or interference.
--
With kind regards,
Robin Verlangen
www.robinverlangen.nl
Counter Column Family Inconsistent Node
[default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=93683) = (counter=FILEUPLOAD, value=347) = (counter=MQUPLOAD, value=14961065367) = (counter=PAGES, value=183089568) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=23972) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943)
Re: Counter Column Family Inconsistent Node
yeah, sorry about that... pushed click before I added my comments. I have a cluster of 5 nodes using 0.8.4 where I am using counters. One one of my nodes, every time I do a list command I get different results. The counters jump all over the place. Any ideas? I have run nodetool repair on all nodes. Thanks! Ryan On Tue, Aug 16, 2011 at 1:18 PM, Ryan Lowe ryanjl...@gmail.com wrote: [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=93683) = (counter=FILEUPLOAD, value=347) = (counter=MQUPLOAD, value=14961065367) = (counter=PAGES, value=183089568) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=23972) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943)
Re: Counter Column Family Inconsistent Node
May be the same as https://issues.apache.org/jira/browse/CASSANDRA-3006 ? On Tue, Aug 16, 2011 at 12:20 PM, Ryan Lowe ryanjl...@gmail.com wrote: yeah, sorry about that... pushed click before I added my comments. I have a cluster of 5 nodes using 0.8.4 where I am using counters. One one of my nodes, every time I do a list command I get different results. The counters jump all over the place. Any ideas? I have run nodetool repair on all nodes. Thanks! Ryan On Tue, Aug 16, 2011 at 1:18 PM, Ryan Lowe ryanjl...@gmail.com wrote: [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=93683) = (counter=FILEUPLOAD, value=347) = (counter=MQUPLOAD, value=14961065367) = (counter=PAGES, value=183089568) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=23972) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Counter Column Family Inconsistent Node
Actually I think it was more related to our servers getting their time out of sync... after finding this article: http://ria101.wordpress.com/2011/02/08/cassandra-the-importance-of-system-clocks-avoiding-oom-and-how-to-escape-oom-meltdown/ I checked our servers, and sure enough, 2 of them were out of sync with each other with more than a 2 min difference! I reconfigured ntp and I think I am back in business. Thanks though! Ryan On Tue, Aug 16, 2011 at 2:53 PM, Jonathan Ellis jbel...@gmail.com wrote: May be the same as https://issues.apache.org/jira/browse/CASSANDRA-3006 ? On Tue, Aug 16, 2011 at 12:20 PM, Ryan Lowe ryanjl...@gmail.com wrote: yeah, sorry about that... pushed click before I added my comments. I have a cluster of 5 nodes using 0.8.4 where I am using counters. One one of my nodes, every time I do a list command I get different results. The counters jump all over the place. Any ideas? I have run nodetool repair on all nodes. Thanks! Ryan On Tue, Aug 16, 2011 at 1:18 PM, Ryan Lowe ryanjl...@gmail.com wrote: [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=93683) = (counter=FILEUPLOAD, value=347) = (counter=MQUPLOAD, value=14961065367) = (counter=PAGES, value=183089568) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=23972) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) 1 Row Returned. [default@Race] list CounterCF; Using default limit of 100 --- RowKey: Stats = (counter=APP, value=7503) = (counter=FILEUPLOAD, value=155) = (counter=MQUPLOAD, value=4726775) = (counter=PAGES, value=131948) = (counter=REST, value=3) = (counter=SOAP, value=44) = (counter=WS, value=1943) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Counter Column
On 27/06/2011 19:19, Sylvain Lebresne wrote: Let me make that simpler. Don't ever use replicate_on_write=false (even if you think that it is what you want, there is a good chance it's not). Obviously, the default is replicate_on_write=true. I may be wrong. But with 0.8.0, I think the default is replicate_on_write=false, you have to declare it explicitly. -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
Re: Counter Column
On Tue, Jun 28, 2011 at 12:53 PM, Donal Zang zan...@ihep.ac.cn wrote: On 27/06/2011 19:19, Sylvain Lebresne wrote: Let me make that simpler. Don't ever use replicate_on_write=false (even if you think that it is what you want, there is a good chance it's not). Obviously, the default is replicate_on_write=true. I may be wrong. But with 0.8.0, I think the default is replicate_on_write=false, you have to declare it explicitly. No, after having checked, you are right, the default is false and it is a bug (literally a bug, the default is true in the code, but it doesn't get applied correctly and it end up defaulting to false uniquely because it is the default value of a boolean in java -- https://issues.apache.org/jira/browse/CASSANDRA-2835). That bug will be fixed in 0.8.2. I sincerely apologize about that, you have to explicitly set replicate_on_write to true for now. The rest stays true. -- Sylvain -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
Re: Counter Column
On 27/06/2011 17:04, Artem Orobets wrote: Hi! As I know, we use counter column only with replication factor ALL, so is it mean that we can't read data while any replica will fail? you can use any consistency level, using replicate_on_write=true when create the counter column family. -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
Re: Counter Column
if i write at ALL and read at ONE,is that setting required ? thanks Le 27 juin 2011 17:22, Donal Zang zan...@ihep.ac.cn a écrit : On 27/06/2011 17:04, Artem Orobets wrote: Hi! As I know, we use counter column only with replication factor ALL, so is it mean that we can't read data while any replica will fail? you can use any consistency level, using replicate_on_write=true when create the counter column family. -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
Re: Counter Column
Let me make that simpler. Don't ever use replicate_on_write=false (even if you think that it is what you want, there is a good chance it's not). Obviously, the default is replicate_on_write=true. -- Sylvain On Mon, Jun 27, 2011 at 7:03 PM, Philippe watche...@gmail.com wrote: if i write at ALL and read at ONE,is that setting required ? thanks Le 27 juin 2011 17:22, Donal Zang zan...@ihep.ac.cn a écrit : On 27/06/2011 17:04, Artem Orobets wrote: Hi! As I know, we use counter column only with replication factor ALL, so is it mean that we can't read data while any replica will fail? you can use any consistency level, using replicate_on_write=true when create the counter column family. -- Donal Zang Computing Center, IHEP 19B YuquanLu, Shijingshan District,Beijing, 100049 zan...@ihep.ac.cn 86 010 8823 6018
Counter Column in Cassandra
Hi, All I am newbie to cassandra. I have a simple question but don't find any clear answer by searching google: What's the meaning of counter column in Cassandra? Best
Re: Counter Column in Cassandra
It's a column whose content represents a distributed counter. http://wiki.apache.org/cassandra/Counters On Mon, Jun 13, 2011 at 8:29 AM, Sijie YANG iyan...@gmail.com wrote: Hi, All I am newbie to cassandra. I have a simple question but don't find any clear answer by searching google: What's the meaning of counter column in Cassandra? Best
Re: expiring + counter column?
sorry to beat on the dead horse. I looked at the link referred from #2103 : https://issues.apache.org/jira/browse/CASSANDRA-2101 I agree with the reasoning in #2101 that the ultimate issue is that delete and counter adds are not commutative. since by definition we can't achieve predictable behavior with deletes + counter, can we redefine the behavior of counter deletes, so that we can always guarantee the declared behavior? --- specifically: *we define that once a counter column is deleted, you can never add to it again.* attempts to add to a dead counter throws an exception all future adds are just ignored. i.e. a counter column has only one life, until all tombstones are purged from system, after which it is possible for the counter to have a new incarnation. basically instead of solving the problem raised in #2103, we declare openly that it's unsolvable (which is true), and make the code reflect this fact. I think this behavior would satisfy most use cases of counters. so instead of relying on the advice to developers: do not do updates for a period after deletes, otherwise it probably wont' work, we enforce this into the code. the same logic can be carried over into expiring column, since they are essentially automatically inserted deletes. that way #2103 could be solved I'm attaching an example below, you can refer to them if needed. Thanks a lot Yang example: for simplicity we assume there is only one column family , one column, so we omit column name and cf name in our notation, assume all counterColumns have a delta value of 1, we only mark their ttl now. so c(123) means a counter column of ttl=1, adding a delta of 1. d(456) means a tombstone with ttl=456. then we can have the following operations operationresult after operation -- c(1)count=1 d(2)count = null ( counter not present ) c(3)count = null ( add on dead counter ignored) --- if the 2 adds arrive out of order , we would still guarantee eventual consistency: operationresult after operation c(1)count=1 c(3)count=2 (we have 2 adds, each with delta=1) d(2)count=null (deleted) -- at the end of both scenarios, the result is guaranteed to be null; note that in the second scenario, line 2 shows a snapshot where we have a state with count=2, which scenario 1 never sees this. this is fine, since even regular columns can have this situation (just consider if the counter columns were inserts/overwrites instead ) On Fri, May 27, 2011 at 5:57 PM, Jonathan Ellis jbel...@gmail.com wrote: No. See comments to https://issues.apache.org/jira/browse/CASSANDRA-2103 On Fri, May 27, 2011 at 7:29 PM, Yang tedd...@gmail.com wrote: is this combination feature available , or on track ? thanks Yang -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: expiring + counter column?
errata: so c(123) means a counter column of ttl=1, so c(123) means a counter column of ttl=123, On Sun, May 29, 2011 at 12:24 AM, Yang tedd...@gmail.com wrote: sorry to beat on the dead horse. I looked at the link referred from #2103 : https://issues.apache.org/jira/browse/CASSANDRA-2101 I agree with the reasoning in #2101 that the ultimate issue is that delete and counter adds are not commutative. since by definition we can't achieve predictable behavior with deletes + counter, can we redefine the behavior of counter deletes, so that we can always guarantee the declared behavior? --- specifically: *we define that once a counter column is deleted, you can never add to it again.* attempts to add to a dead counter throws an exception all future adds are just ignored. i.e. a counter column has only one life, until all tombstones are purged from system, after which it is possible for the counter to have a new incarnation. basically instead of solving the problem raised in #2103, we declare openly that it's unsolvable (which is true), and make the code reflect this fact. I think this behavior would satisfy most use cases of counters. so instead of relying on the advice to developers: do not do updates for a period after deletes, otherwise it probably wont' work, we enforce this into the code. the same logic can be carried over into expiring column, since they are essentially automatically inserted deletes. that way #2103 could be solved I'm attaching an example below, you can refer to them if needed. Thanks a lot Yang example: for simplicity we assume there is only one column family , one column, so we omit column name and cf name in our notation, assume all counterColumns have a delta value of 1, we only mark their ttl now. so c(123) means a counter column of ttl=1, adding a delta of 1. d(456) means a tombstone with ttl=456. then we can have the following operations operationresult after operation -- c(1)count=1 d(2)count = null ( counter not present ) c(3)count = null ( add on dead counter ignored) --- if the 2 adds arrive out of order , we would still guarantee eventual consistency: operationresult after operation c(1)count=1 c(3)count=2 (we have 2 adds, each with delta=1) d(2)count=null (deleted) -- at the end of both scenarios, the result is guaranteed to be null; note that in the second scenario, line 2 shows a snapshot where we have a state with count=2, which scenario 1 never sees this. this is fine, since even regular columns can have this situation (just consider if the counter columns were inserts/overwrites instead ) On Fri, May 27, 2011 at 5:57 PM, Jonathan Ellis jbel...@gmail.com wrote: No. See comments to https://issues.apache.org/jira/browse/CASSANDRA-2103 On Fri, May 27, 2011 at 7:29 PM, Yang tedd...@gmail.com wrote: is this combination feature available , or on track ? thanks Yang -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: expiring + counter column?
sorry in the notation, instead of ttl I mean timestamp On Sun, May 29, 2011 at 12:24 AM, Yang tedd...@gmail.com wrote: sorry to beat on the dead horse. I looked at the link referred from #2103 : https://issues.apache.org/jira/browse/CASSANDRA-2101 I agree with the reasoning in #2101 that the ultimate issue is that delete and counter adds are not commutative. since by definition we can't achieve predictable behavior with deletes + counter, can we redefine the behavior of counter deletes, so that we can always guarantee the declared behavior? --- specifically: *we define that once a counter column is deleted, you can never add to it again.* attempts to add to a dead counter throws an exception all future adds are just ignored. i.e. a counter column has only one life, until all tombstones are purged from system, after which it is possible for the counter to have a new incarnation. basically instead of solving the problem raised in #2103, we declare openly that it's unsolvable (which is true), and make the code reflect this fact. I think this behavior would satisfy most use cases of counters. so instead of relying on the advice to developers: do not do updates for a period after deletes, otherwise it probably wont' work, we enforce this into the code. the same logic can be carried over into expiring column, since they are essentially automatically inserted deletes. that way #2103 could be solved I'm attaching an example below, you can refer to them if needed. Thanks a lot Yang example: for simplicity we assume there is only one column family , one column, so we omit column name and cf name in our notation, assume all counterColumns have a delta value of 1, we only mark their ttl now. so c(123) means a counter column of ttl=1, adding a delta of 1. d(456) means a tombstone with ttl=456. then we can have the following operations operationresult after operation -- c(1)count=1 d(2)count = null ( counter not present ) c(3)count = null ( add on dead counter ignored) --- if the 2 adds arrive out of order , we would still guarantee eventual consistency: operationresult after operation c(1)count=1 c(3)count=2 (we have 2 adds, each with delta=1) d(2)count=null (deleted) -- at the end of both scenarios, the result is guaranteed to be null; note that in the second scenario, line 2 shows a snapshot where we have a state with count=2, which scenario 1 never sees this. this is fine, since even regular columns can have this situation (just consider if the counter columns were inserts/overwrites instead ) On Fri, May 27, 2011 at 5:57 PM, Jonathan Ellis jbel...@gmail.com wrote: No. See comments to https://issues.apache.org/jira/browse/CASSANDRA-2103 On Fri, May 27, 2011 at 7:29 PM, Yang tedd...@gmail.com wrote: is this combination feature available , or on track ? thanks Yang -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: expiring + counter column?
Without commenting on the other parts of the design, this part is not possible attempts to add to a dead counter throws an exception All write operations are no look operations (write to the log, update memtables) we never look at the SSTables. It goes against the architecture of the write path to require a read from disk. Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 29 May 2011, at 20:04, Yang wrote: sorry in the notation, instead of ttl I mean timestamp On Sun, May 29, 2011 at 12:24 AM, Yang tedd...@gmail.com wrote: sorry to beat on the dead horse. I looked at the link referred from #2103 : https://issues.apache.org/jira/browse/CASSANDRA-2101 I agree with the reasoning in #2101 that the ultimate issue is that delete and counter adds are not commutative. since by definition we can't achieve predictable behavior with deletes + counter, can we redefine the behavior of counter deletes, so that we can always guarantee the declared behavior? --- specifically: we define that once a counter column is deleted, you can never add to it again. attempts to add to a dead counter throws an exception all future adds are just ignored. i.e. a counter column has only one life, until all tombstones are purged from system, after which it is possible for the counter to have a new incarnation. basically instead of solving the problem raised in #2103, we declare openly that it's unsolvable (which is true), and make the code reflect this fact. I think this behavior would satisfy most use cases of counters. so instead of relying on the advice to developers: do not do updates for a period after deletes, otherwise it probably wont' work, we enforce this into the code. the same logic can be carried over into expiring column, since they are essentially automatically inserted deletes. that way #2103 could be solved I'm attaching an example below, you can refer to them if needed. Thanks a lot Yang example: for simplicity we assume there is only one column family , one column, so we omit column name and cf name in our notation, assume all counterColumns have a delta value of 1, we only mark their ttl now. so c(123) means a counter column of ttl=1, adding a delta of 1. d(456) means a tombstone with ttl=456. then we can have the following operations operationresult after operation -- c(1)count=1 d(2)count = null ( counter not present ) c(3)count = null ( add on dead counter ignored) --- if the 2 adds arrive out of order , we would still guarantee eventual consistency: operationresult after operation c(1)count=1 c(3)count=2 (we have 2 adds, each with delta=1) d(2)count=null (deleted) -- at the end of both scenarios, the result is guaranteed to be null; note that in the second scenario, line 2 shows a snapshot where we have a state with count=2, which scenario 1 never sees this. this is fine, since even regular columns can have this situation (just consider if the counter columns were inserts/overwrites instead ) On Fri, May 27, 2011 at 5:57 PM, Jonathan Ellis jbel...@gmail.com wrote: No. See comments to https://issues.apache.org/jira/browse/CASSANDRA-2103 On Fri, May 27, 2011 at 7:29 PM, Yang tedd...@gmail.com wrote: is this combination feature available , or on track ? thanks Yang -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: expiring + counter column?
yeah, then maybe we can make that a silent omission. less desirable, but
still better than unpredicted behavior. (this is not that bad: currently you
can't know whether a write result really reached a quorum, i.e. become
effective, anyway)
regarding we never look at SStables, I think right now counter adds do
require a read on SStables, although asynchronously:
StorageProxy:
private static void applyCounterMutation(final IMutation mutation, final
MultimapInetAddress, InetAddress hintedEndpoints, final
IWriteResponseHandler responseHandler, final String localDataCenter, final
ConsistencyLevel consistency_level, boolean executeOnMutationStage) {
..
sendToHintedEndpoints(cm.makeReplicationMutation(), hintedEndpoints,
responseHandler, localDataCenter, false, consistency_level);
}
CounterMutation.java:
public RowMutation makeReplicationMutation() throws IOException {
Table table = Table.open(readCommand.table);
Row row = readCommand.getRow(table);
}
I think the getRow() line above does what the .pdf design doc in the JIRA
described: replication to other replicas (non-leaders) replicate only the
**sum** that I own, not individual delta that I just received. actually I'm
not quite understanding why this approach was chosen, since it makes each
write into read---write (when getReplicateOnWrite() ) , which can be slow.
I'm still trying to understand that
Thanks
Yang
On Sun, May 29, 2011 at 3:45 AM, aaron morton aa...@thelastpickle.comwrote:
Without commenting on the other parts of the design, this part is not
possible attempts to add to a dead counter throws an exception
All write operations are no look operations (write to the log, update
memtables) we never look at the SSTables. It goes against the architecture
of the write path to require a read from disk.
Cheers
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 29 May 2011, at 20:04, Yang wrote:
sorry in the notation, instead of ttl I mean timestamp
On Sun, May 29, 2011 at 12:24 AM, Yang tedd...@gmail.com wrote:
sorry to beat on the dead horse.
I looked at the link referred from #2103 :
https://issues.apache.org/jira/browse/CASSANDRA-2101
I agree with the reasoning in #2101 that the ultimate issue is that delete
and counter adds are not commutative. since by definition we can't achieve
predictable behavior with deletes + counter, can we redefine the behavior
of counter deletes, so that we can always guarantee the declared behavior?
--- specifically:
*we define that once a counter column is deleted, you can never add to it
again.* attempts to add to a dead counter throws an exception all
future adds are just ignored. i.e. a counter column has only one life,
until all tombstones are purged from system, after which it is possible for
the counter to have a new incarnation. basically instead of solving the
problem raised in #2103, we declare openly that it's unsolvable (which is
true), and make the code reflect this fact.
I think this behavior would satisfy most use cases of counters. so instead
of relying on the advice to developers: do not do updates for a period
after deletes, otherwise it probably wont' work, we enforce this into the
code.
the same logic can be carried over into expiring column, since they are
essentially automatically inserted deletes. that way #2103 could be solved
I'm attaching an example below, you can refer to them if needed.
Thanks a lot
Yang
example:
for simplicity we assume there is only one column family , one column, so
we omit column name and cf name in our notation, assume all counterColumns
have a delta value of 1, we only mark their ttl now. so c(123) means a
counter column of ttl=1, adding a delta of 1. d(456) means a tombstone with
ttl=456.
then we can have the following operations
operationresult after operation
--
c(1)count=1
d(2)count = null ( counter not present )
c(3)count = null ( add on dead counter
ignored)
---
if the 2 adds arrive out of order , we would still guarantee eventual
consistency:
operationresult after operation
c(1)count=1
c(3)count=2 (we have 2 adds, each with
delta=1)
d(2)count=null (deleted)
--
at the end of both scenarios, the result is guaranteed to be null;
note that in the second scenario, line 2 shows a snapshot where we have a
state with count=2, which scenario 1 never sees
Re: expiring + counter column?
The comment around line 448 in StorageProxy
// We do the replication on another stage because it
involves a read (see CM.makeReplicationMutation)
// and we want to avoid blocking too much the MUTATION stage
The read happens on another stage, it is not part of the mutation.
And the test before that checks shouldReplicateOnWrite for the CF's involved in
the mutation, which defaults to false.
See also the comments for StorageProxy.mutateCounter() and this comment which I
*think* is still valid
https://issues.apache.org/jira/browse/CASSANDRA-1909?focusedCommentId=12976727page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12976727
Cheers
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 30 May 2011, at 06:41, Yang wrote:
yeah, then maybe we can make that a silent omission. less desirable, but
still better than unpredicted behavior. (this is not that bad: currently you
can't know whether a write result really reached a quorum, i.e. become
effective, anyway)
regarding we never look at SStables, I think right now counter adds do
require a read on SStables, although asynchronously:
StorageProxy:
private static void applyCounterMutation(final IMutation mutation, final
MultimapInetAddress, InetAddress hintedEndpoints, final
IWriteResponseHandler responseHandler, final String localDataCenter, final
ConsistencyLevel consistency_level, boolean executeOnMutationStage) {
..
sendToHintedEndpoints(cm.makeReplicationMutation(), hintedEndpoints,
responseHandler, localDataCenter, false, consistency_level);
}
CounterMutation.java:
public RowMutation makeReplicationMutation() throws IOException {
Table table = Table.open(readCommand.table);
Row row = readCommand.getRow(table);
}
I think the getRow() line above does what the .pdf design doc in the JIRA
described: replication to other replicas (non-leaders) replicate only the
**sum** that I own, not individual delta that I just received. actually I'm
not quite understanding why this approach was chosen, since it makes each
write into read---write (when getReplicateOnWrite() ) , which can be slow.
I'm still trying to understand that
Thanks
Yang
On Sun, May 29, 2011 at 3:45 AM, aaron morton aa...@thelastpickle.com wrote:
Without commenting on the other parts of the design, this part is not
possible attempts to add to a dead counter throws an exception
All write operations are no look operations (write to the log, update
memtables) we never look at the SSTables. It goes against the architecture of
the write path to require a read from disk.
Cheers
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 29 May 2011, at 20:04, Yang wrote:
sorry in the notation, instead of ttl I mean timestamp
On Sun, May 29, 2011 at 12:24 AM, Yang tedd...@gmail.com wrote:
sorry to beat on the dead horse.
I looked at the link referred from #2103 :
https://issues.apache.org/jira/browse/CASSANDRA-2101
I agree with the reasoning in #2101 that the ultimate issue is that delete
and counter adds are not commutative. since by definition we can't achieve
predictable behavior with deletes + counter, can we redefine the behavior of
counter deletes, so that we can always guarantee the declared behavior? ---
specifically:
we define that once a counter column is deleted, you can never add to it
again. attempts to add to a dead counter throws an exception all
future adds are just ignored. i.e. a counter column has only one life,
until all tombstones are purged from system, after which it is possible for
the counter to have a new incarnation. basically instead of solving the
problem raised in #2103, we declare openly that it's unsolvable (which is
true), and make the code reflect this fact.
I think this behavior would satisfy most use cases of counters. so instead
of relying on the advice to developers: do not do updates for a period
after deletes, otherwise it probably wont' work, we enforce this into the
code.
the same logic can be carried over into expiring column, since they are
essentially automatically inserted deletes. that way #2103 could be solved
I'm attaching an example below, you can refer to them if needed.
Thanks a lot
Yang
example:
for simplicity we assume there is only one column family , one column, so we
omit column name and cf name in our notation, assume all counterColumns have
a delta value of 1, we only mark their ttl now. so c(123) means a counter
column of ttl=1, adding a delta of 1. d(456) means a tombstone with ttl=456.
then we can have the following operations
operationresult after
Re: expiring + counter column?
How about implementing a freezing mechanism on counter columns. If there are no more increments within freeze seconds after the last increments (it would be orders or day or so); the column would lock itself on increments and won't accept increment. And after this freeze perioid, the ttl should work fine. The column will be gone forever after freeze + ttl seconds. On Sat, May 28, 2011 at 2:57 AM, Jonathan Ellis jbel...@gmail.com wrote: No. See comments to https://issues.apache.org/jira/browse/CASSANDRA-2103 On Fri, May 27, 2011 at 7:29 PM, Yang tedd...@gmail.com wrote: is this combination feature available , or on track ? thanks Yang -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
expiring + counter column?
is this combination feature available , or on track ? thanks Yang
Re: expiring + counter column?
No. See comments to https://issues.apache.org/jira/browse/CASSANDRA-2103 On Fri, May 27, 2011 at 7:29 PM, Yang tedd...@gmail.com wrote: is this combination feature available , or on track ? thanks Yang -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Counter Column family Cassandra 0.8 PHP Support ?
I'm not a php type person, but I can help a little with thrift. Install thrift 0.6 and then run this in the interface/ directory of the cassandra source... thrift --gen php cassandra.thrift You should end up with the interface/gen-php/ Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 17 May 2011, at 11:26, bhanu choudhary wrote: I am using counters to read the counts on my website dynamically. I am looking for phpcassandra client(?) that supports counters natively. I was looking if any PHP developer could give me a lead in generating the PHP client code required using thrift. Thanks in advance!
