Re: Versioning in cassandra while indexing ?

2015-01-21 Thread Kai Wang
depending on your data model, static column night be useful. https://issues.apache.org/jira/plugins/servlet/mobile#issue/CASSANDRA-6561 On Jan 21, 2015 2:56 AM, "Pandian R" wrote: > Hi, > > I just wanted to know if there is any kind of versioning system in > cassandra while indexing new data(like

Re: Versioning in cassandra while indexing ?

2015-01-21 Thread Pandian R
Awesome. Thanks a lot Graham. Will use the clock timestamp for versioning :) On Wed, Jan 21, 2015 at 2:02 PM, graham sanderson wrote: > I believe you can use “USING TIMESTAMP XXX” with your inserts which will > set the actual cell write times to the timestamp you provide. Then at least > on read

Re: Versioning in cassandra while indexing ?

2015-01-21 Thread graham sanderson
I believe you can use “USING TIMESTAMP XXX” with your inserts which will set the actual cell write times to the timestamp you provide. Then at least on read you’ll get the “latest” value… you may or may not incur an actual write of the old data to disk, but either way it’ll get cleaned up for yo

Versioning in cassandra while indexing ?

2015-01-20 Thread Pandian R
Hi, I just wanted to know if there is any kind of versioning system in cassandra while indexing new data(like the one we have for ElasticSearch, for example). For example, I have a series of payloads each coming with an id and 'updatedAt' timestamp. I just want to maintain the latest state of any

Re: Versioning in cassandra

2013-09-04 Thread dawood abdullah
Thanks for the quick response Michael, looks like I have to go with the solution you have given of maps, as performance is pretty critical for our application and we do not have enough time to test. Appreciate your help. Regards, Dawood On Wed, Sep 4, 2013 at 7:33 PM, Laing, Michael wrote: > Da

Re: Versioning in cassandra

2013-09-04 Thread Laing, Michael
Dawood, In general that will work. However it does mean that you 1) read the old version 2) update the new version and 3) write the archive version. Step 2 is a problem: what if someone else has updated the old version after step 1? and there are 3 atomic operations required, at least. However,

Re: Versioning in cassandra

2013-09-04 Thread dawood abdullah
Michael, Your approach solves the problem, thanks for the solution. I was thinking of another approach as well where in I would create another column family say file_archive, so whenever an update is made to the File table, I will create a new version in the File and move the old version to the ne

Re: Versioning in cassandra

2013-09-03 Thread Laing, Michael
I use the technique described in my previous message to handle millions of messages and their versions. Actually, I use timeuuid's instead of timestamps, as they have more 'uniqueness'. Also I index my maps by a timeuuid that is the complement (based on a future date) of a current timeuuid. Since

Re: Versioning in cassandra

2013-09-03 Thread Vivek Mishra
create table file(id text , parentid text,contenttype text,version timestamp, descr text, name text, PRIMARY KEY(id,version) ) WITH CLUSTERING ORDER BY (version DESC); insert into file (id, parentid, version, contenttype, descr, name) values ('f2', 'd1', '2011-03-06', 'pdf', 'f2 file', 'file1'); i

Re: Versioning in cassandra

2013-09-03 Thread Laing, Michael
try the following. -ml -- put this in and run using 'cqlsh -f DROP KEYSPACE latest; CREATE KEYSPACE latest WITH replication = { 'class': 'SimpleStrategy', 'replication_factor' : 1 }; USE latest; CREATE TABLE file ( parentid text, -- row_key, same for each version id text, --

Re: Versioning in cassandra

2013-09-03 Thread dawood abdullah
I have tried with both the options creating secondary index and also tried adding parentid to primary key, but I am getting all the files with parentid 'yyy', what I want is the latest version of file with the combination of parentid, fileid. Say below are the records inserted in the file table: i

Re: Versioning in cassandra

2013-09-03 Thread Vivek Mishra
My bad. I did miss out to read "latest version" part. -Vivek On Tue, Sep 3, 2013 at 11:20 PM, dawood abdullah wrote: > I have tried with both the options creating secondary index and also tried > adding parentid to primary key, but I am getting all the files with > parentid 'yyy', what I want i

Re: Versioning in cassandra

2013-09-03 Thread Vivek Mishra
create secondary index over parentid. OR make it part of clustering key -Vivek On Tue, Sep 3, 2013 at 10:42 PM, dawood abdullah wrote: > Jan, > > The solution you gave works spot on, but there is one more requirement I > forgot to mention. Following is my table structure > > CREATE TABLE file (

Re: Versioning in cassandra

2013-09-03 Thread dawood abdullah
Jan, The solution you gave works spot on, but there is one more requirement I forgot to mention. Following is my table structure CREATE TABLE file ( id text, contenttype text, createdby text, createdtime timestamp, description text, name text, parentid text, version timestamp, P

Re: Versioning in cassandra

2013-09-02 Thread dawood abdullah
In my case version can be timestamp as well. What do you suggest version number to be, do you see any problems if I keep version as counter / timestamp ? On Tue, Sep 3, 2013 at 12:22 AM, Jan Algermissen wrote: > > On 02.09.2013, at 20:44, dawood abdullah > wrote: > > > Requirement is like I ha

Re: Versioning in cassandra

2013-09-02 Thread Jan Algermissen
On 02.09.2013, at 20:44, dawood abdullah wrote: > Requirement is like I have a column family say File > > create table file(id text primary key, fname text, version int, mimetype > text, content text); > > Say, I have few records inserted, when I modify an existing record (content > is modif

Re: Versioning in cassandra

2013-09-02 Thread dawood abdullah
Requirement is like I have a column family say File create table file(id text primary key, fname text, version int, mimetype text, content text); Say, I have few records inserted, when I modify an existing record (content is modified) a new version needs to be created. As I need to have provision

Re: Versioning in cassandra

2013-09-02 Thread Jan Algermissen
Hi Dawood, On 02.09.2013, at 16:36, dawood abdullah wrote: > Hi > I have a requirement of versioning to be done in Cassandra. > > Following is my column family definition > > create table file_details(id text primary key, fname text, version int, > mimetype text); > > I have a secondary inde

Versioning in cassandra

2013-09-02 Thread dawood abdullah
Hi I have a requirement of versioning to be done in Cassandra. Following is my column family definition *create table file_details(id text primary key, fname text, version int, mimetype text);* I have a secondary index created on fname column. Whenever I do an insert for the same 'fname', the