Re: Cassandra Mutation object decoding

[Sanal Vasudevan]

Hi Nate,

Thank you.

I can give it a try.
Any examples you can point me to using QueryProcessor to read operations
from the CommitLogs?

Best regards,
Sanal

On Thu, Nov 24, 2016 at 1:22 PM, Nate McCall  wrote:

> > I must say that it is really encouraging to get your thoughts.
> > Thanks a ton Benjamin, Jacques-Henri, Jordan Nate and Chris.
> >
> > I do not have access on the client side where the CQL is executed.
>
> QueryHandler (I called it QueryProcessor incorrectly in my initial
> reply) is server side:
> https://github.com/apache/cassandra/blob/cassandra-3.0/
> src/java/org/apache/cassandra/cql3/QueryHandler.java
>
> implemented by the QueryProcessor (which you would most likely want to
> extend and override):
> https://github.com/apache/cassandra/blob/cassandra-3.0/
> src/java/org/apache/cassandra/cql3/QueryProcessor.java
>
> Nothing is stopping you from continuing with processing mutations
> somewhere else though, I just think the above would be a good place to
> start and is supported-ish API.
>



-- 
Sanal Vasudevan Nair

Re: Cassandra Mutation object decoding

[Nate McCall]

> I must say that it is really encouraging to get your thoughts.
> Thanks a ton Benjamin, Jacques-Henri, Jordan Nate and Chris.
>
> I do not have access on the client side where the CQL is executed.

QueryHandler (I called it QueryProcessor incorrectly in my initial
reply) is server side:
https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/cql3/QueryHandler.java

implemented by the QueryProcessor (which you would most likely want to
extend and override):
https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/cql3/QueryProcessor.java

Nothing is stopping you from continuing with processing mutations
somewhere else though, I just think the above would be a good place to
start and is supported-ish API.

Re: Cassandra Mutation object decoding

[Sanal Vasudevan]

I must say that it is really encouraging to get your thoughts.
Thanks a ton Benjamin, Jacques-Henri, Jordan Nate and Chris.

I do not have access on the client side where the CQL is executed.
One of my requirements is that my app should not to affect the performance
of the cassandra cluster or have very minimal overhead.
I am given access to the commit logs and CDC logs (under
//data/cdc_raw). I can access the database to query the metadata.

I understand that using the Mutation object is risky due to changes in
newer releases. Considering no (or minimal) load on C* cluster and
performance of my app, I am leaning more towards Mutation.
CASSANDRA-8844 suggests use of CommitLogReader and implementing a
CommitLogReadHandler interface which pushes the
Mutation object.
Are you guys aware how we could use the CDC feature without decoding
Mutation?
Just want to make sure I am not missing some functionality available in the
CDC feature and I am using the CDC feature in the expected fashion.

Just looking at the state of Mutation object,  this is what I get:
DELETE CQL:
mutation.getPartitionUpdates().partitionUpdate.columns().isEmpty() : true
INSERT/UPDATE CQL:
mutation.getPartitionUpdates().partitionUpdate.columns().isEmpty() : false
I am checking internally with my team whether I can live with INSERT/UPDATE
classified as upsert (as Jacques-Henri did earlier).

I am able to decode partition key, ksName, cfName, ColumnData and Column
definition from the Mutation object.

Thanks folks, great help from this community.

Best regards,
Sanal

On Wed, Nov 23, 2016 at 8:36 PM, Benjamin Lerer  wrote:

> >
> > My goal is to reconstruct the CQL operation from the Mutation object.
> > So that I can trigger the same action on another NoSQL target like
> MongoDB.
> >
>
> There are different way of keeping your 2 database in sync. Unfortunatly,
> they all have some trade offs (as always ;-))
>
>
>1. If you have controle on the client side, you could wrap the driver
>and add some code that convert the query and write it to the other
> database
>at the same time. The main problem with that approach is that a write
> can
>succeed on one of the database but not on the other. Which means that
> you
>will need a mechanism to resolve those problems.
>2. On the Cassandra side you could, as Nate suggested, extends the
>QueryProcessor in order to log the mutations to a log file. As the
>QueryProcessor has access to the prepared statement cache and to the
> bind
>parameter you should be able to extract the information you need. Some
> of
>the problems of that approach are:
>   1. You cannot reprocess already inserted data
>   2. You will probably have to use a replication log to deal with the
>   cases where the other database is unreachable
>   3. It might slow down your query processing and take some of your
>   band width at critical time (heavy write)
>   3. Use a fake index as Jacques-Henri suggested. It will allow to
>easily reprocess already inserted data so you will not need some
>replication logs (at the same time having to rebuild the index might
> slow
>down your database). The main issues for that solution are:
>1. All the tables that you want to replicate will have to have that
>   index and you cannot automatically update the schemas on your
> other database
>   2. It might slow down your query processing and take some of your
>   band width at critical time (heavy write)
>4. Read the commitlogs to recreate the mutation statements (your initial
>approach). The main problem is that it is simply not easy to do and
> might
>break up with new major releases. You will also have to make sure that
> the
>files do not disappear before you have processed them.
>5. Try a Datawarehouse/ETL approach to synchronized your data.
>CASSANDRA-8844 added support for CDC (Change Data Capture) which might
> help
>you there. Unfortunatly, I have not really worked on it so I cannot help
>you much there.
>
> There might be some other approach that are worth considering but they did
> not come to my mind.
>
> Hope it helps
>
> Benjamin
>
> PS: MongoDB ... Seriously ??? ;-)
>



-- 
Sanal Vasudevan Nair

Re: Cassandra Mutation object decoding

[Benjamin Lerer]

>
> My goal is to reconstruct the CQL operation from the Mutation object.
> So that I can trigger the same action on another NoSQL target like MongoDB.
>

There are different way of keeping your 2 database in sync. Unfortunatly,
they all have some trade offs (as always ;-))


   1. If you have controle on the client side, you could wrap the driver
   and add some code that convert the query and write it to the other database
   at the same time. The main problem with that approach is that a write can
   succeed on one of the database but not on the other. Which means that you
   will need a mechanism to resolve those problems.
   2. On the Cassandra side you could, as Nate suggested, extends the
   QueryProcessor in order to log the mutations to a log file. As the
   QueryProcessor has access to the prepared statement cache and to the bind
   parameter you should be able to extract the information you need. Some of
   the problems of that approach are:
  1. You cannot reprocess already inserted data
  2. You will probably have to use a replication log to deal with the
  cases where the other database is unreachable
  3. It might slow down your query processing and take some of your
  band width at critical time (heavy write)
  3. Use a fake index as Jacques-Henri suggested. It will allow to
   easily reprocess already inserted data so you will not need some
   replication logs (at the same time having to rebuild the index might slow
   down your database). The main issues for that solution are:
   1. All the tables that you want to replicate will have to have that
  index and you cannot automatically update the schemas on your
other database
  2. It might slow down your query processing and take some of your
  band width at critical time (heavy write)
   4. Read the commitlogs to recreate the mutation statements (your initial
   approach). The main problem is that it is simply not easy to do and might
   break up with new major releases. You will also have to make sure that the
   files do not disappear before you have processed them.
   5. Try a Datawarehouse/ETL approach to synchronized your data.
   CASSANDRA-8844 added support for CDC (Change Data Capture) which might help
   you there. Unfortunatly, I have not really worked on it so I cannot help
   you much there.

There might be some other approach that are worth considering but they did
not come to my mind.

Hope it helps

Benjamin

PS: MongoDB ... Seriously ??? ;-)

RE: Cassandra Mutation object decoding

[Jacques-Henri Berthemet]

I worked on a custom PerRowSecondaryIndex to synchronize data to another system 
and there was no way to know the difference between an insert and an update at 
this level (I'm using Cassandra 2.2). The solution for me was simply to us an 
upsert operation in the target system.

There is also the QueryHandler interface that would allow you to intercept the 
queries, but I found it difficult to use because you have to make you how query 
interpreter.

However, with Cassandra 3.x the Index interface was refactored and you now have 
insert and update methods, you even have the old and new rows in the update!
Check the SASI index implementation: 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/sasi/SASIIndex.java

If I'm not mistaken the Mutation are only used with thrift and don't have the 
notion of insert or updates with columnFamilies. Currently CQL queries use the 
native protocol: 
https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=doc/native_protocol_v3.spec.
 If you want to stay at protocol level you need to check native protocol 
implementation and how to decode it.


--
Jacques-Henri Berthemet

-Original Message-
From: J. D. Jordan [mailto:jeremiah.jor...@gmail.com] 
Sent: mercredi 23 novembre 2016 07:13
To: dev@cassandra.apache.org
Subject: Re: Cassandra Mutation object decoding

You may also want to look at the triggers interface.

> On Nov 22, 2016, at 7:45 PM, Chris Lohfink <chris.lohf...@datastax.com> wrote:
> 
> There are different kinds of tombstones, a partition tombstone is held in
> the MutableDeletionInfo of the PartitionUpdate that you can get from
> deletionInfo() method which returns the private deletionInfo field from the
> holder. There are also row and cell deletions so you have to handle each of
> those. It can be a little non-trivial to work backwards but all the
> information is there in the Mutation (if you have full access to cfmetadata
> etc), may be easier to go directly to whatever output your looking for.
> 
> A lot of the metadata is driven from system though, you need to be either
> be on a Cassandra node or rebuild them with the same cfids. In sstable-tools
> <https://github.com/tolbertam/sstable-tools> we rebuild the tables from the
> source cluster to parse the Mutations for replaying/viewing hints and
> commitlogs. But that had to be a bit (massively) hackier since its for 3.7
> before CASSANDRA-8844 <https://issues.apache.org/jira/browse/CASSANDRA-8844>.
> Its definitely possible but not easy (which is probably why it hasn't been
> added in yet).
> 
> Chris
> 
> On Tue, Nov 22, 2016 at 6:59 PM, Sanal Vasudevan <get2sa...@gmail.com>
> wrote:
> 
>> Hi Bejamin,
>> 
>> Nice to hear from you.
>> 
>> My goal is to reconstruct the CQL operation from the Mutation object.
>> So that I can trigger the same action on another NoSQL target like MongoDB.
>> 
>> Please let me know know if you have ideas?
>> 
>> Many thanks.
>> Sanal
>> 
>> 
>> On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer <
>> benjamin.le...@datastax.com
>>> wrote:
>> 
>>> Hi Sanal,
>>> 
>>> What you want to do is not an easy stuff and it might break with new
>> major
>>> releases.
>>> 
>>> My question would be: why do you want to do that? There might be another
>>> way to reach the same goal.
>>> 
>>> Benjamin
>>> 
>>> On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan <get2sa...@gmail.com>
>>> wrote:
>>> 
>>>> Thank you Vladimir.
>>>> Anyone else has any other ideas as to how this can be done?
>>>> 
>>>> 
>>>> Many thanks,
>>>> Sanal
>>>> 
>>>> 
>>>> On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin <
>> vla...@winguzone.com>
>>>> wrote:
>>>> 
>>>>> Hi Sanal,
>>>>> 
>>>>> 
>>>>> 
>>>>> do we have metadata inside Mutation object to decode whether the
>>> CQL
>>>>> was an INSERT or UPDATE operation?
>>>>> 
>>>>> I'm not sure it's possible to distinguish them - both of them just
>> add
>>>>> data to SSTable.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> Best regards, Vladimir Yudovin,
>>>>> 
>>>>> Winguzone - Hosted Cloud Cassandra
>>>>> Launch your cluster in minutes.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>&

Re: Cassandra Mutation object decoding

[J. D. Jordan]

You may also want to look at the triggers interface.

> On Nov 22, 2016, at 7:45 PM, Chris Lohfink  wrote:
> 
> There are different kinds of tombstones, a partition tombstone is held in
> the MutableDeletionInfo of the PartitionUpdate that you can get from
> deletionInfo() method which returns the private deletionInfo field from the
> holder. There are also row and cell deletions so you have to handle each of
> those. It can be a little non-trivial to work backwards but all the
> information is there in the Mutation (if you have full access to cfmetadata
> etc), may be easier to go directly to whatever output your looking for.
> 
> A lot of the metadata is driven from system though, you need to be either
> be on a Cassandra node or rebuild them with the same cfids. In sstable-tools
>  we rebuild the tables from the
> source cluster to parse the Mutations for replaying/viewing hints and
> commitlogs. But that had to be a bit (massively) hackier since its for 3.7
> before CASSANDRA-8844 .
> Its definitely possible but not easy (which is probably why it hasn't been
> added in yet).
> 
> Chris
> 
> On Tue, Nov 22, 2016 at 6:59 PM, Sanal Vasudevan 
> wrote:
> 
>> Hi Bejamin,
>> 
>> Nice to hear from you.
>> 
>> My goal is to reconstruct the CQL operation from the Mutation object.
>> So that I can trigger the same action on another NoSQL target like MongoDB.
>> 
>> Please let me know know if you have ideas?
>> 
>> Many thanks.
>> Sanal
>> 
>> 
>> On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer <
>> benjamin.le...@datastax.com
>>> wrote:
>> 
>>> Hi Sanal,
>>> 
>>> What you want to do is not an easy stuff and it might break with new
>> major
>>> releases.
>>> 
>>> My question would be: why do you want to do that? There might be another
>>> way to reach the same goal.
>>> 
>>> Benjamin
>>> 
>>> On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
>>> wrote:
>>> 
 Thank you Vladimir.
 Anyone else has any other ideas as to how this can be done?
 
 
 Many thanks,
 Sanal
 
 
 On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin <
>> vla...@winguzone.com>
 wrote:
 
> Hi Sanal,
> 
> 
> 
> do we have metadata inside Mutation object to decode whether the
>>> CQL
> was an INSERT or UPDATE operation?
> 
> I'm not sure it's possible to distinguish them - both of them just
>> add
> data to SSTable.
> 
> 
> 
> 
> 
> Best regards, Vladimir Yudovin,
> 
> Winguzone - Hosted Cloud Cassandra
> Launch your cluster in minutes.
> 
> 
> 
> 
> 
>  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> lt;get2sa...@gmail.com wrote 
> 
> 
> 
> 
> Hi there,
> 
> 
> 
> I am trying to read the Commit logs to decode the original CQL which
 used.
> 
> I get to the point an implemention of CommitLogReadHandler is able to
 push
> 
> back Mutation objects from the Commit logs.
> 
> 
> 
> Questions:
> 
> 1) CQL: delete from myks.mytable where key1 = 1;
> 
> For the above CQL, the Mutation object has zero objects of
> 
> org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> 
> Is this the only way to detect a DELETE operation? or we have any
>> other
> 
> metadata to indicate a DELETE operation?
> 
> mutation.getPartitionUpdates().forEach(rows - {
>>> if(rows.isEmpty())
> 
> System.out.println("May be a DELETE operation") });
> 
> 2) Likewise do we have metadata inside Mutation object to decode
>>> whether
> 
> the CQL was an INSERT or UPDATE operation?
> 
> 
> 
> Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> 
> (MutableDeletionInfo) may have some information but deletionInfo is
> private.
> 
> 
> 
> Basically, I am looking for help to find a way to classify Mutation
 object
> 
> to INSERT/UPDATE/DELETE with related column and key information.
> 
> 
> 
> Many thanks.
> 
> --
> 
> Sanal
> 
> 
> 
> 
> 
> 
> 
 
 
 --
 Sanal Vasudevan Nair
 
>>> 
>> 
>> 
>> 
>> --
>> Sanal Vasudevan Nair
>> 

Re: Cassandra Mutation object decoding

[Nate McCall]

Have you explored the QueryProcessor interface? That is a legitimate
extension point and may be a more suitable layer at which to
integrate.

On Wed, Nov 23, 2016 at 1:59 PM, Sanal Vasudevan  wrote:
> Hi Bejamin,
>
> Nice to hear from you.
>
> My goal is to reconstruct the CQL operation from the Mutation object.
> So that I can trigger the same action on another NoSQL target like MongoDB.
>
> Please let me know know if you have ideas?
>
> Many thanks.
> Sanal
>
>
> On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer > wrote:
>
>> Hi Sanal,
>>
>> What you want to do is not an easy stuff and it might break with new major
>> releases.
>>
>> My question would be: why do you want to do that? There might be another
>> way to reach the same goal.
>>
>> Benjamin
>>
>> On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
>> wrote:
>>
>> > Thank you Vladimir.
>> > Anyone else has any other ideas as to how this can be done?
>> >
>> >
>> > Many thanks,
>> > Sanal
>> >
>> >
>> > On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin 
>> > wrote:
>> >
>> > > Hi Sanal,
>> > >
>> > >
>> > >
>> > > do we have metadata inside Mutation object to decode whether the
>> CQL
>> > > was an INSERT or UPDATE operation?
>> > >
>> > > I'm not sure it's possible to distinguish them - both of them just add
>> > > data to SSTable.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > Best regards, Vladimir Yudovin,
>> > >
>> > > Winguzone - Hosted Cloud Cassandra
>> > > Launch your cluster in minutes.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
>> > > lt;get2sa...@gmail.com wrote 
>> > >
>> > >
>> > >
>> > >
>> > > Hi there,
>> > >
>> > >
>> > >
>> > > I am trying to read the Commit logs to decode the original CQL which
>> > used.
>> > >
>> > > I get to the point an implemention of CommitLogReadHandler is able to
>> > push
>> > >
>> > > back Mutation objects from the Commit logs.
>> > >
>> > >
>> > >
>> > > Questions:
>> > >
>> > > 1) CQL: delete from myks.mytable where key1 = 1;
>> > >
>> > > For the above CQL, the Mutation object has zero objects of
>> > >
>> > > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
>> > >
>> > > Is this the only way to detect a DELETE operation? or we have any other
>> > >
>> > > metadata to indicate a DELETE operation?
>> > >
>> > >  mutation.getPartitionUpdates().forEach(rows - {
>> if(rows.isEmpty())
>> > >
>> > > System.out.println("May be a DELETE operation") });
>> > >
>> > > 2) Likewise do we have metadata inside Mutation object to decode
>> whether
>> > >
>> > > the CQL was an INSERT or UPDATE operation?
>> > >
>> > >
>> > >
>> > > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
>> > >
>> > > (MutableDeletionInfo) may have some information but deletionInfo is
>> > > private.
>> > >
>> > >
>> > >
>> > > Basically, I am looking for help to find a way to classify Mutation
>> > object
>> > >
>> > > to INSERT/UPDATE/DELETE with related column and key information.
>> > >
>> > >
>> > >
>> > > Many thanks.
>> > >
>> > > --
>> > >
>> > > Sanal
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>> >
>> > --
>> > Sanal Vasudevan Nair
>> >
>>
>
>
>
> --
> Sanal Vasudevan Nair

Re: Cassandra Mutation object decoding

[Chris Lohfink]

There are different kinds of tombstones, a partition tombstone is held in
the MutableDeletionInfo of the PartitionUpdate that you can get from
deletionInfo() method which returns the private deletionInfo field from the
holder. There are also row and cell deletions so you have to handle each of
those. It can be a little non-trivial to work backwards but all the
information is there in the Mutation (if you have full access to cfmetadata
etc), may be easier to go directly to whatever output your looking for.

A lot of the metadata is driven from system though, you need to be either
be on a Cassandra node or rebuild them with the same cfids. In sstable-tools
 we rebuild the tables from the
source cluster to parse the Mutations for replaying/viewing hints and
commitlogs. But that had to be a bit (massively) hackier since its for 3.7
before CASSANDRA-8844 .
Its definitely possible but not easy (which is probably why it hasn't been
added in yet).

Chris

On Tue, Nov 22, 2016 at 6:59 PM, Sanal Vasudevan 
wrote:

> Hi Bejamin,
>
> Nice to hear from you.
>
> My goal is to reconstruct the CQL operation from the Mutation object.
> So that I can trigger the same action on another NoSQL target like MongoDB.
>
> Please let me know know if you have ideas?
>
> Many thanks.
> Sanal
>
>
> On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer <
> benjamin.le...@datastax.com
> > wrote:
>
> > Hi Sanal,
> >
> > What you want to do is not an easy stuff and it might break with new
> major
> > releases.
> >
> > My question would be: why do you want to do that? There might be another
> > way to reach the same goal.
> >
> > Benjamin
> >
> > On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
> > wrote:
> >
> > > Thank you Vladimir.
> > > Anyone else has any other ideas as to how this can be done?
> > >
> > >
> > > Many thanks,
> > > Sanal
> > >
> > >
> > > On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin <
> vla...@winguzone.com>
> > > wrote:
> > >
> > > > Hi Sanal,
> > > >
> > > >
> > > >
> > > > do we have metadata inside Mutation object to decode whether the
> > CQL
> > > > was an INSERT or UPDATE operation?
> > > >
> > > > I'm not sure it's possible to distinguish them - both of them just
> add
> > > > data to SSTable.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Best regards, Vladimir Yudovin,
> > > >
> > > > Winguzone - Hosted Cloud Cassandra
> > > > Launch your cluster in minutes.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> > > > lt;get2sa...@gmail.com wrote 
> > > >
> > > >
> > > >
> > > >
> > > > Hi there,
> > > >
> > > >
> > > >
> > > > I am trying to read the Commit logs to decode the original CQL which
> > > used.
> > > >
> > > > I get to the point an implemention of CommitLogReadHandler is able to
> > > push
> > > >
> > > > back Mutation objects from the Commit logs.
> > > >
> > > >
> > > >
> > > > Questions:
> > > >
> > > > 1) CQL: delete from myks.mytable where key1 = 1;
> > > >
> > > > For the above CQL, the Mutation object has zero objects of
> > > >
> > > > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> > > >
> > > > Is this the only way to detect a DELETE operation? or we have any
> other
> > > >
> > > > metadata to indicate a DELETE operation?
> > > >
> > > >  mutation.getPartitionUpdates().forEach(rows - {
> > if(rows.isEmpty())
> > > >
> > > > System.out.println("May be a DELETE operation") });
> > > >
> > > > 2) Likewise do we have metadata inside Mutation object to decode
> > whether
> > > >
> > > > the CQL was an INSERT or UPDATE operation?
> > > >
> > > >
> > > >
> > > > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> > > >
> > > > (MutableDeletionInfo) may have some information but deletionInfo is
> > > > private.
> > > >
> > > >
> > > >
> > > > Basically, I am looking for help to find a way to classify Mutation
> > > object
> > > >
> > > > to INSERT/UPDATE/DELETE with related column and key information.
> > > >
> > > >
> > > >
> > > > Many thanks.
> > > >
> > > > --
> > > >
> > > > Sanal
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Sanal Vasudevan Nair
> > >
> >
>
>
>
> --
> Sanal Vasudevan Nair
>

Re: Cassandra Mutation object decoding

[Sanal Vasudevan]

Hi Bejamin,

Nice to hear from you.

My goal is to reconstruct the CQL operation from the Mutation object.
So that I can trigger the same action on another NoSQL target like MongoDB.

Please let me know know if you have ideas?

Many thanks.
Sanal


On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer  wrote:

> Hi Sanal,
>
> What you want to do is not an easy stuff and it might break with new major
> releases.
>
> My question would be: why do you want to do that? There might be another
> way to reach the same goal.
>
> Benjamin
>
> On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
> wrote:
>
> > Thank you Vladimir.
> > Anyone else has any other ideas as to how this can be done?
> >
> >
> > Many thanks,
> > Sanal
> >
> >
> > On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin 
> > wrote:
> >
> > > Hi Sanal,
> > >
> > >
> > >
> > > do we have metadata inside Mutation object to decode whether the
> CQL
> > > was an INSERT or UPDATE operation?
> > >
> > > I'm not sure it's possible to distinguish them - both of them just add
> > > data to SSTable.
> > >
> > >
> > >
> > >
> > >
> > > Best regards, Vladimir Yudovin,
> > >
> > > Winguzone - Hosted Cloud Cassandra
> > > Launch your cluster in minutes.
> > >
> > >
> > >
> > >
> > >
> > >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> > > lt;get2sa...@gmail.com wrote 
> > >
> > >
> > >
> > >
> > > Hi there,
> > >
> > >
> > >
> > > I am trying to read the Commit logs to decode the original CQL which
> > used.
> > >
> > > I get to the point an implemention of CommitLogReadHandler is able to
> > push
> > >
> > > back Mutation objects from the Commit logs.
> > >
> > >
> > >
> > > Questions:
> > >
> > > 1) CQL: delete from myks.mytable where key1 = 1;
> > >
> > > For the above CQL, the Mutation object has zero objects of
> > >
> > > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> > >
> > > Is this the only way to detect a DELETE operation? or we have any other
> > >
> > > metadata to indicate a DELETE operation?
> > >
> > >  mutation.getPartitionUpdates().forEach(rows - {
> if(rows.isEmpty())
> > >
> > > System.out.println("May be a DELETE operation") });
> > >
> > > 2) Likewise do we have metadata inside Mutation object to decode
> whether
> > >
> > > the CQL was an INSERT or UPDATE operation?
> > >
> > >
> > >
> > > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> > >
> > > (MutableDeletionInfo) may have some information but deletionInfo is
> > > private.
> > >
> > >
> > >
> > > Basically, I am looking for help to find a way to classify Mutation
> > object
> > >
> > > to INSERT/UPDATE/DELETE with related column and key information.
> > >
> > >
> > >
> > > Many thanks.
> > >
> > > --
> > >
> > > Sanal
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Sanal Vasudevan Nair
> >
>



-- 
Sanal Vasudevan Nair

Re: Cassandra Mutation object decoding

[Benjamin Lerer]

Hi Sanal,

What you want to do is not an easy stuff and it might break with new major
releases.

My question would be: why do you want to do that? There might be another
way to reach the same goal.

Benjamin

On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
wrote:

> Thank you Vladimir.
> Anyone else has any other ideas as to how this can be done?
>
>
> Many thanks,
> Sanal
>
>
> On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin 
> wrote:
>
> > Hi Sanal,
> >
> >
> >
> > do we have metadata inside Mutation object to decode whether the CQL
> > was an INSERT or UPDATE operation?
> >
> > I'm not sure it's possible to distinguish them - both of them just add
> > data to SSTable.
> >
> >
> >
> >
> >
> > Best regards, Vladimir Yudovin,
> >
> > Winguzone - Hosted Cloud Cassandra
> > Launch your cluster in minutes.
> >
> >
> >
> >
> >
> >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> > lt;get2sa...@gmail.com wrote 
> >
> >
> >
> >
> > Hi there,
> >
> >
> >
> > I am trying to read the Commit logs to decode the original CQL which
> used.
> >
> > I get to the point an implemention of CommitLogReadHandler is able to
> push
> >
> > back Mutation objects from the Commit logs.
> >
> >
> >
> > Questions:
> >
> > 1) CQL: delete from myks.mytable where key1 = 1;
> >
> > For the above CQL, the Mutation object has zero objects of
> >
> > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> >
> > Is this the only way to detect a DELETE operation? or we have any other
> >
> > metadata to indicate a DELETE operation?
> >
> >  mutation.getPartitionUpdates().forEach(rows - { if(rows.isEmpty())
> >
> > System.out.println("May be a DELETE operation") });
> >
> > 2) Likewise do we have metadata inside Mutation object to decode whether
> >
> > the CQL was an INSERT or UPDATE operation?
> >
> >
> >
> > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> >
> > (MutableDeletionInfo) may have some information but deletionInfo is
> > private.
> >
> >
> >
> > Basically, I am looking for help to find a way to classify Mutation
> object
> >
> > to INSERT/UPDATE/DELETE with related column and key information.
> >
> >
> >
> > Many thanks.
> >
> > --
> >
> > Sanal
> >
> >
> >
> >
> >
> >
> >
>
>
> --
> Sanal Vasudevan Nair
>

Re: Cassandra Mutation object decoding

[Sanal Vasudevan]

Thank you Vladimir.
Anyone else has any other ideas as to how this can be done?


Many thanks,
Sanal


On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin 
wrote:

> Hi Sanal,
>
>
>
> do we have metadata inside Mutation object to decode whether the CQL
> was an INSERT or UPDATE operation?
>
> I'm not sure it's possible to distinguish them - both of them just add
> data to SSTable.
>
>
>
>
>
> Best regards, Vladimir Yudovin,
>
> Winguzone - Hosted Cloud Cassandra
> Launch your cluster in minutes.
>
>
>
>
>
>  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> lt;get2sa...@gmail.com wrote 
>
>
>
>
> Hi there,
>
>
>
> I am trying to read the Commit logs to decode the original CQL which used.
>
> I get to the point an implemention of CommitLogReadHandler is able to push
>
> back Mutation objects from the Commit logs.
>
>
>
> Questions:
>
> 1) CQL: delete from myks.mytable where key1 = 1;
>
> For the above CQL, the Mutation object has zero objects of
>
> org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
>
> Is this the only way to detect a DELETE operation? or we have any other
>
> metadata to indicate a DELETE operation?
>
>  mutation.getPartitionUpdates().forEach(rows - { if(rows.isEmpty())
>
> System.out.println("May be a DELETE operation") });
>
> 2) Likewise do we have metadata inside Mutation object to decode whether
>
> the CQL was an INSERT or UPDATE operation?
>
>
>
> Josh Mckenzie indicated that PartitionUpdate.deletionInfo
>
> (MutableDeletionInfo) may have some information but deletionInfo is
> private.
>
>
>
> Basically, I am looking for help to find a way to classify Mutation object
>
> to INSERT/UPDATE/DELETE with related column and key information.
>
>
>
> Many thanks.
>
> --
>
> Sanal
>
>
>
>
>
>
>


-- 
Sanal Vasudevan Nair

Re: Cassandra Mutation object decoding

[Vladimir Yudovin]

Hi Sanal,



do we have metadata inside Mutation object to decode whether the CQL was an 
INSERT or UPDATE operation?

I'm not sure it's possible to distinguish them - both of them just add data to 
SSTable.





Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





 On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan 
get2sa...@gmail.com wrote 




Hi there, 

 

I am trying to read the Commit logs to decode the original CQL which used. 

I get to the point an implemention of CommitLogReadHandler is able to push 

back Mutation objects from the Commit logs. 

 

Questions: 

1) CQL: delete from myks.mytable where key1 = 1; 

For the above CQL, the Mutation object has zero objects of 

org.apache.cassandra.db.rows.Row inside ParitionUpdate object. 

Is this the only way to detect a DELETE operation? or we have any other 

metadata to indicate a DELETE operation? 

 mutation.getPartitionUpdates().forEach(rows - { if(rows.isEmpty()) 

System.out.println("May be a DELETE operation") }); 

2) Likewise do we have metadata inside Mutation object to decode whether 

the CQL was an INSERT or UPDATE operation? 

 

Josh Mckenzie indicated that PartitionUpdate.deletionInfo 

(MutableDeletionInfo) may have some information but deletionInfo is private. 

 

Basically, I am looking for help to find a way to classify Mutation object 

to INSERT/UPDATE/DELETE with related column and key information. 

 

Many thanks. 

-- 

Sanal 

Cassandra Mutation object decoding

[Sanal Vasudevan]

Hi there,

I am trying to read the Commit logs to decode the original CQL which used.
I get to the point an implemention of  CommitLogReadHandler is able to push
back Mutation objects from the Commit logs.

Questions:
1) CQL: delete from myks.mytable where key1  = 1;
For the above CQL, the Mutation object has zero objects of
org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
Is this the only way to detect a DELETE operation? or we have any other
metadata to indicate a DELETE operation?
mutation.getPartitionUpdates().forEach(rows -> { if(rows.isEmpty())
System.out.println("May be a DELETE operation") });
2) Likewise do we have metadata inside Mutation object to decode whether
the CQL was an INSERT or UPDATE operation?

Josh Mckenzie indicated that PartitionUpdate.deletionInfo
(MutableDeletionInfo) may have some information but deletionInfo is private.

Basically, I am looking for help to find a way to classify Mutation object
to INSERT/UPDATE/DELETE with related column and key information.

Many thanks.
-- 
Sanal