Migrating to CQL and Non Compact Storage

[Anuj Wadehra]

Hi
We are on 2.0.14 and Thrift. We are planning to migrate to CQL soon but facing 
some challenges.
We have a cf with a mix of statically defined columns and dynamic columns 
(created at run time). For reading dynamic columns in CQL, we have two options:
1. Drop all columns and make the table schema less. This way, we will get a Cql 
row for each column defined for a row key--As mentioned here: 
http://www.datastax.com/dev/blog/thrift-to-cql3
2.Migrate entire data to a new non compact storage table and create collections 
for dynamic columns in new table.
In our case, we have observed that approach 2 causes 3 times slower performance 
in Range scan queries used by Spark. This is not acceptable. Cassandra 3 has 
optimized storage engine but we are not comfortable moving to 3.x in production.
Moreover, data migration to new table using Spark takes hours. 

Any suggestions for the two issues?

ThanksAnuj

Sent from Yahoo Mail on Android

Re: DataStax OpsCenter with Apache Cassandra

[Anuj Wadehra]

Thanks Jeff. If one needs to use OpsCenter with 2.2 or earlier versions of 
Apache Cassandra, is he required to buy license for it separately? What are the 
options if someone wants to use OpsCenter with Apache Distributed 3.x 
(commercial use)?

ThanksAnuj

Sent from Yahoo Mail on Android 
 
  On Sun, 10 Apr, 2016 at 10:42 PM, Jeff Jirsa 
wrote:   It is possible to use OpsCenter for open source / community versions 
up to 2.2.x. It will not be possible in 3.0+


From:  Anuj Wadehra
Reply-To:  "user@cassandra.apache.org"
Date:  Sunday, April 10, 2016 at 9:28 AM
To:  User
Subject:  DataStax OpsCenter with Apache Cassandra

  

Re: DataStax OpsCenter with Apache Cassandra

[Jeff Jirsa]

It is possible to use OpsCenter for open source / community versions up to 
2.2.x. It will not be possible in 3.0+



From:  Anuj Wadehra
Reply-To:  "user@cassandra.apache.org"
Date:  Sunday, April 10, 2016 at 9:28 AM
To:  User
Subject:  DataStax OpsCenter with Apache Cassandra

Hi, 

Is it possible to use DataStax OpsCenter for monitoring Apache distributed 
Cassandra in Production? 

OR

 Is it possible to use DataStax OpsCenter if you are not using DataStax 
Enterprise in production?


Thanks
Anuj



smime.p7s
Description: S/MIME cryptographic signature

DataStax OpsCenter with Apache Cassandra

[Anuj Wadehra]

Hi,
Is it possible to use DataStax OpsCenter for monitoring Apache distributed 
Cassandra in Production? 
OR
 Is it possible to use DataStax OpsCenter if you are not using DataStax 
Enterprise in production?

ThanksAnuj

Datastax OpsCenter - Can't connect to Cassandra All host(s) tried for query failed

[okan özdinç]

   Hello Dear


down votefavorite


I have a two node and I installed DataStax OpsCenter on 10.5.0.201 but *I
give error ( Unable connect to any seed nodes )* when I create new cluster
on OpsCenter. Can you check my config ?

*address.yaml ( 10.5.0.201 )* [image: enter image description here]


*address.yaml ( 10.5.0.203 )* [image: enter image description here]


*nodetool status* [image: enter image description here]


*DataStax OpsCenter* [image: enter image description here]


*startup.log (10.5.0.201)* [image: enter image description here]


*agent.log ( 10.5.0.201 )* [image: enter image description here]


*Versions*

   - Python 2.6.6
   - java version "1.8.0_74"
   - Java(TM) SE Runtime Environment (build 1.8.0_74-b02) Java HotSpot(TM)
   64-Bit Server VM (build 25.74-b02, mixed mode)
   - OpsCenter 5.2.4

Best Regards

RE: Mapping a continuous range to a discrete value

[Peer, Oded]

Thanks for responding. I was able to hack it out eventually.

I removed the ‘upper’ column from the PK so it is no longer a clustering 
column, and I added a secondary index over it. This assumes there is no overlap 
in the a continuous ranges.
I had to add a ‘dummy’ column with a constant value to be able to use an non-eq 
operator on the ‘upper’ column.
Now that it’s not a clustering column and it has an index I can use the 
following map a continuous range to a discrete value.

CREATE TABLE range_mapping (k int, lower int, upper int, dummy int, 
mapped_value int, PRIMARY KEY (k, lower));
CREATE INDEX upper_index on range_mapping(upper);
CREATE INDEX dummy_index on range_mapping(dummy);

INSERT INTO range_mapping (k, dummy, lower, upper, mapped_value) VALUES (0, 0, 
0, 99, 0);
INSERT INTO range_mapping (k, dummy, lower, upper, mapped_value) VALUES (0, 0, 
100, 199, 100);
INSERT INTO range_mapping (k, dummy, lower, upper, mapped_value) VALUES (0, 0, 
200, 299, 200);

Now for the query:

select * from range_mapping where k = 0 and dummy = 0 and lower <= 150 and 
upper >= 150 allow filtering;

k | lower | dummy | mapped_value | upper
---+---+---+--+---
0 |   100 | 0 |  100 |   199


Oded

From: Henry M [mailto:henrymanm...@gmail.com]
Sent: Friday, April 08, 2016 6:38 AM
To: user@cassandra.apache.org
Subject: Re: Mapping a continuous range to a discrete value

I had to do something similar (in my case it was an IN  query)... I ended up 
writing hack in java to create a custom Expression and injecting into the 
RowFilter of a dummy secondary index (not advisable and very short term but it 
keeps my application code clean). I am keeping my eyes open for the evolution 
of SASI indexes (starting with cassandra 3.4 
https://github.com/apache/cassandra/blob/trunk/doc/SASI.md) which should do 
what you are looking.


On Thu, Apr 7, 2016 at 11:06 AM Mitch Gitman 
> wrote:
I just happened to run into a similar situation myself and I can see it's 
through a bad schema design (and query design) on my part. What I wanted to do 
was narrow down by the range on one clustering column and then by another range 
on the next clustering column. Failing to adequately think through how 
Cassandra stores its sorted rows on disk, I just figured, hey, why not?

The result? The same error message you got. But then, going back over some old 
notes from a DataStax CQL webinar, I came across this (my words):

"You can do selects with combinations of the different primary keys including 
ranges on individual columns. The range will only work if you've narrowed 
things down already by equality on all the prior columns. Cassandra creates a 
composite type to store the column name."

My new solution in response. Create two tables: one that's sorted by (in my 
situation) a high timestamp, the other that's sorted by (in my situation) a low 
timestamp. What had been two clustering columns gets broken up into one 
clustering column each in two different tables. Then I do two queries, one with 
the one range, the other with the other, and I programmatically merge the 
results.

The funny thing is, that was my original design which my most recent, and 
failed, design is replacing. My new solution goes back to my old solution.

On Thu, Apr 7, 2016 at 1:37 AM, Peer, Oded 
> wrote:
I have a table mapping continuous ranges to discrete values.

CREATE TABLE range_mapping (k int, lower int, upper int, mapped_value int, 
PRIMARY KEY (k, lower, upper));
INSERT INTO range_mapping (k, lower, upper, mapped_value) VALUES (0, 0, 99, 0);
INSERT INTO range_mapping (k, lower, upper, mapped_value) VALUES (0, 100, 199, 
100);
INSERT INTO range_mapping (k, lower, upper, mapped_value) VALUES (0, 200, 299, 
200);

I then want to query this table to find mapping of a specific value.
In SQL I would use: select mapped_value from range_mapping where k=0 and ? 
between lower and upper

If the variable is bound to the value 150 then the mapped_value returned is 100.

I can’t use the same type of query in CQL.
Using the query “select * from range_mapping where k = 0 and lower <= 150 and 
upper >= 150;” returns an error "Clustering column "upper" cannot be restricted 
(preceding column "lower" is restricted by a non-EQ relation)"

I thought of using multi-column restrictions but they don’t work as I expected 
as the following query returns two rows instead of the one I expected:

select * from range_mapping where k = 0 and (lower,upper) <= (150,999) and 
(lower,upper) >= (-999,150);

k | lower | upper | mapped_value
---+---+---+--
0 | 0 |99 |0
0 |   100 |   199 |  100

I’d appreciate any thoughts on the subject.