TimeoutException and keyspace exceptions

[Rodrigo Felix]

Hi,

   I'm running a heavy load with about 3200 clients accessing cassandra to
execute read and update queries via YCSB.
   At certain moment, YCSB starts throwing exceptions indefinitely like the
following:

*   [30/07/2013 00:04:50] Read Exception (Thread: 5007; Host:
10.191.54.26): TimedOutException()*
[30/07/2013 00:04:50] Read Exception (Thread: 4840; Host: 10.191.54.26):
TimedOutException()
*[30/07/2013 00:04:50] Read Exception (Thread: 5007; Host: 10.139.3.20):
InvalidRequestException(why:You have not set a keyspace for this session)*
[30/07/2013 00:04:50] Read Exception (Thread: 1644; Host: 10.191.54.26):
TimedOutException()

   I'm using Cassandra 1.1.5 in a cluster with 2 seeds and 2 other nodes
that I add (remove) on demand. Each node is a m1.large machine on Amazon.
   Any suggestion on what it could be?
   Unfortunately, upgrading the EC2 instance type is not an option for me.

Other info:
   rpc_timeout_in_ms: 3
   thrift_framed_transport_size_in_mb: 60
   thrift_max_message_length_in_mb: 64

Att.

*Rodrigo Felix de Almeida*
LSBD - Universidade Federal do Ceará
Project Manager
MBA, CSM, CSPO, SCJP

Re: deleting columns with CAS (2.0.0-beta2)

[Kalpana Suraesh]

Yep, for example, I created a simple users table and then issued a delete
with an IF statement. i.e

CREATE TABLE users (
 user_name varchar,
 password varchar,
 state varchar,
 PRIMARY KEY (user_name)
   );
INSERT INTO users
 (user_name, password, state)
 VALUES ('jsmith', 'ch@ngem3a', 'alive');
DELETE password FROM users WHERE user_name = 'jsmith' IF state = 'alive';

And note that the delete won't go through if the state is something else,
i.e. 'dead', as expected.


On Mon, Jul 29, 2013 at 12:58 AM, aaron morton wrote:

> > I presume it can be done because conditional deletes work through CQL3.
> Out of interest where do you have an example or a reference for this?
>
> Cheers
>
> -
> Aaron Morton
> Cassandra Consultant
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 26/07/2013, at 8:07 AM, Kalpana Suraesh  wrote:
>
> > Hi,
> >
> > I've been playing around with CAS (compare-and-swap) in the 2.0 beta,
> using the Thrift API. I can't at all figure out how to delete columns while
> using CAS, however. I'm able to update/insert columns by adding them to the
> list of updates that is passed as a parameter to
> org.apache.cassandra.thrift.Cassandra.Client#cas(). But there seems to be
> no indication as to how to delete.
> >
> > I presume it can be done because conditional deletes work through CQL3.
> However, I'm interested in using Hector/Java (particularly as CQL3 does not
> yet allow batched conditional statements).
> >
> > Any help would be great!
> >
> > Thanks,
> > Kalpana
>
>


-- 
--
Kalpana Suraesh

Re: make cassandra-cli use 7197 for JMX instead?

[Hiller, Dean]

Ugh, how did I miss that one, it was in the cassandra-cli --helpŠ.never
mind.

Dean

On 7/29/13 11:24 AM, "Hiller, Dean"  wrote:

>I start nodetool with
>
>Cassandra-cli ­p 9158 but it gives warnings about not displaying all
>information because my JMX port is on 7197 instead of 7199 which is in
>use by another process.  How do I make cassandra-cli connect to 7197 for
>the JMX stuff (right now it connets into another process which really
>hoses it up and it crashes in that case instead of nice connection
>refused warnings and staying up).
>
>Thanks,
>Dean
>

Re: Custom commands in cassandra

[Nulik Nol]

hi

> Have you identified issues where throughput or latency is an issue ?
no, I am on design stage of my app and I want to do it the fastest way
possible from the beginning
> Most performance gains are going to be made by getting the data model right.
hope to get it right, and with embedding will get even more performance.
>
>
> Embedding the server will add *a lot* of complexity.

that's a conjecture one would come at first sight, but if you analyze
it , it is the opposite. Complexity increases with code, and
communication between processes (like via socket or memory buffer)
implies more code, so if you embed and call server objects directly,
your code will be simpler. JVM already implements security model
between objects, so no isolation of app code from server code is
necessary.  On the countrary,the performance gains will be much bigger
, specially for lots of queries of small objects, because each query
includes communication overhead.

btw, I already found a good example of embedding, it is implemented in
a graph database called Titan. here are the files to look for:
https://github.com/thinkaurelius/titan/tree/master/titan-cassandra/src/main/java/com/thinkaurelius/titan/diskstorage/cassandra/embedded

Regards

make cassandra-cli use 7197 for JMX instead?

[Hiller, Dean]

I start nodetool with

Cassandra-cli –p 9158 but it gives warnings about not displaying all 
information because my JMX port is on 7197 instead of 7199 which is in use by 
another process.  How do I make cassandra-cli connect to 7197 for the JMX stuff 
(right now it connets into another process which really hoses it up and it 
crashes in that case instead of nice connection refused warnings and staying 
up).

Thanks,
Dean

Re: nodetool cfstats write count ?

[Richard Low]

On 29 July 2013 14:43, Langston, Jim  wrote:

 Running nodetool and looking at the cfstats output, for the
> counters such as write count and read count, do those numbers
> reflect any replication ?
>

>  For instance, if write count shows 3000 and the replication factor
> is 3, is that really 1000 writes ?
>

The counts are the number of operations that particular node has processed.
 So if you sum up all the write counts across all node's cfstats output, it
will be a factor of replication factor too high, assuming all replicas
received all writes.  (A write message will be sent to all replicas, but if
one is down or busy it may not get processed.)

If your cluster is balanced, then you can estimate the total number of
write operations from one cfstats output as

count * num_nodes / replication_factor

So, as in your example, if one node shows write count 3000 and you have RF
3 with e.g. 3 nodes, your cluster will have processed about 3000 writes.

Reads are a bit harder, because, unlike writes, not all nodes necessarily
receive all read requests.  It depends on consistency level, snitch and
value of read_repair_chance.  You can estimate how many nodes will be
involved in each read request though to figure out how many reads have
actually been submitted.

However, I think the numbers exposed by
StorageProxyMBean getWriteOperations() and getReadOperations() would give
you the true number (when summed up over all nodes).  These numbers don't
over count for replication since they count client requests.  I don't think
these are exposed by any nodetool commands though, but you can use any JMX
client to read them.

Richard.

nodetool cfstats write count ?

[Langston, Jim]

Hi all,

Running nodetool and looking at the cfstats output, for the
counters such as write count and read count, do those numbers
reflect any replication ?

For instance, if write count shows 3000 and the replication factor
is 3, is that really 1000 writes ?

Thanks,

Jim

Re: key cache hit rate and BF false positive

[Keith Wright]

Re-sending hoping to get some assistance.  Thanks!

From: Keith Wright mailto:kwri...@nanigans.com>>
Reply-To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Date: Friday, July 26, 2013 11:25 PM
To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Subject: Re: key cache hit rate and BF false positive

Also here is the output from cfhistograms if that helps.  Thanks all!

users/cookie_user_lookup histograms
Offset  SSTables Write Latency  Read Latency  Row Size  
Column Count
1  574961651 0 0 0  
   67267
2   50623075 0 0 0  
   7
36977136 0 0 0  
   866112674
41068933   151 0 0  
   5
5 100458  1752 0 0  
   0
6   2299  8845 0 0  
   0
7 25 36376 0 0  
   0
8  0118417 1 0  
   0
10 097089780 0  
   0
12 0   3154897   706 0  
   0
14 0   6681561  5633 0  
   0
17 0  17522902 62716 0  
   0
20 0  23857810323821 0  
   0
24 0  30173547   1756172 0  
   0
29 0  22315480   7493321 0  
   0
35 0   9574788  20574499 0  
   0
42 0   2816872  2338295315  
   0
50 0764169  15332694 16681  
   0
60 0245998  10826754 50571  
   0
72 0112974  13640480 0  
   0
86 0 76213  12988932 0  
   0
1030 68305  12774730 4  
   0
1240 45879  14726045 0  
   0
1490 34684  14306264 474654029  
   0
1790 24458   8489834 391359834  
   0
2150 21349   3616683 98819  
   0
2580 12373   1471382 0  
   0
3100  8138   2096660 0  
   0
3720  6420  12175498 0  
   0
4460  4981  44092631 0  
   0
5350  4052  19968489 0  
   0
6420  3514  30524161 0  
   0
7700  3411  88732015 0  
   0
9240  3379  36941969 0  
   0
1109   0  3259  88311470 0  
   0
1331   0  2925  34015836 0  
   0
1597   0  2522  25807588 0  
   0
1916   0  1909  24346980 0  
   0
2299   0  1083  28822035 0  
   0
2759   0   646  12627664 0  
   0
3311   0   489   9374038 0  
   0
3973   0   331   5114594 0  
   0
4768   0   797   3119917 0  
   0
5722   0   317   1805150 0  
   0
6866   094   1105472 0  
   0
8239   068701533 0  
   0
9887   028459354   

Re: sstable size change

[Keith Wright]

Version 1.2.4.

Original alter that does nothing:  alter table shard_user_lookup with 
compaction_strategy_options = {'sstable_size_in_mb':256};

Correct alter:  alter table cookie_user_lookup with 
compaction={'sstable_size_in_mb': '256', 'class': 'LeveledCompactionStrategy'};

Thanks

From: aaron morton mailto:aa...@thelastpickle.com>>
Reply-To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Date: Monday, July 29, 2013 4:02 AM
To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Subject: Re: sstable size change

I still would have expected my original alter command to at least have thrown 
an error.  I assume I should open a bug for this?
Sounds like a good idea.

Please describe the version and query you first used.

Thanks for taking the time to update the thread.

Cheers


-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 12:57 AM, Keith Wright 
mailto:kwri...@nanigans.com>> wrote:

FYI.  It appears that the proper command in CQL3 is the following:

alter table cookie_user_lookup with compaction={'sstable_size_in_mb': '256', 
'class': 'LeveledCompactionStrategy'};

I still would have expected my original alter command to at least have thrown 
an error.  I assume I should open a bug for this?

alter table shard_user_lookup with compaction_strategy_options = 
{'sstable_size_in_mb':256};


From: Keith Wright mailto:kwri...@nanigans.com>>
Reply-To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Date: Thursday, July 25, 2013 9:14 AM
To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>, Wei Zhu 
mailto:wz1...@yahoo.com>>
Subject: Re: sstable size change

Unfortunately the table in question is a CQL3 table so cassandra-cli will not 
output its describe:

WARNING: CQL3 tables are intentionally omitted from 'describe' output.
See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details.

However, I did figure out that apparently I was not setting the change 
properly.  I was using the following alter via CQL3:

alter table shard_user_lookup with compaction_strategy_options = 
{'sstable_size_in_mb':256};

But when I did a describe, I did not see the change.

CREATE TABLE shard_user_lookup (
  shard_user_id int,
  shard text,
  creation_time timestamp,
  status int,
  user_id timeuuid,
  PRIMARY KEY (shard_user_id, shard)
) WITH
  bloom_filter_fp_chance=0.10 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.00 AND
  gc_grace_seconds=86400 AND
  read_repair_chance=0.10 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  compaction={'class': 'LeveledCompactionStrategy'} AND
  compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 
'sstable_compression': 'LZ4Compressor'};

If I then execute the following via CQL2, I do see the property.  Seems odd to 
me?

alter table shard_user_lookup with 
compaction_strategy_options:sstable_size_in_mb=256;

CREATE TABLE shard_user_lookup (
  shard_user_id int,
  shard text,
  creation_time timestamp,
  status int,
  user_id timeuuid,
  PRIMARY KEY (shard_user_id, shard)
) WITH
  bloom_filter_fp_chance=0.10 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.00 AND
  gc_grace_seconds=86400 AND
  read_repair_chance=0.10 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  compaction={'sstable_size_in_mb': '256', 'class': 
'LeveledCompactionStrategy'} AND
  compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 
'sstable_compression': 'LZ4Compressor'};


From: Wei Zhu mailto:wz1...@yahoo.com>>
Reply-To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>, Wei Zhu 
mailto:wz1...@yahoo.com>>
Date: Wednesday, July 24, 2013 8:49 PM
To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Subject: Re: sstable size change

what is output of show keyspaces from cassandra-cli, did you see the new value?

  Compaction Strategy: 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy
  Compaction Strategy Options:
sstable_size_in_mb: XXX


From: Keith Wright mailto:kwri...@nanigans.com>>
To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Sent: Wednesday, July 24, 2013 3:44 PM
Subject: Re: sstable size change

Hi all,

   This morning I increased the SSTable size for one of my LCS via an alter 
command and saw at least one compaction run (I did not trigger a compaction via 
nodetool nor upgrades stables nor removing the .json file).  But so far my data 
sizes appear the same at the default 5 MB (see below for output of ls –Sal as 
well as relevant portion of cfstats).   Is this e

RE: AssertionError during ALTER TYPE in 1.2.5

[Adriano Paggi]

Maybe it's because of the "COMPACT STORAGE". Without that, the cells of a CQL3 
table definition will be mapped to a composite taking the clustering part of 
the key and the column name.
So, i.e. in your case the "value" column should be mapped to a cell with name 
column1:value if your def not includes the "COMPACT STORAGE" part. I think that 
the assert tries to ensure that the "alter" applies to a CQL3 table (which 
column names are composite ones), and not the old "compact".

http://www.datastax.com/dev/blog/thrift-to-cql3

Am I wrong?

Adriano.


From: aaron morton [mailto:aa...@thelastpickle.com]
Sent: Lunes, 29 de Julio de 2013 05:23 a.m.
To: user@cassandra.apache.org
Subject: Re: AssertionError during ALTER TYPE in 1.2.5

The error is because the underlying CF is not defined using a composite type 
for the comparator.

CREATE TABLE "RRD" (
 key text,
 column1 blob,
 value blob,
 PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
Looks like this PRIMARY KEY (key, column1) is incorrect.

That says "The Primary Key is made up of a Partition Key and one Clustering 
Column. The Partition Key (storage engine row key) is the 'key' column. The 
Clustering Column is 'column1'. The Clustering Column should be used as the 
first column in a composite type in the storage engine row for each non Primary 
Key column name"

But your CLI def does not use a composite type for the column validator.

Try

CREATE TABLE "RRD" (
 key text,
 column1 blob,
 value blob,
 PRIMARY KEY (key)
) WITH COMPACT STORAGE AND

Hope that helps.


-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 3:48 AM, Sergey Leschenko 
mailto:sergle...@gmail.com>> wrote:


Hi,
I have following column family from pre-CQL times, now using 1.2.5 and
want to investigate ability to use CQL instead of Thrift API.

  ColumnFamily: RRD
 Key Validation Class: org.apache.cassandra.db.marshal.UTF8Type
 Default column value validator: org.apache.cassandra.db.marshal.BytesType
 Columns sorted by: org.apache.cassandra.db.marshal.BytesType
 GC grace seconds: 864000
 Compaction min/max thresholds: 4/32
 Read repair chance: 0.1
 DC Local Read repair chance: 0.0
 Populate IO Cache on flush: false
 Replicate on write: true
 Caching: KEYS_ONLY
 Bloom Filter FP chance: default
 Built indexes: []
 Compaction Strategy:
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
 Compression Options:
   sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor

In cqlsh it looks like

CREATE TABLE "RRD" (
 key text,
 column1 blob,
 value blob,
 PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
 bloom_filter_fp_chance=0.01 AND
 caching='KEYS_ONLY' AND
 comment='' AND
 dclocal_read_repair_chance=0.00 AND
 gc_grace_seconds=864000 AND
 read_repair_chance=0.10 AND
 replicate_on_write='true' AND
 populate_io_cache_on_flush='false' AND
 compaction={'class': 'SizeTieredCompactionStrategy'} AND
 compression={'sstable_compression': 'SnappyCompressor'};

I have already successfully changed type of key from blob to text.
column1 is actually a text string, as I would like to change it type
from blob to text too.

But when I run query  ALTER TABLE  "RRD" ALTER  column1  TYPE text,
cqlsh hangs, and I observe following error in Cassandra log:

ERROR 18:28:56,103 Exception in thread Thread[Thrift:16,5,main]
java.lang.AssertionError
at 
org.apache.cassandra.cql3.statements.AlterTableStatement.announceMigration(AlterTableStatement.java:147)
at 
org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:73)
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:132)
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:143)
at 
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1707)
at 
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4074)
at 
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4062)
at 
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at 
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:199)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

>From source code I see that it  failed assertion:
case COLUMN_ALIAS:
  assert cfDef.isComposite;

So it have to be composite, but there is already exists PRIMARY KEY
(key, c

Re: [RELEASE] Apache Cassandra 1.2.8

[Jeremy Hanna]

The CHANGES and NEWS links pointed to the 1.2.8-tentative.
The 1.2.8 links are:
CHANGES.txt: 
https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/1.2.8
NEWS.txt: 
https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/1.2.8

On 29 Jul 2013, at 04:00, Eric Evans  wrote:

> 
> Regretably, 1.2.7 made its way out with a regression, so we've fast-tracked
> 1.2.8 with the fix[1].
> 
> Source and binary downloads are available from the website.
> 
>  http://cassandra.apache.org/download
> 
> The Debian repository will need to wait until Sylvain's return, I apologize
> for the inconvenience.  If you need the package before then, you can
> download and manually install it from:
> 
>  http://people.apache.org/~eevans
> 
> As usual, please pay attention to the release notes and let us know if
> you encounter any issues.
> 
> Thanks.
> 
> [1]: https://issues.apache.org/jira/browse/CASSANDRA-5814
> [2]: http://goo.gl/bQ3a4i (CHANGES.txt)
> [3]: http://goo.gl/Zj1JFa (NEWS.txt)
> [4]: https://issues.apache.org/jira/browse/CASSANDRA
> 
> -- 
> Eric Evans
> eev...@sym-link.com

Re: Installing Debian package from ASF repo

[Richard Low]

On 29 July 2013 12:00, Pavel Kirienko  wrote:

> Hi,
>
> I failed to install the Debian package of Cassandra 1.2.7 from ASF
> repository because of 404 error.
> APT said:
>
> http://www.apache.org/dist/cassandra/debian/pool/main/c/cassandra/cassandra_1.2.7_all.deb
>  404  Not Found [IP: 192.87.106.229 80]
>
> http://www.apache.org/dist/cassandra/debian/pool/main/c/cassandra/cassandra_1.2.7_all.deb
>  404  Not Found [IP: 140.211.11.131 80]
>
> The directory listing shows that there is no "cassandra_1.2.7_all.deb"
> here: http://www.apache.org/dist/cassandra/debian/pool/main/c/cassandra/
>
> Seems that I need to install it from .deb by hand, right? When 1.2.7 will
> be available in the ASF or DataStax repository?
>

1.2.7 has been removed due to a regression - see the 1.2.8 release
announcement.  However, 1.2.8 debs have to be installed manually until
Sylvain is back.  You can find them at http://people.apache.org/~eevans/.

Richard.

Installing Debian package from ASF repo

[Pavel Kirienko]

Hi,

I failed to install the Debian package of Cassandra 1.2.7 from ASF
repository because of 404 error.
APT said:
http://www.apache.org/dist/cassandra/debian/pool/main/c/cassandra/cassandra_1.2.7_all.deb
404  Not Found [IP: 192.87.106.229 80]
http://www.apache.org/dist/cassandra/debian/pool/main/c/cassandra/cassandra_1.2.7_all.deb
404  Not Found [IP: 140.211.11.131 80]

The directory listing shows that there is no "cassandra_1.2.7_all.deb"
here: http://www.apache.org/dist/cassandra/debian/pool/main/c/cassandra/

Seems that I need to install it from .deb by hand, right? When 1.2.7 will
be available in the ASF or DataStax repository?

Thanks.
Pavel.

Re: Custom commands in cassandra

[aaron morton]

>  My reasons to merge the app
> into cassandra's code is to gain speed so I will be running only one
> java vm process along multiple cores inside a node.
Have you identified issues where throughput or latency is an issue ? 
Most performance gains are going to be made by getting the data model right. 

Embedding the server will add *a lot* of complexity. 
 
Cheers

-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 9:35 AM, Radim Kolar  wrote:

> basic osgi integration is easy
> 
> you need to get osgi compatible container and hookup it to cassandra daemon. 
> Its very easy to do - about 5 lines.
> osgi container can be accessed from network, you need to deploy your 
> application into container on each node and start it up. Then use some RPC 
> mechanism like thrift or JMS for communication with server part of your 
> application.

Re: maximum storage per node

[aaron morton]

> Does anyone have opinions on the maximum amount of data reasonable to store 
> on one Cassandra node?
spinning disk and 1Gbe networking, rule of thumb was 300Gb to 500GB. 

SSD or very fast local disk, 10Gbe networking, optionally JBOD, cassandra 1.2 
and vnodes people are talking about multiple TB's per node. 

>  If there are limitations, what are the reasons for it?   

The main issues were:

* As discussed potentially very long compaction
* As discussed repair taking a very long time to calculate the merkle trees. 
* Potentially taking a very long time to rebuild a new node after one totally 
fails. vNodes address this by increasing the number of nodes that can stream 
data to one bootstrapping. This is really something that has to fit into your 
operations. 

Hope that helps. 
 
-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 5:00 AM, Robert Coli  wrote:

> On Fri, Jul 26, 2013 at 4:23 AM, Romain HARDOUIN  
> wrote:
> Do you have some fairly complex queries to run against your data? 
> Or your need is just to store large pieces of data? (In which case Object 
> Storage like OpenStack Swift could be more appropriate IMHO)  
> 
> Or distributed blob storage like MogileFS.
> 
> https://code.google.com/p/mogilefs/
> 
> =Rob

Re: AssertionError during ALTER TYPE in 1.2.5

[aaron morton]

The error is because the underlying CF is not defined using a composite type 
for the comparator. 

> CREATE TABLE "RRD" (
>  key text,
>  column1 blob,
>  value blob,
>  PRIMARY KEY (key, column1)
> ) WITH COMPACT STORAGE AND
Looks like this PRIMARY KEY (key, column1) is incorrect. 

That says "The Primary Key is made up of a Partition Key and one Clustering 
Column. The Partition Key (storage engine row key) is the 'key' column. The 
Clustering Column is 'column1'. The Clustering Column should be used as the 
first column in a composite type in the storage engine row for each non Primary 
Key column name"

But your CLI def does not use a composite type for the column validator. 

Try

CREATE TABLE "RRD" (
 key text,
 column1 blob,
 value blob,
 PRIMARY KEY (key)
) WITH COMPACT STORAGE AND

Hope that helps. 

 
-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 3:48 AM, Sergey Leschenko  wrote:

> Hi,
> I have following column family from pre-CQL times, now using 1.2.5 and
> want to investigate ability to use CQL instead of Thrift API.
> 
>   ColumnFamily: RRD
>  Key Validation Class: org.apache.cassandra.db.marshal.UTF8Type
>  Default column value validator: org.apache.cassandra.db.marshal.BytesType
>  Columns sorted by: org.apache.cassandra.db.marshal.BytesType
>  GC grace seconds: 864000
>  Compaction min/max thresholds: 4/32
>  Read repair chance: 0.1
>  DC Local Read repair chance: 0.0
>  Populate IO Cache on flush: false
>  Replicate on write: true
>  Caching: KEYS_ONLY
>  Bloom Filter FP chance: default
>  Built indexes: []
>  Compaction Strategy:
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
>  Compression Options:
>sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor
> 
> In cqlsh it looks like
> 
> CREATE TABLE "RRD" (
>  key text,
>  column1 blob,
>  value blob,
>  PRIMARY KEY (key, column1)
> ) WITH COMPACT STORAGE AND
>  bloom_filter_fp_chance=0.01 AND
>  caching='KEYS_ONLY' AND
>  comment='' AND
>  dclocal_read_repair_chance=0.00 AND
>  gc_grace_seconds=864000 AND
>  read_repair_chance=0.10 AND
>  replicate_on_write='true' AND
>  populate_io_cache_on_flush='false' AND
>  compaction={'class': 'SizeTieredCompactionStrategy'} AND
>  compression={'sstable_compression': 'SnappyCompressor'};
> 
> I have already successfully changed type of key from blob to text.
> column1 is actually a text string, as I would like to change it type
> from blob to text too.
> 
> But when I run query  ALTER TABLE  "RRD" ALTER  column1  TYPE text,
> cqlsh hangs, and I observe following error in Cassandra log:
> 
> ERROR 18:28:56,103 Exception in thread Thread[Thrift:16,5,main]
> java.lang.AssertionError
>   at 
> org.apache.cassandra.cql3.statements.AlterTableStatement.announceMigration(AlterTableStatement.java:147)
>   at 
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:73)
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:132)
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:143)
>   at 
> org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1707)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4074)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4062)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:199)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:724)
> 
> From source code I see that it  failed assertion:
> case COLUMN_ALIAS:
>   assert cfDef.isComposite;
> 
> So it have to be composite, but there is already exists PRIMARY KEY
> (key, column1). Did I miss something?
> How can I change type of this column1 ?
> 
> Thanks.
> 
> -- 
> Sergey

Re: cassandra 1.2.5- virtual nodes (num_token) pros/cons?

[aaron morton]

I would *strongly* recommend against upgrading from 0.8 directly to 1.2. 
Skipping a major version is generally not recommended, skipped 3 would seem 
like carelessness. 

> I second Romain, do the upgrade and make sure the health is good first.
+1 but I would also recommend deciding if you actually need to use virtual 
nodes. The shuffle process can take a long time and people have had mixed 
experiences with it. 

If you wanted to move to 1.2 and get vNodes I would consider spinning up a new 
cluster and bulk loading into it. You could do an initial load and then to 
delta loads using snapshots, there would however be a period of stale data in 
the new cluster until the last delta snapshot is loaded. 

Cheers

-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 3:36 AM, David McNelis  wrote:

> I second Romain, do the upgrade and make sure the health is good first.
> 
> If you have or plan to have a large number of nodes, you might consider using 
> fewer than 256 as your initial vnodes amount.  I think that number is 
> inflated from reasonable in the docs, as we've had some people talk about 
> potential performance degradation if you have a large number of nodes and a 
> very high number of vnodes, if I had it to do over again, I'd have done 64 
> vnodes as my default (across 20 nodes).
> 
> Another thing to be very cognizant of before shuffle is disk space.  You 
> *must* have less than 50% used in order to do the shuffle successfully 
> because no data is removed (cleaned) from a node during the shuffle process 
> and the shuffle process essentially doubles the amount of data until you're 
> able to run a clean.
> 
> 
> On Fri, Jul 26, 2013 at 11:25 AM, Romain HARDOUIN  
> wrote:
> Vnodes are a great feature. More nodes are involved during operations such as 
> bootstrap, decommission, etc. 
> DataStax documentation is definitely a must read. 
> That said, If I were you, I'd wait somewhat before to shuffle the ring. I'd 
> focus on cluster upgrade and monitoring the nodes. (number of files handles, 
> memory usage, latency, etc). 
> Upgrading from 0.8 to 1.2 can be tricky, there are so many changes since 
> then. Be careful about compaction strategies you choose and double check the 
> options. 
> 
> Regards, 
> Romain 
> 
> rash aroskar  a écrit sur 25/07/2013 23:25:11 :
> 
> > De : rash aroskar  
> > A : user@cassandra.apache.org, 
> > Date : 25/07/2013 23:25 
> > Objet : cassandra 1.2.5- virtual nodes (num_token) pros/cons? 
> > 
> > Hi, 
> > I am upgrading my cassandra cluster from 0.8 to 1.2.5.  
> > In cassandra 1.2.5 the 'num_token' attribute confuses me.  
> > I understand that it distributes multiple tokens per node but I am 
> > not clear how that is helpful for performance or load balancing. Can
> > anyone elaborate? has anyone used this feature  and knows its 
> > advantages/disadvantages? 
> > 
> > Thanks, 
> > Rashmi
> 

Re: hadoop/cassandra integration using CL_ONE...

[aaron morton]

> Is it possible to use CL_ONE with hadoop/cassandra when doing an M/R job?
That's the default. 
https://github.com/apache/cassandra/blob/cassandra-1.2/src/java/org/apache/cassandra/hadoop/ConfigHelper.java#L383

> And more importantly is there a way to configure that such that if my RF=3, 
> that it only reads from 1 of the nodes in that 3.
That's how it works AFAIK. 

> We have 12 nodes and ideally we would for example hope M/R runs on
> a2, a9, a5, a12 which happen to cover all token ranges.
> 
> This would leave a1, a3, a4, a6, a7, a8, a10, a11 to be free for the 
> real-time work.
You could try only putting slots on those machines, not sure if that would work 
though.

The best way to isolate transactional and analytical load is to use multiple 
DC's. 

Cheers

-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 3:33 AM, "Hiller, Dean"  wrote:

> Is it possible to use CL_ONE with hadoop/cassandra when doing an M/R job? And 
> more importantly is there a way to configure that such that if my RF=3, that 
> it only reads from 1 of the nodes in that 3.
> 
> We have 12 nodes and ideally we would for example hope M/R runs on
> a2, a9, a5, a12 which happen to cover all token ranges.
> 
> This would leave a1, a3, a4, a6, a7, a8, a10, a11 to be free for the 
> real-time work.
> 
> (That was just an example…obviously, I could pick another 4 nodes that cover 
> all token ranges).
> 
> Thanks,
> Dean

Re: sstable size change

[aaron morton]

> I still would have expected my original alter command to at least have thrown 
> an error.  I assume I should open a bug for this?
Sounds like a good idea. 

Please describe the version and query you first used. 

Thanks for taking the time to update the thread. 

Cheers


-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 12:57 AM, Keith Wright  wrote:

> FYI.  It appears that the proper command in CQL3 is the following:
> 
> alter table cookie_user_lookup with compaction={'sstable_size_in_mb': '256', 
> 'class': 'LeveledCompactionStrategy'};
> 
> I still would have expected my original alter command to at least have thrown 
> an error.  I assume I should open a bug for this?
> 
> alter table shard_user_lookup with compaction_strategy_options = 
> {'sstable_size_in_mb':256};
> 
> 
> From: Keith Wright 
> Reply-To: "user@cassandra.apache.org" 
> Date: Thursday, July 25, 2013 9:14 AM
> To: "user@cassandra.apache.org" , Wei Zhu 
> 
> Subject: Re: sstable size change
> 
> Unfortunately the table in question is a CQL3 table so cassandra-cli will not 
> output its describe:
> 
> WARNING: CQL3 tables are intentionally omitted from 'describe' output.
> See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details.
> 
> However, I did figure out that apparently I was not setting the change 
> properly.  I was using the following alter via CQL3:
> 
> alter table shard_user_lookup with compaction_strategy_options = 
> {'sstable_size_in_mb':256};
> 
> But when I did a describe, I did not see the change.   
> 
> CREATE TABLE shard_user_lookup (
>   shard_user_id int,
>   shard text,
>   creation_time timestamp,
>   status int,
>   user_id timeuuid,
>   PRIMARY KEY (shard_user_id, shard)
> ) WITH
>   bloom_filter_fp_chance=0.10 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.00 AND
>   gc_grace_seconds=86400 AND
>   read_repair_chance=0.10 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'class': 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 
> 'sstable_compression': 'LZ4Compressor'};
> 
> If I then execute the following via CQL2, I do see the property.  Seems odd 
> to me?
> 
> alter table shard_user_lookup with 
> compaction_strategy_options:sstable_size_in_mb=256;
> 
> CREATE TABLE shard_user_lookup (
>   shard_user_id int,
>   shard text,
>   creation_time timestamp,
>   status int,
>   user_id timeuuid,
>   PRIMARY KEY (shard_user_id, shard)
> ) WITH
>   bloom_filter_fp_chance=0.10 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.00 AND
>   gc_grace_seconds=86400 AND
>   read_repair_chance=0.10 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '256', 'class': 
> 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 
> 'sstable_compression': 'LZ4Compressor'};
> 
> 
> From: Wei Zhu 
> Reply-To: "user@cassandra.apache.org" , Wei Zhu 
> 
> Date: Wednesday, July 24, 2013 8:49 PM
> To: "user@cassandra.apache.org" 
> Subject: Re: sstable size change
> 
> what is output of show keyspaces from cassandra-cli, did you see the new 
> value?
> 
>   Compaction Strategy: 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy
>   Compaction Strategy Options:
> sstable_size_in_mb: XXX
> 
> From: Keith Wright 
> To: "user@cassandra.apache.org"  
> Sent: Wednesday, July 24, 2013 3:44 PM
> Subject: Re: sstable size change
> 
> Hi all,
> 
>This morning I increased the SSTable size for one of my LCS via an alter 
> command and saw at least one compaction run (I did not trigger a compaction 
> via nodetool nor upgrades stables nor removing the .json file).  But so far 
> my data sizes appear the same at the default 5 MB (see below for output of ls 
> –Sal as well as relevant portion of cfstats).   Is this expected?  I was 
> hoping to see at least one file at the new 256 MB size I set.
> 
> Thanks
> 
> SSTable count: 4965
> SSTables in each level: [0, 10, 112/100, 1027/1000, 3816, 0, 0, 0]
> Space used (live): 29062393142
> Space used (total): 29140547702
> Number of Keys (estimate): 195103104
> Memtable Columns Count: 441483
> Memtable Data Size: 205486218
> Memtable Switch Count: 243
> Read Count: 154226729
> 
> -rw-rw-r--  1 cassandra cassandra 5247564 Jul 18 01:33 
> users-shard_user_lookup-ib-97153-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247454 Jul 23 02:59 
> users-shard_user_lookup-ib-109063-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247421 Jul 20 14:58 
> users-shard_user_lookup-ib-103127-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247415 Jul 17 13:56 
> users-shard_user_lookup-ib-95761-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247379 Jul 21 02:44 
> users-shard_user_lookup-ib-104718-Data.db
> -rw-rw-r-

Re: Cassandra 2.0: Paxos Prepare response always false

[aaron morton]

Thanks for looking into this. 

If you have a way to reproduce this the best thing to do is create a ticket at 
https://issues.apache.org/jira/browse/CASSANDRA as 2.0 is still under 
development. 

Cheers


-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 26/07/2013, at 11:29 AM, Soumava Ghosh  wrote:

> Hi,
> 
> I have test setup where clients randomly make a controlled number of cas() 
> requests (among other requests) at a cluster of cassandra 2.0 servers. After 
> one point, I'm seeing that all requests are pending and my client's 
> throughput has reduced to 0.0 for all kinds of requests. For this specific 
> case I had 10 clients each making around 30 cas() requests per second at a 
> cluster of 72 instances of cassandra. 
> 
> Clients are set up to register a request as a success after the cas() call 
> returns with CASResult.success = true, else an exception is thrown. Since I 
> see that no client requests were actually registered and no exceptions were 
> thrown, which indicates that the cas() call itself is hung.
> 
> On the server side, I see Paxos logs as follows - they go on for 50 log files 
> for each of the servers involved, and they span at least an hour. I have 
> marked a particular instance where the prepare response is true but the 
> propose response is false from all the involved servers:
> 
> At the Paxos Initiator:  None of the files among the 50 system logs have the 
> phrase 'Propose response true', these logs just go on and on.
> 
> DEBUG [RequestResponseStage:110] 2013-07-25 15:09:05,332 PrepareCallback.java 
> (line 58) Prepare response PrepareResponse(true, 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> 1a0c4220-f561-11e2-a409-019f62d610d7, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,])), 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> d2093120-f576-11e2-a57e-a154d605509d, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,]))) from 
> /17.163.7.195
> 
> DEBUG [RequestResponseStage:92] 2013-07-25 15:09:05,346 PrepareCallback.java 
> (line 58) Prepare response PrepareResponse(true, 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> 1a0c4220-f561-11e2-a409-019f62d610d7, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,])), 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> d2093120-f576-11e2-a57e-a154d605509d, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,]))) from 
> /17.163.7.184
> 
> DEBUG [RequestResponseStage:98] 2013-07-25 15:09:05,347 PrepareCallback.java 
> (line 58) Prepare response PrepareResponse(true, 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> 1a0c4220-f561-11e2-a409-019f62d610d7, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,])), 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> d2093120-f576-11e2-a57e-a154d605509d, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,]))) from 
> /17.163.7.20
> 
> DEBUG [RequestResponseStage:93] 2013-07-25 15:09:05,350 ProposeCallback.java 
> (line 44) Propose response false from /17.163.7.20
> DEBUG [RequestResponseStage:100] 2013-07-25 15:09:05,350 ProposeCallback.java 
> (line 44) Propose response false from /17.163.7.184
> DEBUG [RequestResponseStage:111] 2013-07-25 15:09:05,350 ProposeCallback.java 
> (line 44) Propose response false from /17.163.7.195
> 
> DEBUG [RequestResponseStage:102] 2013-07-25 15:09:05,351 PrepareCallback.java 
> (line 58) Prepare response PrepareResponse(true, 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> 1a0c4220-f561-11e2-a409-019f62d610d7, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,])), 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> d20c3e60-f576-11e2-9bbe-bf2ad4fe6707, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,]))) from 
> /17.163.7.195
> 
> DEBUG [RequestResponseStage:107] 2013-07-25 15:09:05,352 PrepareCallback.java 
> (line 58) Prepare response PrepareResponse(true, 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> 1a0c4220-f561-11e2-a409-019f62d610d7, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,])), 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> d20c3e60-f576-11e2-9bbe-bf2ad4fe6707, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,]))) from 
> /17.163.7.20
> 
> DEBUG [RequestResponseStage:108] 2013-07-25 15:09:05,352 PrepareCallback.java 
> (line 58) Prepare response PrepareResponse(true, 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> 1a0c4220-f561-11e2-a409-019f62d610d7, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,])), 
> Commit(d145fe46f5d02a54b5ea95852f94c402, 
> d20c3e60-f576-11e2-9bbe-bf2ad4fe6707, ColumnFamily(P 
> [81d271b2125c59cb0003:false:27@1374780817218000,]))) from 
> /17.163.7.184
> 
> DEBUG [RequestResponseStage:104] 2013-07-25 15:09:05,352 ProposeCallback.java 
> (line 44) 

Re: deleting columns with CAS (2.0.0-beta2)

[aaron morton]

> I presume it can be done because conditional deletes work through CQL3. 
Out of interest where do you have an example or a reference for this?

Cheers

-
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 26/07/2013, at 8:07 AM, Kalpana Suraesh  wrote:

> Hi,
> 
> I've been playing around with CAS (compare-and-swap) in the 2.0 beta, using 
> the Thrift API. I can't at all figure out how to delete columns while using 
> CAS, however. I'm able to update/insert columns by adding them to the list of 
> updates that is passed as a parameter to 
> org.apache.cassandra.thrift.Cassandra.Client#cas(). But there seems to be no 
> indication as to how to delete.
> 
> I presume it can be done because conditional deletes work through CQL3. 
> However, I'm interested in using Hector/Java (particularly as CQL3 does not 
> yet allow batched conditional statements).
> 
> Any help would be great!
> 
> Thanks,
> Kalpana