date:20150113


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276261#comment-14276261
 ] 

Benedict commented on CASSANDRA-8614:
-

Why not check java.version?

 Select optimal CRC32 implementation at runtime
 --

 Key: CASSANDRA-8614
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8614
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Ariel Weisberg
  Labels: performance
 Attachments: 8614.patch


 JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec 
 per core in my quick and dirty test. PureJavaCRC32 is  800 megabytes/sec if 
 I recall and it has a lookup table that evicts random cache lines every time 
 it runs.
 In order to capture the benefit of that when it is available we can select a 
 CRC32 implementation at startup in a static block.
 If JDK 8 is not what is running we can fall back to the existing 
 PureJavaCRC32 implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL

2015-01-13 Thread Alex P (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276502#comment-14276502
 ] 

Alex P commented on CASSANDRA-3025:
---

[~lexlythius] please drop me an email (alexp @ datastax) to follow up on this.

 PHP/PDO driver for Cassandra CQL
 

 Key: CASSANDRA-3025
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Mikko Koppanen
Assignee: Mikko Koppanen
  Labels: php
 Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, 
 pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, pdo_cassandra-0.2.0.tgz, 
 pdo_cassandra-0.2.1.tgz, php_test_results_20110818_2317.txt


 Hello,
 attached is the initial version of the PDO driver for Cassandra CQL language. 
 This is a native PHP extension written in what I would call a combination of 
 C and C++, due to PHP being C. The thrift API used is the C++.
 The API looks roughly following:
 {code}
 ?php
 $db = new PDO('cassandra:host=127.0.0.1;port=9160');
 $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor=1;);
 $db-exec (USE mytest);
 $db-exec (CREATE COLUMNFAMILY users (
   my_key varchar PRIMARY KEY,
   full_name varchar ););
   
 $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, 
 :full_name););
 $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
 {code}
 Currently prepared statements are emulated on the client side but I 
 understand that there is a plan to add prepared statements to Cassandra CQL 
 API as well. I will add this feature in to the extension as soon as they are 
 implemented.
 Additional documentation can be found in github 
 https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered 
 MarkDown file. Tests are currently not included in the package file and they 
 can be found in the github for now as well.
 I have created documentation in docbook format as well, but have not yet 
 rendered it.
 Comments and feedback are welcome.
 Thanks,
 Mikko



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Pig: Refactor and deprecate CqlStorage

Pig: Refactor and deprecate CqlStorage

Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-8599


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/359d3bb2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/359d3bb2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/359d3bb2

Branch: refs/heads/trunk
Commit: 359d3bb2a69b856e6dca8a060f6087307808cb5e
Parents: 1cb426b
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 11:24:45 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 11:24:45 2015 -0600

--
 CHANGES.txt |   1 +
 examples/pig/README.txt |  27 +-
 examples/pig/example-script-cql.pig |   6 +-
 examples/pig/test/test_cql_storage.pig  |  12 +-
 .../hadoop/pig/AbstractCassandraStorage.java|   2 +-
 .../cassandra/hadoop/pig/CqlNativeStorage.java  | 520 +-
 .../apache/cassandra/hadoop/pig/CqlStorage.java | 693 +--
 7 files changed, 542 insertions(+), 719 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/359d3bb2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f2e25c4..e070eaf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.3
+ * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599)
  * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537)
  * Fix case-sensitivity of index name on CREATE and DROP INDEX
statements (CASSANDRA-8365)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/359d3bb2/examples/pig/README.txt
--
diff --git a/examples/pig/README.txt b/examples/pig/README.txt
index 2ae9824..1553a9f 100644
--- a/examples/pig/README.txt
+++ b/examples/pig/README.txt
@@ -35,7 +35,7 @@ for input and output:
 CassandraStorage
 
 
-The CassandraStorage class is for any non-CQL3 ColumnFamilies you may have.  
For CQL3 support, refer to the CqlStorage section.
+The CassandraStorage class is for any non-CQL3 ColumnFamilies you may have.  
For CQL3 support, refer to the CqlNativeStorage section.
 
 examples/pig$ bin/pig_cassandra -x local example-script.pig
 
@@ -95,15 +95,24 @@ PIG_INPUT_SPLIT_SIZE: this sets the split size passed to 
Hadoop, controlling
   the amount of mapper tasks created.  This can also be 
set in the LOAD url by
   adding the 'split_size=X' parameter, where X is an 
integer amount for the size.
 
-CqlStorage
-==
-
-The CqlStorage class is somewhat similar to CassandraStorage, but it can work 
with CQL3-defined ColumnFamilies.  The main difference is in the URL format:
-
-cql://[username:password@]keyspace/columnfamily[?[page_size=size][columns=col1,col2][output_query=prepared_statement][where_clause=clause][split_size=size][use_secondary=true|false][partitioner=partitioner]]
+CqlNativeStorage
+
 
+The CqlNativeStorage class is somewhat similar to CassandraStorage, but it can 
work with CQL3-defined ColumnFamilies.  The main difference is in the URL 
format:
+
+cql://[username:password@]keyspace/columnfamily
+
[?[page_size=size][columns=col1,col2][output_query=prepared_statement]
+
[where_clause=clause][split_size=size][partitioner=partitioner][use_secondary=true|false]
+
[init_address=host][native_port=native_port][core_conns=core_conns]
+
[max_conns=max_conns][min_simult_reqs=min_simult_reqs][max_simult_reqs=max_simult_reqs]
+
[native_timeout=native_timeout][native_read_timeout=native_read_timeout][rec_buff_size=rec_buff_size]
+
[send_buff_size=send_buff_size][solinger=solinger][tcp_nodelay=tcp_nodelay][reuse_address=reuse_address]
+
[keep_alive=keep_alive][auth_provider=auth_provider][trust_store_path=trust_store_path]
+
[key_store_path=key_store_path][trust_store_password=trust_store_password]
+
[key_store_password=key_store_password][cipher_suites=cipher_suites][input_cql=input_cql]
+[columns=columns][where_clause=where_clause]]
 Which in grunt, the simplest example would look like:
 
-grunt rows = LOAD 'cql://MyKeyspace/MyColumnFamily' USING CqlStorage();
+grunt rows = LOAD 'cql://MyKeyspace/MyColumnFamily' USING CqlNativeStorage();
 
-CqlStorage handles wide rows automatically and thus has no separate flag for 
this.
+CqlNativeStorage handles wide rows automatically and thus has no separate flag 
for this.

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8a4abdda
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8a4abdda
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8a4abdda

Branch: refs/heads/trunk
Commit: 8a4abdda1e83f0dfb7f9e253f947f3f6c84d752e
Parents: 42e483a 359d3bb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 11:28:07 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 11:28:07 2015 -0600

--
 CHANGES.txt |   1 +
 examples/pig/README.txt |  27 +-
 examples/pig/example-script-cql.pig |   6 +-
 examples/pig/test/test_cql_storage.pig  |  12 +-
 .../hadoop/pig/AbstractCassandraStorage.java|   2 +-
 .../cassandra/hadoop/pig/CqlNativeStorage.java  | 520 +-
 .../apache/cassandra/hadoop/pig/CqlStorage.java | 689 +--
 7 files changed, 542 insertions(+), 715 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8a4abdda/CHANGES.txt
--
diff --cc CHANGES.txt
index 4a7d8ca,e070eaf..b1f5465
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,53 -1,5 +1,54 @@@
 +3.0
 + * Add ReadFailureException to native protocol, respond
 +   immediately when replicas encounter errors while handling
 +   a read request (CASSANDRA-7886)
 + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308)
 + * Allow mixing token and partition key restrictions (CASSANDRA-7016)
 + * Support index key/value entries on map collections (CASSANDRA-8473)
 + * Modernize schema tables (CASSANDRA-8261)
 + * Support for user-defined aggregation functions (CASSANDRA-8053)
 + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419)
 + * Refactor SelectStatement, return IN results in natural order instead
 +   of IN value list order (CASSANDRA-7981)
 + * Support UDTs, tuples, and collections in user-defined
 +   functions (CASSANDRA-7563)
 + * Fix aggregate fn results on empty selection, result column name,
 +   and cqlsh parsing (CASSANDRA-8229)
 + * Mark sstables as repaired after full repair (CASSANDRA-7586)
 + * Extend Descriptor to include a format value and refactor reader/writer
 +   APIs (CASSANDRA-7443)
 + * Integrate JMH for microbenchmarks (CASSANDRA-8151)
 + * Keep sstable levels when bootstrapping (CASSANDRA-7460)
 + * Add Sigar library and perform basic OS settings check on startup 
(CASSANDRA-7838)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 
7929,
 +   7924, 7812, 8063, 7813, 7708)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7028)
 + * Make incremental repair default (CASSANDRA-7250)
 + * Enable code coverage thru JaCoCo (CASSANDRA-7226)
 + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) 
 + * Shorten SSTable path (CASSANDRA-6962)
 + * Use unsafe mutations for most unit tests (CASSANDRA-6969)
 + * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
 + * Fail on very large batch sizes (CASSANDRA-8011)
 + * Improve concurrency of repair (CASSANDRA-6455, 8208)
 +
 +
  2.1.3
+  * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599)
   * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537)
   * Fix case-sensitivity of index name on CREATE and DROP INDEX
 statements (CASSANDRA-8365)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8a4abdda/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
--

[1/3] cassandra git commit: Pig: Refactor and deprecate CqlStorage

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 1cb426b98 - 359d3bb2a
  refs/heads/trunk 42e483a4e - 8a4abdda1


Pig: Refactor and deprecate CqlStorage

Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-8599


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/359d3bb2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/359d3bb2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/359d3bb2

Branch: refs/heads/cassandra-2.1
Commit: 359d3bb2a69b856e6dca8a060f6087307808cb5e
Parents: 1cb426b
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 11:24:45 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 11:24:45 2015 -0600

--
 CHANGES.txt |   1 +
 examples/pig/README.txt |  27 +-
 examples/pig/example-script-cql.pig |   6 +-
 examples/pig/test/test_cql_storage.pig  |  12 +-
 .../hadoop/pig/AbstractCassandraStorage.java|   2 +-
 .../cassandra/hadoop/pig/CqlNativeStorage.java  | 520 +-
 .../apache/cassandra/hadoop/pig/CqlStorage.java | 693 +--
 7 files changed, 542 insertions(+), 719 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/359d3bb2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f2e25c4..e070eaf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.3
+ * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599)
  * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537)
  * Fix case-sensitivity of index name on CREATE and DROP INDEX
statements (CASSANDRA-8365)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/359d3bb2/examples/pig/README.txt
--
diff --git a/examples/pig/README.txt b/examples/pig/README.txt
index 2ae9824..1553a9f 100644
--- a/examples/pig/README.txt
+++ b/examples/pig/README.txt
@@ -35,7 +35,7 @@ for input and output:
 CassandraStorage
 
 
-The CassandraStorage class is for any non-CQL3 ColumnFamilies you may have.  
For CQL3 support, refer to the CqlStorage section.
+The CassandraStorage class is for any non-CQL3 ColumnFamilies you may have.  
For CQL3 support, refer to the CqlNativeStorage section.
 
 examples/pig$ bin/pig_cassandra -x local example-script.pig
 
@@ -95,15 +95,24 @@ PIG_INPUT_SPLIT_SIZE: this sets the split size passed to 
Hadoop, controlling
   the amount of mapper tasks created.  This can also be 
set in the LOAD url by
   adding the 'split_size=X' parameter, where X is an 
integer amount for the size.
 
-CqlStorage
-==
-
-The CqlStorage class is somewhat similar to CassandraStorage, but it can work 
with CQL3-defined ColumnFamilies.  The main difference is in the URL format:
-
-cql://[username:password@]keyspace/columnfamily[?[page_size=size][columns=col1,col2][output_query=prepared_statement][where_clause=clause][split_size=size][use_secondary=true|false][partitioner=partitioner]]
+CqlNativeStorage
+
 
+The CqlNativeStorage class is somewhat similar to CassandraStorage, but it can 
work with CQL3-defined ColumnFamilies.  The main difference is in the URL 
format:
+
+cql://[username:password@]keyspace/columnfamily
+
[?[page_size=size][columns=col1,col2][output_query=prepared_statement]
+
[where_clause=clause][split_size=size][partitioner=partitioner][use_secondary=true|false]
+
[init_address=host][native_port=native_port][core_conns=core_conns]
+
[max_conns=max_conns][min_simult_reqs=min_simult_reqs][max_simult_reqs=max_simult_reqs]
+
[native_timeout=native_timeout][native_read_timeout=native_read_timeout][rec_buff_size=rec_buff_size]
+
[send_buff_size=send_buff_size][solinger=solinger][tcp_nodelay=tcp_nodelay][reuse_address=reuse_address]
+
[keep_alive=keep_alive][auth_provider=auth_provider][trust_store_path=trust_store_path]
+
[key_store_path=key_store_path][trust_store_password=trust_store_password]
+
[key_store_password=key_store_password][cipher_suites=cipher_suites][input_cql=input_cql]
+[columns=columns][where_clause=where_clause]]
 Which in grunt, the simplest example would look like:
 
-grunt rows = LOAD 'cql://MyKeyspace/MyColumnFamily' USING CqlStorage();
+grunt rows = LOAD 'cql://MyKeyspace/MyColumnFamily' USING CqlNativeStorage();
 
-CqlStorage handles wide rows automatically and thus has no separate flag for 
this.
+CqlNativeStorage handles wide rows

[jira] [Commented] (CASSANDRA-8594) pidfile is never filled by cassandra


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275582#comment-14275582
 ] 

Brandon Williams commented on CASSANDRA-8594:
-

I think Cyril is talking about the debian init script, however:

bq. we use start-stop-daemon without asking it to create the pidfile

[This 
line|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/debian/init#L96]
 does ask it to create the pidfile, and it wouldn't be able to stop the service 
without it.  I can confirm that -p does write the pid to the file.

 pidfile is never filled by cassandra
 

 Key: CASSANDRA-8594
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8594
 Project: Cassandra
  Issue Type: Bug
Reporter: Cyril Scetbon
Assignee: Cyril Scetbon
 Fix For: 2.0.13


 The pid file is never filled by cassandra. there is only a File object that 
 is created with [those 
 lines|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/src/java/org/apache/cassandra/service/CassandraDaemon.java#L498-L501]
 Here is a 
 [fix|https://github.com/cscetbon/cassandra/commit/d0c5e0c9be00e48e6d0cd0de208c53274f1919c0.patch]
  that writes the current PID into the pidfile



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8594) pidfile is never filled by cassandra

2015-01-13 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275592#comment-14275592
 ] 

Joshua McKenzie commented on CASSANDRA-8594:


Right - I read that as
{noformat}
/usr/sbin/cassandra -b -p $PIDFILE
{noformat}
pushing through to the install's bin/cassandra based on the contents of 
cassandra.install
{noformat}
bin/cassandra usr/sbin
{noformat}
meaning it would all settle out with the -p writing to disk at the line I 
linked;  I should have been more clean on that entire chain.

 pidfile is never filled by cassandra
 

 Key: CASSANDRA-8594
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8594
 Project: Cassandra
  Issue Type: Bug
Reporter: Cyril Scetbon
Assignee: Cyril Scetbon
 Fix For: 2.0.13


 The pid file is never filled by cassandra. there is only a File object that 
 is created with [those 
 lines|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/src/java/org/apache/cassandra/service/CassandraDaemon.java#L498-L501]
 Here is a 
 [fix|https://github.com/cscetbon/cassandra/commit/d0c5e0c9be00e48e6d0cd0de208c53274f1919c0.patch]
  that writes the current PID into the pidfile



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8599) Refactor or fix CqlStorage


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-8599.
-
Resolution: Fixed
  Reviewer: Brandon Williams

Committed with minor cleanup, and I added an explicit logger.warn about the 
deprecation so users are more likely to see it.

 Refactor or fix CqlStorage
 --

 Key: CASSANDRA-8599
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Brandon Williams
Assignee: Alex Liu
 Fix For: 2.1.3

 Attachments: 8599-2.1-branch.txt, 8599-v2-2.1-branch.txt


 In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF 
 references with CIF, since CNS was broken otherwise.  But this means CS no 
 longer works since it's not a simple drop in replacement (but have CNS work 
 is better than having them both broken by a class that doesn't exist.)  We 
 can't just deprecate and remove CS either, because CNS extends it.  We either 
 need to fix CS to work with CIF, or we need to refactor CNS so that we can 
 just remove CS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8594) pidfile is never filled by cassandra

2015-01-13 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275575#comment-14275575
 ] 

Joshua McKenzie commented on CASSANDRA-8594:


[This 
line|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/bin/cassandra#L150]
 should be writing the pid file to disk - I gather from your responses that 
it's not doing so in your environment?

 pidfile is never filled by cassandra
 

 Key: CASSANDRA-8594
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8594
 Project: Cassandra
  Issue Type: Bug
Reporter: Cyril Scetbon
Assignee: Cyril Scetbon
 Fix For: 2.0.13


 The pid file is never filled by cassandra. there is only a File object that 
 is created with [those 
 lines|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/src/java/org/apache/cassandra/service/CassandraDaemon.java#L498-L501]
 Here is a 
 [fix|https://github.com/cscetbon/cassandra/commit/d0c5e0c9be00e48e6d0cd0de208c53274f1919c0.patch]
  that writes the current PID into the pidfile



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8558) deleted row still can be selected out


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275650#comment-14275650
 ] 

Benedict commented on CASSANDRA-8558:
-

There is already a bunch of ugliness that attempts to address this on flushing. 
Whether or not it works, though, I'm not certain :)

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
 Fix For: 2.0.12, 2.1.3

 Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8558) deleted row still can be selected out


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275644#comment-14275644
 ] 

Tyler Hobbs commented on CASSANDRA-8558:


So, I may be missing something, but it seems like we still have a problem.  A 
range tombstone can start far enough before the slice start that it gets a 
different index entry, in which case we won't ever see it.  It seems like to 
handle arbitrary range tombstones with the current on-disk format, we would 
need to read every RT before the slice start to check for overlap.  Normally, 
for CQL3, this isn't a problem, because RTs only cover a row.  But with slice 
deletions in Thrift, we should be able to see this problem.

Am I missing something about how we handle this, [~slebresne]?

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
 Fix For: 2.0.12, 2.1.3

 Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8448) Comparison method violates its general contract in AbstractEndpointSnitch


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275626#comment-14275626
 ] 

Benedict commented on CASSANDRA-8448:
-

The current patch is a no-op...? sortByProximityWithScore needs to create a 
HashMap (no need to be concurrent) and populate it with the current data, then 
create a new DynamicEndpointSnitch with that snapshot and call 
super.sortByProximity on that (with the necessary gymnastics to make that 
happen, since semantically you can't call super on another object...)

 Comparison method violates its general contract in AbstractEndpointSnitch
 ---

 Key: CASSANDRA-8448
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8448
 Project: Cassandra
  Issue Type: Bug
Reporter: J.B. Langston
Assignee: Brandon Williams
 Fix For: 2.0.12, 2.1.3

 Attachments: 8448.txt


 Seen in both 1.2 and 2.0.  The error is occurring here: 
 https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/locator/AbstractEndpointSnitch.java#L49
 {code}
 ERROR [Thrift:9] 2014-12-04 20:12:28,732 CustomTThreadPoolServer.java (line 
 219) Error occurred during processing of message.
 com.google.common.util.concurrent.UncheckedExecutionException: 
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2199)
   at com.google.common.cache.LocalCache.get(LocalCache.java:3932)
   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3936)
   at 
 com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4806)
   at 
 org.apache.cassandra.service.ClientState.authorize(ClientState.java:352)
   at 
 org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:224)
   at 
 org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:218)
   at 
 org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:202)
   at 
 org.apache.cassandra.thrift.CassandraServer.createMutationList(CassandraServer.java:822)
   at 
 org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:954)
   at com.datastax.bdp.server.DseServer.batch_mutate(DseServer.java:576)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3922)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3906)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.IllegalArgumentException: Comparison method violates its 
 general contract!
   at java.util.TimSort.mergeHi(TimSort.java:868)
   at java.util.TimSort.mergeAt(TimSort.java:485)
   at java.util.TimSort.mergeCollapse(TimSort.java:410)
   at java.util.TimSort.sort(TimSort.java:214)
   at java.util.TimSort.sort(TimSort.java:173)
   at java.util.Arrays.sort(Arrays.java:659)
   at java.util.Collections.sort(Collections.java:217)
   at 
 org.apache.cassandra.locator.AbstractEndpointSnitch.sortByProximity(AbstractEndpointSnitch.java:49)
   at 
 org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithScore(DynamicEndpointSnitch.java:157)
   at 
 org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithBadness(DynamicEndpointSnitch.java:186)
   at 
 org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximity(DynamicEndpointSnitch.java:151)
   at 
 org.apache.cassandra.service.StorageProxy.getLiveSortedEndpoints(StorageProxy.java:1408)
   at 
 org.apache.cassandra.service.StorageProxy.getLiveSortedEndpoints(StorageProxy.java:1402)
   at 
 org.apache.cassandra.service.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:148)
   at 
 org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1223)
   at 
 org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1165)
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:255)
   at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:225)
   at org.apache.cassandra.auth.Auth.selectUser(Auth.java:243)
   at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:84)
   at

[jira] [Updated] (CASSANDRA-8613) Regression in mixed single and multi-column relation support

2015-01-13 Thread Philip Thompson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8613:
---
Tester: Shawn Kumar

 Regression in mixed single and multi-column relation support
 

 Key: CASSANDRA-8613
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8613
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Benjamin Lerer
 Fix For: 2.1.3, 2.0.13


 In 2.0.6 through 2.0.8, a query like the following was supported:
 {noformat}
 SELECT * FROM mytable WHERE clustering_0 = ? AND (clustering_1, clustering_2) 
  (?, ?)
 {noformat}
 However, after CASSANDRA-6875, you'll get the following error:
 {noformat}
 Clustering columns may not be skipped in multi-column relations. They should 
 appear in the PRIMARY KEY order. Got (c, d)  (0, 0)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8617) Cassandra inserts throwing You have not logged in

2015-01-13 Thread Neeraj Khandelwal (JIRA)

Neeraj Khandelwal created CASSANDRA-8617:


 Summary: Cassandra inserts throwing You have not logged in
 Key: CASSANDRA-8617
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8617
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Neeraj Khandelwal


Just to mention that I am using Ya-Cassandra PDO driver but I think the issue 
has nothing to do with.

We are running multiple workers which is basically an infinite loop and keeps 
inserting data into Cassandra. For a while everything runs fine but I keep 
getting You have not logged in for a batch of inserts and then again 
everything runs smoothly. This is happening very frequently.

Additional info:
Connection management - I have static class variable for the connection and the 
connection itself is persistent. So, the class variable is assigned connection 
only the first time and after that it is reused.

Cluster information - 3 nodes having v2.0.8 and 1 node v2.0.9

Please let me know if I am missing anything or extra details will be needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Group for anticompaction by the compaction strategy we have.

2015-01-13 Thread marcuse

Repository: cassandra
Updated Branches:
  refs/heads/trunk 5839a4dbe - b8dd3bd75


Group for anticompaction by the compaction strategy we have.

Patch by marcuse; reviewed by yukim for CASSANDRA-8578


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8dd3bd7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8dd3bd7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8dd3bd7

Branch: refs/heads/trunk
Commit: b8dd3bd75f61c20350db37b784a667636fb6a24d
Parents: 5839a4d
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Jan 8 08:35:44 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Jan 14 08:36:11 2015 +0100

--
 CHANGES.txt| 1 +
 .../cassandra/db/compaction/WrappingCompactionStrategy.java| 5 +
 .../cassandra/db/compaction/LeveledCompactionStrategyTest.java | 6 +++---
 3 files changed, 9 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8dd3bd7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b1f5465..7b0e398 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Group sstables for anticompaction correctly (CASSANDRA-8578)
  * Add ReadFailureException to native protocol, respond
immediately when replicas encounter errors while handling
a read request (CASSANDRA-7886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8dd3bd7/src/java/org/apache/cassandra/db/compaction/WrappingCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/WrappingCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/WrappingCompactionStrategy.java
index 55095a7..9d83a47 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/WrappingCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/WrappingCompactionStrategy.java
@@ -358,6 +358,11 @@ public final class WrappingCompactionStrategy extends 
AbstractCompactionStrategy
 return new ScannerList(scanners);
 }
 
+public CollectionCollectionSSTableReader 
groupSSTablesForAntiCompaction(CollectionSSTableReader sstablesToGroup)
+{
+return unrepaired.groupSSTablesForAntiCompaction(sstablesToGroup);
+}
+
 public ListAbstractCompactionStrategy getWrappedStrategies()
 {
 return Arrays.asList(repaired, unrepaired);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8dd3bd7/test/unit/org/apache/cassandra/db/compaction/LeveledCompactionStrategyTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/LeveledCompactionStrategyTest.java
 
b/test/unit/org/apache/cassandra/db/compaction/LeveledCompactionStrategyTest.java
index 1eca4e6..6ea8168 100644
--- 
a/test/unit/org/apache/cassandra/db/compaction/LeveledCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/LeveledCompactionStrategyTest.java
@@ -121,10 +121,10 @@ public class LeveledCompactionStrategyTest
 }
 
 waitForLeveling(cfs);
-LeveledCompactionStrategy strategy = (LeveledCompactionStrategy) 
cfs.getCompactionStrategy();
+WrappingCompactionStrategy strategy = (WrappingCompactionStrategy) 
cfs.getCompactionStrategy();
 // Checking we're not completely bad at math
-assert strategy.getLevelSize(1)  0;
-assert strategy.getLevelSize(2)  0;
+assert strategy.getSSTableCountPerLevel()[1]  0;
+assert strategy.getSSTableCountPerLevel()[2]  0;
 
 CollectionCollectionSSTableReader groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(cfs.getSSTables());
 for (CollectionSSTableReader sstableGroup : groupedSSTables)

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError

2015-01-13 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276597#comment-14276597
 ] 

Marcus Eriksson commented on CASSANDRA-8548:


ok, problem is that you have atleast one 18GB partition, meaning we can't 
cancel compactions and the scrubbing fails

{noformat}
INFO [CompactionExecutor:16] 2015-01-13 17:08:24,068 CompactionController.java 
(line 192) Compacting large row bitcoin/okcoin_order_book_btc_usd:2014-06 
(18051090068 bytes) incrementally
{noformat}



 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at

[jira] [Commented] (CASSANDRA-8558) deleted row still can be selected out


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275732#comment-14275732
 ] 

Tyler Hobbs commented on CASSANDRA-8558:


Ah, nevermind, this comment explains how that's handled pretty well: 
https://issues.apache.org/jira/browse/CASSANDRA-3708?focusedCommentId=13258303page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13258303

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
 Fix For: 2.0.12, 2.1.3

 Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Remove unused DEFAULT_CQL_PAGE_LIMIT from CqlRecordReader

Remove unused DEFAULT_CQL_PAGE_LIMIT from CqlRecordReader

Patch by Rekha Joshi, reviewed by brandonwilliams for CASSANDRA-7269


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7e6d9eb8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7e6d9eb8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7e6d9eb8

Branch: refs/heads/trunk
Commit: 7e6d9eb8422442c0ea4fc8424e41258ff1fdc981
Parents: 359d3bb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 12:50:39 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 12:53:10 2015 -0600

--
 src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java | 2 --
 1 file changed, 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e6d9eb8/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
index 6a1f5bf..234cba3 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
@@ -69,8 +69,6 @@ public class CqlRecordReader extends RecordReaderLong, Row
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CqlRecordReader.class);
 
-public static final int DEFAULT_CQL_PAGE_LIMIT = 1000;
-
 private ColumnFamilySplit split;
 private RowIterator rowIterator;

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5839a4db
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5839a4db
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5839a4db

Branch: refs/heads/trunk
Commit: 5839a4dbe1e4044b9b6f1ee5ca3e3d520f693508
Parents: 8a4abdd 7e6d9eb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 13:55:30 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 13:55:30 2015 -0600

--
 src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java | 2 --
 1 file changed, 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5839a4db/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--

[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log

[
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275726#comment-14275726
]

Ariel Weisberg commented on CASSANDRA-6809:
---

I finished my review. Comments are in the pull request. It looks good and could
ship as is. I have some thoughts about potential scope creep I would advocate
for. Also some other directions for enhancement the commit log could go in as
well as some reservations about performance in some cases. I only just noticed
CommitLog stress so I need to check that out so I can understand the numbers
and what is being tested.

RE CASSANDRA-7075 multiple CL disks. I see this as a work around for not
having RAID-0 of the volumes being used for the CL and that is it. And that may
introduce it's own load balancing issues as well as a mess of code for
scattering/gathering mutations that I am less comfortable with. Writing a CL
pipeline that can do the maximum supported sequential IO to a single file is
doable, and if I had a choice it is what I would rather write. From a user
perspective it is a nice feature to not to be forced to provide a RAID volume
and to me that should be the primary motivation.

Also fascinating (to me) piece of trivia. When I tested in the past I could
call force() on a mapped byte buffer far fewer times then I could call force()
on a FileChannel. So if I had a battery backed disk controller and I appended a
page (in a preallocated file) and called force() in a loop with a
MappedByteBuffer it would do a few hundreds syncs a second, but with
FileChannel.force it would do a few thousand. MBB was slow enough to be a
concern for synchronous commits.

Compressed Commit Log
-

Key: CASSANDRA-6809
URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
Project: Cassandra
Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
Labels: performance
Fix For: 3.0

Attachments: logtest.txt

It seems an unnecessary oversight that we don't compress the commit log.
Doing so should improve throughput, but some care will need to be taken to
ensure we use as much of a segment as possible. I propose decoupling the
writing of the records from the segments. Basically write into a (queue of)
DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X
MB written to the CL (where X is ordinarily CLS size), and then pack as many
of the compressed chunks into a CLS as possible.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-6809) Compressed Commit Log

[
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275726#comment-14275726
]

Ariel Weisberg edited comment on CASSANDRA-6809 at 1/13/15 7:26 PM:

I finished my review. Comments are in the pull request. It looks good and could
ship as is modulo some minor changes and if the the right tests are there (or
get created). I have some thoughts about potential scope creep I would advocate
for. Also some other directions for enhancement the commit log could go in as
well as some reservations about performance in some cases. I only just noticed
CommitLog stress so I need to check that out so I can understand the numbers
and what is being tested.

was (Author: aweisberg):
I finished my review. Comments are in the pull request. It looks good and could
ship as is. I have some thoughts about potential scope creep I would advocate
for. Also some other directions for enhancement the commit log could go in as
well as some reservations about performance in some cases. I only just noticed
CommitLog stress so I need to check that out so I can understand the numbers
and what is being tested.

Compressed Commit Log
-

Attachments: logtest.txt

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7523) add date and time types

2015-01-13 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275812#comment-14275812
 ] 

Joshua McKenzie commented on CASSANDRA-7523:


Driver PR discussion: [here|https://github.com/datastax/python-driver/pull/241]
Squashed/rebased branch for Cassandra: 
[here|https://github.com/josh-mckenzie/cassandra/compare/7523_rebase]

 add date and time types
 ---

 Key: CASSANDRA-7523
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7523
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Jonathan Ellis
Assignee: Joshua McKenzie
Priority: Minor
  Labels: client-impacting, docs
 Fix For: 2.1.3


 http://www.postgresql.org/docs/9.1/static/datatype-datetime.html
 (we already have timestamp; interval is out of scope for now, and see 
 CASSANDRA-6350 for discussion on timestamp-with-time-zone.  but date/time 
 should be pretty easy to add.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8414) Avoid loops over array backed iterators that call iter.remove()

2015-01-13 Thread Richard Low (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Low updated CASSANDRA-8414:
---
Fix Version/s: 2.0.12

 Avoid loops over array backed iterators that call iter.remove()
 ---

 Key: CASSANDRA-8414
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8414
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jimmy Mårdell
  Labels: performance
 Fix For: 2.0.12, 2.1.3

 Attachments: cassandra-2.0-8414-1.txt, cassandra-2.0-8414-2.txt, 
 cassandra-2.0-8414-3.txt, cassandra-2.0-8414-4.txt, cassandra-2.0-8414-5.txt, 
 cassandra-2.1-8414-5.txt, cassandra-2.1-8414-6.txt


 I noticed from sampling that sometimes compaction spends almost all of its 
 time in iter.remove() in ColumnFamilyStore.removeDeletedStandard. It turns 
 out that the cf object is using ArrayBackedSortedColumns, so deletes are from 
 an ArrayList. If the majority of your columns are GCable tombstones then this 
 is O(n^2). The data structure should be changed or a copy made to avoid this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275809#comment-14275809
]

Ariel Weisberg commented on CASSANDRA-7438:
---

bq. Making freeCapacity a per-segment field: Then I'd prefer to reverse the
stuff - i.e. add an allocatedBytes field to each segment. Operations would
get the (calculated) free capacity as a parameter and act on that value. Or
were you thinking about dividing capacity by number of segments and use that as
the max capacity for each segment?
The goal of splitting the locks and tables into segments is to eliminate any
globally shared cache lines that are written to by common operations on the
cache. Having every put modify the free capacity introduces a potential point
of contention. Only way to really understand the impact is to have a good micro
benchmark you trust and try it both ways.

I think that you would split the capacity across the segments. Do exactly what
you are doing now, but do the check inside the segment. Since puts are allowed
to fail I don't think you have to do anything else.

bq. Regarding rehash/iterators: Could be simply worked around by counting the
number of active iterators and just don't rehash while an iterator is active.
That's better than e.g. returning duplicate keys or keys not at all - i.e.
people relying on that functionality.
This is an OHC not a C* issue. I think from C*'s perspective it can be wrong
rarely and it doesn't matter since it doesn't effect correctness. Definitely
worth documenting though.

bq. I lean towards removing the new tables implementation in OHC. It has the
big drawback that it only a allows a specific number of entries per bucket
(e.g. 8). But I'd like to defer that decision after some tests on a NUMA
machine.
You are on to something in terms of making a faster hash table, but it doesn't
seem like huge win given the short length of most chains (1, or 2) and the
overhead of the allocator and locking etc. It would show up in a
micro-benchmark, but not in C*. I would like to stick with linked for C* for
now since it's easy to understand and I've looked at it a few times.

I think I already sent you a link to this
https://www.cs.cmu.edu/~dga/papers/silt-sosp2011.pdf but there are a lot of
ideas there for dense hash tables. You can chain together multiple buckets so
the entries per bucket becomes a function of cache line size.

Serializing Row cache alternative (Fully off heap)
--

Key: CASSANDRA-7438
URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
Project: Cassandra
Issue Type: Improvement
Components: Core
Environment: Linux
Reporter: Vijay
Assignee: Robert Stupp
Labels: performance
Fix For: 3.0

Attachments: 0001-CASSANDRA-7438.patch, tests.zip

Currently SerializingCache is partially off heap, keys are still stored in
JVM heap as BB,
* There is a higher GC costs for a reasonably big cache.
* Some users have used the row cache efficiently in production for better
results, but this requires careful tunning.
* Overhead in Memory for the cache entries are relatively high.
So the proposal for this ticket is to move the LRU cache logic completely off
heap and use JNI to interact with cache. We might want to ensure that the new
implementation match the existing API's (ICache), and the implementation
needs to have safe memory access, low overhead in memory and less memcpy's
(As much as possible).
We might also want to make this cache configurable.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Remove unused DEFAULT_CQL_PAGE_LIMIT from CqlRecordReader

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 359d3bb2a - 7e6d9eb84
  refs/heads/trunk 8a4abdda1 - 5839a4dbe


Remove unused DEFAULT_CQL_PAGE_LIMIT from CqlRecordReader

Patch by Rekha Joshi, reviewed by brandonwilliams for CASSANDRA-7269


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7e6d9eb8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7e6d9eb8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7e6d9eb8

Branch: refs/heads/cassandra-2.1
Commit: 7e6d9eb8422442c0ea4fc8424e41258ff1fdc981
Parents: 359d3bb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 12:50:39 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 12:53:10 2015 -0600

--
 src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java | 2 --
 1 file changed, 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e6d9eb8/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
index 6a1f5bf..234cba3 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
@@ -69,8 +69,6 @@ public class CqlRecordReader extends RecordReaderLong, Row
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CqlRecordReader.class);
 
-public static final int DEFAULT_CQL_PAGE_LIMIT = 1000;
-
 private ColumnFamilySplit split;
 private RowIterator rowIterator;

[jira] [Commented] (CASSANDRA-7974) Enable tooling to detect hot partitions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275849#comment-14275849
 ] 

Brandon Williams commented on CASSANDRA-7974:
-

This mostly looks good, but a few things: we aren't supposed to return any kind 
of internals over JMX, only simple types.  While String getKeyValidator() makes 
it easier to print stuff, I don't think that's the right approach or something 
that should be exposed via JMX, either.  Using the tracing stage for this seems 
kind of lazy - do we even really need an async call?

 Enable tooling to detect hot partitions
 ---

 Key: CASSANDRA-7974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7974
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Chris Lohfink
 Fix For: 2.1.3

 Attachments: 7974.txt, cassandra-2.1-7974v2.txt


 Sometimes you know you have a hot partition by the load on a replica set, but 
 have no way of determining which partition it is.  Tracing is inadequate for 
 this without a lot of post-tracing analysis that might not yield results.  
 Since we already include stream-lib for HLL in compaction metadata, it 
 shouldn't be too hard to wire up topK for X seconds via jmx/nodetool and then 
 return the top partitions hit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276400#comment-14276400
 ] 

Ariel Weisberg commented on CASSANDRA-8614:
---

I checked Netty's PlatformDependent for how they did it and copied that. The 
question would be why they did it that way? Maybe to avoid error prone string 
parsing? I m fine with switching to parsing the java.version string.

 Select optimal CRC32 implementation at runtime
 --

 Key: CASSANDRA-8614
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8614
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Ariel Weisberg
  Labels: performance
 Attachments: 8614.patch


 JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec 
 per core in my quick and dirty test. PureJavaCRC32 is  800 megabytes/sec if 
 I recall and it has a lookup table that evicts random cache lines every time 
 it runs.
 In order to capture the benefit of that when it is available we can select a 
 CRC32 implementation at startup in a static block.
 If JDK 8 is not what is running we can fall back to the existing 
 PureJavaCRC32 implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8616) sstable2json may result in commit log segments be written

2015-01-13 Thread Russ Hatch (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276270#comment-14276270
 ] 

Russ Hatch commented on CASSANDRA-8616:
---

This indeed seems to be the case with 2.0.10, and it's even happening if you 
invoke sstable2json incorrectly.

To repro I ran cassandra-stress, and killed the cassandra process while stress 
was still running. Then I ran sstable2json (with a bad sstable filename, and 
with a valid one) and new commitlog files are created.

{noformat}
root@ce986ee973c1:/cassandra# ls -ltr /var/lib/cassandra/commitlog | wc -l
11
root@ce986ee973c1:/cassandra# bin/sstable2json thisfiledoesntexist.db  
/dev/null
Exception in thread main java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at 
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:235)
at 
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:204)
at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:452)
^C
root@ce986ee973c1:/cassandra# ls -ltr /var/lib/cassandra/commitlog | wc -l
12
{noformat}

 sstable2json may result in commit log segments be written
 -

 Key: CASSANDRA-8616
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8616
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Russ Hatch
 Fix For: 2.0.13


 There was a report of sstable2json causing commitlog segments to be written 
 out when run.  I haven't attempted to reproduce this yet, so that's all I 
 know for now.  Since sstable2json loads the conf and schema, I'm thinking 
 that it may inadvertently be triggering the commitlog code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime

[
https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276400#comment-14276400
]

Ariel Weisberg edited comment on CASSANDRA-8614 at 1/14/15 2:38 AM:

I checked Netty's PlatformDependent for how they did it and copied that. The
question would be why they did it that way? Maybe to avoid error prone string
parsing? I'm fine with switching to parsing the java.version string.

was (Author: aweisberg):
I checked Netty's PlatformDependent for how they did it and copied that. The
question would be why they did it that way? Maybe to avoid error prone string
parsing? I m fine with switching to parsing the java.version string.

Select optimal CRC32 implementation at runtime
--

Key: CASSANDRA-8614
URL: https://issues.apache.org/jira/browse/CASSANDRA-8614
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Ariel Weisberg
Labels: performance
Attachments: 8614.patch

JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec
per core in my quick and dirty test. PureJavaCRC32 is 800 megabytes/sec if
I recall and it has a lookup table that evicts random cache lines every time
it runs.
In order to capture the benefit of that when it is available we can select a
CRC32 implementation at startup in a static block.
If JDK 8 is not what is running we can fall back to the existing
PureJavaCRC32 implementation.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8616) sstable2json may result in commit log segments be written

2015-01-13 Thread Russ Hatch (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch updated CASSANDRA-8616:
--
Reproduced In: 2.0.10, 2.1.3  (was: 2.0.10)

 sstable2json may result in commit log segments be written
 -

 Key: CASSANDRA-8616
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8616
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Russ Hatch
 Fix For: 2.0.13


 There was a report of sstable2json causing commitlog segments to be written 
 out when run.  I haven't attempted to reproduce this yet, so that's all I 
 know for now.  Since sstable2json loads the conf and schema, I'm thinking 
 that it may inadvertently be triggering the commitlog code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8616) sstable2json may result in commit log segments be written

2015-01-13 Thread Russ Hatch (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276283#comment-14276283
 ] 

Russ Hatch commented on CASSANDRA-8616:
---

Same goes for 2.1

{noformat}
root@e69321a30198:/cassandra# ls -ltr data/commitlog/   
total 262144
-rw-r--r-- 1 root root 33554432 Jan 14 00:37 CommitLog-4-1421195804516.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804525.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804530.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804529.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804528.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804531.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804526.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804527.log
root@e69321a30198:/cassandra# ls -ltr data/commitlog/ | wc -l
9
root@e69321a30198:/cassandra# tools/bin/sstable2json 
data/data/keyspace1/standard1-85148e309b8511e4b94691de6f1c3a1d/keyspace1-standard1-ka-1-Data.db
  /dev/null
root@e69321a30198:/cassandra# ls -ltr data/commitlog/
total 262148
-rw-r--r-- 1 root root 33554432 Jan 14 00:37 CommitLog-4-1421195804516.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804525.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804530.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804529.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804528.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804531.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804526.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:38 CommitLog-4-1421195804527.log
-rw-r--r-- 1 root root 33554432 Jan 14 00:42 CommitLog-4-1421196133229.log
root@e69321a30198:/cassandra# ls -ltr data/commitlog/ | wc -l
10
{noformat}

 sstable2json may result in commit log segments be written
 -

 Key: CASSANDRA-8616
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8616
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Russ Hatch
 Fix For: 2.0.13


 There was a report of sstable2json causing commitlog segments to be written 
 out when run.  I haven't attempted to reproduce this yet, so that's all I 
 know for now.  Since sstable2json loads the conf and schema, I'm thinking 
 that it may inadvertently be triggering the commitlog code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dominic Letz updated CASSANDRA-8603:

Attachment: cassandra-2.0.11-8603.txt

 Cut tombstone memory footprint in half for cql deletes
 --

 Key: CASSANDRA-8603
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Dominic Letz
  Labels: tombstone
 Attachments: cassandra-2.0.11-8603.txt


 As CQL does not yet support range deletes every delete from CQL results in a 
 Semi-RangeTombstone which actually has the same start and end values - but 
 until today they are copies. Effectively doubling the required heap memory to 
 store the RangeTombstone.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8559) OOM caused by large tombstone warning.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dominic Letz updated CASSANDRA-8559:

Attachment: cassandra-2.0.11-8559.txt

Added a variable tombstone_warn_message_len to limit the length of the 
generated string representation of DeletionInfo. I'm not completely happy with 
the naming of the variable as it really only affects DeletionInfo::toString(). 
Food for thought.

 OOM caused by large tombstone warning.
 --

 Key: CASSANDRA-8559
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8559
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.0.11 / 2.1
Reporter: Dominic Letz
 Fix For: 2.0.12

 Attachments: Selection_048.png, cassandra-2.0.11-8559.txt, 
 stacktrace.log


 When running with high amount of tombstones the error message generation from 
 CASSANDRA-6117 can lead to out of memory situation with the default setting.
 Attached a heapdump viewed in visualvm showing how this construct created two 
 777mb strings to print the error message for a read query and then crashed 
 OOM.
 {code}
 if (respectTombstoneThresholds()  columnCounter.ignored()  
 DatabaseDescriptor.getTombstoneWarnThreshold())
 {
 StringBuilder sb = new StringBuilder();
 CellNameType type = container.metadata().comparator;
 for (ColumnSlice sl : slices)
 {
 assert sl != null;
 sb.append('[');
 sb.append(type.getString(sl.start));
 sb.append('-');
 sb.append(type.getString(sl.finish));
 sb.append(']');
 }
 logger.warn(Read {} live and {} tombstoned cells in {}.{} (see 
 tombstone_warn_threshold). {} columns was requested, slices={}, delInfo={},
 columnCounter.live(), columnCounter.ignored(), 
 container.metadata().ksName, container.metadata().cfName, count, sb, 
 container.deletionInfo());
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8559) OOM caused by large tombstone warning.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dominic Letz updated CASSANDRA-8559:

Labels: tombstone  (was: )

 OOM caused by large tombstone warning.
 --

 Key: CASSANDRA-8559
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8559
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.0.11 / 2.1
Reporter: Dominic Letz
  Labels: tombstone
 Fix For: 2.0.12

 Attachments: Selection_048.png, cassandra-2.0.11-8559.txt, 
 stacktrace.log


 When running with high amount of tombstones the error message generation from 
 CASSANDRA-6117 can lead to out of memory situation with the default setting.
 Attached a heapdump viewed in visualvm showing how this construct created two 
 777mb strings to print the error message for a read query and then crashed 
 OOM.
 {code}
 if (respectTombstoneThresholds()  columnCounter.ignored()  
 DatabaseDescriptor.getTombstoneWarnThreshold())
 {
 StringBuilder sb = new StringBuilder();
 CellNameType type = container.metadata().comparator;
 for (ColumnSlice sl : slices)
 {
 assert sl != null;
 sb.append('[');
 sb.append(type.getString(sl.start));
 sb.append('-');
 sb.append(type.getString(sl.finish));
 sb.append(']');
 }
 logger.warn(Read {} live and {} tombstoned cells in {}.{} (see 
 tombstone_warn_threshold). {} columns was requested, slices={}, delInfo={},
 columnCounter.live(), columnCounter.ignored(), 
 container.metadata().ksName, container.metadata().cfName, count, sb, 
 container.deletionInfo());
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8547) Make RangeTombstone.Tracker.isDeleted() faster


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dominic Letz updated CASSANDRA-8547:

Labels: tombstone  (was: )

 Make RangeTombstone.Tracker.isDeleted() faster
 --

 Key: CASSANDRA-8547
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8547
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: 2.0.11
Reporter: Dominic Letz
Assignee: Dominic Letz
  Labels: tombstone
 Fix For: 2.1.3

 Attachments: Selection_044.png, cassandra-2.0.11-8547.txt, 
 cassandra-2.1-8547.txt, rangetombstone.tracker.txt


 During compaction and repairs with many tombstones an exorbitant amount of 
 time is spend in RangeTombstone.Tracker.isDeleted().
 The amount of time spend there can be so big that compactions and repairs 
 look stalled and the time remaining time estimated frozen at the same value 
 for days.
 Using visualvm I've been sample profiling the code during execution and both 
 in Compaction as well as during repairs found this. (point in time backtraces 
 attached)
 Looking at the code the problem is obviously the linear scanning:
 {code}
 public boolean isDeleted(Column column)
 {
 for (RangeTombstone tombstone : ranges)
 {
 if (comparator.compare(column.name(), tombstone.min) = 0
  comparator.compare(column.name(), tombstone.max) = 0
  tombstone.maxTimestamp() = column.timestamp())
 {
 return true;
 }
 }
 return false;
 }
 {code}
 I would like to propose to change this and instead use a sorted list (e.g. 
 RangeTombstoneList) here instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8546) RangeTombstoneList becoming bottleneck on tombstone heavy tasks


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dominic Letz updated CASSANDRA-8546:

Labels: tombstone  (was: )

 RangeTombstoneList becoming bottleneck on tombstone heavy tasks
 ---

 Key: CASSANDRA-8546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8546
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: 2.0.11 / 2.1
Reporter: Dominic Letz
Assignee: Joshua McKenzie
  Labels: tombstone
 Fix For: 2.1.3

 Attachments: cassandra-2.0.11-8546.txt, cassandra-2.1-8546.txt, 
 rangetombstonelist_compaction.png, rangetombstonelist_mutation.png, 
 rangetombstonelist_read.png, tombstone_test.tgz


 I would like to propose a change of the data structure used in the 
 RangeTombstoneList to store and insert tombstone ranges to something with at 
 least O(log N) insert in the middle and at near O(1) and start AND end. Here 
 is why:
 When having tombstone heavy work-loads the current implementation of 
 RangeTombstoneList becomes a bottleneck with slice queries.
 Scanning the number of tombstones up to the default maximum (100k) can take 
 up to 3 minutes of how addInternal() scales on insertion of middle and start 
 elements.
 The attached test shows that with 50k deletes from both sides of a range.
 INSERT 1...11
 flush()
 DELETE 1...5
 DELETE 11...6
 While one direction performs ok (~400ms on my notebook):
 {code}
 SELECT * FROM timeseries WHERE name = 'a' ORDER BY timestamp DESC LIMIT 1
 {code}
 The other direction underperforms (~7seconds on my notebook)
 {code}
 SELECT * FROM timeseries WHERE name = 'a' ORDER BY timestamp ASC LIMIT 1
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes

Dominic Letz created CASSANDRA-8603:
---

 Summary: Cut tombstone memory footprint in half for cql deletes
 Key: CASSANDRA-8603
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Dominic Letz


As CQL does not yet support range deletes every delete from CQL results in a 
Semi-RangeTombstone which actually has the same start and end values - but 
until today they are copies. Effectively doubling the required heap memory to 
store the RangeTombstone.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dominic Letz updated CASSANDRA-8603:

Attachment: cassandra-2.1-8603.txt

Attached trivial patches for 2.0.11 and 2.1 

 Cut tombstone memory footprint in half for cql deletes
 --

 Key: CASSANDRA-8603
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Dominic Letz
  Labels: tombstone
 Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt


 As CQL does not yet support range deletes every delete from CQL results in a 
 Semi-RangeTombstone which actually has the same start and end values - but 
 until today they are copies. Effectively doubling the required heap memory to 
 store the RangeTombstone.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7679) Batch DDL

2015-01-13 Thread Jorge Bay (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275227#comment-14275227
 ] 

Jorge Bay commented on CASSANDRA-7679:
--

Consider the example using any driver with a round robin policy:
{code:java}
session.execute(CREATE TABLE tbl1 ...);
session.execute(INSERT INTO tbl1...);
{code}

In a multinode cluster, this will fail. [This includes any getting started 
example that you can 
find|http://www.datastax.com/documentation/developer/java-driver/2.1/java-driver/quick_start/qsSimpleClientAddSession_t.html].
 I don't think its a good idea to force users to wait an undetermined amount of 
time (even if its small) for them to be sure that the schema change was applied 
in the cluster. This prevents tools similar to Rails Migrations (as a general 
pattern) to be used with C*.

IMO, the batching (allowing multiple schema changes in one request) is not as 
important as allowing users know that the schema change was applied in the 
cluster.

 Batch DDL
 -

 Key: CASSANDRA-7679
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7679
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp

 Just an idea: To improve speed of DDL in clusters with lots of 
 Keyspaces/Tables/Columns it might help to collect a bunch of schema changes 
 and propagate them as a single bunch of changes.
 Such a DDL batch would
 # execute DDLs locally and collect all mutations
 # broadcast all mutations at once
 # schema agreement
 # return listSchemaChange via native protocol to the client
 So {{DefsTables.mergeSchemaInternal}} (which seems to be the expensive part) 
 would only execute once per DDL batch on each node.
 DDL batches would not be atomic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8607) Introduce unit tests or dtests for cassandra-stress

Benedict created CASSANDRA-8607:
---

 Summary: Introduce unit tests or dtests for cassandra-stress
 Key: CASSANDRA-8607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8607
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Benedict


Right now we don't have any tests in place to ensure we don't break stress. In 
general we've relied on simply noticing problems as part of the development 
process, since we've typically developed features in tandem with related c* 
functionality, so we would see if we'd gotten it wrong. However now that it is 
being used more widely, we could do with some automated testing to ensure we 
haven't accidentally broken anything. If we mock up some non-random random 
classes and permit them to be swapped in wherever necessary we could construct 
some known dataset / visitation etc behaviours that elicit all of the edge 
cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little

Benedict created CASSANDRA-8608:
---

 Summary: Fix cassandra-stress bug introduced by 7964, and cleanup 
related code a little
 Key: CASSANDRA-8608
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8608
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict


7964 had a very basic mistake present when wiring up writes - the limit was 
not reset, so only the first row would be returned (the fact _ANY_ row was 
returned was itself sort-of a bug in this scenario, and I've changed this 
also). At the same time I've changed the definition of limit to lastRow so that 
the related functionality is clearer and more obviously correct.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8605) nullpointer exception in thread pool executor


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict resolved CASSANDRA-8605.
-
Resolution: Duplicate

 nullpointer exception in thread pool executor
 -

 Key: CASSANDRA-8605
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8605
 Project: Cassandra
  Issue Type: Bug
 Environment: Debian stable (wheezy)
Reporter: Anderson Cummins
 Fix For: 2.1.2


 i see the following exception messages in /var/log/cassandra/system.log:
 ERROR [CompactionExecutor:123] 2015-01-12 06:02:37,863 
 CassandraDaemon.java:153 - Exception in thread 
 Thread[CompactionExecutor:123,1,main]
 java.lang.NullPointerException: null
 at 
 org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:475)
  ~[apache-cassandra-2.1.2.jar:2.1.2]
 at 
 org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:463)
  ~[apache-cassandra-2.1.2.jar:2.1.2]
 at 
 org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:274)
  ~[apache-cassandra-2.1.2.jar:2.1.2]
 at 
 org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1088)
  ~[apache-cassandra-2.1.2.jar:2.1.2]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
 ~[na:1.8.0_25] at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
 ~[na:1.8.0_25]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8606) sstablesplit does not remove original sstable

2015-01-13 Thread Marcus Eriksson (JIRA)

Marcus Eriksson created CASSANDRA-8606:
--

 Summary: sstablesplit does not remove original sstable
 Key: CASSANDRA-8606
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8606
 Project: Cassandra
  Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 2.1.3


sstablesplit leaves the original file on disk, it should not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8608) Fix cassandra-stress bug introduced by 7964, and cleanup related code a little


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275240#comment-14275240
 ] 

Benedict commented on CASSANDRA-8608:
-

Branch available [here|https://github.com/belliottsmith/cassandra/tree/8608]

 Fix cassandra-stress bug introduced by 7964, and cleanup related code a little
 --

 Key: CASSANDRA-8608
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8608
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict

 7964 had a very basic mistake present when wiring up writes - the limit was 
 not reset, so only the first row would be returned (the fact _ANY_ row was 
 returned was itself sort-of a bug in this scenario, and I've changed this 
 also). At the same time I've changed the definition of limit to lastRow so 
 that the related functionality is clearer and more obviously correct.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7679) Batch DDL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275283#comment-14275283
 ] 

Sylvain Lebresne commented on CASSANDRA-7679:
-

I'm confuse for 2 reasons:
# You example has only a single schema change, so I fail to see how batching 
schema change would have anything to do with it
# That example should *not* fail in multinode clusters with the java driver (it 
might fail in other driver, but that's the one you're pointing to) or it's a 
bug of the java driver.

bq. the batching (allowing multiple schema changes in one request) is not as 
important as allowing users know that the schema change was applied in the 
cluster

You can already do that by checking for schema agreement (which is what the 
java driver does before returning from the {{CREATE TABLE}} statement). So yes, 
it's the responsability of your driver to check for schema agreement, but the 
reason it's that way are that:
* if we were to do it Cassandra side, we wouldn't have much other way to do it 
anyway (at least given the current way schema are handled). So you don't lose 
anything by doing it client side.
* doing it client side is more flexible: the client can have a timeout on how 
long it waits before giving up on agreement (without needing C* to implement a 
new setting) and it can choose between doing that check (being synchronous) or 
not doing that check (If being asynchronous is actually ok).


 Batch DDL
 -

 Key: CASSANDRA-7679
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7679
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp

 Just an idea: To improve speed of DDL in clusters with lots of 
 Keyspaces/Tables/Columns it might help to collect a bunch of schema changes 
 and propagate them as a single bunch of changes.
 Such a DDL batch would
 # execute DDLs locally and collect all mutations
 # broadcast all mutations at once
 # schema agreement
 # return listSchemaChange via native protocol to the client
 So {{DefsTables.mergeSchemaInternal}} (which seems to be the expensive part) 
 would only execute once per DDL batch on each node.
 DDL batches would not be atomic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8414) Avoid loops over array backed iterators that call iter.remove()

2015-01-13 Thread JIRA


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275024#comment-14275024
 ] 

Jimmy Mårdell commented on CASSANDRA-8414:
--

Right, of course. v6 attached.

 Avoid loops over array backed iterators that call iter.remove()
 ---

 Key: CASSANDRA-8414
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8414
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jimmy Mårdell
  Labels: performance
 Fix For: 2.1.3

 Attachments: cassandra-2.0-8414-1.txt, cassandra-2.0-8414-2.txt, 
 cassandra-2.0-8414-3.txt, cassandra-2.0-8414-4.txt, cassandra-2.0-8414-5.txt, 
 cassandra-2.1-8414-5.txt, cassandra-2.1-8414-6.txt


 I noticed from sampling that sometimes compaction spends almost all of its 
 time in iter.remove() in ColumnFamilyStore.removeDeletedStandard. It turns 
 out that the cf object is using ArrayBackedSortedColumns, so deletes are from 
 an ArrayList. If the majority of your columns are GCable tombstones then this 
 is O(n^2). The data structure should be changed or a copy made to avoid this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8566) node crash (while auto-compaction?)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275073#comment-14275073
 ] 

Benedict commented on CASSANDRA-8566:
-

Were these commands run during the time covered by cass.log? If so, it looks 
like the problem is now different, presumably because the system state has 
become unstable due to the prior crash. The node was failing during startup 
cleaning up compaction leftovers (this likely needs a separate bug filing to 
resolve).

 node crash (while auto-compaction?)
 ---

 Key: CASSANDRA-8566
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8566
 Project: Cassandra
  Issue Type: Bug
 Environment: Linux CentOS 6.6 64bit, Cassandra 2.1.2 (release)
Reporter: Dmitri Dmitrienko
 Fix For: 2.1.3

 Attachments: 1.log, 1.tar.gz


 As data size became 20-24GB/node this issue started happening quite 
 frequently. With 7GB/node I didn't notice any crashes.
 HEAP size was 10GB, now increased to 16GB and it didn't help.
 Log is attached



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8604) Batch Schema Changes at protocol level

2015-01-13 Thread Jorge Bay (JIRA)

Jorge Bay created CASSANDRA-8604:


 Summary: Batch Schema Changes at protocol level
 Key: CASSANDRA-8604
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8604
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jorge Bay


It would be nice to have a way to send to a cluster several DDL queries in 1 
request and get a response when the schema change has been applied on all the 
(live) nodes.

This could be helpful for dev teams that are used to create schema changes db 
scripts and deploy them as part of a version change (ie: Rails migrations).

Also it can avoid race conditions between schema changes and querying that 
schema (in test env or other deployments).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8414) Avoid loops over array backed iterators that call iter.remove()

2015-01-13 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Mårdell updated CASSANDRA-8414:
-
Attachment: cassandra-2.1-8414-6.txt

 Avoid loops over array backed iterators that call iter.remove()
 ---

 Key: CASSANDRA-8414
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8414
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Low
Assignee: Jimmy Mårdell
  Labels: performance
 Fix For: 2.1.3

 Attachments: cassandra-2.0-8414-1.txt, cassandra-2.0-8414-2.txt, 
 cassandra-2.0-8414-3.txt, cassandra-2.0-8414-4.txt, cassandra-2.0-8414-5.txt, 
 cassandra-2.1-8414-5.txt, cassandra-2.1-8414-6.txt


 I noticed from sampling that sometimes compaction spends almost all of its 
 time in iter.remove() in ColumnFamilyStore.removeDeletedStandard. It turns 
 out that the cf object is using ArrayBackedSortedColumns, so deletes are from 
 an ArrayList. If the majority of your columns are GCable tombstones then this 
 is O(n^2). The data structure should be changed or a copy made to avoid this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

2015-01-13 Thread Robert Stupp (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275047#comment-14275047
]

Robert Stupp commented on CASSANDRA-7438:
-

Thanks for the review. Many useful hints in it :)

I'll reduce configuration stuff in C* integration as suggested and add some
default-by-system-property mechanism (as suggested).

Making freeCapacity a per-segment field:
Then I'd prefer to reverse the stuff - i.e. add an allocatedBytes field to
each segment. Operations would get the (calculated) free capacity as a
parameter and act on that value. Or were you thinking about dividing capacity
by number of segments and use that as the max capacity for each segment?

Regarding rehash/iterators: Could be simply worked around by counting the
number of active iterators and just don't rehash while an iterator is active.
That's better than e.g. returning duplicate keys or keys not at all - i.e.
people relying on that functionality.

I just started JMH without any additional parameters. It's called during Maven
test phase (unless you specify -DskipTests).

You're right. Murmur3 + UTF8 need more tests.

Didn't notice that that fastutil is that fat. Already replaced with an own
implementation.

I lean towards removing the new tables implementation in OHC. It has the big
drawback that it only a allows a specific number of entries per bucket (e.g.
8). But I'd like to defer that decision after some tests on a NUMA machine.

Serializing Row cache alternative (Fully off heap)
--

Attachments: 0001-CASSANDRA-7438.patch, tests.zip

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8605) nullpointer exception in thread pool executor

2015-01-13 Thread Anderson Cummins (JIRA)

Anderson Cummins created CASSANDRA-8605:
---

 Summary: nullpointer exception in thread pool executor
 Key: CASSANDRA-8605
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8605
 Project: Cassandra
  Issue Type: Bug
 Environment: Debian stable (wheezy)
Reporter: Anderson Cummins
 Fix For: 2.1.2


i see the following exception messages in /var/log/cassandra/system.log:

ERROR [CompactionExecutor:123] 2015-01-12 06:02:37,863 CassandraDaemon.java:153 
- Exception in thread Thread[CompactionExecutor:123,1,main]
java.lang.NullPointerException: null
at 
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:475)
 ~[apache-cassandra-2.1.2.jar:2.1.2]
at 
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:463)
 ~[apache-cassandra-2.1.2.jar:2.1.2]
at 
org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:274)
 ~[apache-cassandra-2.1.2.jar:2.1.2]
at 
org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1088)
 ~[apache-cassandra-2.1.2.jar:2.1.2]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_25] at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_25]





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275190#comment-14275190
 ] 

Sylvain Lebresne commented on CASSANDRA-8603:
-

Have you actually tested and validated that this was actually used? Because 
unless I'm missing something no range tombstone generated by CQL should have 
the same start and stop (it wouldn't entirely illegal to have a RT with the 
same start and stop, but this RT would select a single cell and bug excluded, 
we use cell tombstones for that case, not range tombstones).

 Cut tombstone memory footprint in half for cql deletes
 --

 Key: CASSANDRA-8603
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8603
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Dominic Letz
  Labels: tombstone
 Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt


 As CQL does not yet support range deletes every delete from CQL results in a 
 Semi-RangeTombstone which actually has the same start and end values - but 
 until today they are copies. Effectively doubling the required heap memory to 
 store the RangeTombstone.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8604) Batch Schema Changes at protocol level

2015-01-13 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-8604.
--
Resolution: Duplicate

 Batch Schema Changes at protocol level
 --

 Key: CASSANDRA-8604
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8604
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jorge Bay
  Labels: client-impacting

 It would be nice to have a way to send to a cluster several DDL queries in 1 
 request and get a response when the schema change has been applied on all the 
 (live) nodes.
 This could be helpful for dev teams that are used to create schema changes db 
 scripts and deploy them as part of a version change (ie: Rails migrations).
 Also it can avoid race conditions between schema changes and querying that 
 schema (in test env or other deployments).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8604) Batch Schema Changes at protocol level

2015-01-13 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275143#comment-14275143
 ] 

Aleksey Yeschenko commented on CASSANDRA-8604:
--

See CASSANDRA-7679. Will be closing this one as duplicate, and you can keep 
discussing reopening the ticket there.

 Batch Schema Changes at protocol level
 --

 Key: CASSANDRA-8604
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8604
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jorge Bay
  Labels: client-impacting

 It would be nice to have a way to send to a cluster several DDL queries in 1 
 request and get a response when the schema change has been applied on all the 
 (live) nodes.
 This could be helpful for dev teams that are used to create schema changes db 
 scripts and deploy them as part of a version change (ie: Rails migrations).
 Also it can avoid race conditions between schema changes and querying that 
 schema (in test env or other deployments).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8068) Allow to create authenticator which is aware of the client connection

2015-01-13 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe resolved CASSANDRA-8068.

Resolution: Won't Fix

The IAuthenticator interface is already quite significantly changed by 
CASSANDRA-7653 and as there doesn't seem to be a really compelling use case for 
this, closing it as wontfix. Feel free to re-open if there's a genuine 
requirement.

 Allow to create authenticator which is aware of the client connection
 -

 Key: CASSANDRA-8068
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8068
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jacek Lewandowski
Assignee: Jacek Lewandowski
Priority: Minor
  Labels: security

 Currently, the authenticator interface doesn't allow to make a decision 
 according to the client connection properties (especially the client host 
 name or address). 
 The idea is to add the interface which extends the current SASL aware 
 authenticator interface with additional method to set the client connection. 
 ServerConnection then could supply the connection to the authenticator if the 
 authenticator implements that interface. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7032) Improve vnode allocation

[
https://issues.apache.org/jira/browse/CASSANDRA-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275343#comment-14275343
]

Benedict commented on CASSANDRA-7032:
-

Sounds very promising! Have you tried this on asymmetric cluster configs? i.e.
with racks or DCs of unequal size? Had a glance at the code and it looks like
not, but in case I misread... it would be great to factor this in.

Improve vnode allocation

Key: CASSANDRA-7032
URL: https://issues.apache.org/jira/browse/CASSANDRA-7032
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Benedict
Assignee: Branimir Lambov
Labels: performance, vnodes
Fix For: 3.0

Attachments: TestVNodeAllocation.java, TestVNodeAllocation.java,
TestVNodeAllocation.java, TestVNodeAllocation.java

It's been known for a little while that random vnode allocation causes
hotspots of ownership. It should be possible to improve dramatically on this
with deterministic allocation. I have quickly thrown together a simple greedy
algorithm that allocates vnodes efficiently, and will repair hotspots in a
randomly allocated cluster gradually as more nodes are added, and also
ensures that token ranges are fairly evenly spread between nodes (somewhat
tunably so). The allocation still permits slight discrepancies in ownership,
but it is bound by the inverse of the size of the cluster (as opposed to
random allocation, which strangely gets worse as the cluster size increases).
I'm sure there is a decent dynamic programming solution to this that would be
even better.
If on joining the ring a new node were to CAS a shared table where a
canonical allocation of token ranges lives after running this (or a similar)
algorithm, we could then get guaranteed bounds on the ownership distribution
in a cluster. This will also help for CASSANDRA-6696.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275372#comment-14275372
 ] 

Andrei Ivanov commented on CASSANDRA-8548:
--

Didn't work for me. Scrubing throws the assertion error.
{code}
aivanov@cass:~/timeseries$ nodetool -h localhost scrub bitcoin
Exception in thread main java.lang.AssertionError: 
[SSTableReader(path='/var/lib/cassandra/data/bitcoin/okcoin_order_book_btc_usd/bitcoin-okcoin_order_book_btc_usd-jb-538-Data.db'),
 
SSTableReader(path='/var/lib/cassandra/data/bitcoin/okcoin_order_book_btc_usd/bitcoin-okcoin_order_book_btc_usd-jb-710-Data.db'),
 
SSTableReader(path='/var/lib/cassandra/data/bitcoin/okcoin_order_book_btc_usd/bitcoin-okcoin_order_book_btc_usd-jb-627-Data.db'),
 
SSTableReader(path='/var/lib/cassandra/data/bitcoin/okcoin_order_book_btc_usd/bitcoin-okcoin_order_book_btc_usd-jb-437-Data.db')]
at 
org.apache.cassandra.db.ColumnFamilyStore$13.call(ColumnFamilyStore.java:2132)
at 
org.apache.cassandra.db.ColumnFamilyStore$13.call(ColumnFamilyStore.java:2129)
at 
org.apache.cassandra.db.ColumnFamilyStore.runWithCompactionsDisabled(ColumnFamilyStore.java:2111)
at 
org.apache.cassandra.db.ColumnFamilyStore.markAllCompacting(ColumnFamilyStore.java:2142)
at 
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:215)
at 
org.apache.cassandra.db.compaction.CompactionManager.performScrub(CompactionManager.java:233)
at 
org.apache.cassandra.db.ColumnFamilyStore.scrub(ColumnFamilyStore.java:1120)
at 
org.apache.cassandra.service.StorageService.scrub(StorageService.java:2169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at 
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at 
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at 
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at 
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at 
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at 
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548

[jira] [Updated] (CASSANDRA-8609) Remove depency of hadoop to internals (Cell/CellName)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8609:

Assignee: Alex Liu

 Remove depency of hadoop to internals (Cell/CellName)
 -

 Key: CASSANDRA-8609
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8609
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Alex Liu
 Fix For: 3.0


 For some reason most of the Hadoop code (ColumnFamilyRecordReader, 
 CqlStorage, ...) uses the {{Cell}} and {{CellName}} classes. That dependency 
 is entirely artificial: all this code is really client code that communicate 
 with Cassandra over thrift/native protocol and there is thus no reason for it 
 to use internal classes. And in fact, thoses classes are used in a very crude 
 way, as a {{PairByteBuffer, ByteBuffer}} really.
 But this dependency is really painful when we make changes to the internals. 
 Further, every time we do so, I believe we break some of those the APIs due 
 to the change. This has been painful for CASSANDRA-5417 and this is now 
 painful for CASSANDRA-8099. But while I somewhat hack over it in 
 CASSANDRA-5417, this was a mistake and we should have removed the depency 
 back then. So let do that now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError

2015-01-13 Thread Sebastian Estevez (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275461#comment-14275461
 ] 

Sebastian Estevez commented on CASSANDRA-8548:
--

[~aivanov93] can you post your logs since startup?

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at

[jira] [Comment Edited] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError

2015-01-13 Thread Sebastian Estevez (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275461#comment-14275461
 ] 

Sebastian Estevez edited comment on CASSANDRA-8548 at 1/13/15 4:11 PM:
---

[~aivanov93] can you post your logs since startup for this node?


was (Author: sebastian.este...@datastax.com):
[~aivanov93] can you post your logs since startup?

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by:

[jira] [Created] (CASSANDRA-8609) Remove depency of hadoop to internals (Cell/CellName)

Sylvain Lebresne created CASSANDRA-8609:
---

 Summary: Remove depency of hadoop to internals (Cell/CellName)
 Key: CASSANDRA-8609
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8609
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
 Fix For: 3.0


For some reason most of the Hadoop code (ColumnFamilyRecordReader, CqlStorage, 
...) uses the {{Cell}} and {{CellName}} classes. That dependency is entirely 
artificial: all this code is really client code that communicate with Cassandra 
over thrift/native protocol and there is thus no reason for it to use internal 
classes. And in fact, thoses classes are used in a very crude way, as a 
{{PairByteBuffer, ByteBuffer}} really.

But this dependency is really painful when we make changes to the internals. 
Further, every time we do so, I believe we break some of those the APIs due to 
the change. This has been painful for CASSANDRA-5417 and this is now painful 
for CASSANDRA-8099. But while I somewhat hack over it in CASSANDRA-5417, this 
was a mistake and we should have removed the depency back then. So let do that 
now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8610) Allow IF EXISTS for UPDATE statements

2015-01-13 Thread DOAN DuyHai (JIRA)

DOAN DuyHai created CASSANDRA-8610:
--

 Summary: Allow IF EXISTS for UPDATE statements
 Key: CASSANDRA-8610
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8610
 Project: Cassandra
  Issue Type: Improvement
  Components: API
 Environment: Cassandra 2.1.2
Reporter: DOAN DuyHai
Priority: Minor


While creating a hands-on exercice for Cassandra, I was facing a quite annoying 
limitation. 

 Let's take this table:

{code:sql}
CREATE TABLE killrchat.chat_rooms( 
room_name text,
creation_date timestamp,
banner text,
creator text,
participants settext,
PRIMARY KEY(room_name));
{code}

Upon a new participant joining the room, to be concurrency-proof (avoiding 
mutating the participants set if the room is deleted concurrently), I would 
like to issue this query:

{code:sql}

 UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE room_name 
= 'games' IF EXISTS;
{code}

 Unfortunately the clause IF EXISTS is not allowed for UPDATE statements. 
Similarly I tried

{code:sql}

 UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE room_name 
= 'games' IF room_name='games';
{code}

 It doesn't work either, it is not allowed to use one column of the primary key 
as condition column for LWT (why ? mystery).

 So far, the only work-around I found is:

{code:sql}

 UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE room_name 
= 'games' IF name='games';
{code}

 I added an extra column called *name* which is just the duplicate of the 
partition key *room_name*. It does work but is not very elegant.

 I believe there are legit use cases for UPDATE ... IF EXISTS;





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8099) Refactor and modernize the storage engine


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-8099:

Attachment: 8099-nit

A small thing I noticed, that this nicely helps tidy up something that's bugged 
me before and can (and should) be cleaned up with this IMO: all of the 
sub-comparators in ClusteringComparator can be removed, and the class itself 
can simply implement ComparatorClusterable, since all of the things that can 
be compared now implement Clusterable (perhaps one sub-comparator could be used 
if directly comparing ClusteringPrefix is very common in high-traffic code 
paths, but it's probably not necessary). 

.rowComparator can be removed already and .atomComparator can be removed by 
only making MergeIterator accept ? super In instead of In. 

clusterableComparator would then be subsumed by the enclosing class; everywhere 
it's referred to you can simply delete .clusteringComparator and it will work.

I'm sure it's out of date now, but I've posted the diff anyway. The summary is:
{noformat}
 src/java/org/apache/cassandra/db/ClusteringComparator.java| 33 
++---
 src/java/org/apache/cassandra/db/atoms/AtomIterators.java |  2 +-
 src/java/org/apache/cassandra/db/filters/NamesPartitionFilter.java|  2 +-
 src/java/org/apache/cassandra/db/partitions/AtomicBTreePartition.java |  4 ++--
 src/java/org/apache/cassandra/utils/MergeIterator.java| 10 
+-
 5 files changed, 15 insertions(+), 36 deletions(-)
{noformat}

 Refactor and modernize the storage engine
 -

 Key: CASSANDRA-8099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8099
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 3.0

 Attachments: 8099-nit


 The current storage engine (which for this ticket I'll loosely define as the 
 code implementing the read/write path) is suffering from old age. One of the 
 main problem is that the only structure it deals with is the cell, which 
 completely ignores the more high level CQL structure that groups cell into 
 (CQL) rows.
 This leads to many inefficiencies, like the fact that during a reads we have 
 to group cells multiple times (to count on replica, then to count on the 
 coordinator, then to produce the CQL resultset) because we forget about the 
 grouping right away each time (so lots of useless cell names comparisons in 
 particular). But outside inefficiencies, having to manually recreate the CQL 
 structure every time we need it for something is hindering new features and 
 makes the code more complex that it should be.
 Said storage engine also has tons of technical debt. To pick an example, the 
 fact that during range queries we update {{SliceQueryFilter.count}} is pretty 
 hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has 
 to go into to simply remove the last query result.
 So I want to bite the bullet and modernize this storage engine. I propose to 
 do 2 main things:
 # Make the storage engine more aware of the CQL structure. In practice, 
 instead of having partitions be a simple iterable map of cells, it should be 
 an iterable list of row (each being itself composed of per-column cells, 
 though obviously not exactly the same kind of cell we have today).
 # Make the engine more iterative. What I mean here is that in the read path, 
 we end up reading all cells in memory (we put them in a ColumnFamily object), 
 but there is really no reason to. If instead we were working with iterators 
 all the way through, we could get to a point where we're basically 
 transferring data from disk to the network, and we should be able to reduce 
 GC substantially.
 Please note that such refactor should provide some performance improvements 
 right off the bat but it's not it's primary goal either. It's primary goal is 
 to simplify the storage engine and adds abstraction that are better suited to 
 further optimizations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8611) give streaming_socket_timeout_in_ms a non-zero default

2015-01-13 Thread Jeremy Hanna (JIRA)

Jeremy Hanna created CASSANDRA-8611:
---

 Summary: give streaming_socket_timeout_in_ms a non-zero default
 Key: CASSANDRA-8611
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8611
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jeremy Hanna


Sometimes as mentioned in CASSANDRA-8472 streams will hang.  We have 
streaming_socket_timeout_in_ms which can retry after a timeout.  It would be 
good to make a default non-zero value.  We don't want to paper over problems, 
but streams sometimes hang and you don't want long running streaming operations 
to just fail - as in repairs or bootstraps.

streaming_socket_timeout_in_ms should be based on the tcp idle timeout so it 
shouldn't be a problem to set it to on the order of minutes.  Also the socket 
should only be open during the actual streaming and not during operations such 
as merkle tree generation.  We can set it to a conservative value and people 
can set it more aggressively as needed.  Disabling as a default, in my opinion, 
is too conservative.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Fix loading set types in pig with the 2.1 client driver.

Fix loading set types in pig with the 2.1 client driver.

Patch by Artem Aliev, reviewed by brandonwilliams for CASSANDRA-8577


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1cb426b9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1cb426b9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1cb426b9

Branch: refs/heads/trunk
Commit: 1cb426b9831b42b5f368eac51a6e3bebdb1bd62a
Parents: 0757dc7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 10:23:28 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 10:23:28 2015 -0600

--
 .../apache/cassandra/hadoop/cql3/CqlConfigHelper.java   | 11 +++
 .../apache/cassandra/hadoop/cql3/CqlRecordReader.java   | 12 
 .../cassandra/hadoop/pig/AbstractCassandraStorage.java  |  5 -
 .../apache/cassandra/hadoop/pig/CqlNativeStorage.java   |  4 
 4 files changed, 31 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1cb426b9/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
index 2be811f..7d65663 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
@@ -81,6 +81,8 @@ public class CqlConfigHelper
 private static final String INPUT_NATIVE_SSL_KEY_STORE_PASSWARD = 
cassandra.input.native.ssl.key.store.password;
 private static final String INPUT_NATIVE_SSL_CIPHER_SUITES = 
cassandra.input.native.ssl.cipher.suites;
 
+private static final String INPUT_NATIVE_PROTOCOL_VERSION = 
cassandra.input.native.protocol.version;
+
 private static final String OUTPUT_CQL = cassandra.output.cql;
 
 /**
@@ -279,6 +281,10 @@ public class CqlConfigHelper
 return conf.get(OUTPUT_CQL);
 }
 
+private static OptionalInteger getProtocolVersion(Configuration conf) {
+return getIntSetting(INPUT_NATIVE_PROTOCOL_VERSION, conf);
+}
+
 public static Cluster getInputCluster(String host, Configuration conf)
 {
 // this method has been left for backward compatibility
@@ -290,6 +296,7 @@ public class CqlConfigHelper
 int port = getInputNativePort(conf);
 OptionalAuthProvider authProvider = getAuthProvider(conf);
 OptionalSSLOptions sslOptions = getSSLOptions(conf);
+OptionalInteger protocolVersion = getProtocolVersion(conf);
 LoadBalancingPolicy loadBalancingPolicy = 
getReadLoadBalancingPolicy(conf, hosts);
 SocketOptions socketOptions = getReadSocketOptions(conf);
 QueryOptions queryOptions = getReadQueryOptions(conf);
@@ -305,6 +312,9 @@ public class CqlConfigHelper
 if (sslOptions.isPresent())
 builder.withSSL(sslOptions.get());
 
+if (protocolVersion.isPresent()) {
+builder.withProtocolVersion(protocolVersion.get());
+}
 builder.withLoadBalancingPolicy(loadBalancingPolicy)
.withSocketOptions(socketOptions)
.withQueryOptions(queryOptions)
@@ -313,6 +323,7 @@ public class CqlConfigHelper
 return builder.build();
 }
 
+
 public static void setInputCoreConnections(Configuration conf, String 
connections)
 {
 conf.set(INPUT_NATIVE_CORE_CONNECTIONS_PER_HOST, connections);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1cb426b9/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
index 9c1118b..6a1f5bf 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
@@ -89,6 +89,7 @@ public class CqlRecordReader extends RecordReaderLong, Row
 
 // partition keys -- key aliases
 private LinkedHashMapString, Boolean partitionBoundColumns = 
Maps.newLinkedHashMap();
+protected int nativeProtocolVersion = 1;
 
 public CqlRecordReader()
 {
@@ -129,6 +130,9 @@ public class CqlRecordReader extends RecordReaderLong, Row
 if (session == null)
   throw new RuntimeException(Can't create connection session);
 
+//get negotiated serialization protocol
+nativeProtocolVersion = 
cluster.getConfiguration().getProtocolOptions().getProtocolVersion();
+
 // If the user provides a CQL query then we will use it without 
validation
 // otherwise we will fall back

[1/3] cassandra git commit: Fix loading set types in pig with the 2.1 client driver.

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 0757dc72f - 1cb426b98
  refs/heads/trunk f1475244f - 42e483a4e


Fix loading set types in pig with the 2.1 client driver.

Patch by Artem Aliev, reviewed by brandonwilliams for CASSANDRA-8577


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1cb426b9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1cb426b9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1cb426b9

Branch: refs/heads/cassandra-2.1
Commit: 1cb426b9831b42b5f368eac51a6e3bebdb1bd62a
Parents: 0757dc7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 10:23:28 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 10:23:28 2015 -0600

--
 .../apache/cassandra/hadoop/cql3/CqlConfigHelper.java   | 11 +++
 .../apache/cassandra/hadoop/cql3/CqlRecordReader.java   | 12 
 .../cassandra/hadoop/pig/AbstractCassandraStorage.java  |  5 -
 .../apache/cassandra/hadoop/pig/CqlNativeStorage.java   |  4 
 4 files changed, 31 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1cb426b9/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
index 2be811f..7d65663 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
@@ -81,6 +81,8 @@ public class CqlConfigHelper
 private static final String INPUT_NATIVE_SSL_KEY_STORE_PASSWARD = 
cassandra.input.native.ssl.key.store.password;
 private static final String INPUT_NATIVE_SSL_CIPHER_SUITES = 
cassandra.input.native.ssl.cipher.suites;
 
+private static final String INPUT_NATIVE_PROTOCOL_VERSION = 
cassandra.input.native.protocol.version;
+
 private static final String OUTPUT_CQL = cassandra.output.cql;
 
 /**
@@ -279,6 +281,10 @@ public class CqlConfigHelper
 return conf.get(OUTPUT_CQL);
 }
 
+private static OptionalInteger getProtocolVersion(Configuration conf) {
+return getIntSetting(INPUT_NATIVE_PROTOCOL_VERSION, conf);
+}
+
 public static Cluster getInputCluster(String host, Configuration conf)
 {
 // this method has been left for backward compatibility
@@ -290,6 +296,7 @@ public class CqlConfigHelper
 int port = getInputNativePort(conf);
 OptionalAuthProvider authProvider = getAuthProvider(conf);
 OptionalSSLOptions sslOptions = getSSLOptions(conf);
+OptionalInteger protocolVersion = getProtocolVersion(conf);
 LoadBalancingPolicy loadBalancingPolicy = 
getReadLoadBalancingPolicy(conf, hosts);
 SocketOptions socketOptions = getReadSocketOptions(conf);
 QueryOptions queryOptions = getReadQueryOptions(conf);
@@ -305,6 +312,9 @@ public class CqlConfigHelper
 if (sslOptions.isPresent())
 builder.withSSL(sslOptions.get());
 
+if (protocolVersion.isPresent()) {
+builder.withProtocolVersion(protocolVersion.get());
+}
 builder.withLoadBalancingPolicy(loadBalancingPolicy)
.withSocketOptions(socketOptions)
.withQueryOptions(queryOptions)
@@ -313,6 +323,7 @@ public class CqlConfigHelper
 return builder.build();
 }
 
+
 public static void setInputCoreConnections(Configuration conf, String 
connections)
 {
 conf.set(INPUT_NATIVE_CORE_CONNECTIONS_PER_HOST, connections);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1cb426b9/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
index 9c1118b..6a1f5bf 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
@@ -89,6 +89,7 @@ public class CqlRecordReader extends RecordReaderLong, Row
 
 // partition keys -- key aliases
 private LinkedHashMapString, Boolean partitionBoundColumns = 
Maps.newLinkedHashMap();
+protected int nativeProtocolVersion = 1;
 
 public CqlRecordReader()
 {
@@ -129,6 +130,9 @@ public class CqlRecordReader extends RecordReaderLong, Row
 if (session == null)
   throw new RuntimeException(Can't create connection session);
 
+//get negotiated serialization protocol
+nativeProtocolVersion =

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/42e483a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/42e483a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/42e483a4

Branch: refs/heads/trunk
Commit: 42e483a4e1dc69e886e4c0d1117d7081abaa3768
Parents: f147524 1cb426b
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 13 10:27:18 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 13 10:27:18 2015 -0600

--
 .../apache/cassandra/hadoop/cql3/CqlConfigHelper.java   | 11 +++
 .../apache/cassandra/hadoop/cql3/CqlRecordReader.java   | 12 
 .../cassandra/hadoop/pig/AbstractCassandraStorage.java  |  5 -
 .../apache/cassandra/hadoop/pig/CqlNativeStorage.java   |  4 
 4 files changed, 31 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/42e483a4/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/42e483a4/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/42e483a4/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java
--

[jira] [Commented] (CASSANDRA-7679) Batch DDL

2015-01-13 Thread Jorge Bay (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275488#comment-14275488
 ] 

Jorge Bay commented on CASSANDRA-7679:
--

Sorry guys, dismiss my comment (insert a shame meme here): The wait for the 
schema agreement after a schema change response was never implemented in the C# 
driver and I though it was the common behaviour across all drivers.

 Batch DDL
 -

 Key: CASSANDRA-7679
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7679
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp

 Just an idea: To improve speed of DDL in clusters with lots of 
 Keyspaces/Tables/Columns it might help to collect a bunch of schema changes 
 and propagate them as a single bunch of changes.
 Such a DDL batch would
 # execute DDLs locally and collect all mutations
 # broadcast all mutations at once
 # schema agreement
 # return listSchemaChange via native protocol to the client
 So {{DefsTables.mergeSchemaInternal}} (which seems to be the expensive part) 
 would only execute once per DDL batch on each node.
 DDL batches would not be atomic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8577) Values of set types not loading correctly into Pig


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-8577.
-
   Resolution: Fixed
 Reviewer: Brandon Williams
Reproduced In: 2.1.2, 2.1.1  (was: 2.1.1, 2.1.2)
 Assignee: Artem Aliev  (was: Brandon Williams)

Alright, committed Artem's patch.  Thanks!

 Values of set types not loading correctly into Pig
 --

 Key: CASSANDRA-8577
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8577
 Project: Cassandra
  Issue Type: Bug
Reporter: Oksana Danylyshyn
Assignee: Artem Aliev
 Fix For: 2.1.3

 Attachments: cassandra-2.1-8577.txt


 Values of set types are not loading correctly from Cassandra (cql3 table, 
 Native protocol v3) into Pig using CqlNativeStorage. 
 When using Cassandra version 2.1.0 only empty values are loaded, and for 
 newer versions (2.1.1 and 2.1.2) the following error is received: 
 org.apache.cassandra.serializers.MarshalException: Unexpected extraneous 
 bytes after set value
 at 
 org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
 Steps to reproduce:
 {code}cqlsh:socialdata CREATE TABLE test (
  key varchar PRIMARY KEY,
  tags setvarchar
);
 cqlsh:socialdata insert into test (key, tags) values ('key', {'Running', 
 'onestep4red', 'running'});
 cqlsh:socialdata select * from test;
  key | tags
 -+---
  key | {'Running', 'onestep4red', 'running'}
 (1 rows){code}
 With version 2.1.0:
 {code}grunt data = load 'cql://socialdata/test' using 
 org.apache.cassandra.hadoop.pig.CqlNativeStorage();
 grunt dump data;
 (key,()){code}
 With version 2.1.2:
 {code}grunt data = load 'cql://socialdata/test' using 
 org.apache.cassandra.hadoop.pig.CqlNativeStorage();
 grunt dump data;
 org.apache.cassandra.serializers.MarshalException: Unexpected extraneous 
 bytes after set value
   at 
 org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
   at 
 org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:27)
   at 
 org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.cassandraToObj(AbstractCassandraStorage.java:796)
   at 
 org.apache.cassandra.hadoop.pig.CqlStorage.cqlColumnToObj(CqlStorage.java:195)
   at 
 org.apache.cassandra.hadoop.pig.CqlNativeStorage.getNext(CqlNativeStorage.java:106)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
   at 
 org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
   at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
   at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
   at 
 org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212){code}
 Expected result:
 {code}(key,(Running,onestep4red,running)){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8612) Read metrics should be updated on all types of reads

Chris Lohfink created CASSANDRA-8612:


 Summary: Read metrics should be updated on all types of reads
 Key: CASSANDRA-8612
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8612
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Chris Lohfink
Priority: Minor


Metrics like sstables per read are not updated on a range slice.  Although 
separating things out for each type of read could make sense like we do for 
latencies, only exposing the metrics for one type can be a little confusing 
when people do a query and see nothing increases.  I think its sufficient to 
use the same metrics for all reads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8099) Refactor and modernize the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275491#comment-14275491
]

Benedict commented on CASSANDRA-8099:
-

Also, I'm sure you're on top of this already, but this could do with a lot more
commenting. Little things like what is meant by a complex field - with a bit
of digging you can determine this is related to collections, but you have to
jump to declaration quite a few times before establishing this. Simple
descriptions of classes like ComplexCellBlock and what a class is used for,
like in CellWriter.

It would really help to reduce the mystery that has surrounded some of the
internal functionality. Right now I don't think anybody is fully conversant
with even the current (post CASSANDRA-5417) functionality besides yourself. I
know this rewrite will help a lot, but I doubt it will fix the problem
entirely. There's a lot of legacy cruft to deal with still, that makes things
complicated, and this rewrite will put almost everyone back to square one.

Refactor and modernize the storage engine
-

Key: CASSANDRA-8099
URL: https://issues.apache.org/jira/browse/CASSANDRA-8099
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 3.0

Attachments: 8099-nit

The current storage engine (which for this ticket I'll loosely define as the
code implementing the read/write path) is suffering from old age. One of the
main problem is that the only structure it deals with is the cell, which
completely ignores the more high level CQL structure that groups cell into
(CQL) rows.
This leads to many inefficiencies, like the fact that during a reads we have
to group cells multiple times (to count on replica, then to count on the
coordinator, then to produce the CQL resultset) because we forget about the
grouping right away each time (so lots of useless cell names comparisons in
particular). But outside inefficiencies, having to manually recreate the CQL
structure every time we need it for something is hindering new features and
makes the code more complex that it should be.
Said storage engine also has tons of technical debt. To pick an example, the
fact that during range queries we update {{SliceQueryFilter.count}} is pretty
hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has
to go into to simply remove the last query result.
So I want to bite the bullet and modernize this storage engine. I propose to
do 2 main things:
# Make the storage engine more aware of the CQL structure. In practice,
instead of having partitions be a simple iterable map of cells, it should be
an iterable list of row (each being itself composed of per-column cells,
though obviously not exactly the same kind of cell we have today).
# Make the engine more iterative. What I mean here is that in the read path,
we end up reading all cells in memory (we put them in a ColumnFamily object),
but there is really no reason to. If instead we were working with iterators
all the way through, we could get to a point where we're basically
transferring data from disk to the network, and we should be able to reduce
GC substantially.
Please note that such refactor should provide some performance improvements
right off the bat but it's not it's primary goal either. It's primary goal is
to simplify the storage engine and adds abstraction that are better suited to
further optimizations.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7338) CFS.getRangeSlice should update latency metrics


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275507#comment-14275507
 ] 

Chris Lohfink commented on CASSANDRA-7338:
--

Could we actually include the EstimatedHistogram version that was cut out in 
v3?  The Metrics histogram is pretty bad at non-normal data like latencies and 
(empirically tested and theoretically) is untrustworthy at 99th percentile.  
For applications that care about the percentiles having the more statistically 
accurate version is beneficial.

 CFS.getRangeSlice should update latency metrics
 ---

 Key: CASSANDRA-7338
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7338
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Priority: Trivial
 Fix For: 2.1 rc2

 Attachments: 7338-3.txt, CASSANDRA-7338-v2.txt, CASSANDRA-7338.txt


 CFS.getRangeSlice doesn't update the CF readLatency metric in the same way as 
 CFS.getColumnFamily does. 
 I may be missing something, but I couldn't see a good reason why this wasn't 
 already the case as without it, SELECT * FROM t WHERE x=y results in the 
 read metrics being incremented, but SELECT * FROM t doesn't.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8067) NullPointerException in KeyCacheSerializer


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275514#comment-14275514
 ] 

Benedict commented on CASSANDRA-8067:
-

This looks like an issue with serializing data for a dropped table. We should 
also be looking up the cfMetadata with our UUID, else we can have some 
weirdness with a serialization crossing a drop/recreate boundary, which would 
be especially problematic with different schema.

 NullPointerException in KeyCacheSerializer
 --

 Key: CASSANDRA-8067
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8067
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Eric Leleu
 Fix For: 2.1.1


 Hi,
 I have this stack trace in the logs of Cassandra server (v2.1)
 {code}
 ERROR [CompactionExecutor:14] 2014-10-06 23:32:02,098 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[CompactionExecutor:14,1,main]
 java.lang.NullPointerException: null
 at 
 org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:475)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:463)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:225)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1061)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown 
 Source) ~[na:1.7.0]
 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) 
 ~[na:1.7.0]
 at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
 [na:1.7.0]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
 [na:1.7.0]
 at java.lang.Thread.run(Unknown Source) [na:1.7.0]
 {code}
 It may not be critical because this error occured in the AutoSavingCache. 
 However the line 475 is about the CFMetaData so it may hide bigger issue...
 {code}
  474 CFMetaData cfm = 
 Schema.instance.getCFMetaData(key.desc.ksname, key.desc.cfname);
  475 cfm.comparator.rowIndexEntrySerializer().serialize(entry, 
 out);
 {code}
 Regards,
 Eric



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8613) Regression in mixed single and multi-column relation support

Tyler Hobbs created CASSANDRA-8613:
--

 Summary: Regression in mixed single and multi-column relation 
support
 Key: CASSANDRA-8613
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8613
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Benjamin Lerer
 Fix For: 2.1.3, 2.0.13


In 2.0.6 through 2.0.8, a query like the following was supported:

{noformat}
SELECT * FROM mytable WHERE clustering_0 = ? AND (clustering_1, clustering_2)  
(?, ?)
{noformat}

However, after CASSANDRA-6875, you'll get the following error:
{noformat}
Clustering columns may not be skipped in multi-column relations. They should 
appear in the PRIMARY KEY order. Got (c, d)  (0, 0)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8613) Regression in mixed single and multi-column relation support


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8613:
---
Since Version: 2.0.9

 Regression in mixed single and multi-column relation support
 

 Key: CASSANDRA-8613
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8613
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Benjamin Lerer
 Fix For: 2.1.3, 2.0.13


 In 2.0.6 through 2.0.8, a query like the following was supported:
 {noformat}
 SELECT * FROM mytable WHERE clustering_0 = ? AND (clustering_1, clustering_2) 
  (?, ?)
 {noformat}
 However, after CASSANDRA-6875, you'll get the following error:
 {noformat}
 Clustering columns may not be skipped in multi-column relations. They should 
 appear in the PRIMARY KEY order. Got (c, d)  (0, 0)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8611) give streaming_socket_timeout_in_ms a non-zero default

2015-01-13 Thread Jeremy Hanna (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275528#comment-14275528
]

Jeremy Hanna commented on CASSANDRA-8611:
-

[~jblangs...@datastax.com] mentioned that it would be good to make sure that it
does time out and that it gives an understandable message in the logs when it
does time out. That way, it can be tracked in case of troubleshooting socket
timeouts/resets at the router level, for example.

give streaming_socket_timeout_in_ms a non-zero default
--

Key: CASSANDRA-8611
URL: https://issues.apache.org/jira/browse/CASSANDRA-8611
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jeremy Hanna

Sometimes as mentioned in CASSANDRA-8472 streams will hang. We have
streaming_socket_timeout_in_ms which can retry after a timeout. It would be
good to make a default non-zero value. We don't want to paper over problems,
but streams sometimes hang and you don't want long running streaming
operations to just fail - as in repairs or bootstraps.
streaming_socket_timeout_in_ms should be based on the tcp idle timeout so it
shouldn't be a problem to set it to on the order of minutes. Also the socket
should only be open during the actual streaming and not during operations
such as merkle tree generation. We can set it to a conservative value and
people can set it more aggressively as needed. Disabling as a default, in my
opinion, is too conservative.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError

2015-01-13 Thread Sebastian Estevez (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276084#comment-14276084
 ] 

Sebastian Estevez commented on CASSANDRA-8548:
--

Does the scrub error only appear in stdout? I don't see it in your log.

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276088#comment-14276088
 ] 

Andrei Ivanov commented on CASSANDRA-8548:
--

Yes

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at 
 org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:108)

[jira] [Issue Comment Deleted] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrei Ivanov updated CASSANDRA-8548:
-
Comment: was deleted

(was: Yes)

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at 
 org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:108)
 at

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276091#comment-14276091
 ] 

Andrei Ivanov commented on CASSANDRA-8548:
--

Yes

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at 
 org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:108)

[jira] [Issue Comment Deleted] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrei Ivanov updated CASSANDRA-8548:
-
Comment: was deleted

(was: Yes)

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at 
 org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:108)
 at

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276089#comment-14276089
 ] 

Andrei Ivanov commented on CASSANDRA-8548:
--

Yes

 Nodetool Cleanup - java.lang.AssertionError
 ---

 Key: CASSANDRA-8548
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
 Project: Cassandra
  Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
 Fix For: 2.0.12

 Attachments: 0001-make-sure-we-unmark-compacting.patch


 Needed to free up some space on a node but getting the dump below when 
 running nodetool cleanup.
 Tried turning on debug to try to obtain additional details in the logs but 
 nothing gets added to the logs when running cleanup. Added: 
 log4j.logger.org.apache.cassandra.db=DEBUG 
 in log4j-server.properties
 See the stack trace below:
 root@cassandra-019:~# nodetool cleanup
 {code}Error occurred during cleanup
 java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
 at java.util.concurrent.FutureTask.report(FutureTask.java:122)
 at java.util.concurrent.FutureTask.get(FutureTask.java:188)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
 at 
 org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
 at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
 at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
 at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
 at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
 at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
 at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
 at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
 at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
 at sun.rmi.transport.Transport$1.run(Transport.java:177)
 at sun.rmi.transport.Transport$1.run(Transport.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
 at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
 at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.IllegalArgumentException
 at java.nio.Buffer.limit(Buffer.java:267)
 at 
 org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:108)

[jira] [Commented] (CASSANDRA-8615) Create -D flag to disable speculative retry by default


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276123#comment-14276123
 ] 

Brandon Williams commented on CASSANDRA-8615:
-

bq. Should investigate and fix the cause instead.

+1, 2.0 has been out quite a while now and I've never heard of anyone being 
against SR - just the opposite in fact.

 Create -D flag to disable speculative retry by default
 --

 Key: CASSANDRA-8615
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8615
 Project: Cassandra
  Issue Type: Improvement
Reporter: J.B. Langston

 Some clusters have shown increased latency with speculative retry enabled.  
 Speculative retry is enabled by default when upgrading from 1.2 to 2.0, and 
 for large clusters it can take a long time to complete a rolling upgrade, 
 during which time speculative retry will be enabled. Therefore it would be 
 helpful to have a -D flag that will disable it by default during an upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8576) Primary Key Pushdown For Hadoop

2015-01-13 Thread Alex Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276140#comment-14276140
 ] 

Alex Liu commented on CASSANDRA-8576:
-

Should it work only for EQ predicates? Should it also include IN predicates?

 Primary Key Pushdown For Hadoop
 ---

 Key: CASSANDRA-8576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8576
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Russell Alexander Spitzer

 I've heard reports from several users that they would like to have predicate 
 pushdown functionality for hadoop (Hive in particular) based services. 
 Example usecase
 Table with wide partitions, one per customer
 Application team has HQL they would like to run on a single customer
 Currently time to complete scales with number of customers since Input Format 
 can't pushdown primary key predicate
 Current implementation requires a full table scan (since it can't recognize 
 that a single partition was specified)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-8614:
--
Attachment: CRC32FactoryTest.java
8614.patch

Compiles on Java 7 and when run on Java 8 you get the intrinsic. There is a 
test case to validate that the two checksums implementations behave the same.



 Select optimal CRC32 implementation at runtime
 --

 Key: CASSANDRA-8614
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8614
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Ariel Weisberg
  Labels: performance
 Attachments: 8614.patch, CRC32FactoryTest.java


 JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec 
 per core in my quick and dirty test. PureJavaCRC32 is  800 megabytes/sec if 
 I recall and it has a lookup table that evicts random cache lines every time 
 it runs.
 In order to capture the benefit of that when it is available we can select a 
 CRC32 implementation at startup in a static block.
 If JDK 8 is not what is running we can fall back to the existing 
 PureJavaCRC32 implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

2015-01-13 Thread Jonathan Ellis (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275862#comment-14275862
 ] 

Jonathan Ellis commented on CASSANDRA-8523:
---

Marking related to CASSANDRA-8494, but may want to resolve as duplicate.

 Writes should be sent to a replacement node while it is streaming in data
 -

 Key: CASSANDRA-8523
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Richard Wagner
Assignee: Brandon Williams
 Fix For: 2.0.12, 2.1.3


 In our operations, we make heavy use of replace_address (or 
 replace_address_first_boot) in order to replace broken nodes. We now realize 
 that writes are not sent to the replacement nodes while they are in hibernate 
 state and streaming in data. This runs counter to what our expectations were, 
 especially since we know that writes ARE sent to nodes when they are 
 bootstrapped into the ring.
 It seems like cassandra should arrange to send writes to a node that is in 
 the process of replacing another node, just like it does for a nodes that are 
 bootstraping. I hesitate to phrase this as we should send writes to a node 
 in hibernate because the concept of hibernate may be useful in other 
 contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
 Among other things, the fact that we don't get writes during this period 
 makes subsequent repairs more expensive, proportional to the number of writes 
 that we miss (and depending on the amount of data that needs to be streamed 
 during replacement and the time it may take to rebuild secondary indexes, we 
 could miss many many hours worth of writes). It also leaves us more exposed 
 to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log

[
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275865#comment-14275865
]

Ariel Weisberg commented on CASSANDRA-6809:
---

Moving conversation to JIRA

Ariel
{noformat}
What is the test coverage, unit or otherwise for compressed/uncompressed
playback? It would be good to see the new code produce the mutations out of
both compressed and uncompressed segments.

I think that this kind of performance work isn't done without a micro benchmark
so we know where we stand and so the next person to work on the CL can track
the impact of their changes. I would really like to see a JMH test added that
evaluates the compressed and uncompressed versions for throughput for tiny
mutations as well as normal (say 1k).

When testing compression I have found that generating compressible data doesn't
seem to work well. I never got the advertised compression rates out of Snappy.
You might need to grab a real corpus like some HTML or the text of alice and
wonderland to give the compressor something reasonable to chew on.

I have some concern about waiting to do compression until the end. It's fine in
the common case and configuration, but there are situations where you want to
start the flush immediately and it would be nice if it compressed and wrote a
few hundred kilobytes at a time. You haven't taken away the mapped option so
latency sensitive users can opt for that and it can be enhanced later if there
is demand.

Compression is single threaded and any time spent in force() can't also be used
to do compression. This can really cut into compression throughput and it is a
function of the latency of force() which varies. With synchronous commits it's
can have an impact.

The ICompressor interface and some of the implementations are also a little
limiting. You should be able to compress the data directly into the memory
mapped file. Snappy java supports that although LZ4 doesn't appear to. It also
makes more sense to compress and then checksum. There is less data to checksum
after compression, and another advantage of getting the order right is that you
can use a JNI call to a native checksum implementation which will be much
faster. You can use a larger checksum like murmur3 over each compressed block.
The checksum will be better then a 4-byte checksum, but use less space overall.

Yet another advantage is that Murmur3 doesn't have a lookup table evicting
random cache lines (on every core) as part of every single mutation that flows
through C*.

I only bring this stuff up because this is a format change and getting it all
done at once might make the amount of multi-version support code that has to be
written smaller as opposed to making these changes incrementally across
multiple versions.
{noformat}
Benedict
{noformat}
I'm not really sure JMH is the right tool for disk bound workloads?

To add to Ariel's excellent points, another thing to consider is recycling the
ByteBuffer we use, and potentially also use a DirectByteBuffer. As it stands
this will create a non-trivial amount of garbage that may live long enough to
be promoted (what with being disk bound).
{noformat}

Ariel
{noformat}
JMH doesn't solve the hard problems of for disk bound workloads certainly. It
doesn't hurt either and it has some support for parameterizing and reporting. I
didn't know about CommitLogStress when I asked about JMH. We don't need to
build something new if something is already in place.

I thought about the ByteBuffer pooling and lack of DBB. ICompressor doesn't
accept DBB for input or output. The allocation is large so it will go straight
into the old generation. C* runs CMS pretty regularly, but maybe not that
regularly. It might make sense to pool two of them and if anymore are needed
beyond that to allocate and discard. Since the allocation is CL segment size
it's feels like a lot of memory to pin for one purpose.
{noformat}

Benedict
{noformat}
CommitLogStress is not very good, but JMH is architected around microbenchmarks
or very small operations, which this doesn't really fit.

We now support DBB in LZ4, so there's no reason to not roll it out here.
Pooling as many as we need makes sense to me, no reason to constrain ourselves
to just 2? Perhaps periodically prune if instances aren't being used. We're
currently pinning as much memory as we're ahead of disk by, although admittedly
here we only need it so long as we're writing to the page cache. Which is
perhaps another optimisation - the buffer should be recycled/discarded as soon
as we've compressed its contents.
{noformat}

Compressed Commit Log
-

Key: CASSANDRA-6809
URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
Project: Cassandra
Issue Type: Improvement
Reporter: Benedict
Assignee:

[jira] [Reopened] (CASSANDRA-8479) Timeout Exception on Node Failure in Remote Data Center

2015-01-13 Thread Jonathan Ellis (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reopened CASSANDRA-8479:
---
  Assignee: Sam Tunnicliffe  (was: Ryan McGuire)

Sam, could this be related to the retry fixes you did recently?

Brandon also mentioned CASSANDRA-7947.

 Timeout Exception on Node Failure in Remote Data Center
 ---

 Key: CASSANDRA-8479
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8479
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core, Tools
 Environment: Unix, Cassandra 2.0.11
Reporter: Amit Singh Chowdhery
Assignee: Sam Tunnicliffe
Priority: Minor
 Attachments: TRACE_LOGS.zip


 Issue Faced :
 We have a Geo-red setup with 2 Data centers having 3 nodes each. When we 
 bring down a single Cassandra node down in DC2 by kill -9 Cassandra-pid, 
 reads fail on DC1 with TimedOutException for a brief amount of time (15-20 
 sec~).
 Reference :
 Already a ticket has been opened/resolved and link is provided below :
 https://issues.apache.org/jira/browse/CASSANDRA-8352
 Activity Done as per Resolution Provided :
 Upgraded to Cassandra 2.0.11 .
 We have two 3 node clusters in two different DCs and if one or more of the 
 nodes go down in one Data Center , ~5-10% traffic failure is observed on the 
 other.
 CL: LOCAL_QUORUM
 RF=3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8558) deleted row still can be selected out


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275869#comment-14275869
 ] 

Tyler Hobbs commented on CASSANDRA-8558:


+1 with a few nits:

* In {{IndexSliceReader.fetchMoreData()}}, it seems like {{gotSome}} should 
only be set to true if we read a non-RangeTombstone.  However, that's a minor 
optimization (if I'm correct) and shouldn't affect correctness, so I won't 
block the patch for that.
* In the 2.1 patch, {{nextKind}} is a bit of a strange name.  I suggest 
{{nextFlags}} instead.
* {{CellNameType.readNextFully()}} is also a confusing name.  It seems like 
{{nextNameIsNotEmpty()}} would be more accurate?
* In the 2.0 patch, indentation is off for the comments in the last two hunks
* A couple of typos: maskes in AtomDeserializer, prefecthed in 
IndexSliceReader (2.1 patch), 

 deleted row still can be selected out
 -

 Key: CASSANDRA-8558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2 
 java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
 Fix For: 2.0.12, 2.1.3

 Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt


 first
 {code}CREATE  KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};
 CREATE  TABLE space1.table3(a int, b int, c text,primary key(a,b));
 CREATE  KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy', 
 'replication_factor': 3};{code}
 second
 {code}CREATE  TABLE space2.table1(a int, b int, c int, primary key(a,b));
 CREATE  TABLE space2.table2(a int, b int, c int, primary key(a,b));
 INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
 drop table space2.table1;
 DELETE FROM space1.table3 where a=1 and b=1;
 drop table space2.table2;
 select * from space1.table3 where a=1 and b=1;{code}
 you will find that the row (a=1 and b=1)  in space1.table3 is not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime

Ariel Weisberg created CASSANDRA-8614:
-

 Summary: Select optimal CRC32 implementation at runtime
 Key: CASSANDRA-8614
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8614
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Ariel Weisberg


JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec 
per core in my quick and dirty test. PureJavaCRC32 is  800 megabytes/sec if I 
recall and it has a lookup table that evicts random cache lines every time it 
runs.

In order to capture the benefit of that when it is available we can select a 
CRC32 implementation at startup in a static block.

If JDK 8 is not what is running we can fall back to the existing PureJavaCRC32 
implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-6809) Compressed Commit Log

[
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275865#comment-14275865
]

Ariel Weisberg edited comment on CASSANDRA-6809 at 1/13/15 8:20 PM:

Moving conversation to JIRA

Ariel
{quote}
What is the test coverage, unit or otherwise for compressed/uncompressed
playback? It would be good to see the new code produce the mutations out of
both compressed and uncompressed segments.

Yet another advantage is that Murmur3 doesn't have a lookup table evicting
random cache lines (on every core) as part of every single mutation that flows
through C*.

I only bring this stuff up because this is a format change and getting it all
done at once might make the amount of multi-version support code that has to be
written smaller as opposed to making these changes incrementally across
multiple versions.
{quote}
Benedict
{quote}
I'm not really sure JMH is the right tool for disk bound workloads?

Ariel
{quote}
JMH doesn't solve the hard problems of for disk bound workloads certainly. It
doesn't hurt either and it has some support for parameterizing and reporting. I
didn't know about CommitLogStress when I asked about JMH. We don't need to
build something new if something is already in place.

Benedict
{quote}
CommitLogStress is not very good, but JMH is architected around microbenchmarks
or very small operations, which this doesn't really fit.

was (Author: aweisberg):
Moving conversation to JIRA

Ariel
{noformat}
What is the test coverage, unit or otherwise for compressed/uncompressed
playback? It would be good to see the new code produce the mutations out of
both compressed and uncompressed

[jira] [Commented] (CASSANDRA-7974) Enable tooling to detect hot partitions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276043#comment-14276043
 ] 

Brandon Williams commented on CASSANDRA-7974:
-

I'd move it, it's kind of a strange thing to expose.

 Enable tooling to detect hot partitions
 ---

 Key: CASSANDRA-7974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7974
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Chris Lohfink
 Fix For: 2.1.3

 Attachments: 7974.txt, cassandra-2.1-7974v2.txt


 Sometimes you know you have a hot partition by the load on a replica set, but 
 have no way of determining which partition it is.  Tracing is inadequate for 
 this without a lot of post-tracing analysis that might not yield results.  
 Since we already include stream-lib for HLL in compaction metadata, it 
 shouldn't be too hard to wire up topK for X seconds via jmx/nodetool and then 
 return the top partitions hit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8615) Create -D flag to disable speculative retry by default

2015-01-13 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276069#comment-14276069
 ] 

Aleksey Yeschenko commented on CASSANDRA-8615:
--

Speculative retry is supposed to help with latency, not affect it negatively.

Is something wrong with our sampling, or has something broken recently in the 
implementation?

Maybe the default (99p) was a poorly chosen one, and should be changed?

Providing another -D feels like a very poor band aid to me. Should investigate 
and fix the cause instead.

 Create -D flag to disable speculative retry by default
 --

 Key: CASSANDRA-8615
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8615
 Project: Cassandra
  Issue Type: Improvement
Reporter: J.B. Langston

 Some clusters have shown increased latency with speculative retry enabled.  
 Speculative retry is enabled by default when upgrading from 1.2 to 2.0, and 
 for large clusters it can take a long time to complete a rolling upgrade, 
 during which time speculative retry will be enabled. Therefore it would be 
 helpful to have a -D flag that will disable it by default during an upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276073#comment-14276073
 ] 

Andrei Ivanov commented on CASSANDRA-8548:
--

This the everything since the restart
{code}
INFO [main] 2015-01-13 17:07:59,108 CassandraDaemon.java (line 135) Logging 
initialized
 INFO [main] 2015-01-13 17:07:59,135 YamlConfigurationLoader.java (line 80) 
Loading settings from file:/etc/cassandra/cassandra.yaml
 INFO [main] 2015-01-13 17:07:59,379 DatabaseDescriptor.java (line 142) Data 
files directories: [/var/lib/cassandra/data]
 INFO [main] 2015-01-13 17:07:59,380 DatabaseDescriptor.java (line 143) Commit 
log directory: /var/lib/cassandra/commitlog
 INFO [main] 2015-01-13 17:07:59,380 DatabaseDescriptor.java (line 183) 
DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
 INFO [main] 2015-01-13 17:07:59,380 DatabaseDescriptor.java (line 197) 
disk_failure_policy is stop
 INFO [main] 2015-01-13 17:07:59,381 DatabaseDescriptor.java (line 198) 
commit_failure_policy is stop
 INFO [main] 2015-01-13 17:07:59,385 DatabaseDescriptor.java (line 268) Global 
memtable threshold is enabled at 1996MB
 INFO [main] 2015-01-13 17:07:59,479 DatabaseDescriptor.java (line 401) Not 
using multi-threaded compaction
 INFO [main] 2015-01-13 17:07:59,646 YamlConfigurationLoader.java (line 80) 
Loading settings from file:/etc/cassandra/cassandra.yaml
 INFO [main] 2015-01-13 17:07:59,659 YamlConfigurationLoader.java (line 80) 
Loading settings from file:/etc/cassandra/cassandra.yaml
 INFO [main] 2015-01-13 17:07:59,670 CassandraDaemon.java (line 160) JVM 
vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.7.0_67
 INFO [main] 2015-01-13 17:07:59,670 CassandraDaemon.java (line 188) Heap size: 
8375238656/8375238656
 INFO [main] 2015-01-13 17:07:59,670 CassandraDaemon.java (line 190) Code Cache 
Non-heap memory: init = 2555904(2496K) used = 661056(645K) committed = 
2555904(2496K) max = 50331648(49152K)
 INFO [main] 2015-01-13 17:07:59,670 CassandraDaemon.java (line 190) Par Eden 
Space Heap memory: init = 1718091776(1677824K) used = 240609576(234970K) 
committed = 1718091776(1677824K) max = 1718091776(1677824K)
 INFO [main] 2015-01-13 17:07:59,671 CassandraDaemon.java (line 190) Par 
Survivor Space Heap memory: init = 214695936(209664K) used = 0(0K) committed = 
214695936(209664K) max = 214695936(209664K)
 INFO [main] 2015-01-13 17:07:59,672 CassandraDaemon.java (line 190) CMS Old 
Gen Heap memory: init = 6442450944(6291456K) used = 0(0K) committed = 
6442450944(6291456K) max = 6442450944(6291456K)
 INFO [main] 2015-01-13 17:07:59,673 CassandraDaemon.java (line 190) CMS Perm 
Gen Non-heap memory: init = 21757952(21248K) used = 14320416(13984K) committed 
= 21757952(21248K) max = 85983232(83968K)
 INFO [main] 2015-01-13 17:07:59,673 CassandraDaemon.java (line 191) Classpath: 
/etc/cassandra:/usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.$
 INFO [main] 2015-01-13 17:08:02,273 CLibrary.java (line 121) JNA mlockall 
successful
 INFO [main] 2015-01-13 17:08:02,287 CacheService.java (line 105) Initializing 
key cache with capacity of 100 MBs.
 INFO [main] 2015-01-13 17:08:02,294 CacheService.java (line 117) Scheduling 
key cache save to each 14400 seconds (going to save all keys).
 INFO [main] 2015-01-13 17:08:02,295 CacheService.java (line 131) Initializing 
row cache with capacity of 0 MBs
 INFO [main] 2015-01-13 17:08:02,301 CacheService.java (line 141) Scheduling 
row cache save to each 0 seconds (going to save all keys).
 INFO [main] 2015-01-13 17:08:02,417 ColumnFamilyStore.java (line 249) 
Initializing system.schema_triggers
 INFO [main] 2015-01-13 17:08:02,441 ColumnFamilyStore.java (line 249) 
Initializing system.compaction_history
 INFO [SSTableBatchOpen:1] 2015-01-13 17:08:02,446 SSTableReader.java (line 
223) Opening 
/var/lib/cassandra/data/system/compaction_history/system-compaction_history-jb-1127
 (631 bytes)
 INFO [SSTableBatchOpen:2] 2015-01-13 17:08:02,449 SSTableReader.java (line 
223) Opening 
/var/lib/cassandra/data/system/compaction_history/system-compaction_history-jb-1125
 (15735 bytes)
 INFO [SSTableBatchOpen:3] 2015-01-13 17:08:02,449 SSTableReader.java (line 
223) Opening 
/var/lib/cassandra/data/system/compaction_history/system-compaction_history-jb-1126
 (528 bytes)
 INFO [main] 2015-01-13 17:08:02,471 ColumnFamilyStore.java (line 249) 
Initializing system.batchlog
 INFO [main] 2015-01-13 17:08:02,476 ColumnFamilyStore.java (line 249) 
Initializing system.sstable_activity
 INFO [SSTableBatchOpen:1] 2015-01-13 17:08:02,478 SSTableReader.java (line 
223) Opening 
/var/lib/cassandra/data/system/sstable_activity/system-sstable_activity-jb-2232 
(932 bytes)
 INFO [SSTableBatchOpen:3] 2015-01-13 17:08:02,478 SSTableReader.java (line 
223) Opening

[jira] [Commented] (CASSANDRA-7974) Enable tooling to detect hot partitions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275987#comment-14275987
 ] 

Chris Lohfink commented on CASSANDRA-7974:
--

It could be made synchronous. I was a little concerned that throwing a lock in 
the read/write path could affect performance enough it would Heisenberg the 
data. 

It being async was to avoid a synchronized block or ConcurrentStreamSummary 
which is slower (jmh shows CSS ~10x) and may become a point of contention 
across a lot of threads. Forcing it through a single thread fixes it and its 
able to actually run a lot faster then it could concurrently.  It being in 
TRACING was just because I was thinking its pretty similar in being a sampling 
of queries and overflow safe to throw away, its been a few months though so I 
can't remember the exact train of thought. Its trivial to add another executor 
though if thats cleaner.

Would it work to convert from byte[] to String in the jmx operation 
(finishLocalSampling)?

 Enable tooling to detect hot partitions
 ---

 Key: CASSANDRA-7974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7974
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Chris Lohfink
 Fix For: 2.1.3

 Attachments: 7974.txt, cassandra-2.1-7974v2.txt


 Sometimes you know you have a hot partition by the load on a replica set, but 
 have no way of determining which partition it is.  Tracing is inadequate for 
 this without a lot of post-tracing analysis that might not yield results.  
 Since we already include stream-lib for HLL in compaction metadata, it 
 shouldn't be too hard to wire up topK for X seconds via jmx/nodetool and then 
 return the top partitions hit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7974) Enable tooling to detect hot partitions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275993#comment-14275993
 ] 

Brandon Williams commented on CASSANDRA-7974:
-

bq. Would it work to convert from byte[] to String in the jmx operation 
(finishLocalSampling)?

I don't think that matters either way, it's the Sampler/SamplerResult objects 
that are the problem.

 Enable tooling to detect hot partitions
 ---

 Key: CASSANDRA-7974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7974
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Chris Lohfink
 Fix For: 2.1.3

 Attachments: 7974.txt, cassandra-2.1-7974v2.txt


 Sometimes you know you have a hot partition by the load on a replica set, but 
 have no way of determining which partition it is.  Tracing is inadequate for 
 this without a lot of post-tracing analysis that might not yield results.  
 Since we already include stream-lib for HLL in compaction metadata, it 
 shouldn't be too hard to wire up topK for X seconds via jmx/nodetool and then 
 return the top partitions hit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7974) Enable tooling to detect hot partitions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276036#comment-14276036
 ] 

Chris Lohfink commented on CASSANDRA-7974:
--

gotcha, I can make a new patch with generic java collections.  is the 
getKeyValidator Ok or should I move that processing into operation?

 Enable tooling to detect hot partitions
 ---

 Key: CASSANDRA-7974
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7974
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Chris Lohfink
 Fix For: 2.1.3

 Attachments: 7974.txt, cassandra-2.1-7974v2.txt


 Sometimes you know you have a hot partition by the load on a replica set, but 
 have no way of determining which partition it is.  Tracing is inadequate for 
 this without a lot of post-tracing analysis that might not yield results.  
 Since we already include stream-lib for HLL in compaction metadata, it 
 shouldn't be too hard to wire up topK for X seconds via jmx/nodetool and then 
 return the top partitions hit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8615) Create -D flag to disable speculative retry by default

2015-01-13 Thread J.B. Langston (JIRA)

J.B. Langston created CASSANDRA-8615:


 Summary: Create -D flag to disable speculative retry by default
 Key: CASSANDRA-8615
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8615
 Project: Cassandra
  Issue Type: Improvement
Reporter: J.B. Langston


Some clusters have shown increased latency with speculative retry enabled.  
Speculative retry is enabled by default when upgrading from 1.2 to 2.0, and for 
large clusters it can take a long time to complete a rolling upgrade, during 
which time speculative retry will be enabled. Therefore it would be helpful to 
have a -D flag that will disable it by default during an upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8578) LeveledCompactionStrategyTest.testGrouperLevels failings

2015-01-13 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276082#comment-14276082
 ] 

Yuki Morishita commented on CASSANDRA-8578:
---

+1

 LeveledCompactionStrategyTest.testGrouperLevels failings
 

 Key: CASSANDRA-8578
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8578
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Philip Thompson
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 3.0

 Attachments: 
 0001-make-sure-we-group-by-the-compaction-strategy-we-hav.patch


 This test is failing on trunk. Here is the jenkins output:
 {code}
  Error Details
 org.apache.cassandra.db.compaction.WrappingCompactionStrategy cannot be cast 
 to org.apache.cassandra.db.compaction.LeveledCompactionStrategy
  Stack Trace
 java.lang.ClassCastException: 
 org.apache.cassandra.db.compaction.WrappingCompactionStrategy cannot be cast 
 to org.apache.cassandra.db.compaction.LeveledCompactionStrategy
   at 
 org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testGrouperLevels(LeveledCompactionStrategyTest.java:124)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime