[jira] [Commented] (CASSANDRA-10788) Upgrade from 2.2.1 to 3.0.0 fails with NullPointerException

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041246#comment-15041246
 ] 

Marcus Eriksson commented on CASSANDRA-10788:
-

I needed a tiny fix to be able to run the unit test alone from IDE: 
https://github.com/krummas/cassandra/commits/stef/10788-3.0

other than that, I am +1 and will commit once dtests pass

> Upgrade from 2.2.1 to 3.0.0 fails with NullPointerException
> ---
>
> Key: CASSANDRA-10788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10788
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle
>Reporter: Tomas Ramanauskas
>Assignee: Stefania
> Fix For: 3.0.1, 3.1
>
>
> I tried to upgrade Cassandra from 2.2.1 to 3.0.0, however, I get this error 
> on startup after Cassandra 3.0 software was installed:
> {code}
> ERROR [main] 2015-11-30 15:44:50,164 CassandraDaemon.java:702 - Exception 
> encountered during startup
> java.lang.NullPointerException: null
>   at org.apache.cassandra.io.util.FileUtils.delete(FileUtils.java:374) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.SystemKeyspace.migrateDataDirs(SystemKeyspace.java:1341)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:180) 
> [apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:561)
>  [apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689) 
> [apache-cassandra-3.0.0.jar:3.0.0]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10674) Materialized View SSTable streaming/leaving status race on decommission

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041245#comment-15041245
 ] 

Joel Knighton commented on CASSANDRA-10674:
---

Thanks for the clarifications.

I agree that your latest comment is the right approach, [~pauloricardomg].

> Materialized View SSTable streaming/leaving status race on decommission
> ---
>
> Key: CASSANDRA-10674
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10674
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Distributed Metadata
>Reporter: Joel Knighton
>Assignee: Paulo Motta
> Fix For: 3.0.1, 3.1
>
> Attachments: leaving-node-debug.log, receiving-node-debug.log
>
>
> On decommission of a node in a cluster with materialized views, it is 
> possible for the decommissioning node to begin streaming sstables for an MV 
> base table before the receiving node is aware of the leaving status.
> The materialized view base/view replica pairing checks pending endpoints to 
> handle the case when an sstable is received from a leaving node; without the 
> leaving message, this check breaks and an exception is thrown. The streamed 
> sstable is never applied.
> Logs from a decommissioning node and a node receiving such a stream are 
> attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10788) Upgrade from 2.2.1 to 3.0.0 fails with NullPointerException

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041257#comment-15041257
 ] 

Stefania commented on CASSANDRA-10788:
--

Thanks for the fix!

> Upgrade from 2.2.1 to 3.0.0 fails with NullPointerException
> ---
>
> Key: CASSANDRA-10788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10788
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle
>Reporter: Tomas Ramanauskas
>Assignee: Stefania
> Fix For: 3.0.1, 3.1
>
>
> I tried to upgrade Cassandra from 2.2.1 to 3.0.0, however, I get this error 
> on startup after Cassandra 3.0 software was installed:
> {code}
> ERROR [main] 2015-11-30 15:44:50,164 CassandraDaemon.java:702 - Exception 
> encountered during startup
> java.lang.NullPointerException: null
>   at org.apache.cassandra.io.util.FileUtils.delete(FileUtils.java:374) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.SystemKeyspace.migrateDataDirs(SystemKeyspace.java:1341)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:180) 
> [apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:561)
>  [apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689) 
> [apache-cassandra-3.0.0.jar:3.0.0]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix bug with range tombstone in reverse queries and Partition test coverage

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 814bc7065 -> 1e978df3a


Fix bug with range tombstone in reverse queries and Partition test coverage

patch by blambov; reviewed by slebresne for CASSANDRA-10059


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e978df3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e978df3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e978df3

Branch: refs/heads/cassandra-3.0
Commit: 1e978df3a80a8201ce95253e3fd822fd804993d7
Parents: 814bc70
Author: Branimir Lambov 
Authored: Wed Nov 11 10:43:54 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Dec 4 10:08:51 2015 +0100

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/RangeTombstoneList.java |  47 +-
 .../apache/cassandra/utils/SearchIterator.java  |   3 +-
 test/unit/org/apache/cassandra/Util.java|  23 +
 .../partition/PartitionImplementationTest.java  | 524 +++
 5 files changed, 566 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e978df3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f6aed18..2527c43 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.1
+ * Fix bug with range tombstones on reverse queries and test coverage for
+   AbstractBTreePartition (CASSANDRA-10059)
  * Remove 64k limit on collection elements (CASSANDRA-10374)
  * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
  * Fix NPE on stream read error (CASSANDRA-10771)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e978df3/src/java/org/apache/cassandra/db/RangeTombstoneList.java
--
diff --git a/src/java/org/apache/cassandra/db/RangeTombstoneList.java 
b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
index c92a296..c67ea33 100644
--- a/src/java/org/apache/cassandra/db/RangeTombstoneList.java
+++ b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
@@ -17,21 +17,16 @@
  */
 package org.apache.cassandra.db;
 
-import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.Arrays;
+import java.util.Collections;
 import java.util.Iterator;
 
 import org.apache.cassandra.utils.AbstractIterator;
 import com.google.common.collect.Iterators;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.apache.cassandra.cache.IMeasurableMemory;
 import org.apache.cassandra.db.rows.*;
-import org.apache.cassandra.io.IVersionedSerializer;
-import org.apache.cassandra.io.util.DataInputPlus;
-import org.apache.cassandra.io.util.DataOutputPlus;
 import org.apache.cassandra.utils.ObjectSizes;
 import org.apache.cassandra.utils.memory.AbstractAllocator;
 
@@ -53,8 +48,6 @@ import org.apache.cassandra.utils.memory.AbstractAllocator;
  */
 public class RangeTombstoneList implements Iterable, 
IMeasurableMemory
 {
-private static final Logger logger = 
LoggerFactory.getLogger(RangeTombstoneList.class);
-
 private static long EMPTY_SIZE = ObjectSizes.measure(new 
RangeTombstoneList(null, 0));
 
 private final ClusteringComparator comparator;
@@ -265,6 +258,8 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 /*
  * Return is the index of the range covering name if name is covered. If 
the return idx is negative,
  * no range cover name and -idx-1 is the index of the first range whose 
start is greater than name.
+ *
+ * Note that bounds are not in the range if they fall on its boundary.
  */
 private int searchInternal(ClusteringPrefix name, int startIdx, int endIdx)
 {
@@ -274,7 +269,9 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 int pos = Arrays.binarySearch(starts, startIdx, endIdx, name, 
comparator);
 if (pos >= 0)
 {
-return pos;
+// Equality only happens for bounds (as used by 
forward/reverseIterator), and bounds are equal only if they
+// are the same or complementary, in either case the bound itself 
is not part of the range.
+return -pos - 1;
 }
 else
 {
@@ -283,7 +280,7 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 if (idx < 0)
 return -1;
 
-return comparator.compare(name, ends[idx]) <= 0 ? idx : -idx-2;
+return comparator.compare(name, ends[idx]) < 0 ? idx : -idx-2;
 }
 }
 
@@ -387,14 +384,14 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 final int start = startIdx < 0 ? -startIdx-1 : startIdx;
 
 if (start >= size)
-

[3/3] cassandra git commit: Merge branch 'cassandra-3.1' into trunk

[slebresne]

Merge branch 'cassandra-3.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/220f253f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/220f253f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/220f253f

Branch: refs/heads/trunk
Commit: 220f253fb30809f55368a30e2594735bc8362718
Parents: 79f4bdb fd5cb43
Author: Sylvain Lebresne 
Authored: Fri Dec 4 10:10:42 2015 +0100
Committer: Sylvain Lebresne 
Committed: Fri Dec 4 10:10:42 2015 +0100

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/RangeTombstoneList.java |  47 +-
 .../apache/cassandra/utils/SearchIterator.java  |   3 +-
 test/unit/org/apache/cassandra/Util.java|  23 +
 .../partition/PartitionImplementationTest.java  | 524 +++
 5 files changed, 566 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/220f253f/CHANGES.txt
--
diff --cc CHANGES.txt
index 08266d3,7a6076b..711bdde
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,19 -1,7 +1,21 @@@
 +3.2
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.1
  Merged from 3.0:
+  * Fix bug with range tombstones on reverse queries and test coverage for
+AbstractBTreePartition (CASSANDRA-10059)
   * Remove 64k limit on collection elements (CASSANDRA-10374)
   * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
   * Fix NPE on stream read error (CASSANDRA-10771)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/220f253f/src/java/org/apache/cassandra/db/RangeTombstoneList.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/220f253f/test/unit/org/apache/cassandra/Util.java
--

[1/3] cassandra git commit: Fix bug with range tombstone in reverse queries and Partition test coverage

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 79f4bdb20 -> 220f253fb


Fix bug with range tombstone in reverse queries and Partition test coverage

patch by blambov; reviewed by slebresne for CASSANDRA-10059


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e978df3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e978df3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e978df3

Branch: refs/heads/trunk
Commit: 1e978df3a80a8201ce95253e3fd822fd804993d7
Parents: 814bc70
Author: Branimir Lambov 
Authored: Wed Nov 11 10:43:54 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Dec 4 10:08:51 2015 +0100

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/RangeTombstoneList.java |  47 +-
 .../apache/cassandra/utils/SearchIterator.java  |   3 +-
 test/unit/org/apache/cassandra/Util.java|  23 +
 .../partition/PartitionImplementationTest.java  | 524 +++
 5 files changed, 566 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e978df3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f6aed18..2527c43 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.1
+ * Fix bug with range tombstones on reverse queries and test coverage for
+   AbstractBTreePartition (CASSANDRA-10059)
  * Remove 64k limit on collection elements (CASSANDRA-10374)
  * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
  * Fix NPE on stream read error (CASSANDRA-10771)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e978df3/src/java/org/apache/cassandra/db/RangeTombstoneList.java
--
diff --git a/src/java/org/apache/cassandra/db/RangeTombstoneList.java 
b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
index c92a296..c67ea33 100644
--- a/src/java/org/apache/cassandra/db/RangeTombstoneList.java
+++ b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
@@ -17,21 +17,16 @@
  */
 package org.apache.cassandra.db;
 
-import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.Arrays;
+import java.util.Collections;
 import java.util.Iterator;
 
 import org.apache.cassandra.utils.AbstractIterator;
 import com.google.common.collect.Iterators;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.apache.cassandra.cache.IMeasurableMemory;
 import org.apache.cassandra.db.rows.*;
-import org.apache.cassandra.io.IVersionedSerializer;
-import org.apache.cassandra.io.util.DataInputPlus;
-import org.apache.cassandra.io.util.DataOutputPlus;
 import org.apache.cassandra.utils.ObjectSizes;
 import org.apache.cassandra.utils.memory.AbstractAllocator;
 
@@ -53,8 +48,6 @@ import org.apache.cassandra.utils.memory.AbstractAllocator;
  */
 public class RangeTombstoneList implements Iterable, 
IMeasurableMemory
 {
-private static final Logger logger = 
LoggerFactory.getLogger(RangeTombstoneList.class);
-
 private static long EMPTY_SIZE = ObjectSizes.measure(new 
RangeTombstoneList(null, 0));
 
 private final ClusteringComparator comparator;
@@ -265,6 +258,8 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 /*
  * Return is the index of the range covering name if name is covered. If 
the return idx is negative,
  * no range cover name and -idx-1 is the index of the first range whose 
start is greater than name.
+ *
+ * Note that bounds are not in the range if they fall on its boundary.
  */
 private int searchInternal(ClusteringPrefix name, int startIdx, int endIdx)
 {
@@ -274,7 +269,9 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 int pos = Arrays.binarySearch(starts, startIdx, endIdx, name, 
comparator);
 if (pos >= 0)
 {
-return pos;
+// Equality only happens for bounds (as used by 
forward/reverseIterator), and bounds are equal only if they
+// are the same or complementary, in either case the bound itself 
is not part of the range.
+return -pos - 1;
 }
 else
 {
@@ -283,7 +280,7 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 if (idx < 0)
 return -1;
 
-return comparator.compare(name, ends[idx]) <= 0 ? idx : -idx-2;
+return comparator.compare(name, ends[idx]) < 0 ? idx : -idx-2;
 }
 }
 
@@ -387,14 +384,14 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 final int start = startIdx < 0 ? -startIdx-1 : startIdx;
 
 if (start >= size)
-return 

[jira] [Commented] (CASSANDRA-10743) Failed upgradesstables (upgrade from 2.2.2 to 3.0.0)

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041313#comment-15041313
 ] 

Marcus Eriksson commented on CASSANDRA-10743:
-

[~gabor.auth] We need the schema to be able to reproduce with your sstables

> Failed upgradesstables (upgrade from 2.2.2 to 3.0.0)
> 
>
> Key: CASSANDRA-10743
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10743
> Project: Cassandra
>  Issue Type: Bug
> Environment: CentOS Linux release 7.1.1503, OpenJDK Runtime 
> Environment (build 1.8.0_65-b17), DSC Cassandra 3.0.0 (tar.gz)
>Reporter: Gábor Auth
> Attachments: faulty-tables.tar.gz
>
>
> {code}
> [cassandra@dc01-rack01-cass01 ~]$ 
> /home/cassandra/dsc-cassandra-3.0.0/bin/nodetool upgradesstables
> error: null
> -- StackTrace --
> java.lang.UnsupportedOperationException
> at 
> org.apache.cassandra.db.rows.CellPath$EmptyCellPath.get(CellPath.java:143)
> at 
> org.apache.cassandra.db.marshal.CollectionType$CollectionPathSerializer.serializedSize(CollectionType.java:226)
> at 
> org.apache.cassandra.db.rows.BufferCell$Serializer.serializedSize(BufferCell.java:325)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.sizeOfComplexColumn(UnfilteredSerializer.java:297)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serializedRowBodySize(UnfilteredSerializer.java:282)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:163)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:144)
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:112)
> at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:121)
> at 
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:57)
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:110)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
> at 
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:397)
> at 
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd5cb43d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd5cb43d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd5cb43d

Branch: refs/heads/trunk
Commit: fd5cb43da87f0dd5e4c8e3e80ec293bcaa23d0fb
Parents: 29c653d 1e978df
Author: Sylvain Lebresne 
Authored: Fri Dec 4 10:10:20 2015 +0100
Committer: Sylvain Lebresne 
Committed: Fri Dec 4 10:10:20 2015 +0100

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/RangeTombstoneList.java |  47 +-
 .../apache/cassandra/utils/SearchIterator.java  |   3 +-
 test/unit/org/apache/cassandra/Util.java|  23 +
 .../partition/PartitionImplementationTest.java  | 524 +++
 5 files changed, 566 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd5cb43d/CHANGES.txt
--
diff --cc CHANGES.txt
index bb8653c,2527c43..7a6076b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,6 +1,7 @@@
 -3.0.1
 +3.1
 +Merged from 3.0:
+  * Fix bug with range tombstones on reverse queries and test coverage for
+AbstractBTreePartition (CASSANDRA-10059)
   * Remove 64k limit on collection elements (CASSANDRA-10374)
   * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
   * Fix NPE on stream read error (CASSANDRA-10771)

[jira] [Commented] (CASSANDRA-10788) Upgrade from 2.2.1 to 3.0.0 fails with NullPointerException

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041489#comment-15041489
 ] 

Stefania commented on CASSANDRA-10788:
--

CI completed, it seems OK to me.

> Upgrade from 2.2.1 to 3.0.0 fails with NullPointerException
> ---
>
> Key: CASSANDRA-10788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10788
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle
>Reporter: Tomas Ramanauskas
>Assignee: Stefania
> Fix For: 3.0.1, 3.1
>
>
> I tried to upgrade Cassandra from 2.2.1 to 3.0.0, however, I get this error 
> on startup after Cassandra 3.0 software was installed:
> {code}
> ERROR [main] 2015-11-30 15:44:50,164 CassandraDaemon.java:702 - Exception 
> encountered during startup
> java.lang.NullPointerException: null
>   at org.apache.cassandra.io.util.FileUtils.delete(FileUtils.java:374) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.SystemKeyspace.migrateDataDirs(SystemKeyspace.java:1341)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:180) 
> [apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:561)
>  [apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689) 
> [apache-cassandra-3.0.0.jar:3.0.0]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10666) jmx_test.TestJMX.test_compactionstats is flapping

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-10666:
---

Assignee: Marcus Eriksson

> jmx_test.TestJMX.test_compactionstats is flapping
> -
>
> Key: CASSANDRA-10666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10666
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Marcus Eriksson
> Fix For: 3.0.1, 3.1
>
>
> See the [history for that 
> test|http://cassci.datastax.com/job/cassandra-3.0_dtest/335/testReport/junit/jmx_test/TestJMX/test_compactionstats/history/].
>  On each failure there is something about a problem with {{jolokia}} so 
> that's probably a test environment problem. Still needs to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9752) incremental repair dtest flaps on 2.2

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson resolved CASSANDRA-9752.

   Resolution: Cannot Reproduce
Fix Version/s: (was: 2.2.x)

Looks like this is now stable:
http://cassci.datastax.com/view/trunk/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/junit/incremental_repair_test/history/

> incremental repair dtest flaps on 2.2 
> --
>
> Key: CASSANDRA-9752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9752
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>
> {{incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test}} 
> flaps on 2.2. It's hard to tell what failures are repair-specific, but there 
> are a few distinct failures I've seen recently:
> - [an NPE in 
> StorageService|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/143/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
> - [an NPE in 
> SSTableRewriter|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/135/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/].
>  I believe this is related to CASSANDRA-9730, but someone should confirm this.
> - [an on-disk data size that is too 
> large|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/133/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
> You can find the test itself [here on 
> GitHub|https://github.com/riptano/cassandra-dtest/blob/master/incremental_repair_test.py#L206]
>  and run it with the command
> {code}
> CASSANDRA_VERSION=git:trunk nosetests 
> incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test
> {code}
> Assigning [~yukim], since you're the repair person, but feel free to reassign 
> to whoever's appropriate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10815) WrappingCompactionStrategy can block others when there are lots of sstables

[Severin Leonhardt (JIRA)]

Severin Leonhardt created CASSANDRA-10815:
-

 Summary: WrappingCompactionStrategy can block others when there 
are lots of sstables
 Key: CASSANDRA-10815
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10815
 Project: Cassandra
  Issue Type: Bug
  Components: Compaction
Reporter: Severin Leonhardt


We observe {{nodetool compactionstats}} hanging when there are a lot of 
SSTables in one table. We have about 30.000 SSTables most likely created by an 
incremental repair (why it's that many is still a mystery to us).

Looking at the stacktraces of some selected threads it becomes apparent that a 
single CompactionExecutor blocks several other threads:

{noformat}
"CompactionExecutor:4065" - Thread t@282454
   java.lang.Thread.State: RUNNABLE
at java.util.TimSort.binarySort(TimSort.java:292)
at java.util.TimSort.sort(TimSort.java:235)
at java.util.Arrays.sort(Arrays.java:1512)
at java.util.ArrayList.sort(ArrayList.java:1454)
at java.util.Collections.sort(Collections.java:175)
at 
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.trimToThresholdWithHotness(SizeTieredCompactionStrategy.java:139)
at 
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.mostInterestingBucket(SizeTieredCompactionStrategy.java:119)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getSSTablesForSTCS(LeveledManifest.java:360)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:318)
- locked <25446e7b> (a 
org.apache.cassandra.db.compaction.LeveledManifest)
at 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:101)
at 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:90)
- locked <33506169> (a 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
at 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:77)
- locked <31924612> (a 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
at 
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}

Because this thread has locked <31924612> (a 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy) the following 
threads are blocked:

{noformat}
"CompactionExecutor:4064" - Thread t@282337
   java.lang.Thread.State: BLOCKED
at 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:72)
- waiting to lock <31924612> (a 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
"CompactionExecutor:4065" t@282454
at 
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}

{noformat}
"MemtableFlushWriter:2431" - Thread t@283411
   java.lang.Thread.State: BLOCKED
at 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
- waiting to lock <31924612> (a 
org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
"CompactionExecutor:4065" t@282454
at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
at 
org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
at 
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
at 
org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1541)
at 
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at 

[1/2] cassandra git commit: Fix bug with range tombstone in reverse queries and Partition test coverage

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.1 29c653de6 -> fd5cb43da


Fix bug with range tombstone in reverse queries and Partition test coverage

patch by blambov; reviewed by slebresne for CASSANDRA-10059


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e978df3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e978df3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e978df3

Branch: refs/heads/cassandra-3.1
Commit: 1e978df3a80a8201ce95253e3fd822fd804993d7
Parents: 814bc70
Author: Branimir Lambov 
Authored: Wed Nov 11 10:43:54 2015 +0200
Committer: Sylvain Lebresne 
Committed: Fri Dec 4 10:08:51 2015 +0100

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/RangeTombstoneList.java |  47 +-
 .../apache/cassandra/utils/SearchIterator.java  |   3 +-
 test/unit/org/apache/cassandra/Util.java|  23 +
 .../partition/PartitionImplementationTest.java  | 524 +++
 5 files changed, 566 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e978df3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f6aed18..2527c43 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.1
+ * Fix bug with range tombstones on reverse queries and test coverage for
+   AbstractBTreePartition (CASSANDRA-10059)
  * Remove 64k limit on collection elements (CASSANDRA-10374)
  * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
  * Fix NPE on stream read error (CASSANDRA-10771)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e978df3/src/java/org/apache/cassandra/db/RangeTombstoneList.java
--
diff --git a/src/java/org/apache/cassandra/db/RangeTombstoneList.java 
b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
index c92a296..c67ea33 100644
--- a/src/java/org/apache/cassandra/db/RangeTombstoneList.java
+++ b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
@@ -17,21 +17,16 @@
  */
 package org.apache.cassandra.db;
 
-import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.Arrays;
+import java.util.Collections;
 import java.util.Iterator;
 
 import org.apache.cassandra.utils.AbstractIterator;
 import com.google.common.collect.Iterators;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.apache.cassandra.cache.IMeasurableMemory;
 import org.apache.cassandra.db.rows.*;
-import org.apache.cassandra.io.IVersionedSerializer;
-import org.apache.cassandra.io.util.DataInputPlus;
-import org.apache.cassandra.io.util.DataOutputPlus;
 import org.apache.cassandra.utils.ObjectSizes;
 import org.apache.cassandra.utils.memory.AbstractAllocator;
 
@@ -53,8 +48,6 @@ import org.apache.cassandra.utils.memory.AbstractAllocator;
  */
 public class RangeTombstoneList implements Iterable, 
IMeasurableMemory
 {
-private static final Logger logger = 
LoggerFactory.getLogger(RangeTombstoneList.class);
-
 private static long EMPTY_SIZE = ObjectSizes.measure(new 
RangeTombstoneList(null, 0));
 
 private final ClusteringComparator comparator;
@@ -265,6 +258,8 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 /*
  * Return is the index of the range covering name if name is covered. If 
the return idx is negative,
  * no range cover name and -idx-1 is the index of the first range whose 
start is greater than name.
+ *
+ * Note that bounds are not in the range if they fall on its boundary.
  */
 private int searchInternal(ClusteringPrefix name, int startIdx, int endIdx)
 {
@@ -274,7 +269,9 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 int pos = Arrays.binarySearch(starts, startIdx, endIdx, name, 
comparator);
 if (pos >= 0)
 {
-return pos;
+// Equality only happens for bounds (as used by 
forward/reverseIterator), and bounds are equal only if they
+// are the same or complementary, in either case the bound itself 
is not part of the range.
+return -pos - 1;
 }
 else
 {
@@ -283,7 +280,7 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 if (idx < 0)
 return -1;
 
-return comparator.compare(name, ends[idx]) <= 0 ? idx : -idx-2;
+return comparator.compare(name, ends[idx]) < 0 ? idx : -idx-2;
 }
 }
 
@@ -387,14 +384,14 @@ public class RangeTombstoneList implements 
Iterable, IMeasurable
 final int start = startIdx < 0 ? -startIdx-1 : startIdx;
 
 if (start >= size)
-

[jira] [Resolved] (CASSANDRA-7466) Unit Test Suite Breaks when Run in a Single JVM

[Branimir Lambov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Branimir Lambov resolved CASSANDRA-7466.

Resolution: Not A Problem

> Unit Test Suite Breaks when Run in a Single JVM
> ---
>
> Key: CASSANDRA-7466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7466
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
> Environment: MacOS 10.9.3, IntelliJ IDEA 14, Java 1.8.0_65
>Reporter: Caleb William Rackliffe
>Assignee: Caleb William Rackliffe
>Priority: Minor
>  Labels: unit-test
>
> I pulled down the source and followed 
> http://wiki.apache.org/cassandra/RunningCassandraInIDEA to import C* as an 
> IDEA project. Everything in the tutorial works as it should, but when I tried 
> to run the unit tests in {{test/unit/org.apache.cassandra}}, the suite fails 
> a few tests in, complaining variously about a hodgepodge of things like:
> {noformat}
> FSWriteError in 
> build/test/cassandra/data/cql_test_keyspace/table_1-0fd6a7109a1311e586ed079249e818e0/backups/cql_test_keyspace-table_1.some_index-ka-5-Summary.db
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135)
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:381)
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:377)
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:377)
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:377)
>   at 
> org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:377)
>   at org.apache.cassandra.SchemaLoader.cleanup(SchemaLoader.java:454)
>   at 
> org.apache.cassandra.SchemaLoader.cleanupAndLeaveDirs(SchemaLoader.java:429)
>   at org.apache.cassandra.SchemaLoader.prepareServer(SchemaLoader.java:85)
>   at org.apache.cassandra.SchemaLoader.loadSchema(SchemaLoader.java:61)
>   at org.apache.cassandra.SchemaLoader.loadSchema(SchemaLoader.java:56)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
>   at org.junit.runners.Suite.runChild(Suite.java:117)
>   at org.junit.runners.Suite.runChild(Suite.java:24)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
>   at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
>   at 
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> Caused by: java.nio.file.NoSuchFileException: 
> build/test/cassandra/data/cql_test_keyspace/table_1-0fd6a7109a1311e586ed079249e818e0/backups/cql_test_keyspace-table_1.some_index-ka-5-Summary.db
>   at 
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>   at 
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
>   at 
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
>   at java.nio.file.Files.delete(Files.java:1079)
>   at 
> 

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd5cb43d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd5cb43d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd5cb43d

Branch: refs/heads/cassandra-3.1
Commit: fd5cb43da87f0dd5e4c8e3e80ec293bcaa23d0fb
Parents: 29c653d 1e978df
Author: Sylvain Lebresne 
Authored: Fri Dec 4 10:10:20 2015 +0100
Committer: Sylvain Lebresne 
Committed: Fri Dec 4 10:10:20 2015 +0100

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/RangeTombstoneList.java |  47 +-
 .../apache/cassandra/utils/SearchIterator.java  |   3 +-
 test/unit/org/apache/cassandra/Util.java|  23 +
 .../partition/PartitionImplementationTest.java  | 524 +++
 5 files changed, 566 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd5cb43d/CHANGES.txt
--
diff --cc CHANGES.txt
index bb8653c,2527c43..7a6076b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,6 +1,7 @@@
 -3.0.1
 +3.1
 +Merged from 3.0:
+  * Fix bug with range tombstones on reverse queries and test coverage for
+AbstractBTreePartition (CASSANDRA-10059)
   * Remove 64k limit on collection elements (CASSANDRA-10374)
   * Remove unclear Indexer.indexes() method (CASSANDRA-10690)
   * Fix NPE on stream read error (CASSANDRA-10771)

[jira] [Commented] (CASSANDRA-10477) java.lang.AssertionError in StorageProxy.submitHint

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041410#comment-15041410
 ] 

Sylvain Lebresne commented on CASSANDRA-10477:
--

* The failure detector will never return false for the local host, so the 
changes in the 2nd branch of commitPaxos are unnecessary.
* We're kind of dodging the hint "overload" protection on the paxos path as we 
don't use {{sendToHintedEndpoints}} (which in particular makes the comment on 
{{commitPaxosLocal}} misleading as it suggests otherwise). I think the simplest 
solution is to move the overload test from {{sendToHintedEndpoints}} to some 
{{checkOverloaded()}} method and call that in {{commitPaxos}} too.
* Instead of adding the {{droppable()}} method to {{LocalMutationRunnable}}, we 
should probably use {{MessagingService.DROPPABLE_VERBS.contains(verb)}}.
* In theory, we could still run into the problem of that ticket if 
{{OPTIMIZE_LOCAL_REQUESTS}} is {{false}}. And in fact, I believe this option is 
unsafe since at least CASSANDRA-4753 as we somewhat strongly assume writes to 
the localhost do *not* go through {{MessagingService}}. So I would suggest 
ditching that option. Not only is it unsafe, but it's not used anywhere by the 
code and it's hardcoded so you have to change the code and recompile to even 
use it (which means I doubt anyone has even tried it in a long long time). And 
if we end up needing it in the future, we'll have to figure out how to make it 
safe.
* Why isn't the added assertion in {{WriteCallbackInfo}} on 3.0 not using 
{{!shouldHint}} lie in the 2.1 patch?


> java.lang.AssertionError in StorageProxy.submitHint
> ---
>
> Key: CASSANDRA-10477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10477
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
> Environment: CentOS 6, Oracle JVM 1.8.45
>Reporter: Severin Leonhardt
>Assignee: Ariel Weisberg
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> A few days after updating from 2.0.15 to 2.1.9 we have the following log 
> entry on 2 of 5 machines:
> {noformat}
> ERROR [EXPIRING-MAP-REAPER:1] 2015-10-07 17:01:08,041 
> CassandraDaemon.java:223 - Exception in thread 
> Thread[EXPIRING-MAP-REAPER:1,5,main]
> java.lang.AssertionError: /192.168.11.88
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:949) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.net.MessagingService$5.apply(MessagingService.java:383) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.net.MessagingService$5.apply(MessagingService.java:363) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at org.apache.cassandra.utils.ExpiringMap$1.run(ExpiringMap.java:98) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_45]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {noformat}
> 192.168.11.88 is the broadcast address of the local machine.
> When this is logged the read request latency of the whole cluster becomes 
> very bad, from 6 ms/op to more than 100 ms/op according to OpsCenter. Clients 
> get a lot of timeouts. We need to restart the affected Cassandra node to get 
> back normal read latencies. It seems write latency is not affected.
> Disabling hinted handoff using {{nodetool disablehandoff}} only prevents the 
> assert from being logged. At some point the read latency becomes bad again. 
> Restarting the node where hinted handoff was disabled results in the read 
> latency being better again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10674) Materialized View SSTable streaming/leaving status race on decommission

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041492#comment-15041492
 ] 

Paulo Motta commented on CASSANDRA-10674:
-

Updated branch writing non-paired view mutations only to the batchlog, rebased 
and resubmitted tests.

It would be nice, if possible, re-run the jepsen tests that motivated this 
non-paired endpoint scenario to see if they are still correct.

Thanks!

> Materialized View SSTable streaming/leaving status race on decommission
> ---
>
> Key: CASSANDRA-10674
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10674
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Distributed Metadata
>Reporter: Joel Knighton
>Assignee: Paulo Motta
> Fix For: 3.0.1, 3.1
>
> Attachments: leaving-node-debug.log, receiving-node-debug.log
>
>
> On decommission of a node in a cluster with materialized views, it is 
> possible for the decommissioning node to begin streaming sstables for an MV 
> base table before the receiving node is aware of the leaving status.
> The materialized view base/view replica pairing checks pending endpoints to 
> handle the case when an sstable is received from a leaving node; without the 
> leaving message, this check breaks and an exception is thrown. The streamed 
> sstable is never applied.
> Logs from a decommissioning node and a node receiving such a stream are 
> attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10815) WrappingCompactionStrategy can block others when there are lots of sstables

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10815:
-
Assignee: Marcus Eriksson

> WrappingCompactionStrategy can block others when there are lots of sstables
> ---
>
> Key: CASSANDRA-10815
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10815
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Severin Leonhardt
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.x
>
>
> We observe {{nodetool compactionstats}} hanging when there are a lot of 
> SSTables in one table. We have about 30.000 SSTables most likely created by 
> an incremental repair (why it's that many is still a mystery to us).
> Looking at the stacktraces of some selected threads it becomes apparent that 
> a single CompactionExecutor blocks several other threads:
> {noformat}
> "CompactionExecutor:4065" - Thread t@282454
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.binarySort(TimSort.java:292)
>   at java.util.TimSort.sort(TimSort.java:235)
>   at java.util.Arrays.sort(Arrays.java:1512)
>   at java.util.ArrayList.sort(ArrayList.java:1454)
>   at java.util.Collections.sort(Collections.java:175)
>   at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.trimToThresholdWithHotness(SizeTieredCompactionStrategy.java:139)
>   at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.mostInterestingBucket(SizeTieredCompactionStrategy.java:119)
>   at 
> org.apache.cassandra.db.compaction.LeveledManifest.getSSTablesForSTCS(LeveledManifest.java:360)
>   at 
> org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:318)
>   - locked <25446e7b> (a 
> org.apache.cassandra.db.compaction.LeveledManifest)
>   at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:101)
>   at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:90)
>   - locked <33506169> (a 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:77)
>   - locked <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Because this thread has locked <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) the following 
> threads are blocked:
> {noformat}
> "CompactionExecutor:4064" - Thread t@282337
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:72)
>   - waiting to lock <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:4065" t@282454
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> {noformat}
> "MemtableFlushWriter:2431" - Thread t@283411
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
>   - waiting to lock <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:4065" t@282454
>   at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
>   at 
> org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
>   at 
> 

[jira] [Updated] (CASSANDRA-10815) WrappingCompactionStrategy can block others when there are lots of sstables

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10815:
-
Fix Version/s: 3.x
   2.2.x
   2.1.x

> WrappingCompactionStrategy can block others when there are lots of sstables
> ---
>
> Key: CASSANDRA-10815
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10815
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Severin Leonhardt
> Fix For: 2.1.x, 2.2.x, 3.x
>
>
> We observe {{nodetool compactionstats}} hanging when there are a lot of 
> SSTables in one table. We have about 30.000 SSTables most likely created by 
> an incremental repair (why it's that many is still a mystery to us).
> Looking at the stacktraces of some selected threads it becomes apparent that 
> a single CompactionExecutor blocks several other threads:
> {noformat}
> "CompactionExecutor:4065" - Thread t@282454
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.binarySort(TimSort.java:292)
>   at java.util.TimSort.sort(TimSort.java:235)
>   at java.util.Arrays.sort(Arrays.java:1512)
>   at java.util.ArrayList.sort(ArrayList.java:1454)
>   at java.util.Collections.sort(Collections.java:175)
>   at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.trimToThresholdWithHotness(SizeTieredCompactionStrategy.java:139)
>   at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.mostInterestingBucket(SizeTieredCompactionStrategy.java:119)
>   at 
> org.apache.cassandra.db.compaction.LeveledManifest.getSSTablesForSTCS(LeveledManifest.java:360)
>   at 
> org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:318)
>   - locked <25446e7b> (a 
> org.apache.cassandra.db.compaction.LeveledManifest)
>   at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:101)
>   at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:90)
>   - locked <33506169> (a 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:77)
>   - locked <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Because this thread has locked <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) the following 
> threads are blocked:
> {noformat}
> "CompactionExecutor:4064" - Thread t@282337
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:72)
>   - waiting to lock <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:4065" t@282454
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> {noformat}
> "MemtableFlushWriter:2431" - Thread t@283411
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
>   - waiting to lock <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:4065" t@282454
>   at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
>   at 
> org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
>   at 
> 

[jira] [Commented] (CASSANDRA-8639) Can OOM on CL replay with dense mutations

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041665#comment-15041665
 ] 

Ariel Weisberg commented on CASSANDRA-8639:
---

Tests on 3.0 finished.

> Can OOM on CL replay with dense mutations
> -
>
> Key: CASSANDRA-8639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8639
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: T Jake Luciani
>Assignee: Ariel Weisberg
>Priority: Minor
> Fix For: 2.1.x
>
>
> If you write dense mutations with many clustering keys, the replay of the CL 
> can quickly overwhelm a node on startup.  This looks to be caused by the fact 
> we only ensure there are 1000 mutations in flight at a time. but those 
> mutations could have thousands of cells in them.
> A better approach would be to limit the CL replay to the amount of memory in 
> flight using cell.unsharedHeapSize()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10817) DROP USER is not case-sensitive

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10817:

Fix Version/s: (was: 2.2.4)
   2.2.x

> DROP USER is not case-sensitive
> ---
>
> Key: CASSANDRA-10817
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10817
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Mike Adamson
>Priority: Minor
> Fix For: 2.2.x
>
>
> As per the summary {{DROP USER}} is not case sensitive, so:
> {noformat}
> CREATE USER 'Test';
> LIST USERS;
>  name  | super
> ---+---
>   Test | False
>  cassandra |  True
> DROP USER 'Test';
> InvalidRequest: code=2200 [Invalid query] message="test doesn't exist"
> {noformat}
> {{DROP ROLE}} is case-sensitive and will drop the above user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10799:

Reviewer: Paulo Motta

> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
>  Issue Type: Test
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the 
> driver is not cythonized (installed with {{--no_extensions}}) then the tests 
> are fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10149) Make nodetool cfstats and cfhistograms consistent

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10149:

Fix Version/s: 3.x
   3.0.x

> Make nodetool cfstats and cfhistograms consistent
> -
>
> Key: CASSANDRA-10149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10149
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>  Labels: lhf
> Fix For: 3.x
>
> Attachments: trunk-10149.txt
>
>
> Currently when using nodetool cfstats and cfhistograms (for a keyspace and 
> table) have different syntax.  cfstats uses "keyspace table".  cfhistograms 
> uses "keyspace.table".  We should unify it one way or the other (or both).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10149) Make nodetool cfstats and cfhistograms consistent

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10149:

Fix Version/s: (was: 3.0.x)

> Make nodetool cfstats and cfhistograms consistent
> -
>
> Key: CASSANDRA-10149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10149
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>  Labels: lhf
> Fix For: 3.x
>
> Attachments: trunk-10149.txt
>
>
> Currently when using nodetool cfstats and cfhistograms (for a keyspace and 
> table) have different syntax.  cfstats uses "keyspace table".  cfhistograms 
> uses "keyspace.table".  We should unify it one way or the other (or both).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9879) Add the name of the compressor in the output of sstablemetadata

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041652#comment-15041652
 ] 

Philip Thompson commented on CASSANDRA-9879:


Sorry for the long delay in reviewing this patch. Our review queue wasn't 
catching Patch Available tickets without an assignee. What version is this 
patch targeted for?

> Add the name of the compressor in the output of sstablemetadata
> ---
>
> Key: CASSANDRA-9879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9879
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Nicolas Lalevée
>Priority: Trivial
> Attachments: 
> 0001-add-compressor-name-in-the-output-of-sstablemetadata.patch
>
>
> Here is a simple patch to add to the output of sstablemetadata the name the 
> compressor used for the sstable.
> I also made sstablemetadata embedded into the .deb distribution



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10476) Fix upgrade paging dtest failures on 2.2->3.0 path

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041682#comment-15041682
 ] 

Jim Witschey commented on CASSANDRA-10476:
--

[~blerer] we actually do keep artifacts, but only 20 builds back for disk space 
concerns. I'll see if there's anything I can do about that for this job.

Also, just a note, this could be related to CASSANDRA-10730.

> Fix upgrade paging dtest failures on 2.2->3.0 path
> --
>
> Key: CASSANDRA-10476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10476
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Benjamin Lerer
> Fix For: 3.0.1, 3.1
>
>
> EDIT: this list of failures is no longer current; see comments for current 
> failures.
> The following upgrade tests for paging features fail or flap on the upgrade 
> path from 2.2 to 3.0:
> - {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingSize.test_undefined_page_size_default}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingSize.test_with_more_results_than_page_size}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_multiple_cell_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_cell_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_row_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingDatasetChanges.test_cell_TTL_expiry_during_paging/}}
> I've grouped them all together because I don't know how to tell if they're 
> related; once someone triages them, it may be appropriate to break this out 
> into multiple tickets.
> The failures can be found here:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingData/static_columns_paging_test/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingSize/test_undefined_page_size_default/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/42/testReport/upgrade_tests.paging_test/TestPagingSize/test_with_more_results_than_page_size/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_multiple_cell_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_single_cell_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_single_row_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingDatasetChanges/test_cell_TTL_expiry_during_paging/
> Once [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/586] is 
> merged, these tests should also run with this upgrade path on normal 3.0 
> jobs. Until then, you can run them with the following command:
> {code}
> SKIP=false CASSANDRA_VERSION=binary:2.2.0 UPGRADE_TO=git:cassandra-3.0 
> nosetests 
> upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test 
> upgrade_tests/paging_test.py:TestPagingSize.test_undefined_page_size_default 
> upgrade_tests/paging_test.py:TestPagingSize.test_with_more_results_than_page_size
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_multiple_cell_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_cell_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_row_deletions
> upgrade_tests/paging_test.py:TestPagingDatasetChanges.test_cell_TTL_expiry_during_paging
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10666) jmx_test.TestJMX.test_compactionstats is flapping

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-10666:

Fix Version/s: (was: 3.0.1)
   (was: 3.1)
   3.x
   3.0.x

> jmx_test.TestJMX.test_compactionstats is flapping
> -
>
> Key: CASSANDRA-10666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10666
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> See the [history for that 
> test|http://cassci.datastax.com/job/cassandra-3.0_dtest/335/testReport/junit/jmx_test/TestJMX/test_compactionstats/history/].
>  On each failure there is something about a problem with {{jolokia}} so 
> that's probably a test environment problem. Still needs to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) On dropped mutations, more details should be logged.

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10580:

Reviewer: Paulo Motta

> On dropped mutations, more details should be logged.
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
> Environment: Production
>Reporter: Anubhav Kale
>Priority: Minor
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-10580-Head.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10816) Explicitly handle SSL handshake errors during connect()

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10816:

Fix Version/s: 2.2.x
   2.1.x

> Explicitly handle SSL handshake errors during connect()
> ---
>
> Key: CASSANDRA-10816
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10816
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x, 2.2.x
>
>
> Diagnosing internode SSL issues can be a problem as any messaging 
> {{IOException}} before this patch is just logged to debug, which is likely 
> not enabled in most cases. Logs will just show nothing in case of any 
> problems with SSL certs. 
> Also the implemented retry semantics in {{OutboundTcpConnection}} will not 
> make much sense for SSL handshake errors and will cause unnecessary load for 
> both peers.
> The proposed patch will explicitly catch {{SSLHandshakeException}}, log them 
> to error and abort connect().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-9258:

Reviewer: Branimir Lambov  (was: Aleksey Yeschenko)

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: 0001-pending-ranges.patch
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) On dropped mutations, more details should be logged.

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10580:

Assignee: Anubhav Kale

> On dropped mutations, more details should be logged.
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-10580-Head.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10764) Cassandra Daemon should print JVM arguments

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10764:

Fix Version/s: (was: 2.1.12)
   2.1.x

> Cassandra Daemon should print JVM arguments
> ---
>
> Key: CASSANDRA-10764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10764
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
> Environment: PROD
>Reporter: Anubhav Kale
>Priority: Minor
> Fix For: 2.1.x
>
> Attachments: 10764.patch
>
>
> While debugging, its useful to check if Cassandra indeed is picking up the 
> JVM args supplied to it. Therefore, we should print those on startup.
> Patch is attached, and straightforward.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) On dropped mutations, more details should be logged.

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10580:

Component/s: Coordination

> On dropped mutations, more details should be logged.
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-10580-Head.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10770) Warning TIOStreamTransport.java:112 - Error closing output stream.

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10770:

Labels: thrift  (was: Thrifth)

> Warning TIOStreamTransport.java:112 - Error closing output stream.
> --
>
> Key: CASSANDRA-10770
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10770
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.11, ubuntu 12.04
>Reporter: jean carlo rivera ura
>  Labels: thrift
> Attachments: warning-thrift.txt
>
>
> Hi,  In our cluster cassandra of 6 nodes we are having many times the next 
> warning
> WARN  [Thrift:2477] 2015-11-25 10:06:33,795 TIOStreamTransport.java:112 - 
> Error closing output stream.
> java.net.SocketException: Socket closed
>   at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116) 
> ~[na:1.8.0_60]
>   at java.net.SocketOutputStream.write(SocketOutputStream.java:153) 
> ~[na:1.8.0_60]
>   at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) 
> ~[na:1.8.0_60]
>   at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) 
> ~[na:1.8.0_60]
>   at java.io.FilterOutputStream.close(FilterOutputStream.java:158) 
> ~[na:1.8.0_60]
>   at 
> org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
>  ~[libthrift-0.9.2.jar:0.9.2]
>   at 
> org.apache.cassandra.thrift.TCustomSocket.close(TCustomSocket.java:197) 
> [apache-cassandra-2.1.11.jar:2.1.11]
>   at 
> org.apache.thrift.transport.TFramedTransport.close(TFramedTransport.java:89) 
> [libthrift-0.9.2.jar:0.9.2]
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:231)
>  [apache-cassandra-2.1.11.jar:2.1.11]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_60]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_60]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> We are not able to know why cassandra is closing a socket that was close 
> before. This is hapening mostly when cassandra is having high load of writes.
> Is it that a bug?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10476) Fix upgrade paging dtest failures on 2.2->3.0 path

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041566#comment-15041566
 ] 

Benjamin Lerer commented on CASSANDRA-10476:


On my machine, for the request of {{test_single_partition_deletions}}  the 
response time is at maximum 30 ms which is quite far from the 10 s timeout. As 
the range_request_timeout_in_ms is 1 by default the client timeout will be 
trigger before the server one but my guess is that one of the servers is not 
responding. 

To investigate that further we would need the servers logs. [~mambocab] is 
there a way to store the server logs?
I am also suspecting a bit the machine to be overloaded at sometimes. Do we 
monitor the machines used to run the CI?

> Fix upgrade paging dtest failures on 2.2->3.0 path
> --
>
> Key: CASSANDRA-10476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10476
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Benjamin Lerer
> Fix For: 3.0.1, 3.1
>
>
> EDIT: this list of failures is no longer current; see comments for current 
> failures.
> The following upgrade tests for paging features fail or flap on the upgrade 
> path from 2.2 to 3.0:
> - {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingSize.test_undefined_page_size_default}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingSize.test_with_more_results_than_page_size}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_multiple_cell_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_cell_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_row_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingDatasetChanges.test_cell_TTL_expiry_during_paging/}}
> I've grouped them all together because I don't know how to tell if they're 
> related; once someone triages them, it may be appropriate to break this out 
> into multiple tickets.
> The failures can be found here:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingData/static_columns_paging_test/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingSize/test_undefined_page_size_default/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/42/testReport/upgrade_tests.paging_test/TestPagingSize/test_with_more_results_than_page_size/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_multiple_cell_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_single_cell_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_single_row_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingDatasetChanges/test_cell_TTL_expiry_during_paging/
> Once [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/586] is 
> merged, these tests should also run with this upgrade path on normal 3.0 
> jobs. Until then, you can run them with the following command:
> {code}
> SKIP=false CASSANDRA_VERSION=binary:2.2.0 UPGRADE_TO=git:cassandra-3.0 
> nosetests 
> upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test 
> upgrade_tests/paging_test.py:TestPagingSize.test_undefined_page_size_default 
> upgrade_tests/paging_test.py:TestPagingSize.test_with_more_results_than_page_size
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_multiple_cell_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_cell_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_row_deletions
> upgrade_tests/paging_test.py:TestPagingDatasetChanges.test_cell_TTL_expiry_during_paging
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10122) AssertionError after upgrade to 3.0

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10122:
-
Reviewer: Joshua McKenzie

> AssertionError after upgrade to 3.0
> ---
>
> Key: CASSANDRA-10122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10122
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Sylvain Lebresne
> Fix For: 3.0.1, 3.1
>
> Attachments: node1.log, node2.log, node3.log
>
>
> Upgrade tests are encountering this exception after upgrade from 2.2 HEAD to 
> 3.0 HEAD:
> {noformat}
> ERROR [SharedPool-Worker-4] 2015-08-18 12:33:57,858 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xa5ba2c7a, 
> /127.0.0.1:55048 => /127.0.0.1:9042]
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:520)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:461)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[main/:na]
> at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:583)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:733)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:676) 
> ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:659)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:103)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:76)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.AbstractReadExecutor$AlwaysSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:323)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1599)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1554) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1501) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1420) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:457)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:232)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:202)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:72)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:204)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:470)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:447)
>  ~[main/:na]
> at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:139)
>  ~[main/:na]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  [main/:na]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> 

[jira] [Assigned] (CASSANDRA-10099) Improve concurrency in CompactionStrategyManager

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-10099:
---

Assignee: Marcus Eriksson

> Improve concurrency in CompactionStrategyManager
> 
>
> Key: CASSANDRA-10099
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10099
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Yuki Morishita
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.x
>
>
> Continue discussion from CASSANDRA-9882.
> CompactionStrategyManager(WrappingCompactionStrategy for <3.0) tracks SSTable 
> changes mainly for separating repaired / unrepaired SSTables (+ LCS manages 
> level).
> This is blocking operation, and can lead to block of flush etc. when 
> determining next background task takes longer.
> Explore the way to mitigate this concurrency issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10099) Improve concurrency in CompactionStrategyManager

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-10099:

 Reviewer: Yuki Morishita
Fix Version/s: 2.2.x
   2.1.x

> Improve concurrency in CompactionStrategyManager
> 
>
> Key: CASSANDRA-10099
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10099
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Yuki Morishita
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.x
>
>
> Continue discussion from CASSANDRA-9882.
> CompactionStrategyManager(WrappingCompactionStrategy for <3.0) tracks SSTable 
> changes mainly for separating repaired / unrepaired SSTables (+ LCS manages 
> level).
> This is blocking operation, and can lead to block of flush etc. when 
> determining next background task takes longer.
> Explore the way to mitigate this concurrency issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10099) Improve concurrency in CompactionStrategyManager

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041538#comment-15041538
 ] 

Marcus Eriksson commented on CASSANDRA-10099:
-

main reason we synchronize all/most methods in 
WrappingCompactionStrategy/CompactionStrategyManager is that we need to be able 
to reload the actual compaction strategy safely.

Patch [here|https://github.com/krummas/cassandra/commits/marcuse/10099] which 
replaces the synchronized on the methods with a read/write lock.

Note that this probably has a small chance of creating overlap in LCS since we 
don't see all new sstables as a 'transaction' - but this is safe in 2.1+ as 
long as it is very rare since we drop any sstable that would cause overlap to L0

http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10099-testall/
http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10099-dtest/

> Improve concurrency in CompactionStrategyManager
> 
>
> Key: CASSANDRA-10099
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10099
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Yuki Morishita
> Fix For: 3.x
>
>
> Continue discussion from CASSANDRA-9882.
> CompactionStrategyManager(WrappingCompactionStrategy for <3.0) tracks SSTable 
> changes mainly for separating repaired / unrepaired SSTables (+ LCS manages 
> level).
> This is blocking operation, and can lead to block of flush etc. when 
> determining next background task takes longer.
> Explore the way to mitigate this concurrency issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041536#comment-15041536
 ] 

Stefania edited comment on CASSANDRA-10799 at 12/4/15 1:14 PM:
---

I've adapted the cqlshlib tests to always use the embedded driver even when 
there is a driver installed. Two things to note:

* Because the location of the lib folder containing the driver zip file is 
calculated as relative to {{__file__}}, the soft link created was causing 
problems and so I got rid of it. The reason for this soft link was exclusively 
for importing cqlsh and since the renaming to cqlsh.py in 2.2 this is no longer 
necessary. However in 2.1 we cannot get rid of a soft link (or copy) unless we 
rename cqlsh to cqlsh.py and therefore I did not fix 2.1.

* If running the tests from _pylib/cqlshlib_, like we don on Jenkins, 
_cqlshlib_ is added to {{sys.path}} and this caused an error when importing 
{{cassandra.Cluster}} due to a conflict between the python copy module and 
_cqlshlib/copy.py_. This is a serious problem and I fixed it in 2.1 as well. 
The only reason we don't notice when running cqlsh is that we import 
{{cassandra.Cluster}} after adding cqlshlib to {{sys.path}}.

Patches and CI:

||2.1||2.2||3.0||
|[patch|https://github.com/stef1927/cassandra/commits/10799-2.1]|[patch|https://github.com/stef1927/cassandra/commits/10799-2.2]|[patch|https://github.com/stef1927/cassandra/commits/10799-3.0]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-dtest/]|

2.2 applied cleanly to 3.0 and so I did not create the patches for 3.1 and 
trunk but I can do so after review is completed.



was (Author: stefania):
I've adapted the cqlshlib tests to always use the embedded driver even when 
there is a driver installed. Two things to note:

* Because the location of the lib folder containing the driver zip file is 
calculated as relative to {{__file__}}, the soft link created was causing 
problems and so I got rid of it. The reason for this soft link was exclusively 
for importing cqlsh and since the renaming to cqlsh.py in 2.2 this is no longer 
necessary. However in 2.1 we cannot get rid of a soft link (or copy) unless we 
rename cqlsh to cqlsh.py and therefore I did not fix 2.1.

* If running the tests from _pylib/cqlshlib_, like we don on Jenkins, 
_cqlshlib_ is added to {{sys.path}} and this caused an error when importing 
{{cassandra.Cluster}} due to a conflict between the python copy module and 
_cqlshlib/copy.py_. This is a serious problem and I fixed it in 2.1 as well. 
The only reason we don't notice when running cqlsh is that we import 
{{cassandra.Cluster}} after adding cqlshlib to {{sys.path}}.

Patches and CI:

||2.1||2.2||3.0||
|[patch|https://github.com/stef1927/cassandra/commits/10799-2.1]|[patch|https://github.com/stef1927/cassandra/commits/10799-2.2]|[patch|https://github.com/stef1927/cassandra/commits/10799-3.0]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-dtest/]|

2.2 applied cleanly to 3.0 and so I did not create the patches for 3.1 and 
trunk but I can do so after review is completed.


> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
>  Issue Type: Test
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the 
> driver is not cythonized (installed with {{--no_extensions}}) then the tests 
> are fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10816) Explicitly handle SSL handshake errors during connect()

[Stefan Podkowinski (JIRA)]

Stefan Podkowinski created CASSANDRA-10816:
--

 Summary: Explicitly handle SSL handshake errors during connect()
 Key: CASSANDRA-10816
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10816
 Project: Cassandra
  Issue Type: Bug
  Components: Streaming and Messaging
Reporter: Stefan Podkowinski
Assignee: Stefan Podkowinski


Diagnosing internode SSL issues can be a problem as any messaging 
{{IOException}} before this patch is just logged to debug, which is likely not 
enabled in most cases. Logs will just show nothing in case of any problems with 
SSL certs. 

Also the implemented retry semantics in {{OutboundTcpConnection}} will not make 
much sense for SSL handshake errors and will cause unnecessary load for both 
peers.

The proposed patch will explicitly catch {{SSLHandshakeException}}, log them to 
error and abort connect().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-3.1' into trunk

[marcuse]

Merge branch 'cassandra-3.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52d5eb04
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52d5eb04
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52d5eb04

Branch: refs/heads/trunk
Commit: 52d5eb04f625fffa1855f7e7d3e38eff79ea1828
Parents: 220f253 d1a596a
Author: Marcus Eriksson 
Authored: Fri Dec 4 14:27:26 2015 +0100
Committer: Marcus Eriksson 
Committed: Fri Dec 4 14:27:26 2015 +0100

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SystemKeyspace.java | 10 +---
 .../apache/cassandra/db/SystemKeyspaceTest.java | 27 ++--
 3 files changed, 33 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52d5eb04/CHANGES.txt
--
diff --cc CHANGES.txt
index 711bdde,87d2e82..1607a66
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,19 -1,6 +1,20 @@@
 +3.2
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.1
  Merged from 3.0:
+  * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
   * Fix bug with range tombstones on reverse queries and test coverage for
 AbstractBTreePartition (CASSANDRA-10059)
   * Remove 64k limit on collection elements (CASSANDRA-10374)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/52d5eb04/src/java/org/apache/cassandra/db/SystemKeyspace.java
--

[1/2] cassandra git commit: Fix issue with data dir migration on upgrade

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.1 fd5cb43da -> d1a596a4a


Fix issue with data dir migration on upgrade

Patch by Stefania; reviewed by marcuse for CASSANDRA-10788


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d0da9e76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d0da9e76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d0da9e76

Branch: refs/heads/cassandra-3.1
Commit: d0da9e7612e6fb94fea8298a88215783a8125311
Parents: 1e978df
Author: Stefania Alborghetti 
Authored: Wed Dec 2 09:45:34 2015 +0800
Committer: Marcus Eriksson 
Committed: Fri Dec 4 14:22:26 2015 +0100

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SystemKeyspace.java | 10 +---
 .../apache/cassandra/db/SystemKeyspaceTest.java | 27 ++--
 3 files changed, 33 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2527c43..520afff 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.1
+ * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
  * Fix bug with range tombstones on reverse queries and test coverage for
AbstractBTreePartition (CASSANDRA-10059)
  * Remove 64k limit on collection elements (CASSANDRA-10374)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java 
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 83c8c1c..8a27c9d 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -1324,14 +1324,18 @@ public final class SystemKeyspace
 Iterable dirs = 
Arrays.asList(DatabaseDescriptor.getAllDataFileLocations());
 for (String dataDir : dirs)
 {
-logger.trace("Checking directory {} for old files", dataDir);
+logger.trace("Checking {} for old files", dataDir);
 File dir = new File(dataDir);
 assert dir.exists() : dir + " should have been created by startup 
checks";
 
-for (File ksdir : dir.listFiles((d, n) -> d.isDirectory()))
+for (File ksdir : dir.listFiles((d, n) -> new File(d, 
n).isDirectory()))
 {
-for (File cfdir : ksdir.listFiles((d, n) -> d.isDirectory()))
+logger.trace("Checking {} for old files", ksdir);
+
+for (File cfdir : ksdir.listFiles((d, n) -> new File(d, 
n).isDirectory()))
 {
+logger.trace("Checking {} for old files", cfdir);
+
 if (Descriptor.isLegacyFile(cfdir))
 {
 FileUtils.deleteRecursive(cfdir);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java 
b/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
index 9fc3503..d770610 100644
--- a/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
+++ b/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
@@ -153,6 +153,29 @@ public class SystemKeyspaceTest
 }
 
 @Test
+public void testMigrateEmptyDataDirs() throws IOException
+{
+File dataDir = 
Paths.get(DatabaseDescriptor.getAllDataFileLocations()[0]).toFile();
+if (new File(dataDir, "Emptykeyspace1").exists())
+FileUtils.deleteDirectory(new File(dataDir, "Emptykeyspace1"));
+assertTrue(new File(dataDir, "Emptykeyspace1").mkdirs());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+assertTrue(new File(dataDir, "Emptykeyspace1/table1").mkdirs());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+assertTrue(new File(dataDir, 
"Emptykeyspace1/wrong_file").createNewFile());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+}
+
+@Test
 public void testMigrateDataDirs_2_1() throws IOException
 {
 testMigrateDataDirs("2.1");
@@ -185,9 +208,9 @@ public class SystemKeyspaceTest
 for (String dataDir : dirs)
 {
 File dir = new 

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[marcuse]

Merge branch 'cassandra-3.0' into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d1a596a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d1a596a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d1a596a4

Branch: refs/heads/cassandra-3.1
Commit: d1a596a4ad6717f03917877fc50714bafa55de97
Parents: fd5cb43 d0da9e7
Author: Marcus Eriksson 
Authored: Fri Dec 4 14:27:14 2015 +0100
Committer: Marcus Eriksson 
Committed: Fri Dec 4 14:27:14 2015 +0100

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SystemKeyspace.java | 10 +---
 .../apache/cassandra/db/SystemKeyspaceTest.java | 27 ++--
 3 files changed, 33 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d1a596a4/CHANGES.txt
--
diff --cc CHANGES.txt
index 7a6076b,520afff..87d2e82
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,5 +1,6 @@@
 -3.0.1
 +3.1
 +Merged from 3.0:
+  * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
   * Fix bug with range tombstones on reverse queries and test coverage for
 AbstractBTreePartition (CASSANDRA-10059)
   * Remove 64k limit on collection elements (CASSANDRA-10374)

[1/3] cassandra git commit: Fix issue with data dir migration on upgrade

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 220f253fb -> 52d5eb04f


Fix issue with data dir migration on upgrade

Patch by Stefania; reviewed by marcuse for CASSANDRA-10788


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d0da9e76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d0da9e76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d0da9e76

Branch: refs/heads/trunk
Commit: d0da9e7612e6fb94fea8298a88215783a8125311
Parents: 1e978df
Author: Stefania Alborghetti 
Authored: Wed Dec 2 09:45:34 2015 +0800
Committer: Marcus Eriksson 
Committed: Fri Dec 4 14:22:26 2015 +0100

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SystemKeyspace.java | 10 +---
 .../apache/cassandra/db/SystemKeyspaceTest.java | 27 ++--
 3 files changed, 33 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2527c43..520afff 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.1
+ * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
  * Fix bug with range tombstones on reverse queries and test coverage for
AbstractBTreePartition (CASSANDRA-10059)
  * Remove 64k limit on collection elements (CASSANDRA-10374)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java 
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 83c8c1c..8a27c9d 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -1324,14 +1324,18 @@ public final class SystemKeyspace
 Iterable dirs = 
Arrays.asList(DatabaseDescriptor.getAllDataFileLocations());
 for (String dataDir : dirs)
 {
-logger.trace("Checking directory {} for old files", dataDir);
+logger.trace("Checking {} for old files", dataDir);
 File dir = new File(dataDir);
 assert dir.exists() : dir + " should have been created by startup 
checks";
 
-for (File ksdir : dir.listFiles((d, n) -> d.isDirectory()))
+for (File ksdir : dir.listFiles((d, n) -> new File(d, 
n).isDirectory()))
 {
-for (File cfdir : ksdir.listFiles((d, n) -> d.isDirectory()))
+logger.trace("Checking {} for old files", ksdir);
+
+for (File cfdir : ksdir.listFiles((d, n) -> new File(d, 
n).isDirectory()))
 {
+logger.trace("Checking {} for old files", cfdir);
+
 if (Descriptor.isLegacyFile(cfdir))
 {
 FileUtils.deleteRecursive(cfdir);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java 
b/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
index 9fc3503..d770610 100644
--- a/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
+++ b/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
@@ -153,6 +153,29 @@ public class SystemKeyspaceTest
 }
 
 @Test
+public void testMigrateEmptyDataDirs() throws IOException
+{
+File dataDir = 
Paths.get(DatabaseDescriptor.getAllDataFileLocations()[0]).toFile();
+if (new File(dataDir, "Emptykeyspace1").exists())
+FileUtils.deleteDirectory(new File(dataDir, "Emptykeyspace1"));
+assertTrue(new File(dataDir, "Emptykeyspace1").mkdirs());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+assertTrue(new File(dataDir, "Emptykeyspace1/table1").mkdirs());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+assertTrue(new File(dataDir, 
"Emptykeyspace1/wrong_file").createNewFile());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+}
+
+@Test
 public void testMigrateDataDirs_2_1() throws IOException
 {
 testMigrateDataDirs("2.1");
@@ -185,9 +208,9 @@ public class SystemKeyspaceTest
 for (String dataDir : dirs)
 {
 File dir = new File(dataDir);
-

[2/3] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[marcuse]

Merge branch 'cassandra-3.0' into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d1a596a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d1a596a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d1a596a4

Branch: refs/heads/trunk
Commit: d1a596a4ad6717f03917877fc50714bafa55de97
Parents: fd5cb43 d0da9e7
Author: Marcus Eriksson 
Authored: Fri Dec 4 14:27:14 2015 +0100
Committer: Marcus Eriksson 
Committed: Fri Dec 4 14:27:14 2015 +0100

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SystemKeyspace.java | 10 +---
 .../apache/cassandra/db/SystemKeyspaceTest.java | 27 ++--
 3 files changed, 33 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d1a596a4/CHANGES.txt
--
diff --cc CHANGES.txt
index 7a6076b,520afff..87d2e82
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,5 +1,6 @@@
 -3.0.1
 +3.1
 +Merged from 3.0:
+  * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
   * Fix bug with range tombstones on reverse queries and test coverage for
 AbstractBTreePartition (CASSANDRA-10059)
   * Remove 64k limit on collection elements (CASSANDRA-10374)

[jira] [Created] (CASSANDRA-10817) DROP USER is not case-sensitive

[Mike Adamson (JIRA)]

Mike Adamson created CASSANDRA-10817:


 Summary: DROP USER is not case-sensitive
 Key: CASSANDRA-10817
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10817
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
Reporter: Mike Adamson
Priority: Minor
 Fix For: 2.2.4


As per the summary {{DROP USER}} is not case sensitive, so:
{noformat}
CREATE USER 'Test';
LIST USERS;
 name  | super
---+---
  Test | False
 cassandra |  True
DROP USER 'Test';
InvalidRequest: code=2200 [Invalid query] message="test doesn't exist"
{noformat}
{{DROP ROLE}} is case-sensitive and will drop the above user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10122) AssertionError after upgrade to 3.0

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041593#comment-15041593
 ] 

Sylvain Lebresne commented on CASSANDRA-10122:
--

What the trace (which is btw different from the initial trace the ticket was 
created for but lets ignore that) is telling us is that 
{{ReadCommand.LegacyReadCommandSerializer.serializedSize}} is called with a 
{{version < MessagingService.VERSION_30}}. Except that the only time a 
{{ReadCommand.LegacyReadCommandSerializer}} is even created is in 
{{SinglePartitionReadCommand.createMessage}} with this:
{noformat}
return new MessageOut<>(MessagingService.Verb.READ, this, version < 
MessagingService.VERSION_30 ? legacyReadCommandSerializer : serializer);
{noformat}
so we shouldn't get that error. This suggest that the code somehow doesn't 
always pass the same version when creating a message and later serializing it, 
and indeed, in {{AbstractReadExecutor.makeRequests}}, we pass the version for 
the message creation, but we reuse the message for all endpoints without 
checking that the version is the same for all of them, which is obviously 
wrong. So I've pushed a simple fix for that 
[here|https://github.com/pcmanus/cassandra/commits/10122] (we potentially 
re-create the same message a few times, but it's pretty cheap and the number of 
replica is very small, so it's not worth micro-optimizing imo).

It's pretty obvious from the stack that this is the problem but I haven't 
validated that it fixes the {{upgrade_through_versions_test}} because running 
that test locally is ... (deep breathing) ... problematic. Because:
* you can't pass the local branch you want to the test, it expects a version 
number and build the branch name from that. That means I'd presumably have to 
commit work-in-progress stuff to my local {{cassandra-3.0}} branch to make the 
test happy, but that's quite error-prone.
* it then errors out basically because I haven't exported {{CASSANDRA_DIR}}. 
Easily fixed, but it would be great if the test could reuse the general dtest 
mechanism for that (which checks {{~/.cassandra_dtest}} for that).
* it then errors out with
{noformat}
ERROR: Failure: ValueError (too many values to unpack)
--
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 420, in 
loadTestsFromName
addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in 
importFromPath
return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in 
importFromDir
mod = load_module(part_fqname, fh, filename, desc)
  File "/home/pcmanus/Git/cassandra-dtest/upgrade_through_versions_test.py", 
line 63, in 
_, ref_type, ref = _fullref.split('/')
ValueError: too many values to unpack
{noformat}
That's because it iterates over all the repo git refs and expect all of them to 
be of the form {{x/y/z}}. Sadly I had some that are {{x/y/z/w}}.
* once you've fixed that, it errors out with:
{noformat}
ERROR: rolling_upgrade_test 
(upgrade_through_versions_test.TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD)
--
Traceback (most recent call last):
  File "/home/pcmanus/Git/cassandra-dtest/upgrade_through_versions_test.py", 
line 904, in setUp
switch_jdks(os.environ['CASSANDRA_VERSION'])
  File "/usr/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
KeyError: 'CASSANDRA_VERSION'
 >> begin captured logging << 
{noformat}
that is, it complains you haven't exported {{CASSANDRA_VERSION}}, which is 
ironic cause the test has at the beginning the following lines:
{noformat}
if os.environ.get('CASSANDRA_VERSION'):
debug('CASSANDRA_VERSION is not used by upgrade tests!')
{noformat}
which actually make sense: if you upgrade through version, what version are you 
supposed to set. I ended up commenting all calls to {{switch_jdks}} as it's not 
need for upgrading from 2.1, but it seems that method would also require you to 
manually set {{JAVA7_HOME}} and {{JAVA8_HOME}}. It would be great if it wasn't 
necessary to set up tons of environment variable to have the test running.
* with that fixed, the test appears to actually start running, but it quickly 
errors out with a bunch of error messages like
{noformat}
ServerError: 
{noformat}
which suggest something is wrong with how branches are compiled.

I gave up at that point, and maybe I was particularly unlucky/messed up 
something, but it would be great if we could it more friendly to run this test 
locally (and having it work) ([~rhatch]).

> AssertionError after upgrade to 3.0
> ---
>
> Key: CASSANDRA-10122
> URL: 

[jira] [Commented] (CASSANDRA-10799) 2 cqlshlib tests still failing with cythonized driver installation

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041536#comment-15041536
 ] 

Stefania commented on CASSANDRA-10799:
--

I've adapted the cqlshlib tests to always use the embedded driver even when 
there is a driver installed. Two things to note:

* Because the location of the lib folder containing the driver zip file is 
calculated as relative to {{__file__}}, the soft link created was causing 
problems and so I got rid of it. The reason for this soft link was exclusively 
for importing cqlsh and since the renaming to cqlsh.py in 2.2 this is no longer 
necessary. However in 2.1 we cannot get rid of a soft link (or copy) unless we 
rename cqlsh to cqlsh.py and therefore I did not fix 2.1.

* If running the tests from _pylib/cqlshlib_, like we don on Jenkins, 
_cqlshlib_ is added to {{sys.path}} and this caused an error when importing 
{{cassandra.Cluster}} due to a conflict between the python copy module and 
_cqlshlib/copy.py_. This is a serious problem and I fixed it in 2.1 as well. 
The only reason we don't notice when running cqlsh is that we import 
{{cassandra.Cluster}} after adding cqlshlib to {{sys.path}}.

Patches and CI:

||2.1||2.2||3.0||
|[patch|https://github.com/stef1927/cassandra/commits/10799-2.1]|[patch|https://github.com/stef1927/cassandra/commits/10799-2.2]|[patch|https://github.com/stef1927/cassandra/commits/10799-3.0]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10799-3.0-dtest/]|

2.2 applied cleanly to 3.0 and so I did not create the patches for 3.1 and 
trunk but I can do so after review is completed.


> 2 cqlshlib tests still failing with cythonized driver installation
> --
>
> Key: CASSANDRA-10799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10799
> Project: Cassandra
>  Issue Type: Test
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.2.x, 3.x
>
>
> We still have 2 cqlshlib tests failing on Jenkins:
> http://cassci.datastax.com/job/cassandra-3.0_cqlshlib/lastCompletedBuild/testReport/
> Locally, these tests only fail with a cythonized driver installation. If the 
> driver is not cythonized (installed with {{--no_extensions}}) then the tests 
> are fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix issue with data dir migration on upgrade

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 1e978df3a -> d0da9e761


Fix issue with data dir migration on upgrade

Patch by Stefania; reviewed by marcuse for CASSANDRA-10788


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d0da9e76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d0da9e76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d0da9e76

Branch: refs/heads/cassandra-3.0
Commit: d0da9e7612e6fb94fea8298a88215783a8125311
Parents: 1e978df
Author: Stefania Alborghetti 
Authored: Wed Dec 2 09:45:34 2015 +0800
Committer: Marcus Eriksson 
Committed: Fri Dec 4 14:22:26 2015 +0100

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SystemKeyspace.java | 10 +---
 .../apache/cassandra/db/SystemKeyspaceTest.java | 27 ++--
 3 files changed, 33 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2527c43..520afff 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.1
+ * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
  * Fix bug with range tombstones on reverse queries and test coverage for
AbstractBTreePartition (CASSANDRA-10059)
  * Remove 64k limit on collection elements (CASSANDRA-10374)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java 
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 83c8c1c..8a27c9d 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -1324,14 +1324,18 @@ public final class SystemKeyspace
 Iterable dirs = 
Arrays.asList(DatabaseDescriptor.getAllDataFileLocations());
 for (String dataDir : dirs)
 {
-logger.trace("Checking directory {} for old files", dataDir);
+logger.trace("Checking {} for old files", dataDir);
 File dir = new File(dataDir);
 assert dir.exists() : dir + " should have been created by startup 
checks";
 
-for (File ksdir : dir.listFiles((d, n) -> d.isDirectory()))
+for (File ksdir : dir.listFiles((d, n) -> new File(d, 
n).isDirectory()))
 {
-for (File cfdir : ksdir.listFiles((d, n) -> d.isDirectory()))
+logger.trace("Checking {} for old files", ksdir);
+
+for (File cfdir : ksdir.listFiles((d, n) -> new File(d, 
n).isDirectory()))
 {
+logger.trace("Checking {} for old files", cfdir);
+
 if (Descriptor.isLegacyFile(cfdir))
 {
 FileUtils.deleteRecursive(cfdir);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0da9e76/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java 
b/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
index 9fc3503..d770610 100644
--- a/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
+++ b/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java
@@ -153,6 +153,29 @@ public class SystemKeyspaceTest
 }
 
 @Test
+public void testMigrateEmptyDataDirs() throws IOException
+{
+File dataDir = 
Paths.get(DatabaseDescriptor.getAllDataFileLocations()[0]).toFile();
+if (new File(dataDir, "Emptykeyspace1").exists())
+FileUtils.deleteDirectory(new File(dataDir, "Emptykeyspace1"));
+assertTrue(new File(dataDir, "Emptykeyspace1").mkdirs());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+assertTrue(new File(dataDir, "Emptykeyspace1/table1").mkdirs());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+assertTrue(new File(dataDir, 
"Emptykeyspace1/wrong_file").createNewFile());
+assertEquals(0, numLegacyFiles());
+SystemKeyspace.migrateDataDirs();
+assertEquals(0, numLegacyFiles());
+
+}
+
+@Test
 public void testMigrateDataDirs_2_1() throws IOException
 {
 testMigrateDataDirs("2.1");
@@ -185,9 +208,9 @@ public class SystemKeyspaceTest
 for (String dataDir : dirs)
 {
 File dir = new 

[jira] [Resolved] (CASSANDRA-10815) WrappingCompactionStrategy can block others when there are lots of sstables

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson resolved CASSANDRA-10815.
-
   Resolution: Duplicate
Fix Version/s: (was: 2.2.x)
   (was: 2.1.x)
   (was: 3.x)

> WrappingCompactionStrategy can block others when there are lots of sstables
> ---
>
> Key: CASSANDRA-10815
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10815
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Severin Leonhardt
>Assignee: Marcus Eriksson
>
> We observe {{nodetool compactionstats}} hanging when there are a lot of 
> SSTables in one table. We have about 30.000 SSTables most likely created by 
> an incremental repair (why it's that many is still a mystery to us).
> Looking at the stacktraces of some selected threads it becomes apparent that 
> a single CompactionExecutor blocks several other threads:
> {noformat}
> "CompactionExecutor:4065" - Thread t@282454
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.binarySort(TimSort.java:292)
>   at java.util.TimSort.sort(TimSort.java:235)
>   at java.util.Arrays.sort(Arrays.java:1512)
>   at java.util.ArrayList.sort(ArrayList.java:1454)
>   at java.util.Collections.sort(Collections.java:175)
>   at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.trimToThresholdWithHotness(SizeTieredCompactionStrategy.java:139)
>   at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.mostInterestingBucket(SizeTieredCompactionStrategy.java:119)
>   at 
> org.apache.cassandra.db.compaction.LeveledManifest.getSSTablesForSTCS(LeveledManifest.java:360)
>   at 
> org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:318)
>   - locked <25446e7b> (a 
> org.apache.cassandra.db.compaction.LeveledManifest)
>   at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:101)
>   at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:90)
>   - locked <33506169> (a 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:77)
>   - locked <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Because this thread has locked <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) the following 
> threads are blocked:
> {noformat}
> "CompactionExecutor:4064" - Thread t@282337
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:72)
>   - waiting to lock <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:4065" t@282454
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> {noformat}
> "MemtableFlushWriter:2431" - Thread t@283411
>java.lang.Thread.State: BLOCKED
>   at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:250)
>   - waiting to lock <31924612> (a 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy) owned by 
> "CompactionExecutor:4065" t@282454
>   at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
>   at 
> org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
>   at 
> 

[jira] [Commented] (CASSANDRA-10477) java.lang.AssertionError in StorageProxy.submitHint

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041786#comment-15041786
 ] 

Ariel Weisberg commented on CASSANDRA-10477:


bq. We're kind of dodging the hint "overload" protection on the paxos path as 
we don't use sendToHintedEndpoints (which in particular makes the comment on 
commitPaxosLocal misleading as it suggests otherwise). I think the simplest 
solution is to move the overload test from sendToHintedEndpoints to some 
checkOverloaded() method and call that in commitPaxos too.
Which aspect of hint "overload" protection is missing? [I see it increments a 
counter which I thought was the signal 
upstream.|https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageProxy.java#L976]

Looking at it further is it because it doesn't throw {{OverloadedException}}? 
So a better behavior would be to have the check and exception in a helper 
method and use that in commitPaxos() so that it can now throw 
{{OverloadedException}}?

I do wonder what the unforeseen consequences of having {{CAS}} capable of 
throwing {{OE}} is going to do that we haven't seen or tested before. Where 
this gets interesting is that the read path now throws {{OE}} where it didn't 
before because apparently serial consistency reads can end up calling 
{{beginAndRepairPaxos}}. I need to take a close look at how we test this path 
to make sure it's going to behave well once exercised.

bq. In theory, we could still run into the problem of that ticket if 
OPTIMIZE_LOCAL_REQUESTS is false. And in fact, I believe this option is unsafe 
since at least CASSANDRA-4753 as we somewhat strongly assume writes to the 
localhost do not go through MessagingService. So I would suggest ditching that 
option. Not only is it unsafe, but it's not used anywhere by the code and it's 
hardcoded so you have to change the code and recompile to even use it (which 
means I doubt anyone has even tried it in a long long time). And if we end up 
needing it in the future, we'll have to figure out how to make it safe.
It's already removed from 2.2. Yeah I don't think anyone uses it.

bq. Why isn't the added assertion in WriteCallbackInfo on 3.0 not using 
!shouldHint lie in the 2.1 patch?
It's an oversight from merging.

> java.lang.AssertionError in StorageProxy.submitHint
> ---
>
> Key: CASSANDRA-10477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10477
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
> Environment: CentOS 6, Oracle JVM 1.8.45
>Reporter: Severin Leonhardt
>Assignee: Ariel Weisberg
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> A few days after updating from 2.0.15 to 2.1.9 we have the following log 
> entry on 2 of 5 machines:
> {noformat}
> ERROR [EXPIRING-MAP-REAPER:1] 2015-10-07 17:01:08,041 
> CassandraDaemon.java:223 - Exception in thread 
> Thread[EXPIRING-MAP-REAPER:1,5,main]
> java.lang.AssertionError: /192.168.11.88
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:949) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.net.MessagingService$5.apply(MessagingService.java:383) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.net.MessagingService$5.apply(MessagingService.java:363) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at org.apache.cassandra.utils.ExpiringMap$1.run(ExpiringMap.java:98) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_45]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {noformat}
> 192.168.11.88 is the broadcast address of the local machine.
> When this is logged the read request latency of the whole cluster becomes 
> very bad, from 6 ms/op to more than 100 ms/op according to OpsCenter. Clients 
> get a lot of timeouts. We need to restart the affected Cassandra node to 

[jira] [Commented] (CASSANDRA-10816) Explicitly handle SSL handshake errors during connect()

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041821#comment-15041821
 ] 

Ariel Weisberg commented on CASSANDRA-10816:


I only made a 3.0 branch just to get the tests running. Will address fix 
versions after review.
|[3.0 
code|https://github.com/apache/cassandra/compare/cassandra-3.0...aweisberg:CASSANDRA-10816?expand=1]|[utests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-10816-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-10816-dtest/]|

> Explicitly handle SSL handshake errors during connect()
> ---
>
> Key: CASSANDRA-10816
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10816
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x, 2.2.x
>
>
> Diagnosing internode SSL issues can be a problem as any messaging 
> {{IOException}} before this patch is just logged to debug, which is likely 
> not enabled in most cases. Logs will just show nothing in case of any 
> problems with SSL certs. 
> Also the implemented retry semantics in {{OutboundTcpConnection}} will not 
> make much sense for SSL handshake errors and will cause unnecessary load for 
> both peers.
> The proposed patch will explicitly catch {{SSLHandshakeException}}, log them 
> to error and abort connect().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Handle single-col deletions in MVs correctly

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 d0da9e761 -> 83f8ccc66


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/test/unit/org/apache/cassandra/cql3/ViewTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/ViewTest.java 
b/test/unit/org/apache/cassandra/cql3/ViewTest.java
index 74c6374..101eb3d 100644
--- a/test/unit/org/apache/cassandra/cql3/ViewTest.java
+++ b/test/unit/org/apache/cassandra/cql3/ViewTest.java
@@ -37,13 +37,9 @@ import org.apache.cassandra.concurrent.Stage;
 import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
-import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.db.ColumnFamilyStore;
 import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.db.SystemKeyspace;
-import org.apache.cassandra.serializers.SimpleDateSerializer;
-import org.apache.cassandra.serializers.TimeSerializer;
-import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 import org.junit.After;
 import org.junit.Before;
@@ -54,6 +50,8 @@ import com.datastax.driver.core.ResultSet;
 import com.datastax.driver.core.Row;
 import com.datastax.driver.core.exceptions.InvalidQueryException;
 
+import static org.junit.Assert.assertTrue;
+
 public class ViewTest extends CQLTester
 {
 int protocolVersion = 4;
@@ -106,45 +104,6 @@ public class ViewTest extends CQLTester
 }
 
 @Test
-public void testCaseSensitivity() throws Throwable
-{
-createTable("CREATE TABLE %s (\"theKey\" int, \"theClustering\" int, 
\"theValue\" int, PRIMARY KEY (\"theKey\", \"theClustering\"))");
-
-execute("USE " + keyspace());
-executeNet(protocolVersion, "USE " + keyspace());
-
-execute("INSERT INTO %s (\"theKey\", \"theClustering\", \"theValue\") 
VALUES (?, ?, ?)", 0, 0, 0);
-
-createView("mv_test", "CREATE MATERIALIZED VIEW %s AS SELECT * FROM 
%%s " +
-  "WHERE \"theKey\" IS NOT NULL AND 
\"theClustering\" IS NOT NULL AND \"theValue\" IS NOT NULL " +
-  "PRIMARY KEY (\"theKey\", \"theClustering\")");
-
-while (!SystemKeyspace.isViewBuilt(keyspace(), "mv_test"))
-Thread.sleep(10);
-createView("mv_test2", "CREATE MATERIALIZED VIEW %s AS SELECT 
\"theKey\", \"theClustering\", \"theValue\" FROM %%s " +
-   "WHERE \"theKey\" IS NOT NULL AND 
\"theClustering\" IS NOT NULL AND \"theValue\" IS NOT NULL " +
-   "PRIMARY KEY (\"theKey\", \"theClustering\")");
-while (!SystemKeyspace.isViewBuilt(keyspace(), "mv_test2"))
-Thread.sleep(10);
-
-for (String mvname : Arrays.asList("mv_test", "mv_test2"))
-{
-assertRows(execute("SELECT \"theKey\", \"theClustering\", 
\"theValue\" FROM " + mvname),
-   row(0, 0, 0)
-);
-}
-
-executeNet(protocolVersion, "ALTER TABLE %s RENAME \"theClustering\" 
TO \"Col\"");
-
-for (String mvname : Arrays.asList("mv_test", "mv_test2"))
-{
-assertRows(execute("SELECT \"theKey\", \"Col\", \"theValue\" FROM 
" + mvname),
-   row(0, 0, 0)
-);
-}
-}
-
-@Test
 public void testPartitionTombstone() throws Throwable
 {
 createTable("CREATE TABLE %s (k1 int, c1 int , val int, PRIMARY KEY 
(k1))");
@@ -241,67 +200,6 @@ public class ViewTest extends CQLTester
 }
 
 @Test
-public void testAccessAndSchema() throws Throwable
-{
-createTable("CREATE TABLE %s (" +
-"k int, " +
-"asciival ascii, " +
-"bigintval bigint, " +
-"PRIMARY KEY((k, asciival)))");
-
-execute("USE " + keyspace());
-executeNet(protocolVersion, "USE " + keyspace());
-
-createView("mv1_test", "CREATE MATERIALIZED VIEW %s AS SELECT * FROM 
%%s WHERE bigintval IS NOT NULL AND k IS NOT NULL AND asciival IS NOT NULL 
PRIMARY KEY (bigintval, k, asciival)");
-updateView("INSERT INTO %s(k,asciival,bigintval)VALUES(?,?,?)", 0, 
"foo", 1L);
-
-try
-{
-updateView("INSERT INTO mv1_test(k,asciival,bigintval) 
VALUES(?,?,?)", 1, "foo", 2L);
-Assert.fail("Shouldn't be able to modify a MV directly");
-}
-catch (Exception e)
-{
-}
-
-try
-{
-executeNet(protocolVersion, "ALTER TABLE mv1_test ADD foo text");
-Assert.fail("Should not be able to use alter table with MV");
-}
-catch (Exception e)
-{
-}
-
-try
-{
-executeNet(protocolVersion, "ALTER TABLE mv1_test WITH compaction 
= { 'class' : 'LeveledCompactionStrategy' }");
-   

[2/2] cassandra git commit: Handle single-col deletions in MVs correctly

[tylerhobbs]

Handle single-col deletions in MVs correctly

Patch by Tyler Hobbs; reviewed by Carl Yeksigian for CASSANDRA-10796


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83f8ccc6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83f8ccc6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83f8ccc6

Branch: refs/heads/cassandra-3.0
Commit: 83f8ccc6685bf81a5264c3dfa1969ce061d2bc61
Parents: d0da9e7
Author: Tyler Hobbs 
Authored: Fri Dec 4 11:42:13 2015 -0600
Committer: Tyler Hobbs 
Committed: Fri Dec 4 11:42:13 2015 -0600

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/rows/AbstractCell.java  |   5 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   6 +-
 .../apache/cassandra/db/view/TemporalRow.java   |   6 +-
 src/java/org/apache/cassandra/db/view/View.java |  15 +-
 .../apache/cassandra/db/view/ViewManager.java   |   4 +
 .../apache/cassandra/service/StorageProxy.java  |  14 +-
 .../apache/cassandra/cql3/ViewSchemaTest.java   | 762 +++
 .../org/apache/cassandra/cql3/ViewTest.java | 707 +
 9 files changed, 838 insertions(+), 683 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 520afff..c22d138 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.1
+ * Handle single-column deletions correction in materialized views
+   when the column is part of the view primary key (CASSANDRA-10796)
  * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
  * Fix bug with range tombstones on reverse queries and test coverage for
AbstractBTreePartition (CASSANDRA-10059)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/rows/AbstractCell.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/AbstractCell.java 
b/src/java/org/apache/cassandra/db/rows/AbstractCell.java
index e804b7a..882c0e0 100644
--- a/src/java/org/apache/cassandra/db/rows/AbstractCell.java
+++ b/src/java/org/apache/cassandra/db/rows/AbstractCell.java
@@ -110,7 +110,10 @@ public abstract class AbstractCell extends Cell
  ct.valueComparator().getString(value()),
  livenessInfoString());
 }
-return String.format("[%s=%s %s]", column().name, 
type.getString(value()), livenessInfoString());
+if (isTombstone())
+return String.format("[%s= %s]", column().name, 
livenessInfoString());
+else
+return String.format("[%s=%s %s]", column().name, 
type.getString(value()), livenessInfoString());
 }
 
 private String livenessInfoString()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/rows/AbstractRow.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/AbstractRow.java 
b/src/java/org/apache/cassandra/db/rows/AbstractRow.java
index 8958575..484f981 100644
--- a/src/java/org/apache/cassandra/db/rows/AbstractRow.java
+++ b/src/java/org/apache/cassandra/db/rows/AbstractRow.java
@@ -126,7 +126,11 @@ public abstract class AbstractRow extends 
AbstractCollection impleme
 if (cd.column().isSimple())
 {
 Cell cell = (Cell)cd;
-
sb.append(cell.column().name).append('=').append(cell.column().type.getString(cell.value()));
+sb.append(cell.column().name).append('=');
+if (cell.isTombstone())
+sb.append("");
+else
+sb.append(cell.column().type.getString(cell.value()));
 }
 else
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/view/TemporalRow.java
--
diff --git a/src/java/org/apache/cassandra/db/view/TemporalRow.java 
b/src/java/org/apache/cassandra/db/view/TemporalRow.java
index 01da6e7..8898857 100644
--- a/src/java/org/apache/cassandra/db/view/TemporalRow.java
+++ b/src/java/org/apache/cassandra/db/view/TemporalRow.java
@@ -426,7 +426,11 @@ public class TemporalRow
 
 Collection val = 
values(definition, resolver);
 if (val != null && val.size() == 1)
-return Iterables.getOnlyElement(val).value();
+{
+org.apache.cassandra.db.rows.Cell cell = 
Iterables.getOnlyElement(val);
+ 

[1/3] cassandra git commit: Handle single-col deletions in MVs correctly

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.1 d1a596a4a -> f0d45ea37


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/test/unit/org/apache/cassandra/cql3/ViewTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/ViewTest.java 
b/test/unit/org/apache/cassandra/cql3/ViewTest.java
index 74c6374..101eb3d 100644
--- a/test/unit/org/apache/cassandra/cql3/ViewTest.java
+++ b/test/unit/org/apache/cassandra/cql3/ViewTest.java
@@ -37,13 +37,9 @@ import org.apache.cassandra.concurrent.Stage;
 import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
-import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.db.ColumnFamilyStore;
 import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.db.SystemKeyspace;
-import org.apache.cassandra.serializers.SimpleDateSerializer;
-import org.apache.cassandra.serializers.TimeSerializer;
-import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 import org.junit.After;
 import org.junit.Before;
@@ -54,6 +50,8 @@ import com.datastax.driver.core.ResultSet;
 import com.datastax.driver.core.Row;
 import com.datastax.driver.core.exceptions.InvalidQueryException;
 
+import static org.junit.Assert.assertTrue;
+
 public class ViewTest extends CQLTester
 {
 int protocolVersion = 4;
@@ -106,45 +104,6 @@ public class ViewTest extends CQLTester
 }
 
 @Test
-public void testCaseSensitivity() throws Throwable
-{
-createTable("CREATE TABLE %s (\"theKey\" int, \"theClustering\" int, 
\"theValue\" int, PRIMARY KEY (\"theKey\", \"theClustering\"))");
-
-execute("USE " + keyspace());
-executeNet(protocolVersion, "USE " + keyspace());
-
-execute("INSERT INTO %s (\"theKey\", \"theClustering\", \"theValue\") 
VALUES (?, ?, ?)", 0, 0, 0);
-
-createView("mv_test", "CREATE MATERIALIZED VIEW %s AS SELECT * FROM 
%%s " +
-  "WHERE \"theKey\" IS NOT NULL AND 
\"theClustering\" IS NOT NULL AND \"theValue\" IS NOT NULL " +
-  "PRIMARY KEY (\"theKey\", \"theClustering\")");
-
-while (!SystemKeyspace.isViewBuilt(keyspace(), "mv_test"))
-Thread.sleep(10);
-createView("mv_test2", "CREATE MATERIALIZED VIEW %s AS SELECT 
\"theKey\", \"theClustering\", \"theValue\" FROM %%s " +
-   "WHERE \"theKey\" IS NOT NULL AND 
\"theClustering\" IS NOT NULL AND \"theValue\" IS NOT NULL " +
-   "PRIMARY KEY (\"theKey\", \"theClustering\")");
-while (!SystemKeyspace.isViewBuilt(keyspace(), "mv_test2"))
-Thread.sleep(10);
-
-for (String mvname : Arrays.asList("mv_test", "mv_test2"))
-{
-assertRows(execute("SELECT \"theKey\", \"theClustering\", 
\"theValue\" FROM " + mvname),
-   row(0, 0, 0)
-);
-}
-
-executeNet(protocolVersion, "ALTER TABLE %s RENAME \"theClustering\" 
TO \"Col\"");
-
-for (String mvname : Arrays.asList("mv_test", "mv_test2"))
-{
-assertRows(execute("SELECT \"theKey\", \"Col\", \"theValue\" FROM 
" + mvname),
-   row(0, 0, 0)
-);
-}
-}
-
-@Test
 public void testPartitionTombstone() throws Throwable
 {
 createTable("CREATE TABLE %s (k1 int, c1 int , val int, PRIMARY KEY 
(k1))");
@@ -241,67 +200,6 @@ public class ViewTest extends CQLTester
 }
 
 @Test
-public void testAccessAndSchema() throws Throwable
-{
-createTable("CREATE TABLE %s (" +
-"k int, " +
-"asciival ascii, " +
-"bigintval bigint, " +
-"PRIMARY KEY((k, asciival)))");
-
-execute("USE " + keyspace());
-executeNet(protocolVersion, "USE " + keyspace());
-
-createView("mv1_test", "CREATE MATERIALIZED VIEW %s AS SELECT * FROM 
%%s WHERE bigintval IS NOT NULL AND k IS NOT NULL AND asciival IS NOT NULL 
PRIMARY KEY (bigintval, k, asciival)");
-updateView("INSERT INTO %s(k,asciival,bigintval)VALUES(?,?,?)", 0, 
"foo", 1L);
-
-try
-{
-updateView("INSERT INTO mv1_test(k,asciival,bigintval) 
VALUES(?,?,?)", 1, "foo", 2L);
-Assert.fail("Shouldn't be able to modify a MV directly");
-}
-catch (Exception e)
-{
-}
-
-try
-{
-executeNet(protocolVersion, "ALTER TABLE mv1_test ADD foo text");
-Assert.fail("Should not be able to use alter table with MV");
-}
-catch (Exception e)
-{
-}
-
-try
-{
-executeNet(protocolVersion, "ALTER TABLE mv1_test WITH compaction 
= { 'class' : 'LeveledCompactionStrategy' }");
-   

[jira] [Commented] (CASSANDRA-10122) AssertionError after upgrade to 3.0

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041780#comment-15041780
 ] 

Russ Hatch commented on CASSANDRA-10122:


bq. which is btw different from the initial trace the ticket was created for 
but lets ignore that

Sorry for missing that

bq. because running that test locally is ... (deep breathing) ... problematic

We're looking to make these tests better or possibly just replacing them 
entirely with something like Fallout. Sorry for the pain there.

> AssertionError after upgrade to 3.0
> ---
>
> Key: CASSANDRA-10122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10122
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Sylvain Lebresne
> Fix For: 3.0.1, 3.1
>
> Attachments: node1.log, node2.log, node3.log
>
>
> Upgrade tests are encountering this exception after upgrade from 2.2 HEAD to 
> 3.0 HEAD:
> {noformat}
> ERROR [SharedPool-Worker-4] 2015-08-18 12:33:57,858 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xa5ba2c7a, 
> /127.0.0.1:55048 => /127.0.0.1:9042]
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:520)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:461)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[main/:na]
> at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:583)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:733)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:676) 
> ~[main/:na]
> at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:659)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:103)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:76)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.AbstractReadExecutor$AlwaysSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:323)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1599)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1554) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1501) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1420) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:457)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:232)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:202)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:72)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:204)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:470)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:447)
>  ~[main/:na]
> at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:139)
>  ~[main/:na]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 

[jira] [Commented] (CASSANDRA-9302) Optimize cqlsh COPY FROM, part 3

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041827#comment-15041827
 ] 

Joshua McKenzie commented on CASSANDRA-9302:


[~aholmber] to review.

> Optimize cqlsh COPY FROM, part 3
> 
>
> Key: CASSANDRA-9302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9302
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> We've had some discussion moving to Spark CSV import for bulk load in 3.x, 
> but people need a good bulk load tool now.  One option is to add a separate 
> Java bulk load tool (CASSANDRA-9048), but if we can match that performance 
> from cqlsh I would prefer to leave COPY FROM as the preferred option to which 
> we point people, rather than adding more tools that need to be supported 
> indefinitely.
> Previous work on COPY FROM optimization was done in CASSANDRA-7405 and 
> CASSANDRA-8225.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/4] cassandra git commit: Handle single-col deletions in MVs correctly

[tylerhobbs]

Handle single-col deletions in MVs correctly

Patch by Tyler Hobbs; reviewed by Carl Yeksigian for CASSANDRA-10796


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83f8ccc6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83f8ccc6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83f8ccc6

Branch: refs/heads/trunk
Commit: 83f8ccc6685bf81a5264c3dfa1969ce061d2bc61
Parents: d0da9e7
Author: Tyler Hobbs 
Authored: Fri Dec 4 11:42:13 2015 -0600
Committer: Tyler Hobbs 
Committed: Fri Dec 4 11:42:13 2015 -0600

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/rows/AbstractCell.java  |   5 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   6 +-
 .../apache/cassandra/db/view/TemporalRow.java   |   6 +-
 src/java/org/apache/cassandra/db/view/View.java |  15 +-
 .../apache/cassandra/db/view/ViewManager.java   |   4 +
 .../apache/cassandra/service/StorageProxy.java  |  14 +-
 .../apache/cassandra/cql3/ViewSchemaTest.java   | 762 +++
 .../org/apache/cassandra/cql3/ViewTest.java | 707 +
 9 files changed, 838 insertions(+), 683 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 520afff..c22d138 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.1
+ * Handle single-column deletions correction in materialized views
+   when the column is part of the view primary key (CASSANDRA-10796)
  * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
  * Fix bug with range tombstones on reverse queries and test coverage for
AbstractBTreePartition (CASSANDRA-10059)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/rows/AbstractCell.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/AbstractCell.java 
b/src/java/org/apache/cassandra/db/rows/AbstractCell.java
index e804b7a..882c0e0 100644
--- a/src/java/org/apache/cassandra/db/rows/AbstractCell.java
+++ b/src/java/org/apache/cassandra/db/rows/AbstractCell.java
@@ -110,7 +110,10 @@ public abstract class AbstractCell extends Cell
  ct.valueComparator().getString(value()),
  livenessInfoString());
 }
-return String.format("[%s=%s %s]", column().name, 
type.getString(value()), livenessInfoString());
+if (isTombstone())
+return String.format("[%s= %s]", column().name, 
livenessInfoString());
+else
+return String.format("[%s=%s %s]", column().name, 
type.getString(value()), livenessInfoString());
 }
 
 private String livenessInfoString()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/rows/AbstractRow.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/AbstractRow.java 
b/src/java/org/apache/cassandra/db/rows/AbstractRow.java
index 8958575..484f981 100644
--- a/src/java/org/apache/cassandra/db/rows/AbstractRow.java
+++ b/src/java/org/apache/cassandra/db/rows/AbstractRow.java
@@ -126,7 +126,11 @@ public abstract class AbstractRow extends 
AbstractCollection impleme
 if (cd.column().isSimple())
 {
 Cell cell = (Cell)cd;
-
sb.append(cell.column().name).append('=').append(cell.column().type.getString(cell.value()));
+sb.append(cell.column().name).append('=');
+if (cell.isTombstone())
+sb.append("");
+else
+sb.append(cell.column().type.getString(cell.value()));
 }
 else
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/view/TemporalRow.java
--
diff --git a/src/java/org/apache/cassandra/db/view/TemporalRow.java 
b/src/java/org/apache/cassandra/db/view/TemporalRow.java
index 01da6e7..8898857 100644
--- a/src/java/org/apache/cassandra/db/view/TemporalRow.java
+++ b/src/java/org/apache/cassandra/db/view/TemporalRow.java
@@ -426,7 +426,11 @@ public class TemporalRow
 
 Collection val = 
values(definition, resolver);
 if (val != null && val.size() == 1)
-return Iterables.getOnlyElement(val).value();
+{
+org.apache.cassandra.db.rows.Cell cell = 
Iterables.getOnlyElement(val);
+// 

[jira] [Updated] (CASSANDRA-9302) Optimize cqlsh COPY FROM, part 3

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9302:
---
Reviewer: Adam Holmberg  (was: Tyler Hobbs)

> Optimize cqlsh COPY FROM, part 3
> 
>
> Key: CASSANDRA-9302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9302
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> We've had some discussion moving to Spark CSV import for bulk load in 3.x, 
> but people need a good bulk load tool now.  One option is to add a separate 
> Java bulk load tool (CASSANDRA-9048), but if we can match that performance 
> from cqlsh I would prefer to leave COPY FROM as the preferred option to which 
> we point people, rather than adding more tools that need to be supported 
> indefinitely.
> Previous work on COPY FROM optimization was done in CASSANDRA-7405 and 
> CASSANDRA-8225.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Handle single-col deletions in MVs correctly

[tylerhobbs]

Handle single-col deletions in MVs correctly

Patch by Tyler Hobbs; reviewed by Carl Yeksigian for CASSANDRA-10796


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83f8ccc6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83f8ccc6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83f8ccc6

Branch: refs/heads/cassandra-3.1
Commit: 83f8ccc6685bf81a5264c3dfa1969ce061d2bc61
Parents: d0da9e7
Author: Tyler Hobbs 
Authored: Fri Dec 4 11:42:13 2015 -0600
Committer: Tyler Hobbs 
Committed: Fri Dec 4 11:42:13 2015 -0600

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/rows/AbstractCell.java  |   5 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   6 +-
 .../apache/cassandra/db/view/TemporalRow.java   |   6 +-
 src/java/org/apache/cassandra/db/view/View.java |  15 +-
 .../apache/cassandra/db/view/ViewManager.java   |   4 +
 .../apache/cassandra/service/StorageProxy.java  |  14 +-
 .../apache/cassandra/cql3/ViewSchemaTest.java   | 762 +++
 .../org/apache/cassandra/cql3/ViewTest.java | 707 +
 9 files changed, 838 insertions(+), 683 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 520afff..c22d138 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 3.0.1
+ * Handle single-column deletions correction in materialized views
+   when the column is part of the view primary key (CASSANDRA-10796)
  * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
  * Fix bug with range tombstones on reverse queries and test coverage for
AbstractBTreePartition (CASSANDRA-10059)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/rows/AbstractCell.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/AbstractCell.java 
b/src/java/org/apache/cassandra/db/rows/AbstractCell.java
index e804b7a..882c0e0 100644
--- a/src/java/org/apache/cassandra/db/rows/AbstractCell.java
+++ b/src/java/org/apache/cassandra/db/rows/AbstractCell.java
@@ -110,7 +110,10 @@ public abstract class AbstractCell extends Cell
  ct.valueComparator().getString(value()),
  livenessInfoString());
 }
-return String.format("[%s=%s %s]", column().name, 
type.getString(value()), livenessInfoString());
+if (isTombstone())
+return String.format("[%s= %s]", column().name, 
livenessInfoString());
+else
+return String.format("[%s=%s %s]", column().name, 
type.getString(value()), livenessInfoString());
 }
 
 private String livenessInfoString()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/rows/AbstractRow.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/AbstractRow.java 
b/src/java/org/apache/cassandra/db/rows/AbstractRow.java
index 8958575..484f981 100644
--- a/src/java/org/apache/cassandra/db/rows/AbstractRow.java
+++ b/src/java/org/apache/cassandra/db/rows/AbstractRow.java
@@ -126,7 +126,11 @@ public abstract class AbstractRow extends 
AbstractCollection impleme
 if (cd.column().isSimple())
 {
 Cell cell = (Cell)cd;
-
sb.append(cell.column().name).append('=').append(cell.column().type.getString(cell.value()));
+sb.append(cell.column().name).append('=');
+if (cell.isTombstone())
+sb.append("");
+else
+sb.append(cell.column().type.getString(cell.value()));
 }
 else
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/src/java/org/apache/cassandra/db/view/TemporalRow.java
--
diff --git a/src/java/org/apache/cassandra/db/view/TemporalRow.java 
b/src/java/org/apache/cassandra/db/view/TemporalRow.java
index 01da6e7..8898857 100644
--- a/src/java/org/apache/cassandra/db/view/TemporalRow.java
+++ b/src/java/org/apache/cassandra/db/view/TemporalRow.java
@@ -426,7 +426,11 @@ public class TemporalRow
 
 Collection val = 
values(definition, resolver);
 if (val != null && val.size() == 1)
-return Iterables.getOnlyElement(val).value();
+{
+org.apache.cassandra.db.rows.Cell cell = 
Iterables.getOnlyElement(val);
+ 

[3/4] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[tylerhobbs]

Merge branch 'cassandra-3.0' into cassandra-3.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0d45ea3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0d45ea3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0d45ea3

Branch: refs/heads/trunk
Commit: f0d45ea378ce010c2a5f807cd560c540a1d4ae46
Parents: d1a596a 83f8ccc
Author: Tyler Hobbs 
Authored: Fri Dec 4 11:43:22 2015 -0600
Committer: Tyler Hobbs 
Committed: Fri Dec 4 11:43:22 2015 -0600

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/rows/AbstractCell.java  |   5 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   6 +-
 .../apache/cassandra/db/view/TemporalRow.java   |   6 +-
 src/java/org/apache/cassandra/db/view/View.java |  15 +-
 .../apache/cassandra/db/view/ViewManager.java   |   4 +
 .../apache/cassandra/service/StorageProxy.java  |  14 +-
 .../apache/cassandra/cql3/ViewSchemaTest.java   | 762 +++
 .../org/apache/cassandra/cql3/ViewTest.java | 707 +
 9 files changed, 838 insertions(+), 683 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d45ea3/CHANGES.txt
--
diff --cc CHANGES.txt
index 87d2e82,c22d138..7c7e301
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,6 +1,7 @@@
 -3.0.1
 +3.1
 +Merged from 3.0:
+  * Handle single-column deletions correction in materialized views
+when the column is part of the view primary key (CASSANDRA-10796)
   * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
   * Fix bug with range tombstones on reverse queries and test coverage for
 AbstractBTreePartition (CASSANDRA-10059)

[1/4] cassandra git commit: Handle single-col deletions in MVs correctly

[tylerhobbs]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 1c41a9ac2 -> 4a566139d


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83f8ccc6/test/unit/org/apache/cassandra/cql3/ViewTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/ViewTest.java 
b/test/unit/org/apache/cassandra/cql3/ViewTest.java
index 74c6374..101eb3d 100644
--- a/test/unit/org/apache/cassandra/cql3/ViewTest.java
+++ b/test/unit/org/apache/cassandra/cql3/ViewTest.java
@@ -37,13 +37,9 @@ import org.apache.cassandra.concurrent.Stage;
 import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
-import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.db.ColumnFamilyStore;
 import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.db.SystemKeyspace;
-import org.apache.cassandra.serializers.SimpleDateSerializer;
-import org.apache.cassandra.serializers.TimeSerializer;
-import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 import org.junit.After;
 import org.junit.Before;
@@ -54,6 +50,8 @@ import com.datastax.driver.core.ResultSet;
 import com.datastax.driver.core.Row;
 import com.datastax.driver.core.exceptions.InvalidQueryException;
 
+import static org.junit.Assert.assertTrue;
+
 public class ViewTest extends CQLTester
 {
 int protocolVersion = 4;
@@ -106,45 +104,6 @@ public class ViewTest extends CQLTester
 }
 
 @Test
-public void testCaseSensitivity() throws Throwable
-{
-createTable("CREATE TABLE %s (\"theKey\" int, \"theClustering\" int, 
\"theValue\" int, PRIMARY KEY (\"theKey\", \"theClustering\"))");
-
-execute("USE " + keyspace());
-executeNet(protocolVersion, "USE " + keyspace());
-
-execute("INSERT INTO %s (\"theKey\", \"theClustering\", \"theValue\") 
VALUES (?, ?, ?)", 0, 0, 0);
-
-createView("mv_test", "CREATE MATERIALIZED VIEW %s AS SELECT * FROM 
%%s " +
-  "WHERE \"theKey\" IS NOT NULL AND 
\"theClustering\" IS NOT NULL AND \"theValue\" IS NOT NULL " +
-  "PRIMARY KEY (\"theKey\", \"theClustering\")");
-
-while (!SystemKeyspace.isViewBuilt(keyspace(), "mv_test"))
-Thread.sleep(10);
-createView("mv_test2", "CREATE MATERIALIZED VIEW %s AS SELECT 
\"theKey\", \"theClustering\", \"theValue\" FROM %%s " +
-   "WHERE \"theKey\" IS NOT NULL AND 
\"theClustering\" IS NOT NULL AND \"theValue\" IS NOT NULL " +
-   "PRIMARY KEY (\"theKey\", \"theClustering\")");
-while (!SystemKeyspace.isViewBuilt(keyspace(), "mv_test2"))
-Thread.sleep(10);
-
-for (String mvname : Arrays.asList("mv_test", "mv_test2"))
-{
-assertRows(execute("SELECT \"theKey\", \"theClustering\", 
\"theValue\" FROM " + mvname),
-   row(0, 0, 0)
-);
-}
-
-executeNet(protocolVersion, "ALTER TABLE %s RENAME \"theClustering\" 
TO \"Col\"");
-
-for (String mvname : Arrays.asList("mv_test", "mv_test2"))
-{
-assertRows(execute("SELECT \"theKey\", \"Col\", \"theValue\" FROM 
" + mvname),
-   row(0, 0, 0)
-);
-}
-}
-
-@Test
 public void testPartitionTombstone() throws Throwable
 {
 createTable("CREATE TABLE %s (k1 int, c1 int , val int, PRIMARY KEY 
(k1))");
@@ -241,67 +200,6 @@ public class ViewTest extends CQLTester
 }
 
 @Test
-public void testAccessAndSchema() throws Throwable
-{
-createTable("CREATE TABLE %s (" +
-"k int, " +
-"asciival ascii, " +
-"bigintval bigint, " +
-"PRIMARY KEY((k, asciival)))");
-
-execute("USE " + keyspace());
-executeNet(protocolVersion, "USE " + keyspace());
-
-createView("mv1_test", "CREATE MATERIALIZED VIEW %s AS SELECT * FROM 
%%s WHERE bigintval IS NOT NULL AND k IS NOT NULL AND asciival IS NOT NULL 
PRIMARY KEY (bigintval, k, asciival)");
-updateView("INSERT INTO %s(k,asciival,bigintval)VALUES(?,?,?)", 0, 
"foo", 1L);
-
-try
-{
-updateView("INSERT INTO mv1_test(k,asciival,bigintval) 
VALUES(?,?,?)", 1, "foo", 2L);
-Assert.fail("Shouldn't be able to modify a MV directly");
-}
-catch (Exception e)
-{
-}
-
-try
-{
-executeNet(protocolVersion, "ALTER TABLE mv1_test ADD foo text");
-Assert.fail("Should not be able to use alter table with MV");
-}
-catch (Exception e)
-{
-}
-
-try
-{
-executeNet(protocolVersion, "ALTER TABLE mv1_test WITH compaction 
= { 'class' : 'LeveledCompactionStrategy' }");
-

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[tylerhobbs]

Merge branch 'cassandra-3.0' into cassandra-3.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0d45ea3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0d45ea3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0d45ea3

Branch: refs/heads/cassandra-3.1
Commit: f0d45ea378ce010c2a5f807cd560c540a1d4ae46
Parents: d1a596a 83f8ccc
Author: Tyler Hobbs 
Authored: Fri Dec 4 11:43:22 2015 -0600
Committer: Tyler Hobbs 
Committed: Fri Dec 4 11:43:22 2015 -0600

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/rows/AbstractCell.java  |   5 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   6 +-
 .../apache/cassandra/db/view/TemporalRow.java   |   6 +-
 src/java/org/apache/cassandra/db/view/View.java |  15 +-
 .../apache/cassandra/db/view/ViewManager.java   |   4 +
 .../apache/cassandra/service/StorageProxy.java  |  14 +-
 .../apache/cassandra/cql3/ViewSchemaTest.java   | 762 +++
 .../org/apache/cassandra/cql3/ViewTest.java | 707 +
 9 files changed, 838 insertions(+), 683 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0d45ea3/CHANGES.txt
--
diff --cc CHANGES.txt
index 87d2e82,c22d138..7c7e301
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,6 +1,7 @@@
 -3.0.1
 +3.1
 +Merged from 3.0:
+  * Handle single-column deletions correction in materialized views
+when the column is part of the view primary key (CASSANDRA-10796)
   * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
   * Fix bug with range tombstones on reverse queries and test coverage for
 AbstractBTreePartition (CASSANDRA-10059)

[4/4] cassandra git commit: Merge branch 'cassandra-3.1' into trunk

[tylerhobbs]

Merge branch 'cassandra-3.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a566139
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a566139
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a566139

Branch: refs/heads/trunk
Commit: 4a566139d1763453c01ad297ab98ec633e7befd1
Parents: 1c41a9a f0d45ea
Author: Tyler Hobbs 
Authored: Fri Dec 4 11:43:44 2015 -0600
Committer: Tyler Hobbs 
Committed: Fri Dec 4 11:43:44 2015 -0600

--
 CHANGES.txt |   2 +
 .../apache/cassandra/db/rows/AbstractCell.java  |   5 +-
 .../apache/cassandra/db/rows/AbstractRow.java   |   6 +-
 .../apache/cassandra/db/view/TemporalRow.java   |   6 +-
 src/java/org/apache/cassandra/db/view/View.java |  15 +-
 .../apache/cassandra/db/view/ViewManager.java   |   4 +
 .../apache/cassandra/service/StorageProxy.java  |  14 +-
 .../apache/cassandra/cql3/ViewSchemaTest.java   | 762 +++
 .../org/apache/cassandra/cql3/ViewTest.java | 707 +
 9 files changed, 838 insertions(+), 683 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a566139/CHANGES.txt
--
diff --cc CHANGES.txt
index 6c46183,7c7e301..532a77a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,7 +1,22 @@@
 +3.2
 + * bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(CASSANDRA-8639)
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.1
  Merged from 3.0:
+  * Handle single-column deletions correction in materialized views
+when the column is part of the view primary key (CASSANDRA-10796)
   * Fix issue with datadir migration on upgrade (CASSANDRA-10788)
   * Fix bug with range tombstones on reverse queries and test coverage for
 AbstractBTreePartition (CASSANDRA-10059)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a566139/src/java/org/apache/cassandra/db/view/View.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a566139/src/java/org/apache/cassandra/service/StorageProxy.java
--

[jira] [Commented] (CASSANDRA-8831) Create a system table to expose prepared statements

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041853#comment-15041853
 ] 

Joshua McKenzie commented on CASSANDRA-8831:


Moving from Ready to Commit back to In Progress as I don't see +1 from reviewer 
and it was changed 2 days ago by Anonymous.

Not entirely sure what that's about.

> Create a system table to expose prepared statements
> ---
>
> Key: CASSANDRA-8831
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8831
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Robert Stupp
>  Labels: client-impacting, docs-impacting
> Fix For: 3.x
>
>
> Because drivers abstract from users the handling of up/down nodes, they have 
> to deal with the fact that when a node is restarted (or join), it won't know 
> any prepared statement. Drivers could somewhat ignore that problem and wait 
> for a query to return an error (that the statement is unknown by the node) to 
> re-prepare the query on that node, but it's relatively inefficient because 
> every time a node comes back up, you'll get bad latency spikes due to some 
> queries first failing, then being re-prepared and then only being executed. 
> So instead, drivers (at least the java driver but I believe others do as 
> well) pro-actively re-prepare statements when a node comes up. It solves the 
> latency problem, but currently every driver instance blindly re-prepare all 
> statements, meaning that in a large cluster with many clients there is a lot 
> of duplication of work (it would be enough for a single client to prepare the 
> statements) and a bigger than necessary load on the node that started.
> An idea to solve this it to have a (cheap) way for clients to check if some 
> statements are prepared on the node. There is different options to provide 
> that but what I'd suggest is to add a system table to expose the (cached) 
> prepared statements because:
> # it's reasonably straightforward to implement: we just add a line to the 
> table when a statement is prepared and remove it when it's evicted (we 
> already have eviction listeners). We'd also truncate the table on startup but 
> that's easy enough). We can even switch it to a "virtual table" if/when 
> CASSANDRA-7622 lands but it's trivial to do with a normal table in the 
> meantime.
> # it doesn't require a change to the protocol or something like that. It 
> could even be done in 2.1 if we wish to.
> # exposing prepared statements feels like a genuinely useful information to 
> have (outside of the problem exposed here that is), if only for 
> debugging/educational purposes.
> The exposed table could look something like:
> {noformat}
> CREATE TABLE system.prepared_statements (
>keyspace_name text,
>table_name text,
>prepared_id blob,
>query_string text,
>PRIMARY KEY (keyspace_name, table_name, prepared_id)
> )
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10794) System table name resource_role_permissons_index is spelt wrong!

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10794:
-
Assignee: Sam Tunnicliffe

> System table name resource_role_permissons_index is spelt wrong!
> 
>
> Key: CASSANDRA-10794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Max Bowsher
>Assignee: Sam Tunnicliffe
>Priority: Trivial
> Fix For: 3.0.x, 3.x
>
>
> System table name resource_role_permissons_index is spelt wrong!
> "permissons" is missing an "i"
> Fixing that isn't going to be fun, though :-(



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-9752) incremental repair dtest flaps on 2.2

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey reopened CASSANDRA-9752:
-

> incremental repair dtest flaps on 2.2 
> --
>
> Key: CASSANDRA-9752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9752
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>
> {{incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test}} 
> flaps on 2.2. It's hard to tell what failures are repair-specific, but there 
> are a few distinct failures I've seen recently:
> - [an NPE in 
> StorageService|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/143/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
> - [an NPE in 
> SSTableRewriter|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/135/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/].
>  I believe this is related to CASSANDRA-9730, but someone should confirm this.
> - [an on-disk data size that is too 
> large|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/133/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
> You can find the test itself [here on 
> GitHub|https://github.com/riptano/cassandra-dtest/blob/master/incremental_repair_test.py#L206]
>  and run it with the command
> {code}
> CASSANDRA_VERSION=git:trunk nosetests 
> incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test
> {code}
> Assigning [~yukim], since you're the repair person, but feel free to reassign 
> to whoever's appropriate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10793) ohc should be in the all-pom section of build.xml

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10793:
-
Assignee: Robert Stupp

> ohc should be in the all-pom section of build.xml
> -
>
> Key: CASSANDRA-10793
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10793
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Reporter: Jeremiah Jordan
>Assignee: Robert Stupp
> Fix For: 3.0.x
>
>
> ohc-core/ohc-core-j8 should be included in the  section of the build.xml.  Otherwise when getting cassandra-all from maven 
> row cache doesn't work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9752) incremental repair dtest flaps on 2.2

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041692#comment-15041692
 ] 

Jim Witschey commented on CASSANDRA-9752:
-

It's skipped, marked as hanging CI:

http://cassci.datastax.com/view/trunk/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/

Reopening.

> incremental repair dtest flaps on 2.2 
> --
>
> Key: CASSANDRA-9752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9752
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>
> {{incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test}} 
> flaps on 2.2. It's hard to tell what failures are repair-specific, but there 
> are a few distinct failures I've seen recently:
> - [an NPE in 
> StorageService|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/143/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
> - [an NPE in 
> SSTableRewriter|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/135/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/].
>  I believe this is related to CASSANDRA-9730, but someone should confirm this.
> - [an on-disk data size that is too 
> large|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/133/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
> You can find the test itself [here on 
> GitHub|https://github.com/riptano/cassandra-dtest/blob/master/incremental_repair_test.py#L206]
>  and run it with the command
> {code}
> CASSANDRA_VERSION=git:trunk nosetests 
> incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test
> {code}
> Assigning [~yukim], since you're the repair person, but feel free to reassign 
> to whoever's appropriate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10794) System table name resource_role_permissons_index is spelt wrong!

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041694#comment-15041694
 ] 

Sylvain Lebresne commented on CASSANDRA-10794:
--

[~beobal] I don't suppose we can change that without break shit everywhere?

> System table name resource_role_permissons_index is spelt wrong!
> 
>
> Key: CASSANDRA-10794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Max Bowsher
>Assignee: Sam Tunnicliffe
>Priority: Trivial
> Fix For: 3.0.x, 3.x
>
>
> System table name resource_role_permissons_index is spelt wrong!
> "permissons" is missing an "i"
> Fixing that isn't going to be fun, though :-(



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10817) DROP USER is not case-sensitive

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10817:
-
Assignee: Marcus Eriksson

> DROP USER is not case-sensitive
> ---
>
> Key: CASSANDRA-10817
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10817
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Mike Adamson
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 2.2.x
>
>
> As per the summary {{DROP USER}} is not case sensitive, so:
> {noformat}
> CREATE USER 'Test';
> LIST USERS;
>  name  | super
> ---+---
>   Test | False
>  cassandra |  True
> DROP USER 'Test';
> InvalidRequest: code=2200 [Invalid query] message="test doesn't exist"
> {noformat}
> {{DROP ROLE}} is case-sensitive and will drop the above user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10816) Explicitly handle SSL handshake errors during connect()

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10816:

Reviewer: Ariel Weisberg

> Explicitly handle SSL handshake errors during connect()
> ---
>
> Key: CASSANDRA-10816
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10816
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x, 2.2.x
>
>
> Diagnosing internode SSL issues can be a problem as any messaging 
> {{IOException}} before this patch is just logged to debug, which is likely 
> not enabled in most cases. Logs will just show nothing in case of any 
> problems with SSL certs. 
> Also the implemented retry semantics in {{OutboundTcpConnection}} will not 
> make much sense for SSL handshake errors and will cause unnecessary load for 
> both peers.
> The proposed patch will explicitly catch {{SSLHandshakeException}}, log them 
> to error and abort connect().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7950) Output of nodetool compactionstats and compactionhistory does not work well with long keyspace and column family names.

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7950:
---
Component/s: Tools

> Output of nodetool compactionstats and compactionhistory does not work well 
> with long keyspace and column family names.  
> -
>
> Key: CASSANDRA-7950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7950
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
> Environment: CentOS 5, 64bit, Oracle JDK 7, DSE
>Reporter: Eugene
>Assignee: Michael Shuler
>Priority: Minor
>  Labels: lhf
> Fix For: 2.1.x
>
> Attachments: 7950.patch, nodetool-examples.txt
>
>
> When running these commands:
> nodetool compactionstats
> nodetool compactionhistory
> The output can be difficult to grok due to long keyspace names, column family 
> names, and long values.  I have attached an example.
> It's difficult for both humans and grep/sed/awk/perl to read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8142) prevent the command "cassandra start" from starting a cluster

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-8142:
---
Reviewer: Michael Shuler

> prevent the command "cassandra start" from starting a cluster
> -
>
> Key: CASSANDRA-8142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8142
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Steven Lowenthal
>Assignee: Robert Stupp
> Fix For: 3.x
>
>
> Students often type 
> "sudo service cassandra start" wrong, and type "sudo cassandra start".
> Running a package installation as root messes up their environments.
> since "start" is not a valid option on the "cassandra" command, we should 
> block cassandra from starting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7950) Output of nodetool compactionstats and compactionhistory does not work well with long keyspace and column family names.

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7950:
---
Reviewer: Yuki Morishita

> Output of nodetool compactionstats and compactionhistory does not work well 
> with long keyspace and column family names.  
> -
>
> Key: CASSANDRA-7950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7950
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
> Environment: CentOS 5, 64bit, Oracle JDK 7, DSE
>Reporter: Eugene
>Assignee: Michael Shuler
>Priority: Minor
>  Labels: lhf
> Fix For: 2.1.x
>
> Attachments: 7950.patch, nodetool-examples.txt
>
>
> When running these commands:
> nodetool compactionstats
> nodetool compactionhistory
> The output can be difficult to grok due to long keyspace names, column family 
> names, and long values.  I have attached an example.
> It's difficult for both humans and grep/sed/awk/perl to read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9454) Log WARN on Multi Partition IN clause Queries

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9454:
---
   Reviewer: Robert Stupp
Component/s: CQL

> Log WARN on Multi Partition IN clause Queries
> -
>
> Key: CASSANDRA-9454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9454
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Sebastian Estevez
>Assignee: T Jake Luciani
>Priority: Minor
> Fix For: 2.2.x
>
>
> Similar to CASSANDRA-6487 but for multi-partition queries.
> Show warning (ideally at the client CASSANDRA-8930) when users try to use IN 
> clauses when clustering columns span multiple partitions. The right way to go 
> is async requests per partition.
> **Update**: Unless the query is CL.ONE and all the partition ranges are on 
> the node! In which case multi partition IN is okay.
> This can cause an OOM
> {code}
> ERROR [Thread-388] 2015-05-18 12:11:10,147 CassandraDaemon.java (line 199) 
> Exception in thread Thread[Thread-388,5,main]
> java.lang.OutOfMemoryError: Java heap space
> ERROR [ReadStage:321] 2015-05-18 12:11:10,147 CassandraDaemon.java (line 199) 
> Exception in thread Thread[ReadStage:321,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
> at 
> org.apache.cassandra.io.util.MappedFileDataInput.readBytes(MappedFileDataInput.java:146)
> at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
> at 
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371)
> at 
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
> at 
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:970)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:871)
> at 
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:41)
> at 
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
> at 
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
> at 
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
> at 
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1547)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1376)
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
> at 
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
> at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> {code}
> By flooding heap with:
> {code}org.apache.cassandra.io.sstable.IndexHelper$IndexInfo{code}
> taken from:
> http://stackoverflow.com/questions/30366729/out-of-memory-error-in-cassandra-when-querying-big-rows-containing-a-collection



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10464) "nodetool compactionhistory" output should be sorted on compacted_at column and the timestamp shown in human readable format

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10464:

Reviewer: Yuki Morishita

> "nodetool compactionhistory" output should be sorted on compacted_at column 
> and the timestamp shown in human readable format
> 
>
> Key: CASSANDRA-10464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10464
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Wei Deng
>Priority: Minor
> Fix For: 3.x
>
> Attachments: CASSANDRA-10464-CompactionHistory.patch
>
>
> "nodetool compactionhistory" (introduced in CASSANDRA-5078) is a useful tool 
> for Cassandra DBAs. However, the current output limits its usefulness without 
> some additional parsing.
> We should improve it in the following two areas:
> 1. The output should be sorted on the compacted_at column, so that the most 
> recently finished compaction will show up last (which is what the DBAs would 
> expect);
> 2. The compacted_at column should be printed in human-readable timestamp.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10782) AssertionError at getApproximateKeyCount

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10782:

Component/s: Compaction

> AssertionError at getApproximateKeyCount
> 
>
> Key: CASSANDRA-10782
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10782
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: C* 2.1.11, Debian Wheezy
>Reporter: mlowicki
>
> {code}
> ERROR [CompactionExecutor:9845] 2015-11-28 09:26:10,525 
> CassandraDaemon.java:227 - Exception in thread 
> Thread[CompactionExecutor:9845,1,main]
> java.lang.AssertionError: 
> /var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/system-sstable_activity-ka-6335-Data.db
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:268)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:151)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:236)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_80]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> ~[na:1.7.0_80]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_80]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_80]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10541) cqlshlib tests cannot run on Windows

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041733#comment-15041733
 ] 

Jim Witschey commented on CASSANDRA-10541:
--

The problem seems to be here:

{code}
16:16:27 Caused by: hudson.plugins.git.GitException: Command "git checkout -f 
0be57425173f9afa12864ac830ddfeb0b38e0421" returned status code 128:
16:16:27 stdout: 
16:16:27 stderr: fatal: cannot create directory at 
'test/data/migration-sstables/2.2/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/snapshots/1435298241281-upgrade-3.0.0-SNAPSHOT-2.2.0-rc1-SNAPSHOT':
 Filename too long
{code}

(from 
http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-3.0-paulomotta/19/console)

[~pauloricardomg] I believe you're a more experienced Windows admin than I am 
-- what's a good workaround for this? A quick search turns up this:

http://stackoverflow.com/questions/22575662/filename-too-long-in-git-for-windows

There are a number of suggestions here, but I don't know how to evaluate them.

> cqlshlib tests cannot run on Windows
> 
>
> Key: CASSANDRA-10541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10541
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Benjamin Lerer
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: cqlsh, windows
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> If I try to run the {{cqlshlib}} tests on Windows, I got the following error:
> {quote}
> ==
> ERROR: Failure: AttributeError ('module' object has no attribute 'symlink')
> --
> Traceback (most recent call last):
>   File "C:\Python27\lib\site-packages\nose\loader.py", line 414, in 
> loadTestsFromName
> addr.filename, addr.module)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 47, in 
> importFromPath
> return self.importFromDir(dir_path, fqname)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 94, in 
> importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
>   File "[...]\pylib\cqlshlib\test\__init__.py", line 17, in 
> from .cassconnect import create_test_db, remove_test_db
>   File "[...]\pylib\cqlshlib\test\cassconnect.py", line 22, in 
> from .basecase import cql, cqlsh, cqlshlog, TEST_HOST, TEST_PORT, rundir
>   File "[...]\pylib\cqlshlib\test\basecase.py", line 43, in 
> os.symlink(path_to_cqlsh, modulepath)
> AttributeError: 'module' object has no attribute 'symlink'
> --
> Ran 1 test in 0.002s
> FAILED (errors=1)
> {quote}
> The problem comes from the fact tha Windows has no support for symlinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8639) Can OOM on CL replay with dense mutations

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-8639:
--
Issue Type: Improvement  (was: Bug)

> Can OOM on CL replay with dense mutations
> -
>
> Key: CASSANDRA-8639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8639
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: T Jake Luciani
>Assignee: Ariel Weisberg
>Priority: Minor
> Fix For: 2.1.x
>
>
> If you write dense mutations with many clustering keys, the replay of the CL 
> can quickly overwhelm a node on startup.  This looks to be caused by the fact 
> we only ensure there are 1000 mutations in flight at a time. but those 
> mutations could have thousands of cells in them.
> A better approach would be to limit the CL replay to the amount of memory in 
> flight using cell.unsharedHeapSize()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10811) Prepared SELECT * queries return incorrect columns after schema change

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-10811.
--
Resolution: Duplicate

Agreed with Tyler, it's a duplicate.

> Prepared SELECT * queries return incorrect columns after schema change
> --
>
> Key: CASSANDRA-10811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10811
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 2.2.3
> Java driver 2.1.9
>Reporter: Adam Warski
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> (see also the test case attached)
> When executing the following steps using a single 
> {{com.datastax.driver.core.Session}}:
> 1. create table with column {{a}} and {{c}}
> 2. insert a single row with values for both columns
> 3. select that row using a prepared statement ({{SELECT * FROM table WHERE 
> id=?}})
> 4. alter the table adding a new column {{b}}
> 5. select the row using a prepared statement (preparing the statement again, 
> not re-using the old one)
> The value of the {{c}} column is not returned, instead there's a {{NULL}} 
> (which I suppose is the value of the newly inserted {{b}} column).
> The query returns correct results if:
> * step 3 is skipped, that is there is no prepared select before altering the 
> table
> * a normal, non-prepared select is done
> * a select with explicitly enumerated fields is done
> * a new session is created
> * the new column is alphabetically larger than the existing columns (e.g. 
> {{d}})



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10806) sstableloader can't handle upper case keyspace

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10806:

Reviewer: Yuki Morishita

> sstableloader can't handle upper case keyspace
> --
>
> Key: CASSANDRA-10806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10806
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Alex Liu
>Assignee: Alex Liu
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
> Attachments: CASSANDRA-10806-3.0-branch.txt
>
>
> sstableloader can't handle upper case keyspace. The following shows the 
> endpoint is missing
> {code}
> cassandra/bin/sstableloader 
> /var/folders/zz/zyxvpxvq6csfxvn_n0/T/bulk-write-to-Test1-Words-a9343a5f-62f3-4901-a9c8-ab7dc42a458e/Test1/Words-5
>   -d 127.0.0.1
> objc[7818]: Class JavaLaunchHelper is implemented in both 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/bin/java and 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/jre/lib/libinstrument.dylib.
>  One of the two will be used. Which one is undefined.
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of 
> /var/folders/zz/zyxvpxvq6csfxvn_n0/T/bulk-write-to-Test1-Words-a9343a5f-62f3-4901-a9c8-ab7dc42a458e/Test1/Words-5/ma-1-big-Data.db
>  to []
> Summary statistics: 
>   Connections per host:: 1
>   Total files transferred:  : 0
>   Total bytes transferred:  : 0
>   Total duration (ms):  : 923  
>   Average transfer rate (MB/s): : 0
>   Peak transfer rate (MB/s):: 0 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10807) Stress does not compiles within eclipse

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10807:

Reviewer: T Jake Luciani

> Stress does not compiles within eclipse
> ---
>
> Key: CASSANDRA-10807
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10807
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Attachments: 10807-2.1.txt
>
>
> In Eclipse  {{org.apache.cassandra.stress.settings.Lists}} and 
> {{org.apache.cassandra.stress.settings.Sets}} do not compiles.
> It seems that in some case javac might not fully follow the JLS. I am not 
> sure if it is really the case here but it would be nice if the code was 
> compiling within Eclipse.
> After all, it is possible that there are other developers using Eclipse to 
> work on Cassandra. Who knows? ;-)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10541) cqlshlib tests cannot run on Windows

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041706#comment-15041706
 ] 

Paulo Motta commented on CASSANDRA-10541:
-

I rebased and fixed minor conflicts. It seems the 2.2 windows branch is 
[fixed|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-2.2-paulomotta/],
 while 
[3.0|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-3.0-paulomotta/]
 and 
[trunk|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-trunk-paulomotta/]
 are not able to checkout the rebased version with the following error:

{noformat}
16:16:27 FATAL: Could not checkout 0be57425173f9afa12864ac830ddfeb0b38e0421
16:16:27 hudson.plugins.git.GitException: Could not checkout 
0be57425173f9afa12864ac830ddfeb0b38e0421
{noformat}

That's super-strange, since 2.2 is correctly fetching the new version. Any idea 
of what might be happening here? Could you maybe reset the state somehow so it 
would make a clean pull? [~mambocab]

Thanks!

> cqlshlib tests cannot run on Windows
> 
>
> Key: CASSANDRA-10541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10541
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Benjamin Lerer
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: cqlsh, windows
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> If I try to run the {{cqlshlib}} tests on Windows, I got the following error:
> {quote}
> ==
> ERROR: Failure: AttributeError ('module' object has no attribute 'symlink')
> --
> Traceback (most recent call last):
>   File "C:\Python27\lib\site-packages\nose\loader.py", line 414, in 
> loadTestsFromName
> addr.filename, addr.module)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 47, in 
> importFromPath
> return self.importFromDir(dir_path, fqname)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 94, in 
> importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
>   File "[...]\pylib\cqlshlib\test\__init__.py", line 17, in 
> from .cassconnect import create_test_db, remove_test_db
>   File "[...]\pylib\cqlshlib\test\cassconnect.py", line 22, in 
> from .basecase import cql, cqlsh, cqlshlog, TEST_HOST, TEST_PORT, rundir
>   File "[...]\pylib\cqlshlib\test\basecase.py", line 43, in 
> os.symlink(path_to_cqlsh, modulepath)
> AttributeError: 'module' object has no attribute 'symlink'
> --
> Ran 1 test in 0.002s
> FAILED (errors=1)
> {quote}
> The problem comes from the fact tha Windows has no support for symlinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6246) EPaxos

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-6246:

Reviewer: Sylvain Lebresne

> EPaxos
> --
>
> Key: CASSANDRA-6246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6246
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Blake Eggleston
> Fix For: 3.x
>
>
> One reason we haven't optimized our Paxos implementation with Multi-paxos is 
> that Multi-paxos requires leader election and hence, a period of 
> unavailability when the leader dies.
> EPaxos is a Paxos variant that requires (1) less messages than multi-paxos, 
> (2) is particularly useful across multiple datacenters, and (3) allows any 
> node to act as coordinator: 
> http://sigops.org/sosp/sosp13/papers/p358-moraru.pdf
> However, there is substantial additional complexity involved if we choose to 
> implement it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9879) Add the name of the compressor in the output of sstablemetadata

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9879:
---
Reviewer: Marcus Eriksson

> Add the name of the compressor in the output of sstablemetadata
> ---
>
> Key: CASSANDRA-9879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9879
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Nicolas Lalevée
>Priority: Trivial
> Attachments: 
> 0001-add-compressor-name-in-the-output-of-sstablemetadata.patch
>
>
> Here is a simple patch to add to the output of sstablemetadata the name the 
> compressor used for the sstable.
> I also made sstablemetadata embedded into the .deb distribution



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10358) Allow CQLSSTableWriter.Builder to use custom AbstractSSTableSimpleWriter

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10358:
-
Reviewer: Sylvain Lebresne

> Allow CQLSSTableWriter.Builder to use custom AbstractSSTableSimpleWriter 
> -
>
> Key: CASSANDRA-10358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10358
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Andre Turgeon
>Priority: Minor
> Attachments: SSTableWriterCreationStrategy.patch, patch.txt
>
>
> I've created a patch for your consideration. 
> This change to CQLSSTableWriter allows for a custom 
> AbstractSSTableSimpleWriter to be specified. 
> I needed this for a bulkload process I wrote. I believe the change would be 
> beneficial for other people as well. 
> Below are the reasons I needed a custom implementation of 
> AbstractSSTableSimpleWriter:
> 1) The available implementations of AbstractSSTableSimpleWriter do not 
> provide a way to specify the filename (or rather revision) of the sstable. I 
> needed to control the name because my bulkload process write sstables in 
> parallel (on multiple machines) and I wish to avoid name collisions.
> 2) I discovered a problem with SSTableSimpleUnsortedWriter where it creates 
> invalid level-compaction-style sstables; It allows a partition to span 2 
> sstables which violates the "no overlap of token ranges" constraint of level 
> compaction.   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10794) System table name resource_role_permissons_index is spelt wrong!

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10794:

Priority: Trivial  (was: Minor)

> System table name resource_role_permissons_index is spelt wrong!
> 
>
> Key: CASSANDRA-10794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Max Bowsher
>Priority: Trivial
> Fix For: 3.0.x, 3.x
>
>
> System table name resource_role_permissons_index is spelt wrong!
> "permissons" is missing an "i"
> Fixing that isn't going to be fun, though :-(



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10797) Bootstrap new node fails with OOM when streaming nodes contains thousands of sstables

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10797:

Assignee: Paulo Motta

> Bootstrap new node fails with OOM when streaming nodes contains thousands of 
> sstables
> -
>
> Key: CASSANDRA-10797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10797
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Cassandra 2.1.8.621 w/G1GC
>Reporter: Jose Martinez Poblete
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
> Attachments: 112415_system.log, Heapdump_OOM.zip, Screen Shot 
> 2015-12-01 at 7.34.40 PM.png
>
>
> When adding a new node to an existing DC, it runs OOM after 25-45 minutes
> Upon heapdump revision, it is found the sending nodes are streaming thousands 
> of sstables which in turns blows the bootstrapping node heap 
> {noformat}
> ERROR [RMI Scheduler(0)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [STREAM-IN-/173.36.28.148] 2015-11-24 10:10:44,585 
> StreamSession.java:502 - [Stream #0bb13f50-92cb-11e5-bc8d-f53b7528ffb4] 
> Streaming error occurred
> java.lang.IllegalStateException: Shutdown in progress
> at 
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) 
> ~[na:1.8.0_65]
> at java.lang.Runtime.removeShutdownHook(Runtime.java:239) 
> ~[na:1.8.0_65]
> at 
> org.apache.cassandra.service.StorageService.removeShutdownHook(StorageService.java:747)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector$Killer.killCurrentJVM(JVMStabilityInspector.java:95)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:64)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:66)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:250)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> ERROR [RMI TCP Connection(idle)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [OptionalTasks:1] 2015-11-24 10:10:44,585 CassandraDaemon.java:223 - 
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.IllegalStateException: Shutdown in progress
> {noformat}
> Attached is the Eclipse MAT report as a zipped web page



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8639) Can OOM on CL replay with dense mutations

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041696#comment-15041696
 ] 

T Jake Luciani commented on CASSANDRA-8639:
---

I'm thinking this is more of an improvement than a bug.  Would you be ok with 
me only putting this in 3.1?

> Can OOM on CL replay with dense mutations
> -
>
> Key: CASSANDRA-8639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8639
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: T Jake Luciani
>Assignee: Ariel Weisberg
>Priority: Minor
> Fix For: 2.1.x
>
>
> If you write dense mutations with many clustering keys, the replay of the CL 
> can quickly overwhelm a node on startup.  This looks to be caused by the fact 
> we only ensure there are 1000 mutations in flight at a time. but those 
> mutations could have thousands of cells in them.
> A better approach would be to limit the CL replay to the amount of memory in 
> flight using cell.unsharedHeapSize()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10800) Misguiding error messages on AllowAllAuthenticator usage

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10800:
-
Reviewer: Sam Tunnicliffe

> Misguiding error messages on AllowAllAuthenticator usage
> 
>
> Key: CASSANDRA-10800
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10800
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Priority: Trivial
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> Users trying to execute an operation that requires non-anonymous logins (e.g. 
> create role) receive a "You have to be logged in and not anonymous to perform 
> this request" message when using the default AllowAllAuthenticator. However, 
> loging in to cqlsh using the cassandra user and password does not fix this 
> and will result in the same error message.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: bound maximum in-flight commit log replay mutation bytes to 64 megabytes (tunable via cassandra.commitlog_max_outstanding_replay_bytes)

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 52d5eb04f -> 1c41a9ac2


bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(tunable via cassandra.commitlog_max_outstanding_replay_bytes)

Patch by Ariel Weisberg; reviewed by tjake for CASSANDRA-8639


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1c41a9ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1c41a9ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1c41a9ac

Branch: refs/heads/trunk
Commit: 1c41a9ac2c147ed111d9d8fba53652707dac7df0
Parents: 52d5eb0
Author: Ariel Weisberg 
Authored: Tue Nov 24 15:17:10 2015 -0500
Committer: T Jake Luciani 
Committed: Fri Dec 4 11:49:41 2015 -0500

--
 CHANGES.txt |   1 +
 NEWS.txt|   3 +-
 .../db/commitlog/CommitLogReplayer.java | 131 +++---
 .../cassandra/db/RecoveryManagerTest.java   | 137 +++
 4 files changed, 222 insertions(+), 50 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1c41a9ac/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1607a66..6c46183 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.2
+ * bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(CASSANDRA-8639)
  * Normalize all scripts (CASSANDRA-10679)
  * Make compression ratio much more accurate (CASSANDRA-10225)
  * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1c41a9ac/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 2a5970d..8830c99 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -18,7 +18,8 @@ using the provided 'sstableupgrade' tool.
 
 New features
 
-
+   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
+ tunable via cassandra.commitlog_max_outstanding_replay_bytes
- Support for type casting has been added to the selection clause.
 
 Upgrading

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1c41a9ac/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
index 2668bba..5010696 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
@@ -29,6 +29,7 @@ import java.util.concurrent.Future;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.zip.CRC32;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
 import com.google.common.collect.HashMultimap;
 import com.google.common.collect.Iterables;
@@ -54,7 +55,6 @@ import org.apache.cassandra.io.compress.ICompressor;
 import org.apache.cassandra.io.util.ChannelProxy;
 import org.apache.cassandra.io.util.DataInputBuffer;
 import org.apache.cassandra.io.util.FileDataInput;
-import org.apache.cassandra.io.util.NIODataInputStream;
 import org.apache.cassandra.io.util.RandomAccessReader;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.cassandra.utils.JVMStabilityInspector;
@@ -65,13 +65,17 @@ import static 
org.apache.cassandra.utils.FBUtilities.updateChecksumInt;
 
 public class CommitLogReplayer
 {
+@VisibleForTesting
+public static long MAX_OUTSTANDING_REPLAY_BYTES = 
Long.getLong("cassandra.commitlog_max_outstanding_replay_bytes", 1024 * 1024 * 
64);
+@VisibleForTesting
+public static MutationInitiator mutationInitiator = new 
MutationInitiator();
 static final String IGNORE_REPLAY_ERRORS_PROPERTY = 
"cassandra.commitlog.ignorereplayerrors";
 private static final Logger logger = 
LoggerFactory.getLogger(CommitLogReplayer.class);
 private static final int MAX_OUTSTANDING_REPLAY_COUNT = 
Integer.getInteger("cassandra.commitlog_max_outstanding_replay_count", 1024);
 private static final int LEGACY_END_OF_SEGMENT_MARKER = 0;
 
 private final Set keyspacesRecovered;
-private final List futures;
+private final Queue futures;
 private final Map invalidMutations;
 private final AtomicInteger replayedCount;
 private final Map cfPositions;
@@ -79,14 +83,74 @@ public class CommitLogReplayer
 private final CRC32 checksum;
 private byte[] buffer;
 private byte[] uncompressedBuffer;
+private long 

[jira] [Updated] (CASSANDRA-8581) Null pointer in cassandra.hadoop.ColumnFamilyRecoderWriter

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-8581:
---
Reviewer: Philip Thompson

> Null pointer in cassandra.hadoop.ColumnFamilyRecoderWriter
> --
>
> Key: CASSANDRA-8581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8581
> Project: Cassandra
>  Issue Type: Bug
>Reporter: xiangdong Huang
>Assignee: Brandon Williams
>  Labels: hadoop
> Fix For: 2.1.x
>
> Attachments: difference, 屏幕快照 2015-01-08 下午7.59.29.png, 屏幕快照 
> 2015-01-08 下午8.01.15.png, 屏幕快照 2015-01-08 下午8.07.23.png
>
>
> When I run examples/hadoop_word_count. I find that ReducerToFilesystem is 
> correct but when I use ReducerToCassandra, the program will call loadYaml().
> The reason is that the program catch a exception at line 196 of 
> ColumnFamilyRecoderWriter.java. 
> Then it check why the exception occur, then it loadYaml to check if the disk 
> is broken...
> However, the exception is NullPointerException. because the client is not 
> initialized.
>  
> So we need a check to judge whether the client is null. 
> (
> The exception, original code and fixed code are in the attachments.
> )



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9294) Streaming errors should log the root cause

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9294:
---
Reviewer: Yuki Morishita

> Streaming errors should log the root cause
> --
>
> Key: CASSANDRA-9294
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9294
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Brandon Williams
>Assignee: Paulo Motta
> Fix For: 3.2, 2.1.x, 2.2.x, 3.0.x
>
>
> Currently, when a streaming error occurs all you get is something like:
> {noformat}
> java.util.concurrent.ExecutionException: 
> org.apache.cassandra.streaming.StreamException: Stream failed
> {noformat}
> Instead, we should log the root cause.  Was the connection reset by peer, did 
> it timeout, etc?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8639) Can OOM on CL replay with dense mutations

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041696#comment-15041696
 ] 

T Jake Luciani edited comment on CASSANDRA-8639 at 12/4/15 4:46 PM:


I'm thinking this is more of an improvement than a bug.  Would you be ok with 
me only putting this in 3.2?


was (Author: tjake):
I'm thinking this is more of an improvement than a bug.  Would you be ok with 
me only putting this in 3.1?

> Can OOM on CL replay with dense mutations
> -
>
> Key: CASSANDRA-8639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8639
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: T Jake Luciani
>Assignee: Ariel Weisberg
>Priority: Minor
> Fix For: 3.2
>
>
> If you write dense mutations with many clustering keys, the replay of the CL 
> can quickly overwhelm a node on startup.  This looks to be caused by the fact 
> we only ensure there are 1000 mutations in flight at a time. but those 
> mutations could have thousands of cells in them.
> A better approach would be to limit the CL replay to the amount of memory in 
> flight using cell.unsharedHeapSize()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10790) querying on partitioning key and secondary index slow

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10790:

Issue Type: Improvement  (was: Bug)

> querying on partitioning key and secondary index slow 
> --
>
> Key: CASSANDRA-10790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Jan Cetkovsky
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> If large amount on entries exists in table, querying on partitioning key and 
> (low cardinality) secondary index is significantly slower than just on the 
> partitioning key, even with limit 1. It seems to do a full scan on both 
> indexes and combine and filter this result.
> In the example below, there is roughly 400M records on a given node - as you 
> can see the two queries yield the same result, with the refining secondary 
> key condition increasing the execution time roughly 35x. It seems to me that 
> plain filtering of the pk query result should be more efficient especially 
> with low cardinality secondary indexes (but it might be an assumption based 
> on my modeling approach).
> [cqlsh 4.1.1 | Cassandra 2.0.14 | CQL spec 3.1.1 | Thrift protocol 19.39.0]
> {code}
> CREATE TABLE fil (
>   nm text,
>   dir boolean,
>   id timeuuid,
>   attr map,
>   cntt text,
>   del boolean,
>   exp timestamp,
>   gid text,
>   perm int,
>   seg map,
>   size bigint,
>   uid text,
>   PRIMARY KEY ((nm), dir, id)
> ) WITH CLUSTERING ORDER BY (dir ASC, id DESC) AND
>   bloom_filter_fp_chance=0.01 AND
>   caching='KEYS_ONLY' AND
>   comment='table used to store file information' AND
>   dclocal_read_repair_chance=0.10 AND
>   gc_grace_seconds=864000 AND
>   index_interval=128 AND
>   read_repair_chance=0.00 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   default_time_to_live=0 AND
>   speculative_retry='99.0PERCENTILE' AND
>   memtable_flush_period_in_ms=0 AND
>   compaction={'class': 'SizeTieredCompactionStrategy'} AND
>   compression={'chunk_length_kb': '512', 'crc_check_chance': '0.5', 
> 'sstable_compression': 'SnappyCompressor'};
> CREATE INDEX fil_del_idx ON fil (del);
> cqlsh:pronto> select id, nm, attr, seg, exp, dir, uid, gid, perm, cntt from 
> fil where nm='/dir_108/aePzC_62755108' and dir=false and del=false limit 1;
>  id   | nm  | attr | seg  
>  | exp  | dir   | 
> uid   | gid| perm | cntt
> --+-+--+---+--+---+---++--+--
>  e22d17f3-7f77-11e5-8313-0522d849d63b | /dir_108/aePzC_62755108 | null | 
> {e23d6ba0-7f77-11e5-8313-0522d849d63b: 22528} | 2016-10-29 19:34:17-0700 | 
> False | test_user | test_group | 1911 | application/octet-stream
> (1 rows)
> Tracing session: e0287850-88cd-11e5-a7d5-2360c9ef4b33
>  activity 
>   
>   
> | timestamp| source   
> | source_elapsed
> --+--+--+
>   
>   
>   
>  execute_cql3_query | 15:42:31,643 | 10.65.230.23 
> |  0
>   
>   Parsing select id, 
> nm, attr, seg, exp, dir, uid, gid, perm, cntt from fil where 
> nm='/dir_108/aePzC_62755108' and dir=false and del=false limit 1; | 
> 15:42:31,643 | 10.65.230.23 |105
>   
>   
>   
> Preparing statement | 15:42:31,644 | 

[jira] [Commented] (CASSANDRA-8639) Can OOM on CL replay with dense mutations

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041726#comment-15041726
 ] 

Ariel Weisberg commented on CASSANDRA-8639:
---

I think that is fine. We can always backport if there is demand.

> Can OOM on CL replay with dense mutations
> -
>
> Key: CASSANDRA-8639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8639
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: T Jake Luciani
>Assignee: Ariel Weisberg
>Priority: Minor
> Fix For: 2.1.x
>
>
> If you write dense mutations with many clustering keys, the replay of the CL 
> can quickly overwhelm a node on startup.  This looks to be caused by the fact 
> we only ensure there are 1000 mutations in flight at a time. but those 
> mutations could have thousands of cells in them.
> A better approach would be to limit the CL replay to the amount of memory in 
> flight using cell.unsharedHeapSize()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10812) CompactionInterruptedException related to secondary index build during rolling upgrade

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10812:

Summary: CompactionInterruptedException related to secondary index build 
during rolling upgrade  (was: CompactionInterruptedException related to 
seconary index build during rolling upgrade)

> CompactionInterruptedException related to secondary index build during 
> rolling upgrade
> --
>
> Key: CASSANDRA-10812
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10812
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Russ Hatch
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log
>
>
> I'm not certain if this is a problem or not, but during upgrade tests we're 
> observing the following exception:
> {noformat}
> INFO  [RMI TCP Connection(4)-127.0.0.1] 2015-12-02 09:07:11,067 
> Keyspace.java:600 - adding secondary index table cf.vals to operation
> DEBUG [CompactionExecutor:2] 2015-12-02 09:07:11,071 CompactionTask.java:146 
> - Compacting (be950fe0-990e-11e5-9474-ff4e6f1f6740) 
> [/tmp/dtest-yLzF8G/test/node1/data/upgrade/cf-6a7d8130990e11e5ac03ff4e6f1f6740/upgrade-cf-ka-1-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1] 2015-12-02 09:07:11,083 
> CompactionManager.java:1436 - Compaction interrupted: Secondary index 
> build@6a7d8130-990e-11e5-ac03-ff4e6f1f6740(upgrade, cf, 221970/1166280)bytes
> ERROR [SecondaryIndexManagement:1] 2015-12-02 09:07:11,096 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[SecondaryIndexManagement:1,5,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
> interrupted: Secondary index 
> build@6a7d8130-990e-11e5-ac03-ff4e6f1f6740(upgrade, cf, 221970/1
> 166280)bytes
> at 
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368) 
> ~[main/:na]
> at 
> org.apache.cassandra.index.internal.CassandraIndex.buildBlocking(CassandraIndex.java:671)
>  ~[main/:na]
> at 
> org.apache.cassandra.index.internal.CassandraIndex.lambda$getBuildIndexTask$214(CassandraIndex.java:641)
>  ~[main/:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_66]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_66]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_66]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
> interrupted: Secondary index 
> build@6a7d8130-990e-11e5-ac03-ff4e6f1f6740(upgrade, cf, 221970/1166280)bytes
> at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> ~[na:1.8.0_66]
> at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
> ~[na:1.8.0_66]
> at 
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:364) 
> ~[main/:na]
> ... 6 common frames omitted
> Caused by: org.apache.cassandra.db.compaction.CompactionInterruptedException: 
> Compaction interrupted: Secondary index 
> build@6a7d8130-990e-11e5-ac03-ff4e6f1f6740(upgrade, cf, 221970/1166280)bytes
> at 
> org.apache.cassandra.index.SecondaryIndexBuilder.build(SecondaryIndexBuilder.java:67)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1269)
>  ~[main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_66]
> ... 4 common frames omitted
> {noformat}
> These tests are typically writing and reading data continuously and 
> restarting nodes (so perhaps this is a routine exception under those 
> circumstances?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8639) Can OOM on CL replay with dense mutations

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-8639:
--
Fix Version/s: (was: 2.1.x)
   3.2

> Can OOM on CL replay with dense mutations
> -
>
> Key: CASSANDRA-8639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8639
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: T Jake Luciani
>Assignee: Ariel Weisberg
>Priority: Minor
> Fix For: 3.2
>
>
> If you write dense mutations with many clustering keys, the replay of the CL 
> can quickly overwhelm a node on startup.  This looks to be caused by the fact 
> we only ensure there are 1000 mutations in flight at a time. but those 
> mutations could have thousands of cells in them.
> A better approach would be to limit the CL replay to the amount of memory in 
> flight using cell.unsharedHeapSize()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)