[jira] [Updated] (CASSANDRA-12073) [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial results

[Pavel Yaskevich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-12073:

   Resolution: Fixed
Fix Version/s: (was: 3.x)
   3.9
   Status: Resolved  (was: Patch Available)

Committed. Thanks, [~doanduyhai]!

> [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial 
> results
> ---
>
> Key: CASSANDRA-12073
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12073
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.7
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
> Fix For: 3.9
>
> Attachments: patch_PREFIX_search_with_CONTAINS_mode.txt, 
> patch_PREFIX_search_with_CONTAINS_mode_V2.txt
>
>
> {noformat}
> cqlsh:music> CREATE TABLE music.albums (
> id uuid PRIMARY KEY,
> artist text,
> country text,
> quality text,
> status text,
> title text,
> year int
> );
> cqlsh:music> CREATE CUSTOM INDEX albums_artist_idx ON music.albums (artist) 
> USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 
> 'CONTAINS', 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> cqlsh:music> SELECT * FROM albums WHERE artist like 'lady%'  LIMIT 100;
>  id   | artist| country| quality 
> | status| title | year
> --+---++-+---+---+--
>  372bb0ab-3263-41bc-baad-bb520ddfa787 | Lady Gaga |USA |  normal 
> |  Official |   Red and Blue EP | 2006
>  1a4abbcd-b5de-4c69-a578-31231e01ff09 | Lady Gaga |Unknown |  normal 
> | Promotion |Poker Face | 2008
>  31f4a0dc-9efc-48bf-9f5e-bfc09af42b82 | Lady Gaga |USA |  normal 
> |  Official |   The Cherrytree Sessions | 2009
>  8ebfaebd-28d0-477d-b735-469661ce6873 | Lady Gaga |Unknown |  normal 
> |  Official |Poker Face | 2009
>  98107d82-e0dd-46bc-a273-1577578984c7 | Lady Gaga |USA |  normal 
> |  Official |   Just Dance: The Remixes | 2008
>  a76af0f2-f5c5-4306-974a-e3c17158e6c6 | Lady Gaga |  Italy |  normal 
> |  Official |  The Fame | 2008
>  849ee019-8b15-4767-8660-537ab9710459 | Lady Gaga |USA |  normal 
> |  Official |Christmas Tree | 2008
>  4bad59ac-913f-43da-9d48-89adc65453d2 | Lady Gaga |  Australia |  normal 
> |  Official | Eh Eh | 2009
>  80327731-c450-457f-bc12-0a8c21fd9c5d | Lady Gaga |USA |  normal 
> |  Official | Just Dance Remixes Part 2 | 2008
>  3ad33659-e932-4d31-a040-acab0e23c3d4 | Lady Gaga |Unknown |  normal 
> |  null |Just Dance | 2008
>  9adce7f6-6a1d-49fd-b8bd-8f6fac73558b | Lady Gaga | United Kingdom |  normal 
> |  Official |Just Dance | 2009
> (11 rows)
> {noformat}
> *SASI* says that there are only 11 artists whose name starts with {{lady}}.
> However, in the data set, there are:
> * Lady Pank
> * Lady Saw
> * Lady Saw
> * Ladyhawke
> * Ladytron
> * Ladysmith Black Mambazo
> * Lady Gaga
> * Lady Sovereign
> etc ...
> By debugging the source code, the issue is in 
> {{OnDiskIndex.TermIterator::computeNext()}}
> {code:java}
> for (;;)
> {
> if (currentBlock == null)
> return endOfData();
> if (offset >= 0 && offset < currentBlock.termCount())
> {
> DataTerm currentTerm = currentBlock.getTerm(nextOffset());
> if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
> continue;
> // flip the flag right on the first bounds match
> // to avoid expensive comparisons
> checkLower = false;
> if (checkUpper && !e.isUpperSatisfiedBy(currentTerm))
> return endOfData();
> return currentTerm;
> }
> nextBlock();
> }
> {code}
>  So the {{endOfData()}} conditions are:
> * currentBlock == null
> * checkUpper && !e.isUpperSatisfiedBy(currentTerm)
> The problem is that {{e::isUpperSatisfiedBy}} is checking not only whether 
> the term match but also returns *false* when it's a *partial term* !
> {code:java}
> public boolean isUpperSatisfiedBy(OnDiskIndex.DataTerm term)
> {
> if (!hasUpper())
> return true;
> if (nonMatchingPartial(term))
> return false;
> int cmp = term.compareTo(validator, upper.value, false);
>  

[2/2] cassandra git commit: Merge branch cassandra-3.9 into trunk

[xedin]

Merge branch cassandra-3.9 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/71f3c91d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/71f3c91d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/71f3c91d

Branch: refs/heads/trunk
Commit: 71f3c91d142daf10fbd9d1e99d5e111d4be8bf96
Parents: 73b5cab 7107646
Author: Pavel Yaskevich 
Authored: Mon Jul 4 21:26:29 2016 -0700
Committer: Pavel Yaskevich 
Committed: Mon Jul 4 21:26:29 2016 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/index/sasi/disk/OnDiskIndex.java  |  7 
 .../cassandra/index/sasi/plan/Expression.java   | 11 --
 .../unit/org/apache/cassandra/SchemaLoader.java | 35 +++
 .../cassandra/index/sasi/SASIIndexTest.java | 30 +++-
 .../index/sasi/disk/OnDiskIndexTest.java| 36 
 6 files changed, 108 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/71f3c91d/CHANGES.txt
--
diff --cc CHANGES.txt
index 8726a45,68854b3..290a2d1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
 +3.10
 + * Remove pre-startup check for open JMX port (CASSANDRA-12074)
 +
  3.9
+  * Fix SASI PREFIX search in CONTAINS mode with partial terms 
(CASSANDRA-12073)
   * Increase size of flushExecutor thread pool (CASSANDRA-12071)
  Merged from 3.0:
   * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)

cassandra git commit: Fix SASI PREFIX search in CONTAINS mode with partial terms

[xedin]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.9 0702e4580 -> 7107646ac


Fix SASI PREFIX search in CONTAINS mode with partial terms

patch by doanduyhai; reviewed by xedin for CASSANDRA-12073


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7107646a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7107646a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7107646a

Branch: refs/heads/cassandra-3.9
Commit: 7107646ace81fe8f9e1de1e87c5dc4cdfd9f6607
Parents: 0702e45
Author: Pavel Yaskevich 
Authored: Mon Jul 4 14:14:59 2016 -0700
Committer: Pavel Yaskevich 
Committed: Mon Jul 4 21:24:20 2016 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/index/sasi/disk/OnDiskIndex.java  |  7 
 .../cassandra/index/sasi/plan/Expression.java   | 11 --
 .../unit/org/apache/cassandra/SchemaLoader.java | 35 +++
 .../cassandra/index/sasi/SASIIndexTest.java | 30 +++-
 .../index/sasi/disk/OnDiskIndexTest.java| 36 
 6 files changed, 108 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ee5a4af..68854b3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)
 Merged from 3.0:
  * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
--
diff --git a/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java 
b/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
index 80092ef..4d43cd9 100644
--- a/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
@@ -756,6 +756,13 @@ public class OnDiskIndex implements 
Iterable, Closeable
 {
 DataTerm currentTerm = currentBlock.getTerm(nextOffset());
 
+// we need to step over all of the partial terms, in 
PREFIX mode,
+// encountered by the query until upper-bound tells us to 
stop
+if (e.getOp() == Op.PREFIX && currentTerm.isPartial())
+continue;
+
+// haven't reached the start of the query range yet, let's
+// keep skip the current term until lower bound is 
satisfied
 if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
 continue;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
--
diff --git a/src/java/org/apache/cassandra/index/sasi/plan/Expression.java 
b/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
index ce420d1..cc156ee 100644
--- a/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
+++ b/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
@@ -322,9 +322,6 @@ public class Expression
 if (!hasLower())
 return true;
 
-if (nonMatchingPartial(term))
-return false;
-
 int cmp = term.compareTo(validator, lower.value, false);
 return cmp > 0 || cmp == 0 && lower.inclusive;
 }
@@ -334,9 +331,6 @@ public class Expression
 if (!hasUpper())
 return true;
 
-if (nonMatchingPartial(term))
-return false;
-
 int cmp = term.compareTo(validator, upper.value, false);
 return cmp < 0 || cmp == 0 && upper.inclusive;
 }
@@ -385,11 +379,6 @@ public class Expression
 && exclusions.equals(o.exclusions);
 }
 
-private boolean nonMatchingPartial(OnDiskIndex.DataTerm term)
-{
-return term.isPartial() && operation == Op.PREFIX;
-}
-
 public static class Bound
 {
 public final ByteBuffer value;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java 
b/test/unit/org/apache/cassandra/SchemaLoader.java
index 6aea343..28fc8d5 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -645,6 +645,41 @@ public class 

[1/2] cassandra git commit: Fix SASI PREFIX search in CONTAINS mode with partial terms

[xedin]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 73b5cabbc -> 71f3c91d1


Fix SASI PREFIX search in CONTAINS mode with partial terms

patch by doanduyhai; reviewed by xedin for CASSANDRA-12073


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7107646a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7107646a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7107646a

Branch: refs/heads/trunk
Commit: 7107646ace81fe8f9e1de1e87c5dc4cdfd9f6607
Parents: 0702e45
Author: Pavel Yaskevich 
Authored: Mon Jul 4 14:14:59 2016 -0700
Committer: Pavel Yaskevich 
Committed: Mon Jul 4 21:24:20 2016 -0700

--
 CHANGES.txt |  1 +
 .../cassandra/index/sasi/disk/OnDiskIndex.java  |  7 
 .../cassandra/index/sasi/plan/Expression.java   | 11 --
 .../unit/org/apache/cassandra/SchemaLoader.java | 35 +++
 .../cassandra/index/sasi/SASIIndexTest.java | 30 +++-
 .../index/sasi/disk/OnDiskIndexTest.java| 36 
 6 files changed, 108 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ee5a4af..68854b3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)
 Merged from 3.0:
  * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
--
diff --git a/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java 
b/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
index 80092ef..4d43cd9 100644
--- a/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
@@ -756,6 +756,13 @@ public class OnDiskIndex implements 
Iterable, Closeable
 {
 DataTerm currentTerm = currentBlock.getTerm(nextOffset());
 
+// we need to step over all of the partial terms, in 
PREFIX mode,
+// encountered by the query until upper-bound tells us to 
stop
+if (e.getOp() == Op.PREFIX && currentTerm.isPartial())
+continue;
+
+// haven't reached the start of the query range yet, let's
+// keep skip the current term until lower bound is 
satisfied
 if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
 continue;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
--
diff --git a/src/java/org/apache/cassandra/index/sasi/plan/Expression.java 
b/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
index ce420d1..cc156ee 100644
--- a/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
+++ b/src/java/org/apache/cassandra/index/sasi/plan/Expression.java
@@ -322,9 +322,6 @@ public class Expression
 if (!hasLower())
 return true;
 
-if (nonMatchingPartial(term))
-return false;
-
 int cmp = term.compareTo(validator, lower.value, false);
 return cmp > 0 || cmp == 0 && lower.inclusive;
 }
@@ -334,9 +331,6 @@ public class Expression
 if (!hasUpper())
 return true;
 
-if (nonMatchingPartial(term))
-return false;
-
 int cmp = term.compareTo(validator, upper.value, false);
 return cmp < 0 || cmp == 0 && upper.inclusive;
 }
@@ -385,11 +379,6 @@ public class Expression
 && exclusions.equals(o.exclusions);
 }
 
-private boolean nonMatchingPartial(OnDiskIndex.DataTerm term)
-{
-return term.isPartial() && operation == Op.PREFIX;
-}
-
 public static class Bound
 {
 public final ByteBuffer value;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7107646a/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java 
b/test/unit/org/apache/cassandra/SchemaLoader.java
index 6aea343..28fc8d5 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -645,6 +645,41 @@ public class SchemaLoader
   

[jira] [Commented] (CASSANDRA-11850) cannot use cql since upgrading python to 2.7.11+

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361976#comment-15361976
 ] 

Stefania commented on CASSANDRA-11850:
--

2.1 tests are fine but in 2.2+ we have some behavioral changes in the driver 
that cause problems, mostly related to utf-8 encoding and 
[PYTHON-458|https://datastax-oss.atlassian.net/browse/PYTHON-458] / 
CASSANDRA-10686.

Some fixes went into the patches linked above, whilst the fixes in the dtest 
code are [here|https://github.com/stef1927/cassandra-dtest/tree/11850]. I've 
relaunched the tests.

[~pauloricardomg], given that PYTHON-458 is now available, are we still 
expecting an operation timeout and schema mismatch warning in 
[{{test_refresh_schema_on_timeout_error}}|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11850-cqlsh-2.2-cqlsh-tests/lastCompletedBuild/cython=no/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_refresh_schema_on_timeout_error/]?
 I added the following in cqlsh 
[{{perform_simple_stement}}|https://github.com/stef1927/cassandra/commit/ba02cd1b96e329b533141cf78c43eb960e5d4d61#diff-1cce67f7d76864f07aaf4d986d6fc051R1256]
 but {{future.is_schema_agreed}} is always true (I think the driver refreshes 
the metadata before returning the statement result):

{code}
future = self.session.execute_async(statement, trace=self.tracing_enabled)
result = future.result()

if not future.is_schema_agreed:
self.refresh_schema_metadata_best_effort()
{code}

> cannot use cql since upgrading python to 2.7.11+
> 
>
> Key: CASSANDRA-11850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11850
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Development
>Reporter: Andrew Madison
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> OS: Debian GNU/Linux stretch/sid 
> Kernel: 4.5.0-2-amd64 #1 SMP Debian 4.5.4-1 (2016-05-16) x86_64 GNU/Linux
> Python version: 2.7.11+ (default, May  9 2016, 15:54:33)
> [GCC 5.3.1 20160429]
> cqlsh --version: cqlsh 5.0.1
> cassandra -v: 3.5 (also occurs with 3.0.6)
> Issue:
> when running cqlsh, it returns the following error:
> cqlsh -u dbarpt_usr01
> Password: *
> Connection error: ('Unable to connect to any servers', {'odbasandbox1': 
> TypeError('ref() does not take keyword arguments',)})
> I cleared PYTHONPATH:
> python -c "import json; print dir(json); print json.__version__"
> ['JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__', 
> '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', 
> '_default_decoder', '_default_encoder', 'decoder', 'dump', 'dumps', 
> 'encoder', 'load', 'loads', 'scanner']
> 2.0.9
> Java based clients can connect to Cassandra with no issue. Just CQLSH and 
> Python clients cannot.
> nodetool status also works.
> Thank you for your help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: combine all the architecture alignment checks into one place

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 6c16d2745 -> 73b5cabbc


combine all the architecture alignment checks into one place


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/73b5cabb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/73b5cabb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/73b5cabb

Branch: refs/heads/trunk
Commit: 73b5cabbcdf476632300c8d73fa5edf3fb447bdc
Parents: 6c16d27
Author: Dave Brosius 
Authored: Mon Jul 4 19:29:26 2016 -0400
Committer: Dave Brosius 
Committed: Mon Jul 4 19:29:26 2016 -0400

--
 .../org/apache/cassandra/io/util/Memory.java| 20 -
 .../apache/cassandra/utils/Architecture.java| 44 
 .../cassandra/utils/FastByteOperations.java |  5 +--
 .../cassandra/utils/memory/MemoryUtil.java  | 22 --
 4 files changed, 61 insertions(+), 30 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b5cabb/src/java/org/apache/cassandra/io/util/Memory.java
--
diff --git a/src/java/org/apache/cassandra/io/util/Memory.java 
b/src/java/org/apache/cassandra/io/util/Memory.java
index 78950ce..bd87f5e 100644
--- a/src/java/org/apache/cassandra/io/util/Memory.java
+++ b/src/java/org/apache/cassandra/io/util/Memory.java
@@ -22,6 +22,8 @@ import java.nio.ByteBuffer;
 import java.nio.ByteOrder;
 
 import net.nicoulaj.compilecommand.annotations.Inline;
+
+import org.apache.cassandra.utils.Architecture;
 import org.apache.cassandra.utils.FastByteOperations;
 import org.apache.cassandra.utils.concurrent.Ref;
 import org.apache.cassandra.utils.memory.MemoryUtil;
@@ -51,17 +53,9 @@ public class Memory implements AutoCloseable
 private static final long BYTE_ARRAY_BASE_OFFSET = 
unsafe.arrayBaseOffset(byte[].class);
 
 private static final boolean bigEndian = 
ByteOrder.nativeOrder().equals(ByteOrder.BIG_ENDIAN);
-private static final boolean unaligned;
 
 public static final ByteBuffer[] NO_BYTE_BUFFERS = new ByteBuffer[0];
 
-static
-{
-String arch = System.getProperty("os.arch");
-unaligned = arch.equals("i386") || arch.equals("x86")
-|| arch.equals("amd64") || arch.equals("x86_64") || 
arch.equals("s390x");
-}
-
 protected long peer;
 // size of the memory region
 protected final long size;
@@ -113,7 +107,7 @@ public class Memory implements AutoCloseable
 public void setLong(long offset, long l)
 {
 checkBounds(offset, offset + 8);
-if (unaligned)
+if (Architecture.IS_UNALIGNED)
 {
 unsafe.putLong(peer + offset, l);
 }
@@ -152,7 +146,7 @@ public class Memory implements AutoCloseable
 public void setInt(long offset, int l)
 {
 checkBounds(offset, offset + 4);
-if (unaligned)
+if (Architecture.IS_UNALIGNED)
 {
 unsafe.putInt(peer + offset, l);
 }
@@ -183,7 +177,7 @@ public class Memory implements AutoCloseable
 public void setShort(long offset, short l)
 {
 checkBounds(offset, offset + 2);
-if (unaligned)
+if (Architecture.IS_UNALIGNED)
 {
 unsafe.putShort(peer + offset, l);
 }
@@ -258,7 +252,7 @@ public class Memory implements AutoCloseable
 public long getLong(long offset)
 {
 checkBounds(offset, offset + 8);
-if (unaligned) {
+if (Architecture.IS_UNALIGNED) {
 return unsafe.getLong(peer + offset);
 } else {
 return getLongByByte(peer + offset);
@@ -290,7 +284,7 @@ public class Memory implements AutoCloseable
 public int getInt(long offset)
 {
 checkBounds(offset, offset + 4);
-if (unaligned) {
+if (Architecture.IS_UNALIGNED) {
 return unsafe.getInt(peer + offset);
 } else {
 return getIntByByte(peer + offset);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b5cabb/src/java/org/apache/cassandra/utils/Architecture.java
--
diff --git a/src/java/org/apache/cassandra/utils/Architecture.java 
b/src/java/org/apache/cassandra/utils/Architecture.java
new file mode 100644
index 000..3173f8e
--- /dev/null
+++ b/src/java/org/apache/cassandra/utils/Architecture.java
@@ -0,0 +1,44 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements.  See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership.  The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this 

cassandra git commit: c* uses commons-lang3, not commons-lang

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk b73f58135 -> 6c16d2745


c* uses commons-lang3, not commons-lang


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c16d274
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c16d274
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c16d274

Branch: refs/heads/trunk
Commit: 6c16d2745b403d0192c780886b875cc629c0227c
Parents: b73f581
Author: Dave Brosius 
Authored: Mon Jul 4 17:23:46 2016 -0400
Committer: Dave Brosius 
Committed: Mon Jul 4 17:23:46 2016 -0400

--
 src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c16d274/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
index a914cc9..6acbd0d 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java
@@ -23,7 +23,7 @@ import java.util.concurrent.atomic.AtomicInteger;
 import java.util.zip.CRC32;
 
 import com.google.common.annotations.VisibleForTesting;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 

[jira] [Commented] (CASSANDRA-7019) Improve tombstone compactions

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361744#comment-15361744
 ] 

Branimir Lambov commented on CASSANDRA-7019:


Rebased and uploaded again to the same branch:
|[code|https://github.com/blambov/cassandra/tree/7019-rebased]|[utests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-7019-rebased-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-7019-rebased-dtest/]|


> Improve tombstone compactions
> -
>
> Key: CASSANDRA-7019
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7019
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Branimir Lambov
>  Labels: compaction, fallout
> Fix For: 3.x
>
> Attachments: 7019-2-system.log, 7019-debug.log, cell.tar.gz, 
> control.tar.gz, none.tar.gz, row.tar.gz, temp-plot.html
>
>
> When there are no other compactions to do, we trigger a single-sstable 
> compaction if there is more than X% droppable tombstones in the sstable.
> In this ticket we should try to include overlapping sstables in those 
> compactions to be able to actually drop the tombstones. Might only be doable 
> with LCS (with STCS we would probably end up including all sstables)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12131) system_schema corruption causing nodes to not restart

[Tom van der Woerdt (JIRA)]

Tom van der Woerdt created CASSANDRA-12131:
--

 Summary: system_schema corruption causing nodes to not restart
 Key: CASSANDRA-12131
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12131
 Project: Cassandra
  Issue Type: Bug
Reporter: Tom van der Woerdt
Priority: Critical


Symptoms :
* Existing nodes fail to restart
* system_schema has broken data
* `nodetool describecluster` shows a full disagreement

This happened on two clusters I manage, and so far I have no idea why. I'll 
describe symptoms and info on what I did to (partially) resolve this. Hope the 
actual bug can get fixed.

All clusters run with the binary distribution from cassandra.apache.org. One 
cluster runs on CentOS 6, the other CentOS 7, but both with Java 8u77. The 
issue was seen on version 3.0.4 and during an upgrade from 3.0.6 to 3.0.7.

** Cluster 1 **
Version: 3.0.4
Hardware: 2 datacenters, 3 machines each
Network: 1Gbit, <1ms within the dc, <20ms cross-dc

This happened several months ago. I found out the hard way that every node had 
a different schema_version when I tried to restart a node and it didn't come 
back. Assuming it was just a single unhappy node, I ignored it and restarted a 
second node (in a different datacenter) which also did not come back.

I like my quorums so I didn't restart the other nodes. `nodetool 
describecluster` showed that every node had a different schema version. 
Querying system_schema showed a lot of records with their keys set to 
`\0\0\0\0(...)\0\0`. Cassandra logs indicated corrupted data, which was then 
fixed by running scrub.

Of course that didn't actually fix the data and using CQL I removed most of the 
rows in system_schema that looked wrong. After doing that `nodetool 
describecluster' agreed on a schema version again. I've attached the python 
script I used to remove the records from the 'columns' table (fix.py), similar 
scripts were used for other tables.

That didn't actually remove all the records, some proved impossible to delete :
{code}
# Partial output from the query "select * from system_schema.columns"
|   regular |   -1 |text
 system_distributed | 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
 | 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
 | none |exception_stacktrace 
|   regular |   -1 |text
 system_distributed | 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
 | 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 | none |   
  finished_at 
|   regular |   -1 |   timestamp
 system_distributed | 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
 | 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 | none |   
keyspace_name
{code}

... so I just left those there as it doesn't seem to impact the cluster other 
than spewing this error every minute :
{code}
ERROR [CompactionExecutor:20] 2016-07-04 14:19:59,798 CassandraDaemon.java:201 
- Exception in thread Thread[CompactionExecutor:20,1,main]
java.lang.AssertionError: Invalid clustering for the table: 
org.apache.cassandra.db.Clustering$2@661b79a
at 
org.apache.cassandra.db.Clustering$Serializer.serialize(Clustering.java:136) 
~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:159)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:144) 
~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:112) 
~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52) 
~[apache-cassandra-3.0.7.jar:3.0.7]
{code}

The cluster works fine now, minus the phantom rows and minutely error on every 
node. As for the two boxes that got killed, they were `removenode`d and added 
back, somewhere in this process.

** Cluster 2 **
Version: 3.0.6
Hardware: 3 datacenters, 13 machines total
Network: 1Gbit, <1ms within the dc, <50ms cross-dc

This is a cluster I use for tests, which involves doing a lot of keyspace 
changes. While doing a 3.0.6->3.0.7 upgrade this morning I noticed that the 
first box I wanted to upgrade immediately didn't come back. Instead of trying 
to fix it (or just rebuilding the cluster) I left it like that and am now 
filing this report.

Startup on this node fails with :
{code}
ERROR [main] 2016-07-04 09:58:44,306 

[jira] [Commented] (CASSANDRA-12130) SASI related tests failing since CASSANDRA-11820

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361347#comment-15361347
 ] 

Sam Tunnicliffe commented on CASSANDRA-12130:
-

As this seems to be purely a problem with the unit tests, I haven't run the 
dtest CI jobs.

||branch||testall||
|[12130-3.9|https://github.com/beobal/cassandra/tree/12130-3.9]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12130-3.9-testall]|
|[12130-trunk|https://github.com/beobal/cassandra/tree/12130-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12130-trunk-testall]|


> SASI related tests failing since CASSANDRA-11820
> 
>
> Key: CASSANDRA-12130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12130
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> Since CASSANDRA-11820 was committed, a number of tests covering SASI have 
> been failing. In both {{SASIIndexTest}} and {{SSTableFlushObserverTest}}, 
> rows are built using an unsorted builder, which assumes that the columns are 
> added in clustering order. However, in both cases, this is not true and the 
> additional checks added to {{UnfilteredSerializer::serializeRowBody}} by 
> CASSANDRA-11820 now trigger assertion errors and, ultimately, failing tests. 
> In addition, {{SASIIndexTest}} reuses a single table in multiple tests and 
> performs its cleanup in the tear down method. When the assertion error is 
> triggered, the tear down is not run, leaving data in the table and causing 
> other failures in subsequent tests. 
> Patch to follow shortly...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12130) SASI related tests failing since CASSANDRA-11820

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-12130:

Status: Patch Available  (was: Open)

> SASI related tests failing since CASSANDRA-11820
> 
>
> Key: CASSANDRA-12130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12130
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> Since CASSANDRA-11820 was committed, a number of tests covering SASI have 
> been failing. In both {{SASIIndexTest}} and {{SSTableFlushObserverTest}}, 
> rows are built using an unsorted builder, which assumes that the columns are 
> added in clustering order. However, in both cases, this is not true and the 
> additional checks added to {{UnfilteredSerializer::serializeRowBody}} by 
> CASSANDRA-11820 now trigger assertion errors and, ultimately, failing tests. 
> In addition, {{SASIIndexTest}} reuses a single table in multiple tests and 
> performs its cleanup in the tear down method. When the assertion error is 
> triggered, the tear down is not run, leaving data in the table and causing 
> other failures in subsequent tests. 
> Patch to follow shortly...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11638) Add cassandra-stress test source

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11638:
--
   Resolution: Fixed
Fix Version/s: (was: 3.x)
   3.10
   Status: Resolved  (was: Ready to Commit)

Committed as 
[b73f5813559e5356f2dd50e4c3c15e520d7db0cc|https://github.com/apache/cassandra/commit/b73f5813559e5356f2dd50e4c3c15e520d7db0cc]
 to trunk only, thanks.

> Add cassandra-stress test source
> 
>
> Key: CASSANDRA-11638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11638
> Project: Cassandra
>  Issue Type: Test
>  Components: Tools
>Reporter: Christopher Batey
>Assignee: Christopher Batey
>Priority: Minor
>  Labels: stress
> Fix For: 3.10
>
> Attachments: 
> 0001-Add-a-test-source-directory-for-Cassandra-stress.patch
>
>
> This adds a test root for cassandra-stress and a couple of noddy tests for a 
> jira I did last week to prove it works / fails the build if they fail.
> I put the source in {{tools/stress/test/unit}} and the classes in 
> {{build/test/stress-classes}} (rather than putting them in with the main test 
> classes).
> Patch attached or at: https://github.com/chbatey/cassandra-1/tree/stress-tests



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Add basic testing support for the Cassandra Stress tool

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2aa665aa0 -> b73f58135


Add basic testing support for the Cassandra Stress tool

Patch by Christopher Batey; reviewed by Joel Knighton for CASSANDRA-11638


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b73f5813
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b73f5813
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b73f5813

Branch: refs/heads/trunk
Commit: b73f5813559e5356f2dd50e4c3c15e520d7db0cc
Parents: 2aa665a
Author: Christopher Batey 
Authored: Sun Apr 24 12:52:30 2016 +0100
Committer: Aleksey Yeschenko 
Committed: Mon Jul 4 14:48:52 2016 +0100

--
 build.xml   | 29 +-
 ide/idea-iml-file.xml   |  1 +
 .../cassandra/stress/settings/SettingsNode.java | 18 ++---
 .../stress/settings/OptionReplicationTest.java  | 34 
 .../stress/settings/SettingsNodeTest.java   | 42 
 5 files changed, 108 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b73f5813/build.xml
--
diff --git a/build.xml b/build.xml
index 9603855..1492d23 100644
--- a/build.xml
+++ b/build.xml
@@ -189,6 +189,7 @@
 
 
 
+
 
 
 
@@ -830,12 +831,29 @@
 
 
 
+
 
+

 
 
 
 
+
+
+
+
+
+
+
+
+
+
+
 

 
@@ -854,6 +872,12 @@
 
 
 
+
+
+
+
+



 
   
+  
   
   
+  
   
   
 
@@ -1551,7 +1577,7 @@
   
 
   
   
   
@@ -1824,6 +1850,7 @@
   
   
   
+  
   
   
   

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b73f5813/ide/idea-iml-file.xml
--
diff --git a/ide/idea-iml-file.xml b/ide/idea-iml-file.xml
index f14fe2e..3bb51b4 100644
--- a/ide/idea-iml-file.xml
+++ b/ide/idea-iml-file.xml
@@ -28,6 +28,7 @@
 
 
 
+
 
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b73f5813/tools/stress/src/org/apache/cassandra/stress/settings/SettingsNode.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsNode.java 
b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsNode.java
index 89b7871..a081e55 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/SettingsNode.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/SettingsNode.java
@@ -42,9 +42,8 @@ public class SettingsNode implements Serializable
 try
 {
 String node;
-List tmpNodes = new ArrayList();
-BufferedReader in = new BufferedReader(new 
InputStreamReader(new FileInputStream(options.file.value(;
-try
+List tmpNodes = new ArrayList<>();
+try (BufferedReader in = new BufferedReader(new 
InputStreamReader(new FileInputStream(options.file.value()
 {
 while ((node = in.readLine()) != null)
 {
@@ -53,10 +52,6 @@ public class SettingsNode implements Serializable
 }
 nodes = Arrays.asList(tmpNodes.toArray(new 
String[tmpNodes.size()]));
 }
-finally
-{
-in.close();
-}
 }
 catch(IOException ioe)
 {
@@ -177,13 +172,6 @@ public class SettingsNode implements Serializable
 
 public static Runnable helpPrinter()
 {
-return new Runnable()
-{
-@Override
-public void run()
-{
-printHelp();
-}
-};
+return SettingsNode::printHelp;
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b73f5813/tools/stress/test/unit/org/apache/cassandra/stress/settings/OptionReplicationTest.java
--
diff --git 
a/tools/stress/test/unit/org/apache/cassandra/stress/settings/OptionReplicationTest.java
 
b/tools/stress/test/unit/org/apache/cassandra/stress/settings/OptionReplicationTest.java
new file mode 100644
index 000..803ee18
--- /dev/null
+++ 

[jira] [Created] (CASSANDRA-12130) SASI related tests failing since CASSANDRA-11820

[Sam Tunnicliffe (JIRA)]

Sam Tunnicliffe created CASSANDRA-12130:
---

 Summary: SASI related tests failing since CASSANDRA-11820
 Key: CASSANDRA-12130
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12130
 Project: Cassandra
  Issue Type: Test
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
 Fix For: 3.x


Since CASSANDRA-11820 was committed, a number of tests covering SASI have been 
failing. In both {{SASIIndexTest}} and {{SSTableFlushObserverTest}}, rows are 
built using an unsorted builder, which assumes that the columns are added in 
clustering order. However, in both cases, this is not true and the additional 
checks added to {{UnfilteredSerializer::serializeRowBody}} by CASSANDRA-11820 
now trigger assertion errors and, ultimately, failing tests. 

In addition, {{SASIIndexTest}} reuses a single table in multiple tests and 
performs its cleanup in the tear down method. When the assertion error is 
triggered, the tear down is not run, leaving data in the table and causing 
other failures in subsequent tests. 

Patch to follow shortly...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12113) Cassandra 3.5.0 Repair Error

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361317#comment-15361317
 ] 

Serhat Rıfat Demircan commented on CASSANDRA-12113:
---

On source node:

{noformat}
ERROR [RepairJobTask:18] 2016-06-16 02:56:47,502 RepairSession.java:290 - 
[repair #cc96d270-3352-11e6-b94d-af0e016acd9a] Session completed with the 
following error
org.apache.cassandra.streaming.StreamException: Stream failed
at 
org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310) 
~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
 ~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
 ~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) 
~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
 ~[guava-18.0.jar:na]
at 
org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:189)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:429)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:524) 
~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:217)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
{noformat}

{noformat}
ERROR [StreamReceiveTask:4] 2016-06-16 02:58:01,817 StreamSession.java:519 - 
[Stream #f1c8a720-3353-11e6-b94d-af0e016acd9a] Streaming error occurred
java.lang.RuntimeException: Outgoing stream handler has been closed
at 
org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:142)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.maybeCompleted(StreamSession.java:711)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.taskCompleted(StreamSession.java:667)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:211)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
{noformat}

> Cassandra 3.5.0 Repair Error
> 
>
> Key: CASSANDRA-12113
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12113
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Production
>Reporter: Serhat Rıfat Demircan
>Assignee: Paulo Motta
>
> I got the following error while repairing nodes with the "nodetool repair" 
> command. Error occured on 2 nodes in the cluster which have 9 nodes.
>  
> Interesting thing is corrupted sstable is no more exists one of 2 nodes. 
> Copied existing one to test cluster and restored table from that sstable. No 
> error occured on test cluster.
> {noformat}
> ERROR [StreamReceiveTask:6] 2016-06-16 02:56:47,480 
> StreamReceiveTask.java:215 - Error applying streamed data:
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /var/lib/cassandra/data/keyspace/table/ma-1518-big-Data.db
> at 
> org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:50) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:372) 
> 

[jira] [Comment Edited] (CASSANDRA-12113) Cassandra 3.5.0 Repair Error

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361317#comment-15361317
 ] 

Serhat Rıfat Demircan edited comment on CASSANDRA-12113 at 7/4/16 1:32 PM:
---

On source node:

{noformat}
ERROR [RepairJobTask:18] 2016-06-16 02:56:47,502 RepairSession.java:290 - 
[repair #cc96d270-3352-11e6-b94d-af0e016acd9a] Session completed with the 
following error
org.apache.cassandra.streaming.StreamException: Stream failed
at 
org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310) 
~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
 ~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
 ~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) 
~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
 ~[guava-18.0.jar:na]
at 
org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:189)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:429)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:524) 
~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:217)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
{noformat}

Other node in the cluster:

{noformat}
ERROR [StreamReceiveTask:4] 2016-06-16 02:58:01,817 StreamSession.java:519 - 
[Stream #f1c8a720-3353-11e6-b94d-af0e016acd9a] Streaming error occurred
java.lang.RuntimeException: Outgoing stream handler has been closed
at 
org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:142)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.maybeCompleted(StreamSession.java:711)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamSession.taskCompleted(StreamSession.java:667)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:211)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_91]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
{noformat}


was (Author: serhatd):
On source node:

{noformat}
ERROR [RepairJobTask:18] 2016-06-16 02:56:47,502 RepairSession.java:290 - 
[repair #cc96d270-3352-11e6-b94d-af0e016acd9a] Session completed with the 
following error
org.apache.cassandra.streaming.StreamException: Stream failed
at 
org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310) 
~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
 ~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
 ~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) 
~[guava-18.0.jar:na]
at 
com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
 ~[guava-18.0.jar:na]
at 
org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213)
 ~[apache-cassandra-3.5.0.jar:3.5.0]
at 

[jira] [Updated] (CASSANDRA-11907) 2i behaviour is different in different versions

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-11907:
---
   Resolution: Fixed
Reproduced In: 3.0.6, 2.1.14, 2.2.7  (was: 2.1.14, 2.2.7, 3.0.6)
   Status: Resolved  (was: Patch Available)

Committed into 2.1 at c857919b40b9fb27139424944e9fb6cc58befc48 and merged into 
2.2, 3.0, 3.9 and trunk

> 2i behaviour is different in different versions
> ---
>
> Key: CASSANDRA-11907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11907
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tommy Stendahl
>Assignee: Alex Petrov
>
>  I think I have found more cases where 2i behave different in different 
> Cassandra versions, CASSANDRA-11510 solved one such case but I think there 
> are a few more.
> I get one behaviour with 2.1.14 and Trunk and I think this is the correct 
> one. With 2.2.7 and 3.0.6 the behaviour is different.
> To test this I used ccm to setup one node clusters with the different 
> versions, I prepared each cluster with these commands:
> {code:sql}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1' };
> CREATE TABLE test.table1 (name text,class int,inter text,foo text,power 
> int,PRIMARY KEY (name, class, inter, foo)) WITH CLUSTERING ORDER BY (class 
> DESC, inter ASC);
> CREATE INDEX table1_power ON test.table1 (power) ;
> CREATE TABLE test.table2 (name text,class int,inter text,foo text,power 
> int,PRIMARY KEY (name, class, inter, foo)) WITH CLUSTERING ORDER BY (class 
> DESC, inter ASC);
> CREATE INDEX table2_inter ON test.table2 (inter) ;
> {code}
> I executed two select quieries on each cluster:
> {code:sql}
> SELECT * FROM test.table1 where name='R1' AND class>0 AND class<4 AND 
> inter='int1' AND power=18 ALLOW FILTERING;
> SELECT * FROM test.table2 where name='R1' AND class>0 AND class<4 AND 
> inter='int1' AND foo='aa' ALLOW FILTERING;
> {code}
> On 2.1.14 and Trunk they where successful. But on 2.2.7 and 3.0.6 they 
> failed, the first one with {{InvalidRequest: code=2200 [Invalid query] 
> message="Clustering column "inter" cannot be restricted (preceding column 
> "class" is restricted by a non-EQ relation)"}} and the second one with 
> {{InvalidRequest: code=2200 [Invalid query] message="Clustering column "foo" 
> cannot be restricted (preceding column "inter" is restricted by a non-EQ 
> relation)"}}.
> I could get the queries to execute successfully on 2.2.7 and 3.0.6 by 
> creating two more 2i:
> {code:sql}
> CREATE INDEX table1_inter ON test.table1 (inter) ;
> CREATE INDEX table2_foo ON test.table2 (foo) ;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[4/6] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.9

[blerer]

Merge branch cassandra-3.0 into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/118f1a0d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/118f1a0d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/118f1a0d

Branch: refs/heads/trunk
Commit: 118f1a0d698eddedd70bcd72193bf3796219b9a7
Parents: c86b3e1 9244531
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:33:20 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:34:32 2016 +0200

--
 CHANGES.txt |  1 +
 .../ClusteringColumnRestrictions.java   | 20 ++--
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../SelectMultiColumnRelationTest.java  | 27 ++
 .../cql3/validation/operations/SelectTest.java  | 54 
 5 files changed, 86 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/118f1a0d/CHANGES.txt
--
diff --cc CHANGES.txt
index 475365f,2df77e1..1a03b89
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -5,34 -4,11 +5,35 @@@ Merged from 3.0
  Merged from 2.2:
   * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
  Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 + * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043)
 + * Remove finished incoming streaming connections from MessagingService 
(CASSANDRA-11854)
  
  
 -3.0.8
 - * Fix potential race in schema during new table creation (CASSANDRA-12083)
 +3.8
 + * Improve details in compaction log message (CASSANDRA-12080)
 + * Allow unset values in CQLSSTableWriter (CASSANDRA-11911)
 + * Chunk cache to request compressor-compatible buffers if pool space is 
exhausted (CASSANDRA-11993)
 + * Remove DatabaseDescriptor dependencies from SequentialWriter 
(CASSANDRA-11579)
 + * Move skip_stop_words filter before stemming (CASSANDRA-12078)
 + * Support seek() in EncryptedFileSegmentInputStream (CASSANDRA-11957)
 + * SSTable tools mishandling LocalPartitioner (CASSANDRA-12002)
 + * When SEPWorker assigned work, set thread name to match pool 
(CASSANDRA-11966)
 + * Add cross-DC latency metrics (CASSANDRA-11596)
 + * Allow terms in selection clause (CASSANDRA-10783)
 + * Add bind variables to trace (CASSANDRA-11719)
 + * Switch counter shards' clock to timestamps (CASSANDRA-9811)
 + * Introduce HdrHistogram and response/service/wait separation to stress tool 
(CASSANDRA-11853)
 + * entry-weighers in QueryProcessor should respect partitionKeyBindIndexes 
field (CASSANDRA-11718)
 + * Support older ant versions (CASSANDRA-11807)
 + * Estimate compressed on disk size when deciding if sstable size limit 
reached (CASSANDRA-11623)
 + * cassandra-stress profiles should support case sensitive schemas 
(CASSANDRA-11546)
 + * Remove DatabaseDescriptor dependency from FileUtils (CASSANDRA-11578)
 + * Faster streaming (CASSANDRA-9766)
 + * Add prepared query parameter to trace for "Execute CQL3 prepared query" 
session (CASSANDRA-11425)
 + * Add repaired percentage metric (CASSANDRA-11503)
 + * Add Change-Data-Capture (CASSANDRA-8844)
 +Merged from 3.0:
   * cqlsh: fix error handling in rare COPY FROM failure scenario 
(CASSANDRA-12070)
   * Disable autocompaction during drain (CASSANDRA-11878)
   * Add a metrics timer to MemtablePool and use it to track time spent blocked 
on memory in MemtableAllocator (CASSANDRA-11327)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/118f1a0d/src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
index 837ee13,000..dc349d9
mode 100644,00..100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
@@@ -1,229 -1,0 +1,215 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" 

[1/6] cassandra git commit: Fix filtering on clustering columns when 2i is used

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk dec1bdb20 -> 2aa665aa0


 Fix filtering on clustering columns when 2i is used

Patch by Alex Petrov; reviewed by Benjamin Lerer for CASSANDRA-11907


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c857919b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c857919b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c857919b

Branch: refs/heads/trunk
Commit: c857919b40b9fb27139424944e9fb6cc58befc48
Parents: bd6ad43
Author: Alex Petrov 
Authored: Mon Jul 4 14:15:39 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:15:39 2016 +0200

--
 CHANGES.txt |  1 +
 .../cql3/statements/SelectStatement.java| 10 ++-
 .../cql3/validation/operations/SelectTest.java  | 64 
 3 files changed, 74 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3aa5ea9..0967ce4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.16
+ * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
  * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 
 2.1.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6351bb5..245e64e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1582,6 +1582,7 @@ public class SelectStatement implements CQLStatement
 
 int numberOfRestrictionsEvaluatedWithSlices = 0;
 
+Restriction lastSliceRestriction = null;
 for (ColumnDefinition def : cfm.clusteringColumns())
 {
 // Remove clustering column restrictions that can be handled 
by slices; the remainder will be
@@ -1589,10 +1590,17 @@ public class SelectStatement implements CQLStatement
 Boolean indexed = stmt.restrictedColumns.get(def);
 if (indexed == null)
 break;
-if (!(indexed && stmt.usesSecondaryIndexing) && 
stmt.columnRestrictions[def.position()].canEvaluateWithSlices())
+
+Restriction restriction = 
stmt.columnRestrictions[def.position()];
+if (lastSliceRestriction != null && 
!restriction.equals(lastSliceRestriction))
+break;
+
+if (!(indexed && stmt.usesSecondaryIndexing) && 
restriction.canEvaluateWithSlices())
 {
 stmt.restrictedColumns.remove(def);
 numberOfRestrictionsEvaluatedWithSlices++;
+if (restriction.isSlice())
+lastSliceRestriction = restriction;
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
index 6acab6f..68cf6f8 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
@@ -1264,4 +1264,68 @@ public class SelectTest extends CQLTester
"SELECT * FROM %s WHERE a = 'foo' AND b= 'bar' AND 
c IN (?, ?)",
new String(TOO_BIG.array()), new 
String(TOO_BIG.array()));
 }
+
+@Test
+public void testFilteringWithSecondaryIndex() throws Throwable
+{
+createTable("CREATE TABLE %s (pk int, " +
+"c1 int, " +
+"c2 int, " +
+"c3 int, " +
+"v int, " +
+"PRIMARY KEY (pk, c1, c2, c3))");
+createIndex("CREATE INDEX v_idx_1 ON %s (v);");
+
+for (int i = 1; i <= 5; i++)
+{
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, 1, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, i, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, i, i, i);
+

[3/6] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9244531a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9244531a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9244531a

Branch: refs/heads/trunk
Commit: 9244531ab7bb0f4d07cf9996a9bb89d5e4370f54
Parents: db61372 e06dae8
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:30:55 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:31:46 2016 +0200

--
 CHANGES.txt |  3 +
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../restrictions/PrimaryKeyRestrictionSet.java  | 82 
 .../restrictions/StatementRestrictions.java | 41 --
 .../SelectMultiColumnRelationTest.java  | 33 +++-
 .../cql3/validation/operations/SelectTest.java  | 61 +++
 6 files changed, 144 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/CHANGES.txt
--
diff --cc CHANGES.txt
index f12d704,451575c..2df77e1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,11 +1,20 @@@
 -2.2.8
 +3.0.9
 + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
 + * Fix EOF exception when altering column type (CASSANDRA-11820)
 +Merged from 2.2:
   * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
+ Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 - * Account for partition deletions in tombstone histogram (CASSANDRA-12112)
+ 
  
 -2.2.7
 +3.0.8
 + * Fix potential race in schema during new table creation (CASSANDRA-12083)
 + * cqlsh: fix error handling in rare COPY FROM failure scenario 
(CASSANDRA-12070)
 + * Disable autocompaction during drain (CASSANDRA-11878)
 + * Add a metrics timer to MemtablePool and use it to track time spent blocked 
on memory in MemtableAllocator (CASSANDRA-11327)
 + * Fix upgrading schema with super columns with non-text subcomparators 
(CASSANDRA-12023)
 + * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
 +Merged from 2.2:
   * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755)
   * Validate bloom_filter_fp_chance against lowest supported
 value when the table is created (CASSANDRA-11920)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
index 4329b6e,51e2ce4..9d33bb1
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
@@@ -473,11 -471,11 +473,11 @@@ public abstract class MultiColumnRestri
  }
  
  @Override
 -public final void addIndexExpressionTo(List 
expressions,
 -   SecondaryIndexManager 
indexManager,
 -   QueryOptions options) throws 
InvalidRequestException
 +public final void addRowFilterTo(RowFilter filter,
 + SecondaryIndexManager indexManager,
 + QueryOptions options) throws 
InvalidRequestException
  {
- throw invalidRequest("Slice restrictions are not supported on 
indexed columns");
+ throw invalidRequest("Multi-column slice restrictions cannot be 
used for filtering.");
  }
  
  @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
index 061284b,e1cbc29..a5f4a24
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
@@@ -25,14 -26,11 +25,13 @@@ import org.apache.cassandra.config.Colu
  import org.apache.cassandra.cql3.QueryOptions;
  import org.apache.cassandra.cql3.functions.Function;
  import org.apache.cassandra.cql3.statements.Bound;
 -import org.apache.cassandra.db.IndexExpression;
 -import org.apache.cassandra.db.composites.*;
 -import org.apache.cassandra.db.composites.Composite.EOC;
 -import org.apache.cassandra.db.index.SecondaryIndexManager;
 +import org.apache.cassandra.db.*;
 +import 

[6/6] cassandra git commit: Merge branch cassandra-3.9 into trunk

[blerer]

Merge branch cassandra-3.9 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2aa665aa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2aa665aa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2aa665aa

Branch: refs/heads/trunk
Commit: 2aa665aa00faa01be4aee4bc3560d3a26a7dc46a
Parents: dec1bdb 0702e45
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:44:06 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:44:26 2016 +0200

--
 CHANGES.txt |  1 +
 .../ClusteringColumnRestrictions.java   | 20 ++--
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../SelectMultiColumnRelationTest.java  | 27 ++
 .../cql3/validation/operations/SelectTest.java  | 54 
 5 files changed, 86 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2aa665aa/CHANGES.txt
--

[5/6] cassandra git commit: Merge branch 'cassandra-3.9' of http://git-wip-us.apache.org/repos/asf/cassandra into cassandra-3.9

[blerer]

Merge branch 'cassandra-3.9' of 
http://git-wip-us.apache.org/repos/asf/cassandra into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0702e458
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0702e458
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0702e458

Branch: refs/heads/trunk
Commit: 0702e4580c8b51dcd2ea462de3f5fcd1e79038a5
Parents: 118f1a0 44e475c
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:35:19 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:35:19 2016 +0200

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 10 +-
 2 files changed, 10 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0702e458/CHANGES.txt
--

[2/6] cassandra git commit: Merge branch cassandra-2.1 into cassandra-2.2

[blerer]

Merge branch cassandra-2.1 into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e06dae81
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e06dae81
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e06dae81

Branch: refs/heads/trunk
Commit: e06dae81fb08870ef6a6596b1557b88fc7762302
Parents: 65f8bb6 c857919
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:20:34 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:22:55 2016 +0200

--
 CHANGES.txt |  1 +
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../restrictions/PrimaryKeyRestrictionSet.java  | 81 
 .../restrictions/StatementRestrictions.java | 42 --
 .../SelectMultiColumnRelationTest.java  | 32 +++-
 .../cql3/validation/operations/SelectTest.java  | 61 +++
 6 files changed, 142 insertions(+), 77 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/CHANGES.txt
--
diff --cc CHANGES.txt
index 13a1c4f,0967ce4..451575c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,41 -1,9 +1,42 @@@
 -2.1.16
 +2.2.8
 + * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
 +Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 -
 -2.1.15
   * Account for partition deletions in tombstone histogram (CASSANDRA-12112)
 +
 +
 +2.2.7
 + * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755)
 + * Validate bloom_filter_fp_chance against lowest supported
 +   value when the table is created (CASSANDRA-11920)
 + * RandomAccessReader: call isEOF() only when rebuffering, not for every read 
operation (CASSANDRA-12013)
 + * Don't send erroneous NEW_NODE notifications on restart (CASSANDRA-11038)
 + * StorageService shutdown hook should use a volatile variable 
(CASSANDRA-11984)
 + * Persist local metadata earlier in startup sequence (CASSANDRA-11742)
 + * Run CommitLog tests with different compression settings (CASSANDRA-9039)
 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
 + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)
 + * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
 + * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
   * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043)
   * Remove finished incoming streaming connections from MessagingService 
(CASSANDRA-11854)
   * Don't try to get sstables for non-repairing column families 
(CASSANDRA-12077)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
index 96e6f2b,000..51e2ce4
mode 100644,00..100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
@@@ -1,515 -1,0 +1,515 @@@
 +/*
 + * 

[jira] [Updated] (CASSANDRA-11303) New inbound throughput parameters for streaming

[Satoshi Konno (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satoshi Konno updated CASSANDRA-11303:
--
Attachment: 11303_inbound_patch_for_trunk_20160704.diff

> New inbound throughput parameters for streaming
> ---
>
> Key: CASSANDRA-11303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11303
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Configuration
>Reporter: Satoshi Konno
>Priority: Minor
> Attachments: 11303_inbound_limit_debug_20160419.log, 
> 11303_inbound_nolimit_debug_20160419.log, 
> 11303_inbound_patch_for_trunk_20160419.diff, 
> 11303_inbound_patch_for_trunk_20160525.diff, 
> 11303_inbound_patch_for_trunk_20160704.diff, 
> 200vs40inboundstreamthroughput.png, cassandra_inbound_stream.diff
>
>
> Hi,
> To specify stream throughputs of a node more clearly, I would like to add the 
> following new inbound parameters like existing outbound parameters in the 
> cassandra.yaml.
> - stream_throughput_inbound_megabits_per_sec
> - inter_dc_stream_throughput_outbound_megabits_per_sec  
> We use only the existing outbound parameters now, but it is difficult to 
> control the total throughputs of a node.　In our production network, some 
> critical alerts occurs when a node exceed the specified total throughput 
> which is the sum of the input and output throughputs.
> In our operation of Cassandra, the alerts occurs during the bootstrap or 
> repair processing when a new node is added. In the worst case, we have to 
> stop the operation of the exceed node.
> I have attached the patch under consideration. I would like to add a new 
> limiter class, StreamInboundRateLimiter, and use the limiter class in 
> StreamDeserializer class. I use Row::dataSize( ）to get the input throughput 
> in StreamDeserializer::newPartition(), but I am not sure whether the 
> dataSize() returns the correct data size.
> Can someone please tell me how to do it ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[4/9] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[blerer]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db613724
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db613724
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db613724

Branch: refs/heads/cassandra-3.9
Commit: db6137249837c255a687db7636471e4f7a15f53b
Parents: 6b24109 65f8bb6
Author: Dave Brosius 
Authored: Fri Jul 1 16:14:13 2016 -0400
Committer: Dave Brosius 
Committed: Fri Jul 1 16:14:13 2016 -0400

--
 lib/licenses/logback-classic-1.1.2.txt | 70 -
 lib/licenses/logback-classic-1.1.3.txt | 70 +
 lib/licenses/logback-core-1.1.2.txt| 70 -
 lib/licenses/logback-core-1.1.3.txt| 70 +
 4 files changed, 140 insertions(+), 140 deletions(-)
--

[8/9] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.9

[blerer]

Merge branch cassandra-3.0 into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/118f1a0d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/118f1a0d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/118f1a0d

Branch: refs/heads/cassandra-3.9
Commit: 118f1a0d698eddedd70bcd72193bf3796219b9a7
Parents: c86b3e1 9244531
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:33:20 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:34:32 2016 +0200

--
 CHANGES.txt |  1 +
 .../ClusteringColumnRestrictions.java   | 20 ++--
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../SelectMultiColumnRelationTest.java  | 27 ++
 .../cql3/validation/operations/SelectTest.java  | 54 
 5 files changed, 86 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/118f1a0d/CHANGES.txt
--
diff --cc CHANGES.txt
index 475365f,2df77e1..1a03b89
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -5,34 -4,11 +5,35 @@@ Merged from 3.0
  Merged from 2.2:
   * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
  Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 + * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043)
 + * Remove finished incoming streaming connections from MessagingService 
(CASSANDRA-11854)
  
  
 -3.0.8
 - * Fix potential race in schema during new table creation (CASSANDRA-12083)
 +3.8
 + * Improve details in compaction log message (CASSANDRA-12080)
 + * Allow unset values in CQLSSTableWriter (CASSANDRA-11911)
 + * Chunk cache to request compressor-compatible buffers if pool space is 
exhausted (CASSANDRA-11993)
 + * Remove DatabaseDescriptor dependencies from SequentialWriter 
(CASSANDRA-11579)
 + * Move skip_stop_words filter before stemming (CASSANDRA-12078)
 + * Support seek() in EncryptedFileSegmentInputStream (CASSANDRA-11957)
 + * SSTable tools mishandling LocalPartitioner (CASSANDRA-12002)
 + * When SEPWorker assigned work, set thread name to match pool 
(CASSANDRA-11966)
 + * Add cross-DC latency metrics (CASSANDRA-11596)
 + * Allow terms in selection clause (CASSANDRA-10783)
 + * Add bind variables to trace (CASSANDRA-11719)
 + * Switch counter shards' clock to timestamps (CASSANDRA-9811)
 + * Introduce HdrHistogram and response/service/wait separation to stress tool 
(CASSANDRA-11853)
 + * entry-weighers in QueryProcessor should respect partitionKeyBindIndexes 
field (CASSANDRA-11718)
 + * Support older ant versions (CASSANDRA-11807)
 + * Estimate compressed on disk size when deciding if sstable size limit 
reached (CASSANDRA-11623)
 + * cassandra-stress profiles should support case sensitive schemas 
(CASSANDRA-11546)
 + * Remove DatabaseDescriptor dependency from FileUtils (CASSANDRA-11578)
 + * Faster streaming (CASSANDRA-9766)
 + * Add prepared query parameter to trace for "Execute CQL3 prepared query" 
session (CASSANDRA-11425)
 + * Add repaired percentage metric (CASSANDRA-11503)
 + * Add Change-Data-Capture (CASSANDRA-8844)
 +Merged from 3.0:
   * cqlsh: fix error handling in rare COPY FROM failure scenario 
(CASSANDRA-12070)
   * Disable autocompaction during drain (CASSANDRA-11878)
   * Add a metrics timer to MemtablePool and use it to track time spent blocked 
on memory in MemtableAllocator (CASSANDRA-11327)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/118f1a0d/src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
index 837ee13,000..dc349d9
mode 100644,00..100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/ClusteringColumnRestrictions.java
@@@ -1,229 -1,0 +1,215 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS 

[3/9] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[blerer]

Merge branch 'cassandra-2.1' into cassandra-2.2

Conflicts:
lib/licenses/crc32ex-0.1.1.txt
lib/licenses/snappy-java-1.0.5.2.txt
lib/licenses/snappy-java-1.0.5.txt
lib/licenses/snappy-java-1.1.1.7.txt
lib/licenses/thrift-server-0.3.5.txt
lib/licenses/thrift-server-0.3.7.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/65f8bb6c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/65f8bb6c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/65f8bb6c

Branch: refs/heads/cassandra-3.9
Commit: 65f8bb6c4635fee4e17b2023915701dce59bc3c8
Parents: da20fec bd6ad43
Author: Dave Brosius 
Authored: Fri Jul 1 16:10:55 2016 -0400
Committer: Dave Brosius 
Committed: Fri Jul 1 16:10:55 2016 -0400

--

--

[7/9] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9244531a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9244531a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9244531a

Branch: refs/heads/cassandra-3.9
Commit: 9244531ab7bb0f4d07cf9996a9bb89d5e4370f54
Parents: db61372 e06dae8
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:30:55 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:31:46 2016 +0200

--
 CHANGES.txt |  3 +
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../restrictions/PrimaryKeyRestrictionSet.java  | 82 
 .../restrictions/StatementRestrictions.java | 41 --
 .../SelectMultiColumnRelationTest.java  | 33 +++-
 .../cql3/validation/operations/SelectTest.java  | 61 +++
 6 files changed, 144 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/CHANGES.txt
--
diff --cc CHANGES.txt
index f12d704,451575c..2df77e1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,11 +1,20 @@@
 -2.2.8
 +3.0.9
 + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
 + * Fix EOF exception when altering column type (CASSANDRA-11820)
 +Merged from 2.2:
   * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
+ Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 - * Account for partition deletions in tombstone histogram (CASSANDRA-12112)
+ 
  
 -2.2.7
 +3.0.8
 + * Fix potential race in schema during new table creation (CASSANDRA-12083)
 + * cqlsh: fix error handling in rare COPY FROM failure scenario 
(CASSANDRA-12070)
 + * Disable autocompaction during drain (CASSANDRA-11878)
 + * Add a metrics timer to MemtablePool and use it to track time spent blocked 
on memory in MemtableAllocator (CASSANDRA-11327)
 + * Fix upgrading schema with super columns with non-text subcomparators 
(CASSANDRA-12023)
 + * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
 +Merged from 2.2:
   * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755)
   * Validate bloom_filter_fp_chance against lowest supported
 value when the table is created (CASSANDRA-11920)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
index 4329b6e,51e2ce4..9d33bb1
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
@@@ -473,11 -471,11 +473,11 @@@ public abstract class MultiColumnRestri
  }
  
  @Override
 -public final void addIndexExpressionTo(List 
expressions,
 -   SecondaryIndexManager 
indexManager,
 -   QueryOptions options) throws 
InvalidRequestException
 +public final void addRowFilterTo(RowFilter filter,
 + SecondaryIndexManager indexManager,
 + QueryOptions options) throws 
InvalidRequestException
  {
- throw invalidRequest("Slice restrictions are not supported on 
indexed columns");
+ throw invalidRequest("Multi-column slice restrictions cannot be 
used for filtering.");
  }
  
  @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
index 061284b,e1cbc29..a5f4a24
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
@@@ -25,14 -26,11 +25,13 @@@ import org.apache.cassandra.config.Colu
  import org.apache.cassandra.cql3.QueryOptions;
  import org.apache.cassandra.cql3.functions.Function;
  import org.apache.cassandra.cql3.statements.Bound;
 -import org.apache.cassandra.db.IndexExpression;
 -import org.apache.cassandra.db.composites.*;
 -import org.apache.cassandra.db.composites.Composite.EOC;
 -import org.apache.cassandra.db.index.SecondaryIndexManager;
 +import org.apache.cassandra.db.*;
 +import 

[1/9] cassandra git commit: update license file versions

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.9 44e475c1a -> 0702e4580


update license file versions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bd6ad43b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bd6ad43b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bd6ad43b

Branch: refs/heads/cassandra-3.9
Commit: bd6ad43b9cda883d5aa151fe8580de2f8784e85f
Parents: 40b18d5
Author: Dave Brosius 
Authored: Fri Jul 1 15:53:29 2016 -0400
Committer: Dave Brosius 
Committed: Fri Jul 1 15:53:29 2016 -0400

--
 lib/licenses/snappy-java-1.0.5.2.txt | 209 ++
 lib/licenses/snappy-java-1.0.5.txt   | 209 --
 lib/licenses/thrift-server-0.3.5.txt | 202 -
 lib/licenses/thrift-server-0.3.7.txt | 202 +
 4 files changed, 411 insertions(+), 411 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bd6ad43b/lib/licenses/snappy-java-1.0.5.2.txt
--
diff --git a/lib/licenses/snappy-java-1.0.5.2.txt 
b/lib/licenses/snappy-java-1.0.5.2.txt
new file mode 100644
index 000..d5c4984
--- /dev/null
+++ b/lib/licenses/snappy-java-1.0.5.2.txt
@@ -0,0 +1,209 @@
+
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other modifications
+  represent, as a whole, an original work of authorship. For the purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+ 

[6/9] cassandra git commit: Merge branch cassandra-2.1 into cassandra-2.2

[blerer]

Merge branch cassandra-2.1 into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e06dae81
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e06dae81
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e06dae81

Branch: refs/heads/cassandra-3.9
Commit: e06dae81fb08870ef6a6596b1557b88fc7762302
Parents: 65f8bb6 c857919
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:20:34 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:22:55 2016 +0200

--
 CHANGES.txt |  1 +
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../restrictions/PrimaryKeyRestrictionSet.java  | 81 
 .../restrictions/StatementRestrictions.java | 42 --
 .../SelectMultiColumnRelationTest.java  | 32 +++-
 .../cql3/validation/operations/SelectTest.java  | 61 +++
 6 files changed, 142 insertions(+), 77 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/CHANGES.txt
--
diff --cc CHANGES.txt
index 13a1c4f,0967ce4..451575c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,41 -1,9 +1,42 @@@
 -2.1.16
 +2.2.8
 + * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
 +Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 -
 -2.1.15
   * Account for partition deletions in tombstone histogram (CASSANDRA-12112)
 +
 +
 +2.2.7
 + * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755)
 + * Validate bloom_filter_fp_chance against lowest supported
 +   value when the table is created (CASSANDRA-11920)
 + * RandomAccessReader: call isEOF() only when rebuffering, not for every read 
operation (CASSANDRA-12013)
 + * Don't send erroneous NEW_NODE notifications on restart (CASSANDRA-11038)
 + * StorageService shutdown hook should use a volatile variable 
(CASSANDRA-11984)
 + * Persist local metadata earlier in startup sequence (CASSANDRA-11742)
 + * Run CommitLog tests with different compression settings (CASSANDRA-9039)
 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
 + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)
 + * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
 + * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
   * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043)
   * Remove finished incoming streaming connections from MessagingService 
(CASSANDRA-11854)
   * Don't try to get sstables for non-repairing column families 
(CASSANDRA-12077)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
index 96e6f2b,000..51e2ce4
mode 100644,00..100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
@@@ -1,515 -1,0 +1,515 @@@
 +/*
 

[5/9] cassandra git commit: Fix filtering on clustering columns when 2i is used

[blerer]

 Fix filtering on clustering columns when 2i is used

Patch by Alex Petrov; reviewed by Benjamin Lerer for CASSANDRA-11907


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c857919b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c857919b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c857919b

Branch: refs/heads/cassandra-3.9
Commit: c857919b40b9fb27139424944e9fb6cc58befc48
Parents: bd6ad43
Author: Alex Petrov 
Authored: Mon Jul 4 14:15:39 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:15:39 2016 +0200

--
 CHANGES.txt |  1 +
 .../cql3/statements/SelectStatement.java| 10 ++-
 .../cql3/validation/operations/SelectTest.java  | 64 
 3 files changed, 74 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3aa5ea9..0967ce4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.16
+ * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
  * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 
 2.1.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6351bb5..245e64e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1582,6 +1582,7 @@ public class SelectStatement implements CQLStatement
 
 int numberOfRestrictionsEvaluatedWithSlices = 0;
 
+Restriction lastSliceRestriction = null;
 for (ColumnDefinition def : cfm.clusteringColumns())
 {
 // Remove clustering column restrictions that can be handled 
by slices; the remainder will be
@@ -1589,10 +1590,17 @@ public class SelectStatement implements CQLStatement
 Boolean indexed = stmt.restrictedColumns.get(def);
 if (indexed == null)
 break;
-if (!(indexed && stmt.usesSecondaryIndexing) && 
stmt.columnRestrictions[def.position()].canEvaluateWithSlices())
+
+Restriction restriction = 
stmt.columnRestrictions[def.position()];
+if (lastSliceRestriction != null && 
!restriction.equals(lastSliceRestriction))
+break;
+
+if (!(indexed && stmt.usesSecondaryIndexing) && 
restriction.canEvaluateWithSlices())
 {
 stmt.restrictedColumns.remove(def);
 numberOfRestrictionsEvaluatedWithSlices++;
+if (restriction.isSlice())
+lastSliceRestriction = restriction;
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
index 6acab6f..68cf6f8 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
@@ -1264,4 +1264,68 @@ public class SelectTest extends CQLTester
"SELECT * FROM %s WHERE a = 'foo' AND b= 'bar' AND 
c IN (?, ?)",
new String(TOO_BIG.array()), new 
String(TOO_BIG.array()));
 }
+
+@Test
+public void testFilteringWithSecondaryIndex() throws Throwable
+{
+createTable("CREATE TABLE %s (pk int, " +
+"c1 int, " +
+"c2 int, " +
+"c3 int, " +
+"v int, " +
+"PRIMARY KEY (pk, c1, c2, c3))");
+createIndex("CREATE INDEX v_idx_1 ON %s (v);");
+
+for (int i = 1; i <= 5; i++)
+{
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, 1, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, i, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, i, i, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, i, i, 

[2/9] cassandra git commit: update license txt file version numbers

[blerer]

update license txt file version numbers


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/da20fece
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/da20fece
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/da20fece

Branch: refs/heads/cassandra-3.9
Commit: da20fece744ab37d7355daa75e1717c08c83e4de
Parents: 82cdc86
Author: Dave Brosius 
Authored: Fri Jul 1 16:08:35 2016 -0400
Committer: Dave Brosius 
Committed: Fri Jul 1 16:08:35 2016 -0400

--
 lib/licenses/logback-classic-1.1.2.txt | 70 -
 lib/licenses/logback-classic-1.1.3.txt | 70 +
 lib/licenses/logback-core-1.1.2.txt| 70 -
 lib/licenses/logback-core-1.1.3.txt| 70 +
 4 files changed, 140 insertions(+), 140 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/da20fece/lib/licenses/logback-classic-1.1.2.txt
--
diff --git a/lib/licenses/logback-classic-1.1.2.txt 
b/lib/licenses/logback-classic-1.1.2.txt
deleted file mode 100644
index 79e486c..000
--- a/lib/licenses/logback-classic-1.1.2.txt
+++ /dev/null
@@ -1,70 +0,0 @@
-Eclipse Public License - v 1.0
-
-THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC 
LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM 
CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
-
-1. DEFINITIONS
-
-"Contribution" means:
-
-a) in the case of the initial Contributor, the initial code and documentation 
distributed under this Agreement, and
-b) in the case of each subsequent Contributor:
-i) changes to the Program, and
-ii) additions to the Program;
-where such changes and/or additions to the Program originate from and are 
distributed by that particular Contributor. A Contribution 'originates' from a 
Contributor if it was added to the Program by such Contributor itself or anyone 
acting on such Contributor's behalf. Contributions do not include additions to 
the Program which: (i) are separate modules of software distributed in 
conjunction with the Program under their own license agreement, and (ii) are 
not derivative works of the Program.
-"Contributor" means any person or entity that distributes the Program.
-
-"Licensed Patents" mean patent claims licensable by a Contributor which are 
necessarily infringed by the use or sale of its Contribution alone or when 
combined with the Program.
-
-"Program" means the Contributions distributed in accordance with this 
Agreement.
-
-"Recipient" means anyone who receives the Program under this Agreement, 
including all Contributors.
-
-2. GRANT OF RIGHTS
-
-a) Subject to the terms of this Agreement, each Contributor hereby grants 
Recipient a non-exclusive, worldwide, royalty-free copyright license to 
reproduce, prepare derivative works of, publicly display, publicly perform, 
distribute and sublicense the Contribution of such Contributor, if any, and 
such derivative works, in source code and object code form.
-b) Subject to the terms of this Agreement, each Contributor hereby grants 
Recipient a non-exclusive, worldwide, royalty-free patent license under 
Licensed Patents to make, use, sell, offer to sell, import and otherwise 
transfer the Contribution of such Contributor, if any, in source code and 
object code form. This patent license shall apply to the combination of the 
Contribution and the Program if, at the time the Contribution is added by the 
Contributor, such addition of the Contribution causes such combination to be 
covered by the Licensed Patents. The patent license shall not apply to any 
other combinations which include the Contribution. No hardware per se is 
licensed hereunder.
-c) Recipient understands that although each Contributor grants the licenses to 
its Contributions set forth herein, no assurances are provided by any 
Contributor that the Program does not infringe the patent or other intellectual 
property rights of any other entity. Each Contributor disclaims any liability 
to Recipient for claims brought by any other entity based on infringement of 
intellectual property rights or otherwise. As a condition to exercising the 
rights and licenses granted hereunder, each Recipient hereby assumes sole 
responsibility to secure any other intellectual property rights needed, if any. 
For example, if a third party patent license is required to allow Recipient to 
distribute the Program, it is Recipient's responsibility to acquire that 
license before distributing the Program.
-d) Each Contributor represents that to its knowledge it has sufficient 
copyright rights in its Contribution, if any, to grant the 

[9/9] cassandra git commit: Merge branch 'cassandra-3.9' of http://git-wip-us.apache.org/repos/asf/cassandra into cassandra-3.9

[blerer]

Merge branch 'cassandra-3.9' of 
http://git-wip-us.apache.org/repos/asf/cassandra into cassandra-3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0702e458
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0702e458
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0702e458

Branch: refs/heads/cassandra-3.9
Commit: 0702e4580c8b51dcd2ea462de3f5fcd1e79038a5
Parents: 118f1a0 44e475c
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:35:19 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:35:19 2016 +0200

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 10 +-
 2 files changed, 10 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0702e458/CHANGES.txt
--

[2/3] cassandra git commit: Merge branch cassandra-2.1 into cassandra-2.2

[blerer]

Merge branch cassandra-2.1 into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e06dae81
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e06dae81
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e06dae81

Branch: refs/heads/cassandra-3.0
Commit: e06dae81fb08870ef6a6596b1557b88fc7762302
Parents: 65f8bb6 c857919
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:20:34 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:22:55 2016 +0200

--
 CHANGES.txt |  1 +
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../restrictions/PrimaryKeyRestrictionSet.java  | 81 
 .../restrictions/StatementRestrictions.java | 42 --
 .../SelectMultiColumnRelationTest.java  | 32 +++-
 .../cql3/validation/operations/SelectTest.java  | 61 +++
 6 files changed, 142 insertions(+), 77 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/CHANGES.txt
--
diff --cc CHANGES.txt
index 13a1c4f,0967ce4..451575c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,41 -1,9 +1,42 @@@
 -2.1.16
 +2.2.8
 + * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
 +Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 -
 -2.1.15
   * Account for partition deletions in tombstone histogram (CASSANDRA-12112)
 +
 +
 +2.2.7
 + * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755)
 + * Validate bloom_filter_fp_chance against lowest supported
 +   value when the table is created (CASSANDRA-11920)
 + * RandomAccessReader: call isEOF() only when rebuffering, not for every read 
operation (CASSANDRA-12013)
 + * Don't send erroneous NEW_NODE notifications on restart (CASSANDRA-11038)
 + * StorageService shutdown hook should use a volatile variable 
(CASSANDRA-11984)
 + * Persist local metadata earlier in startup sequence (CASSANDRA-11742)
 + * Run CommitLog tests with different compression settings (CASSANDRA-9039)
 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
 + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)
 + * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
 + * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
   * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043)
   * Remove finished incoming streaming connections from MessagingService 
(CASSANDRA-11854)
   * Don't try to get sstables for non-repairing column families 
(CASSANDRA-12077)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
index 96e6f2b,000..51e2ce4
mode 100644,00..100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
@@@ -1,515 -1,0 +1,515 @@@
 +/*
 

[3/3] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9244531a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9244531a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9244531a

Branch: refs/heads/cassandra-3.0
Commit: 9244531ab7bb0f4d07cf9996a9bb89d5e4370f54
Parents: db61372 e06dae8
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:30:55 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:31:46 2016 +0200

--
 CHANGES.txt |  3 +
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../restrictions/PrimaryKeyRestrictionSet.java  | 82 
 .../restrictions/StatementRestrictions.java | 41 --
 .../SelectMultiColumnRelationTest.java  | 33 +++-
 .../cql3/validation/operations/SelectTest.java  | 61 +++
 6 files changed, 144 insertions(+), 78 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/CHANGES.txt
--
diff --cc CHANGES.txt
index f12d704,451575c..2df77e1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,11 +1,20 @@@
 -2.2.8
 +3.0.9
 + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
 + * Fix EOF exception when altering column type (CASSANDRA-11820)
 +Merged from 2.2:
   * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
+ Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 - * Account for partition deletions in tombstone histogram (CASSANDRA-12112)
+ 
  
 -2.2.7
 +3.0.8
 + * Fix potential race in schema during new table creation (CASSANDRA-12083)
 + * cqlsh: fix error handling in rare COPY FROM failure scenario 
(CASSANDRA-12070)
 + * Disable autocompaction during drain (CASSANDRA-11878)
 + * Add a metrics timer to MemtablePool and use it to track time spent blocked 
on memory in MemtableAllocator (CASSANDRA-11327)
 + * Fix upgrading schema with super columns with non-text subcomparators 
(CASSANDRA-12023)
 + * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
 +Merged from 2.2:
   * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755)
   * Validate bloom_filter_fp_chance against lowest supported
 value when the table is created (CASSANDRA-11920)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
index 4329b6e,51e2ce4..9d33bb1
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
@@@ -473,11 -471,11 +473,11 @@@ public abstract class MultiColumnRestri
  }
  
  @Override
 -public final void addIndexExpressionTo(List 
expressions,
 -   SecondaryIndexManager 
indexManager,
 -   QueryOptions options) throws 
InvalidRequestException
 +public final void addRowFilterTo(RowFilter filter,
 + SecondaryIndexManager indexManager,
 + QueryOptions options) throws 
InvalidRequestException
  {
- throw invalidRequest("Slice restrictions are not supported on 
indexed columns");
+ throw invalidRequest("Multi-column slice restrictions cannot be 
used for filtering.");
  }
  
  @Override

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9244531a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
index 061284b,e1cbc29..a5f4a24
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/PrimaryKeyRestrictionSet.java
@@@ -25,14 -26,11 +25,13 @@@ import org.apache.cassandra.config.Colu
  import org.apache.cassandra.cql3.QueryOptions;
  import org.apache.cassandra.cql3.functions.Function;
  import org.apache.cassandra.cql3.statements.Bound;
 -import org.apache.cassandra.db.IndexExpression;
 -import org.apache.cassandra.db.composites.*;
 -import org.apache.cassandra.db.composites.Composite.EOC;
 -import org.apache.cassandra.db.index.SecondaryIndexManager;
 +import org.apache.cassandra.db.*;
 +import 

[1/3] cassandra git commit: Fix filtering on clustering columns when 2i is used

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 db6137249 -> 9244531ab


 Fix filtering on clustering columns when 2i is used

Patch by Alex Petrov; reviewed by Benjamin Lerer for CASSANDRA-11907


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c857919b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c857919b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c857919b

Branch: refs/heads/cassandra-3.0
Commit: c857919b40b9fb27139424944e9fb6cc58befc48
Parents: bd6ad43
Author: Alex Petrov 
Authored: Mon Jul 4 14:15:39 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:15:39 2016 +0200

--
 CHANGES.txt |  1 +
 .../cql3/statements/SelectStatement.java| 10 ++-
 .../cql3/validation/operations/SelectTest.java  | 64 
 3 files changed, 74 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3aa5ea9..0967ce4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.16
+ * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
  * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 
 2.1.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6351bb5..245e64e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1582,6 +1582,7 @@ public class SelectStatement implements CQLStatement
 
 int numberOfRestrictionsEvaluatedWithSlices = 0;
 
+Restriction lastSliceRestriction = null;
 for (ColumnDefinition def : cfm.clusteringColumns())
 {
 // Remove clustering column restrictions that can be handled 
by slices; the remainder will be
@@ -1589,10 +1590,17 @@ public class SelectStatement implements CQLStatement
 Boolean indexed = stmt.restrictedColumns.get(def);
 if (indexed == null)
 break;
-if (!(indexed && stmt.usesSecondaryIndexing) && 
stmt.columnRestrictions[def.position()].canEvaluateWithSlices())
+
+Restriction restriction = 
stmt.columnRestrictions[def.position()];
+if (lastSliceRestriction != null && 
!restriction.equals(lastSliceRestriction))
+break;
+
+if (!(indexed && stmt.usesSecondaryIndexing) && 
restriction.canEvaluateWithSlices())
 {
 stmt.restrictedColumns.remove(def);
 numberOfRestrictionsEvaluatedWithSlices++;
+if (restriction.isSlice())
+lastSliceRestriction = restriction;
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
index 6acab6f..68cf6f8 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
@@ -1264,4 +1264,68 @@ public class SelectTest extends CQLTester
"SELECT * FROM %s WHERE a = 'foo' AND b= 'bar' AND 
c IN (?, ?)",
new String(TOO_BIG.array()), new 
String(TOO_BIG.array()));
 }
+
+@Test
+public void testFilteringWithSecondaryIndex() throws Throwable
+{
+createTable("CREATE TABLE %s (pk int, " +
+"c1 int, " +
+"c2 int, " +
+"c3 int, " +
+"v int, " +
+"PRIMARY KEY (pk, c1, c2, c3))");
+createIndex("CREATE INDEX v_idx_1 ON %s (v);");
+
+for (int i = 1; i <= 5; i++)
+{
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, 1, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, i, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, i, i, i);
+ 

[jira] [Commented] (CASSANDRA-11303) New inbound throughput parameters for streaming

[Satoshi Konno (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361243#comment-15361243
 ] 

Satoshi Konno commented on CASSANDRA-11303:
---

Hi [~pauloricardomg],

I have uploaded a new patch which includes the following changes.

{quote}
- Rename StorageService.(get/set)(Inbound/Outbound)StreamThroughputMbPerSec to 
be consistent with 
DatabaseDescriptor.(get/set)StreamThroughput(Inbound/Outbound)MegabitsPerSec.
- Add stream_throughput_inbound_megabits_per_sec to inter_dc_stream_throughput* 
property description on cassandra.yaml
- Add deprecation javadoc to deprecated StorageServiceMBean methods (see 
forceRepairAsync for example)
- remove additional spaces from throw new IOException("CF " + cfId + " was 
dropped during streaming");
- It seems your IDE removed static qualifiers on StreamReader constants and 
static methods
- Keep old methods on NodeProbe (add deprecation flag) in case they're used 
externally
{quote}

I tried to update the throughput values to the ongoing streams dynamically, but 
I suppose that it is hard to 
do using the singleton pattern because the streams are generated based on the 
peer address. 

I was going to manage a map which has all ongoing peer streams to update the 
throughput values dynamically,
but I think that the method is unreasonable.

In short, I hope that this patch will be accepted without the dynamic 
throughput changes.

Please let me know if you have any suggestions.

> New inbound throughput parameters for streaming
> ---
>
> Key: CASSANDRA-11303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11303
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Configuration
>Reporter: Satoshi Konno
>Priority: Minor
> Attachments: 11303_inbound_limit_debug_20160419.log, 
> 11303_inbound_nolimit_debug_20160419.log, 
> 11303_inbound_patch_for_trunk_20160419.diff, 
> 11303_inbound_patch_for_trunk_20160525.diff, 
> 200vs40inboundstreamthroughput.png, cassandra_inbound_stream.diff
>
>
> Hi,
> To specify stream throughputs of a node more clearly, I would like to add the 
> following new inbound parameters like existing outbound parameters in the 
> cassandra.yaml.
> - stream_throughput_inbound_megabits_per_sec
> - inter_dc_stream_throughput_outbound_megabits_per_sec  
> We use only the existing outbound parameters now, but it is difficult to 
> control the total throughputs of a node.　In our production network, some 
> critical alerts occurs when a node exceed the specified total throughput 
> which is the sum of the input and output throughputs.
> In our operation of Cassandra, the alerts occurs during the bootstrap or 
> repair processing when a new node is added. In the worst case, we have to 
> stop the operation of the exceed node.
> I have attached the patch under consideration. I would like to add a new 
> limiter class, StreamInboundRateLimiter, and use the limiter class in 
> StreamDeserializer class. I use Row::dataSize( ）to get the input throughput 
> in StreamDeserializer::newPartition(), but I am not sure whether the 
> dataSize() returns the correct data size.
> Can someone please tell me how to do it ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Fix filtering on clustering columns when 2i is used

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 65f8bb6c4 -> e06dae81f


 Fix filtering on clustering columns when 2i is used

Patch by Alex Petrov; reviewed by Benjamin Lerer for CASSANDRA-11907


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c857919b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c857919b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c857919b

Branch: refs/heads/cassandra-2.2
Commit: c857919b40b9fb27139424944e9fb6cc58befc48
Parents: bd6ad43
Author: Alex Petrov 
Authored: Mon Jul 4 14:15:39 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:15:39 2016 +0200

--
 CHANGES.txt |  1 +
 .../cql3/statements/SelectStatement.java| 10 ++-
 .../cql3/validation/operations/SelectTest.java  | 64 
 3 files changed, 74 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3aa5ea9..0967ce4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.16
+ * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
  * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 
 2.1.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6351bb5..245e64e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1582,6 +1582,7 @@ public class SelectStatement implements CQLStatement
 
 int numberOfRestrictionsEvaluatedWithSlices = 0;
 
+Restriction lastSliceRestriction = null;
 for (ColumnDefinition def : cfm.clusteringColumns())
 {
 // Remove clustering column restrictions that can be handled 
by slices; the remainder will be
@@ -1589,10 +1590,17 @@ public class SelectStatement implements CQLStatement
 Boolean indexed = stmt.restrictedColumns.get(def);
 if (indexed == null)
 break;
-if (!(indexed && stmt.usesSecondaryIndexing) && 
stmt.columnRestrictions[def.position()].canEvaluateWithSlices())
+
+Restriction restriction = 
stmt.columnRestrictions[def.position()];
+if (lastSliceRestriction != null && 
!restriction.equals(lastSliceRestriction))
+break;
+
+if (!(indexed && stmt.usesSecondaryIndexing) && 
restriction.canEvaluateWithSlices())
 {
 stmt.restrictedColumns.remove(def);
 numberOfRestrictionsEvaluatedWithSlices++;
+if (restriction.isSlice())
+lastSliceRestriction = restriction;
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
index 6acab6f..68cf6f8 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
@@ -1264,4 +1264,68 @@ public class SelectTest extends CQLTester
"SELECT * FROM %s WHERE a = 'foo' AND b= 'bar' AND 
c IN (?, ?)",
new String(TOO_BIG.array()), new 
String(TOO_BIG.array()));
 }
+
+@Test
+public void testFilteringWithSecondaryIndex() throws Throwable
+{
+createTable("CREATE TABLE %s (pk int, " +
+"c1 int, " +
+"c2 int, " +
+"c3 int, " +
+"v int, " +
+"PRIMARY KEY (pk, c1, c2, c3))");
+createIndex("CREATE INDEX v_idx_1 ON %s (v);");
+
+for (int i = 1; i <= 5; i++)
+{
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, 1, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, i, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, i, i, i);
+ 

[2/2] cassandra git commit: Merge branch cassandra-2.1 into cassandra-2.2

[blerer]

Merge branch cassandra-2.1 into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e06dae81
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e06dae81
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e06dae81

Branch: refs/heads/cassandra-2.2
Commit: e06dae81fb08870ef6a6596b1557b88fc7762302
Parents: 65f8bb6 c857919
Author: Benjamin Lerer 
Authored: Mon Jul 4 14:20:34 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:22:55 2016 +0200

--
 CHANGES.txt |  1 +
 .../restrictions/MultiColumnRestriction.java|  2 +-
 .../restrictions/PrimaryKeyRestrictionSet.java  | 81 
 .../restrictions/StatementRestrictions.java | 42 --
 .../SelectMultiColumnRelationTest.java  | 32 +++-
 .../cql3/validation/operations/SelectTest.java  | 61 +++
 6 files changed, 142 insertions(+), 77 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/CHANGES.txt
--
diff --cc CHANGES.txt
index 13a1c4f,0967ce4..451575c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,41 -1,9 +1,42 @@@
 -2.1.16
 +2.2.8
 + * MemoryUtil.getShort() should return an unsigned short also for 
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
 +Merged from 2.1:
+  * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 -
 -2.1.15
   * Account for partition deletions in tombstone histogram (CASSANDRA-12112)
 +
 +
 +2.2.7
 + * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755)
 + * Validate bloom_filter_fp_chance against lowest supported
 +   value when the table is created (CASSANDRA-11920)
 + * RandomAccessReader: call isEOF() only when rebuffering, not for every read 
operation (CASSANDRA-12013)
 + * Don't send erroneous NEW_NODE notifications on restart (CASSANDRA-11038)
 + * StorageService shutdown hook should use a volatile variable 
(CASSANDRA-11984)
 + * Persist local metadata earlier in startup sequence (CASSANDRA-11742)
 + * Run CommitLog tests with different compression settings (CASSANDRA-9039)
 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
 + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)
 + * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
 + * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
   * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043)
   * Remove finished incoming streaming connections from MessagingService 
(CASSANDRA-11854)
   * Don't try to get sstables for non-repairing column families 
(CASSANDRA-12077)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e06dae81/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
index 96e6f2b,000..51e2ce4
mode 100644,00..100644
--- 
a/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
+++ 
b/src/java/org/apache/cassandra/cql3/restrictions/MultiColumnRestriction.java
@@@ -1,515 -1,0 +1,515 @@@
 +/*
 

[jira] [Updated] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-12071:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

committed, thanks

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
> Fix For: 3.9
>
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Increase size of flushExecutor thread pool size

[marcuse]

Increase size of flushExecutor thread pool size

Patch by marcuse; reviewed by Branimir Lambov for CASSANDRA-12071


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44e475c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44e475c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44e475c1

Branch: refs/heads/trunk
Commit: 44e475c1a7aa351763fa5bd97664394186565b41
Parents: c86b3e1
Author: Marcus Eriksson 
Authored: Mon Jul 4 10:49:43 2016 +0200
Committer: Marcus Eriksson 
Committed: Mon Jul 4 14:17:01 2016 +0200

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 10 +-
 2 files changed, 10 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44e475c1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 475365f..b7330f0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Increase size of flushExecutor thread pool (CASSANDRA-12071)
 Merged from 3.0:
  * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
  * Fix EOF exception when altering column type (CASSANDRA-11820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/44e475c1/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 523e15f..010800b 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -125,7 +125,15 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 
 private static final Logger logger = 
LoggerFactory.getLogger(ColumnFamilyStore.class);
 
-private static final ExecutorService flushExecutor = new 
JMXEnabledThreadPoolExecutor(1,
+/*
+We keep a pool of threads for each data directory, size of each pool is 
memtable_flush_writers.
+When flushing we start a Flush runnable in the flushExecutor. Flush 
calculates how to split the
+memtable ranges over the existing data directories and creates a 
FlushRunnable for each of the directories.
+The FlushRunnables are executed in the perDiskflushExecutors and the Flush 
will block until all FlushRunnables
+are finished. By having flushExecutor size the same size as each of the 
perDiskflushExecutors we make sure we can
+have that many flushes going at the same time.
+*/
+private static final ExecutorService flushExecutor = new 
JMXEnabledThreadPoolExecutor(DatabaseDescriptor.getFlushWriters(),

   StageManager.KEEPALIVE,

   TimeUnit.SECONDS,

   new LinkedBlockingQueue(),

[1/3] cassandra git commit: Increase size of flushExecutor thread pool size

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.9 c86b3e183 -> 44e475c1a
  refs/heads/trunk f542a2ea6 -> dec1bdb20


Increase size of flushExecutor thread pool size

Patch by marcuse; reviewed by Branimir Lambov for CASSANDRA-12071


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44e475c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44e475c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44e475c1

Branch: refs/heads/cassandra-3.9
Commit: 44e475c1a7aa351763fa5bd97664394186565b41
Parents: c86b3e1
Author: Marcus Eriksson 
Authored: Mon Jul 4 10:49:43 2016 +0200
Committer: Marcus Eriksson 
Committed: Mon Jul 4 14:17:01 2016 +0200

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 10 +-
 2 files changed, 10 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44e475c1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 475365f..b7330f0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Increase size of flushExecutor thread pool (CASSANDRA-12071)
 Merged from 3.0:
  * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
  * Fix EOF exception when altering column type (CASSANDRA-11820)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/44e475c1/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 523e15f..010800b 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -125,7 +125,15 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 
 private static final Logger logger = 
LoggerFactory.getLogger(ColumnFamilyStore.class);
 
-private static final ExecutorService flushExecutor = new 
JMXEnabledThreadPoolExecutor(1,
+/*
+We keep a pool of threads for each data directory, size of each pool is 
memtable_flush_writers.
+When flushing we start a Flush runnable in the flushExecutor. Flush 
calculates how to split the
+memtable ranges over the existing data directories and creates a 
FlushRunnable for each of the directories.
+The FlushRunnables are executed in the perDiskflushExecutors and the Flush 
will block until all FlushRunnables
+are finished. By having flushExecutor size the same size as each of the 
perDiskflushExecutors we make sure we can
+have that many flushes going at the same time.
+*/
+private static final ExecutorService flushExecutor = new 
JMXEnabledThreadPoolExecutor(DatabaseDescriptor.getFlushWriters(),

   StageManager.KEEPALIVE,

   TimeUnit.SECONDS,

   new LinkedBlockingQueue(),

[3/3] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

[marcuse]

Merge branch 'cassandra-3.9' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dec1bdb2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dec1bdb2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dec1bdb2

Branch: refs/heads/trunk
Commit: dec1bdb20dcbacc8fba1ab8a14c16905a6384a34
Parents: f542a2e 44e475c
Author: Marcus Eriksson 
Authored: Mon Jul 4 14:17:51 2016 +0200
Committer: Marcus Eriksson 
Committed: Mon Jul 4 14:17:51 2016 +0200

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 10 +-
 2 files changed, 10 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dec1bdb2/CHANGES.txt
--
diff --cc CHANGES.txt
index 44b6699,b7330f0..309667e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
 +3.10
 + * Remove pre-startup check for open JMX port (CASSANDRA-12074)
 +
  3.9
+  * Increase size of flushExecutor thread pool (CASSANDRA-12071)
  Merged from 3.0:
   * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
   * Fix EOF exception when altering column type (CASSANDRA-11820)

cassandra git commit: Fix filtering on clustering columns when 2i is used

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 bd6ad43b9 -> c857919b4


 Fix filtering on clustering columns when 2i is used

Patch by Alex Petrov; reviewed by Benjamin Lerer for CASSANDRA-11907


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c857919b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c857919b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c857919b

Branch: refs/heads/cassandra-2.1
Commit: c857919b40b9fb27139424944e9fb6cc58befc48
Parents: bd6ad43
Author: Alex Petrov 
Authored: Mon Jul 4 14:15:39 2016 +0200
Committer: Benjamin Lerer 
Committed: Mon Jul 4 14:15:39 2016 +0200

--
 CHANGES.txt |  1 +
 .../cql3/statements/SelectStatement.java| 10 ++-
 .../cql3/validation/operations/SelectTest.java  | 64 
 3 files changed, 74 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3aa5ea9..0967ce4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.16
+ * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
  * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 
 2.1.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 6351bb5..245e64e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -1582,6 +1582,7 @@ public class SelectStatement implements CQLStatement
 
 int numberOfRestrictionsEvaluatedWithSlices = 0;
 
+Restriction lastSliceRestriction = null;
 for (ColumnDefinition def : cfm.clusteringColumns())
 {
 // Remove clustering column restrictions that can be handled 
by slices; the remainder will be
@@ -1589,10 +1590,17 @@ public class SelectStatement implements CQLStatement
 Boolean indexed = stmt.restrictedColumns.get(def);
 if (indexed == null)
 break;
-if (!(indexed && stmt.usesSecondaryIndexing) && 
stmt.columnRestrictions[def.position()].canEvaluateWithSlices())
+
+Restriction restriction = 
stmt.columnRestrictions[def.position()];
+if (lastSliceRestriction != null && 
!restriction.equals(lastSliceRestriction))
+break;
+
+if (!(indexed && stmt.usesSecondaryIndexing) && 
restriction.canEvaluateWithSlices())
 {
 stmt.restrictedColumns.remove(def);
 numberOfRestrictionsEvaluatedWithSlices++;
+if (restriction.isSlice())
+lastSliceRestriction = restriction;
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c857919b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
index 6acab6f..68cf6f8 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
@@ -1264,4 +1264,68 @@ public class SelectTest extends CQLTester
"SELECT * FROM %s WHERE a = 'foo' AND b= 'bar' AND 
c IN (?, ?)",
new String(TOO_BIG.array()), new 
String(TOO_BIG.array()));
 }
+
+@Test
+public void testFilteringWithSecondaryIndex() throws Throwable
+{
+createTable("CREATE TABLE %s (pk int, " +
+"c1 int, " +
+"c2 int, " +
+"c3 int, " +
+"v int, " +
+"PRIMARY KEY (pk, c1, c2, c3))");
+createIndex("CREATE INDEX v_idx_1 ON %s (v);");
+
+for (int i = 1; i <= 5; i++)
+{
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, 1, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, 1, i, i);
+execute("INSERT INTO %s (pk, c1, c2, c3, v) VALUES (?, ?, ?, ?, 
?)", 1, 1, i, i, i);
+ 

[jira] [Commented] (CASSANDRA-11907) 2i behaviour is different in different versions

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361231#comment-15361231
 ] 

Benjamin Lerer commented on CASSANDRA-11907:


Thanks for the patch. +1

> 2i behaviour is different in different versions
> ---
>
> Key: CASSANDRA-11907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11907
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tommy Stendahl
>Assignee: Alex Petrov
>
>  I think I have found more cases where 2i behave different in different 
> Cassandra versions, CASSANDRA-11510 solved one such case but I think there 
> are a few more.
> I get one behaviour with 2.1.14 and Trunk and I think this is the correct 
> one. With 2.2.7 and 3.0.6 the behaviour is different.
> To test this I used ccm to setup one node clusters with the different 
> versions, I prepared each cluster with these commands:
> {code:sql}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1' };
> CREATE TABLE test.table1 (name text,class int,inter text,foo text,power 
> int,PRIMARY KEY (name, class, inter, foo)) WITH CLUSTERING ORDER BY (class 
> DESC, inter ASC);
> CREATE INDEX table1_power ON test.table1 (power) ;
> CREATE TABLE test.table2 (name text,class int,inter text,foo text,power 
> int,PRIMARY KEY (name, class, inter, foo)) WITH CLUSTERING ORDER BY (class 
> DESC, inter ASC);
> CREATE INDEX table2_inter ON test.table2 (inter) ;
> {code}
> I executed two select quieries on each cluster:
> {code:sql}
> SELECT * FROM test.table1 where name='R1' AND class>0 AND class<4 AND 
> inter='int1' AND power=18 ALLOW FILTERING;
> SELECT * FROM test.table2 where name='R1' AND class>0 AND class<4 AND 
> inter='int1' AND foo='aa' ALLOW FILTERING;
> {code}
> On 2.1.14 and Trunk they where successful. But on 2.2.7 and 3.0.6 they 
> failed, the first one with {{InvalidRequest: code=2200 [Invalid query] 
> message="Clustering column "inter" cannot be restricted (preceding column 
> "class" is restricted by a non-EQ relation)"}} and the second one with 
> {{InvalidRequest: code=2200 [Invalid query] message="Clustering column "foo" 
> cannot be restricted (preceding column "inter" is restricted by a non-EQ 
> relation)"}}.
> I could get the queries to execute successfully on 2.2.7 and 3.0.6 by 
> creating two more 2i:
> {code:sql}
> CREATE INDEX table1_inter ON test.table1 (inter) ;
> CREATE INDEX table2_foo ON test.table2 (foo) ;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361225#comment-15361225
 ] 

Branimir Lambov commented on CASSANDRA-12071:
-

+1 to commit.

The growing number of threads in this piece of code is a bit worrying, but 
improving that needs some non-trivial restructuring (which should come soon 
enough as part of CASSANDRA-8496).

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
> Fix For: 3.9
>
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11477) MerkleTree mismatch when a cell is shadowed by a range tombstone in different SSTables

[Branimir Lambov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Branimir Lambov resolved CASSANDRA-11477.
-
   Resolution: Duplicate
Reproduced In: 2.2.5, 2.1.13  (was: 2.1.13, 2.2.5)

> MerkleTree mismatch when a cell is shadowed by a range tombstone in different 
> SSTables
> --
>
> Key: CASSANDRA-11477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11477
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Fabien Rousseau
>  Labels: repair
> Attachments: 11477-2.1.patch
>
>
> Below is a script which allows to reproduce the problem:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 text,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3) VALUES ( 'a', 'b', 'c');
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush 
> ccm node1 cqlsh
> USE test_rt;
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair test_rt table1
> # now grep the log and observe that there was some inconstencies detected 
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> The wrong hash will be computed on node1, which will include the previously 
> deleted cell, thus resulting in a MT mismatch.
> This is due to the fact that, in LazilyCompactedRow, the RT is not added into 
> the RT tracker (in fact, it's added only if it is GCable, but should always 
> be added).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11349) MerkleTree mismatch when multiple range tombstones exists for the same partition and interval

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361168#comment-15361168
 ] 

Branimir Lambov commented on CASSANDRA-11349:
-

Tests look ok, all failures are either failing on base or failed very recently.

> MerkleTree mismatch when multiple range tombstones exists for the same 
> partition and interval
> -
>
> Key: CASSANDRA-11349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11349
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Stefan Podkowinski
>  Labels: repair
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 11349-2.1-v2.patch, 11349-2.1-v3.patch, 
> 11349-2.1-v4.patch, 11349-2.1.patch, 11349-2.2-v4.patch
>
>
> We observed that repair, for some of our clusters, streamed a lot of data and 
> many partitions were "out of sync".
> Moreover, the read repair mismatch ratio is around 3% on those clusters, 
> which is really high.
> After investigation, it appears that, if two range tombstones exists for a 
> partition for the same range/interval, they're both included in the merkle 
> tree computation.
> But, if for some reason, on another node, the two range tombstones were 
> already compacted into a single range tombstone, this will result in a merkle 
> tree difference.
> Currently, this is clearly bad because MerkleTree differences are dependent 
> on compactions (and if a partition is deleted and created multiple times, the 
> only way to ensure that repair "works correctly"/"don't overstream data" is 
> to major compact before each repair... which is not really feasible).
> Below is a list of steps allowing to easily reproduce this case:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 float,
> c4 float,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 2);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush 
> ccm node1 cqlsh
> USE test_rt;
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 3);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair
> # now grep the log and observe that there was some inconstencies detected 
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> Consequences of this are a costly repair, accumulating many small SSTables 
> (up to thousands for a rather short period of time when using VNodes, the 
> time for compaction to absorb those small files), but also an increased size 
> on disk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12073) [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial results

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361102#comment-15361102
 ] 

DOAN DuyHai commented on CASSANDRA-12073:
-

[~xedin]  new patch attached (V2) with your inline suggestion + 1 extra unit 
test for {{OnDiskIndex}}

> [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial 
> results
> ---
>
> Key: CASSANDRA-12073
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12073
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.7
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
> Fix For: 3.x
>
> Attachments: patch_PREFIX_search_with_CONTAINS_mode.txt, 
> patch_PREFIX_search_with_CONTAINS_mode_V2.txt
>
>
> {noformat}
> cqlsh:music> CREATE TABLE music.albums (
> id uuid PRIMARY KEY,
> artist text,
> country text,
> quality text,
> status text,
> title text,
> year int
> );
> cqlsh:music> CREATE CUSTOM INDEX albums_artist_idx ON music.albums (artist) 
> USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 
> 'CONTAINS', 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> cqlsh:music> SELECT * FROM albums WHERE artist like 'lady%'  LIMIT 100;
>  id   | artist| country| quality 
> | status| title | year
> --+---++-+---+---+--
>  372bb0ab-3263-41bc-baad-bb520ddfa787 | Lady Gaga |USA |  normal 
> |  Official |   Red and Blue EP | 2006
>  1a4abbcd-b5de-4c69-a578-31231e01ff09 | Lady Gaga |Unknown |  normal 
> | Promotion |Poker Face | 2008
>  31f4a0dc-9efc-48bf-9f5e-bfc09af42b82 | Lady Gaga |USA |  normal 
> |  Official |   The Cherrytree Sessions | 2009
>  8ebfaebd-28d0-477d-b735-469661ce6873 | Lady Gaga |Unknown |  normal 
> |  Official |Poker Face | 2009
>  98107d82-e0dd-46bc-a273-1577578984c7 | Lady Gaga |USA |  normal 
> |  Official |   Just Dance: The Remixes | 2008
>  a76af0f2-f5c5-4306-974a-e3c17158e6c6 | Lady Gaga |  Italy |  normal 
> |  Official |  The Fame | 2008
>  849ee019-8b15-4767-8660-537ab9710459 | Lady Gaga |USA |  normal 
> |  Official |Christmas Tree | 2008
>  4bad59ac-913f-43da-9d48-89adc65453d2 | Lady Gaga |  Australia |  normal 
> |  Official | Eh Eh | 2009
>  80327731-c450-457f-bc12-0a8c21fd9c5d | Lady Gaga |USA |  normal 
> |  Official | Just Dance Remixes Part 2 | 2008
>  3ad33659-e932-4d31-a040-acab0e23c3d4 | Lady Gaga |Unknown |  normal 
> |  null |Just Dance | 2008
>  9adce7f6-6a1d-49fd-b8bd-8f6fac73558b | Lady Gaga | United Kingdom |  normal 
> |  Official |Just Dance | 2009
> (11 rows)
> {noformat}
> *SASI* says that there are only 11 artists whose name starts with {{lady}}.
> However, in the data set, there are:
> * Lady Pank
> * Lady Saw
> * Lady Saw
> * Ladyhawke
> * Ladytron
> * Ladysmith Black Mambazo
> * Lady Gaga
> * Lady Sovereign
> etc ...
> By debugging the source code, the issue is in 
> {{OnDiskIndex.TermIterator::computeNext()}}
> {code:java}
> for (;;)
> {
> if (currentBlock == null)
> return endOfData();
> if (offset >= 0 && offset < currentBlock.termCount())
> {
> DataTerm currentTerm = currentBlock.getTerm(nextOffset());
> if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
> continue;
> // flip the flag right on the first bounds match
> // to avoid expensive comparisons
> checkLower = false;
> if (checkUpper && !e.isUpperSatisfiedBy(currentTerm))
> return endOfData();
> return currentTerm;
> }
> nextBlock();
> }
> {code}
>  So the {{endOfData()}} conditions are:
> * currentBlock == null
> * checkUpper && !e.isUpperSatisfiedBy(currentTerm)
> The problem is that {{e::isUpperSatisfiedBy}} is checking not only whether 
> the term match but also returns *false* when it's a *partial term* !
> {code:java}
> public boolean isUpperSatisfiedBy(OnDiskIndex.DataTerm term)
> {
> if (!hasUpper())
> return true;
> if (nonMatchingPartial(term))
> return false;
> int cmp = term.compareTo(validator, upper.value, false);
> return cmp < 0 || cmp == 0 && 

[jira] [Updated] (CASSANDRA-12073) [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial results

[DOAN DuyHai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

DOAN DuyHai updated CASSANDRA-12073:

Attachment: patch_PREFIX_search_with_CONTAINS_mode_V2.txt

> [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial 
> results
> ---
>
> Key: CASSANDRA-12073
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12073
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.7
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
> Fix For: 3.x
>
> Attachments: patch_PREFIX_search_with_CONTAINS_mode.txt, 
> patch_PREFIX_search_with_CONTAINS_mode_V2.txt
>
>
> {noformat}
> cqlsh:music> CREATE TABLE music.albums (
> id uuid PRIMARY KEY,
> artist text,
> country text,
> quality text,
> status text,
> title text,
> year int
> );
> cqlsh:music> CREATE CUSTOM INDEX albums_artist_idx ON music.albums (artist) 
> USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 
> 'CONTAINS', 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> cqlsh:music> SELECT * FROM albums WHERE artist like 'lady%'  LIMIT 100;
>  id   | artist| country| quality 
> | status| title | year
> --+---++-+---+---+--
>  372bb0ab-3263-41bc-baad-bb520ddfa787 | Lady Gaga |USA |  normal 
> |  Official |   Red and Blue EP | 2006
>  1a4abbcd-b5de-4c69-a578-31231e01ff09 | Lady Gaga |Unknown |  normal 
> | Promotion |Poker Face | 2008
>  31f4a0dc-9efc-48bf-9f5e-bfc09af42b82 | Lady Gaga |USA |  normal 
> |  Official |   The Cherrytree Sessions | 2009
>  8ebfaebd-28d0-477d-b735-469661ce6873 | Lady Gaga |Unknown |  normal 
> |  Official |Poker Face | 2009
>  98107d82-e0dd-46bc-a273-1577578984c7 | Lady Gaga |USA |  normal 
> |  Official |   Just Dance: The Remixes | 2008
>  a76af0f2-f5c5-4306-974a-e3c17158e6c6 | Lady Gaga |  Italy |  normal 
> |  Official |  The Fame | 2008
>  849ee019-8b15-4767-8660-537ab9710459 | Lady Gaga |USA |  normal 
> |  Official |Christmas Tree | 2008
>  4bad59ac-913f-43da-9d48-89adc65453d2 | Lady Gaga |  Australia |  normal 
> |  Official | Eh Eh | 2009
>  80327731-c450-457f-bc12-0a8c21fd9c5d | Lady Gaga |USA |  normal 
> |  Official | Just Dance Remixes Part 2 | 2008
>  3ad33659-e932-4d31-a040-acab0e23c3d4 | Lady Gaga |Unknown |  normal 
> |  null |Just Dance | 2008
>  9adce7f6-6a1d-49fd-b8bd-8f6fac73558b | Lady Gaga | United Kingdom |  normal 
> |  Official |Just Dance | 2009
> (11 rows)
> {noformat}
> *SASI* says that there are only 11 artists whose name starts with {{lady}}.
> However, in the data set, there are:
> * Lady Pank
> * Lady Saw
> * Lady Saw
> * Ladyhawke
> * Ladytron
> * Ladysmith Black Mambazo
> * Lady Gaga
> * Lady Sovereign
> etc ...
> By debugging the source code, the issue is in 
> {{OnDiskIndex.TermIterator::computeNext()}}
> {code:java}
> for (;;)
> {
> if (currentBlock == null)
> return endOfData();
> if (offset >= 0 && offset < currentBlock.termCount())
> {
> DataTerm currentTerm = currentBlock.getTerm(nextOffset());
> if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
> continue;
> // flip the flag right on the first bounds match
> // to avoid expensive comparisons
> checkLower = false;
> if (checkUpper && !e.isUpperSatisfiedBy(currentTerm))
> return endOfData();
> return currentTerm;
> }
> nextBlock();
> }
> {code}
>  So the {{endOfData()}} conditions are:
> * currentBlock == null
> * checkUpper && !e.isUpperSatisfiedBy(currentTerm)
> The problem is that {{e::isUpperSatisfiedBy}} is checking not only whether 
> the term match but also returns *false* when it's a *partial term* !
> {code:java}
> public boolean isUpperSatisfiedBy(OnDiskIndex.DataTerm term)
> {
> if (!hasUpper())
> return true;
> if (nonMatchingPartial(term))
> return false;
> int cmp = term.compareTo(validator, upper.value, false);
> return cmp < 0 || cmp == 0 && upper.inclusive;
> }
> {code}
> By debugging the OnDiskIndex data, I've 

[jira] [Comment Edited] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361046#comment-15361046
 ] 

Marcus Eriksson edited comment on CASSANDRA-12071 at 7/4/16 9:05 AM:
-

{{Flush}} runs on the {{flushExecutor}} and splits the memtable up based on the 
number of data directories on the node and executes the {{FlushRunnables}} on 
the per disk flush executors. It then blocks until all per disk runnables are 
completed to be able to run {{PostFlush}} etc.

Since {{flushExecutor}} size is 1 we can only ever execute a single flush at a 
time.

Patch to bump it to {{memtable_flush_writers}} 
[here|https://github.com/krummas/cassandra/commits/marcuse/12071] - in my tests 
this removes the timeout issues

[~aweisberg] could you try it out?

[~blambov] to review since you might have some ideas for further improvements


was (Author: krummas):
{{Flush}} runs on the {{flushExecutor}} and splits the memtable up based on the 
number of data directories on the node and executes the {{FlushRunnable}}s on 
the per disk flush executors. It then blocks until all per disk runnables are 
completed to be able to run {{PostFlush}} etc.

Since {{flushExecutor}} size is 1 we can only ever execute a single flush at a 
time.

Patch to bump it to {{memtable_flush_writers}} 
[here|https://github.com/krummas/cassandra/commits/marcuse/12071] - in my tests 
this removes the timeout issues

[~aweisberg] could you try it out?

[~blambov] to review since you might have some ideas for further improvements

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
> Fix For: 3.9
>
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361046#comment-15361046
 ] 

Marcus Eriksson edited comment on CASSANDRA-12071 at 7/4/16 9:05 AM:
-

{{Flush}} runs on the {{flushExecutor}} and splits the memtable up based on the 
number of data directories on the node and executes the {{FlushRunnables}} on 
the per disk flush executors. It then blocks until all per disk runnables are 
completed to be able to run {{PostFlush}} etc.

Since {{flushExecutor}} size is 1 we can only ever execute a single flush at a 
time.

Patch to bump it to {{memtable_flush_writers}} 
[here|https://github.com/krummas/cassandra/commits/marcuse/12071] - in my tests 
this removes the timeout issues

[~aweisberg] could you try it out?

[~blambov] to review since you might have some ideas for further improvements

tests here:
http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-12071-dtest/
http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-12071-testall/


was (Author: krummas):
{{Flush}} runs on the {{flushExecutor}} and splits the memtable up based on the 
number of data directories on the node and executes the {{FlushRunnables}} on 
the per disk flush executors. It then blocks until all per disk runnables are 
completed to be able to run {{PostFlush}} etc.

Since {{flushExecutor}} size is 1 we can only ever execute a single flush at a 
time.

Patch to bump it to {{memtable_flush_writers}} 
[here|https://github.com/krummas/cassandra/commits/marcuse/12071] - in my tests 
this removes the timeout issues

[~aweisberg] could you try it out?

[~blambov] to review since you might have some ideas for further improvements

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
> Fix For: 3.9
>
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361046#comment-15361046
 ] 

Marcus Eriksson edited comment on CASSANDRA-12071 at 7/4/16 9:04 AM:
-

{{Flush}} runs on the {{flushExecutor}} and splits the memtable up based on the 
number of data directories on the node and executes the {{FlushRunnable}}s on 
the per disk flush executors. It then blocks until all per disk runnables are 
completed to be able to run {{PostFlush}} etc.

Since {{flushExecutor}} size is 1 we can only ever execute a single flush at a 
time.

Patch to bump it to {{memtable_flush_writers}} 
[here|https://github.com/krummas/cassandra/commits/marcuse/12071] - in my tests 
this removes the timeout issues

[~aweisberg] could you try it out?

[~blambov] to review since you might have some ideas for further improvements


was (Author: krummas):
{{Flush}} runs on the flushExecutor and splits the memtable up based on the 
number of data directories on the node and executes the {{FlushRunnable}}s on 
the per disk flush executors. It then blocks until all per disk runnables are 
completed to be able to run {{PostFlush}} etc.

Since {{flushExecutor}} size is 1 we can only ever execute a single flush at a 
time.

Patch to bump it to {{memtable_flush_writers}} 
[here|https://github.com/krummas/cassandra/commits/marcuse/12071] - in my tests 
this removes the timeout issues

[~aweisberg] could you try it out?

[~blambov] to review since you might have some ideas for further improvements

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
> Fix For: 3.9
>
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-12071:

 Reviewer: Branimir Lambov
Fix Version/s: 3.9
   Status: Patch Available  (was: Open)

{{Flush}} runs on the flushExecutor and splits the memtable up based on the 
number of data directories on the node and executes the {{FlushRunnable}}s on 
the per disk flush executors. It then blocks until all per disk runnables are 
completed to be able to run {{PostFlush}} etc.

Since {{flushExecutor}} size is 1 we can only ever execute a single flush at a 
time.

Patch to bump it to {{memtable_flush_writers}} 
[here|https://github.com/krummas/cassandra/commits/marcuse/12071] - in my tests 
this removes the timeout issues

[~aweisberg] could you try it out?

[~blambov] to review since you might have some ideas for further improvements

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
> Fix For: 3.9
>
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11349) MerkleTree mismatch when multiple range tombstones exists for the same partition and interval

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361016#comment-15361016
 ] 

Branimir Lambov commented on CASSANDRA-11349:
-

Rebased patch here:
|[2.1|https://github.com/blambov/cassandra/tree/11349]|[utests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-11349-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-11349-dtest/]|
|[2.2|https://github.com/blambov/cassandra/tree/11349-2.2]|[utests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-11349-2.2-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-11349-2.2-dtest/]|


> MerkleTree mismatch when multiple range tombstones exists for the same 
> partition and interval
> -
>
> Key: CASSANDRA-11349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11349
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Stefan Podkowinski
>  Labels: repair
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 11349-2.1-v2.patch, 11349-2.1-v3.patch, 
> 11349-2.1-v4.patch, 11349-2.1.patch, 11349-2.2-v4.patch
>
>
> We observed that repair, for some of our clusters, streamed a lot of data and 
> many partitions were "out of sync".
> Moreover, the read repair mismatch ratio is around 3% on those clusters, 
> which is really high.
> After investigation, it appears that, if two range tombstones exists for a 
> partition for the same range/interval, they're both included in the merkle 
> tree computation.
> But, if for some reason, on another node, the two range tombstones were 
> already compacted into a single range tombstone, this will result in a merkle 
> tree difference.
> Currently, this is clearly bad because MerkleTree differences are dependent 
> on compactions (and if a partition is deleted and created multiple times, the 
> only way to ensure that repair "works correctly"/"don't overstream data" is 
> to major compact before each repair... which is not really feasible).
> Below is a list of steps allowing to easily reproduce this case:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 float,
> c4 float,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 2);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush 
> ccm node1 cqlsh
> USE test_rt;
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 3);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair
> # now grep the log and observe that there was some inconstencies detected 
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> Consequences of this are a costly repair, accumulating many small SSTables 
> (up to thousands for a rather short period of time when using VNodes, the 
> time for compaction to absorb those small files), but also an increased size 
> on disk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12129) deb Packages cannot be installed under Ubuntu 16.04 (xenial) due to missing dependency

[Stuart Bishop (JIRA)]

Stuart Bishop created CASSANDRA-12129:
-

 Summary: deb Packages cannot be installed under Ubuntu 16.04 
(xenial) due to missing dependency
 Key: CASSANDRA-12129
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12129
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
 Environment: Ubuntu 16.04 (Xenial)
Reporter: Stuart Bishop


The deb packages depend on python-support, which no longer exists in Ubuntu 
16.04 (xenial).

$ sudo apt install cassandra
Reading package lists... Done
Building dependency tree   
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 cassandra : Depends: python-support (>= 0.90.0) but it is not installable
 Recommends: ntp but it is not going to be installed or
 time-daemon
E: Unable to correct problems, you have held broken packages.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10988) isInclusive and boundsAsComposites in Restriction take bounds in different order

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer resolved CASSANDRA-10988.

Resolution: Fixed

> isInclusive and boundsAsComposites in Restriction take bounds in different 
> order
> 
>
> Key: CASSANDRA-10988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10988
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Vassil Hristov
>Assignee: Alex Petrov
> Fix For: 2.2.x
>
>
> After we've upgraded our cluster to version 2.1.11, we started getting the 
> bellow exceptions for some of our queries. Issue seems to be very similar to 
> CASSANDRA-7284.
> Code to reproduce:
> {code:java}
> createTable("CREATE TABLE %s (" +
> "a text," +
> "b int," +
> "PRIMARY KEY (a, b)" +
> ") WITH COMPACT STORAGE" +
> "AND CLUSTERING ORDER BY (b DESC)");
> execute("insert into %s (a, b) values ('a', 2)");
> execute("SELECT * FROM %s WHERE a = 'a' AND b > 0");
> {code}
> {code:java}
> java.lang.ClassCastException: 
> org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast 
> to org.apache.cassandra.db.composites.CellName
> at 
> org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:188)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.composites.AbstractSimpleCellNameType.makeCellName(AbstractSimpleCellNameType.java:125)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.composites.AbstractCellNameType.makeCellName(AbstractCellNameType.java:254)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.makeExclusiveSliceBound(SelectStatement.java:1197)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.applySliceRestriction(SelectStatement.java:1205)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.processColumnFamily(SelectStatement.java:1283)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:1250)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:299)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:276)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:67)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:238)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:493)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:138)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
>  [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
>  [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_66]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  [apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> at 

[jira] [Commented] (CASSANDRA-11944) sstablesInBounds might not actually give all sstables within the bounds due to having start positions moved in sstables

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360927#comment-15360927
 ] 

Marcus Eriksson commented on CASSANDRA-11944:
-

ping on this [~benedict]

> sstablesInBounds might not actually give all sstables within the bounds due 
> to having start positions moved in sstables
> ---
>
> Key: CASSANDRA-11944
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11944
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> Same problem as with CASSANDRA-11886 - if we try to fetch sstablesInBounds 
> for CANONICAL_SSTABLES, we can miss some actually overlapping sstables. In 
> 3.0+ we state which SSTableSet we want when calling the method.
> Looks like the only issue this could cause is that we include a few too many 
> sstables in compactions that we think contain only droppable tombstones



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)