[jira] [Updated] (CASSANDRA-13151) Unicode unittest fail

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13151:

Status: Ready to Commit  (was: Patch Available)

> Unicode unittest fail
> -
>
> Key: CASSANDRA-13151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13151
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jay Zhuang
>Assignee: Alex Petrov
>
> Following unittests are failed in 3.11 and trunk(4.0) branch
> {noformat}
> SASIIndexTest.testUnicodeSupport
> StandardAnalyzerTest.testTokenizationJaJp1
> StandardAnalyzerTest.testTokenizationJaJp2
> StandardAnalyzerTest.testTokenizationRuRu1
> StandardAnalyzerTest.testTokenizationZnTw1
> {noformat}
> It works fine on my local mac, but not linux server. I guess it's related to 
> Unicode setting. Does anyone have any idea on that? (could it be related to 
> CASSANDRA-11077, CASSANDRA-11431?)
> Here are the failure details
> {noformat}
> $ ant testsome -Dtest.name=org.apache.cassandra.index.sasi.SASIIndexTest 
> -Dtest.methods=testUnicodeSupport
> ...
> [junit] Testcase: 
> testUnicodeSupport(org.apache.cassandra.index.sasi.SASIIndexTest):
> FAILED
> [junit] []
> [junit] junit.framework.AssertionFailedError: []
> [junit] at 
> org.apache.cassandra.index.sasi.SASIIndexTest.testUnicodeSupport(SASIIndexTest.java:1159)
> [junit] at 
> org.apache.cassandra.index.sasi.SASIIndexTest.testUnicodeSupport(SASIIndexTest.java:1122)
> {noformat}
> {noformat}
> $ ant test -Dtest.name=StandardAnalyzerTest
> ...
> [junit] Testcase: 
> testTokenizationJaJp1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<210> but was:<0>
> [junit] junit.framework.AssertionFailedError: expected:<210> but was:<0>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationJaJp1(StandardAnalyzerTest.java:85)
> [junit]
> [junit]
> [junit] Testcase: 
> testTokenizationJaJp2(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<57> but was:<9>
> [junit] junit.framework.AssertionFailedError: expected:<57> but was:<9>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationJaJp2(StandardAnalyzerTest.java:104)
> [junit]
> [junit]
> [junit] Testcase: 
> testTokenizationRuRu1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<456> but was:<0>
> [junit] junit.framework.AssertionFailedError: expected:<456> but was:<0>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationRuRu1(StandardAnalyzerTest.java:120)
> [junit]
> [junit]
> [junit] Testcase: 
> testTokenizationZnTw1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<963> but was:<0>
> [junit] junit.framework.AssertionFailedError: expected:<963> but was:<0>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationZnTw1(StandardAnalyzerTest.java:136)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12883) Remove support for non-JavaScript UDFs

[jaikiran pai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960369#comment-15960369
 ] 

jaikiran pai commented on CASSANDRA-12883:
--

[~snazy], the release notes of 2.2.9 release and the title of this JIRA states 
that non-JavaScripts UDFs won't be supported. The comments in this JIRA and one 
of the _comment_ in the commit [1] seem to indicate that both "java" and 
"javasciprt" languages will be supported going forward (including 2.2.9). 
However, looking at the commit [1] there seems to be an explicit check on the 
"javascript" language. Does that mean if I have a UDF with:

{code}
LANGUAGE java
{code}

it's going  to fail starting 2.2.9? I haven't yet had a chance to try this with 
2.2.9, hence this question.


[1] 
https://github.com/apache/cassandra/commit/6f360b6d239a7b3d3b625e1bcefcaeb2f6b2be1d

> Remove support for non-JavaScript UDFs
> --
>
> Key: CASSANDRA-12883
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12883
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 2.2.9, 3.0.11, 3.10
>
>
> As recently reported in the user mailing list, JSR-223 languages other than 
> JavaScript no longer work since version 3.0.
> The reason is that the sandbox implemented in CASSANDRA-9402 restricts the 
> use of "evil" packages, classes and functions. Unfortunately, even "non-evil" 
> packages from JSR-223 providers are blocked.
> In order to get a JSR-223 provider working fine, we need to allow JSR-223 
> provider specific packages and also allow specific runtime permissions.
> The fact that "arbitrary" JSR-223 providers no longer work since 3.0 has just 
> been reported recently, means that this functionality (i.e. non-JavaSCript 
> JSR-223 UDFs) is obviously not used.
> Therefore I propose to remove support for UDFs that do not use Java or 
> JavaScript in 4.0. This will also allow to specialize scripted UDFs on 
> Nashorn and allow to use its security features, although these are limited, 
> more extensively. (Clarification: this ticket is just about to remove that 
> support)
> Also want to point out that we never "officially" supported UDFs that are not 
> Java or JavaScript.
> Sample error message:
> {code}
> Traceback (most recent call last):
>   File "/usr/bin/cqlsh.py", line 1264, in perform_simple_statement
> result = future.result()
>   File 
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py",
>  line 3650, in result
> raise self._final_exception
> FunctionFailure: Error from server: code=1400 [User Defined Function failure] 
> message="execution of 'e.test123[bigint]' failed: 
> java.security.AccessControlException: access denied: 
> ("java.lang.RuntimePermission" 
> "accessClassInPackage.org.python.jline.console")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13354:

Reviewer: Marcus Eriksson

> LCS estimated compaction tasks does not take number of files into account
> -
>
> Key: CASSANDRA-13354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.2.9
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Attachments: 13354-trunk.txt, patchedTest.png, unpatchedTest.png
>
>
> In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
> taking the size of a SSTable and multiply it by four. This would give 4*160mb 
> with default settings. This calculation is used to determine whether repaired 
> or repaired data is being compacted.
> Now this works well until you take repair into account. Repair streams over 
> many many sstables which could be smaller than the configured SSTable size 
> depending on your use case. In our case we are talking about many thousands 
> of tiny SSTables. As number of files increases one can run into any number of 
> problems, including GC issues, too many open files or plain increase in read 
> latency.
> With the current algorithm we will choose repaired or unrepaired depending on 
> whichever side has more data in it. Even if the repaired files outnumber the 
> unrepaired files by a large margin.
> Similarily, our algorithm that selects compaction candidates takes up to 32 
> SSTables at a time in L0, however our estimated task calculation does not 
> take this number into account. These two mechanisms should be aligned with 
> each other.
> I propose that we take the number of files in L0 into account when estimating 
> remaining tasks. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13188) compaction-stress AssertionError from getMemtableFor()

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960363#comment-15960363
 ] 

Yuki Morishita commented on CASSANDRA-13188:


It is not initialized because Memtable is tied to commit log, so it creates 
commit log in commit log directory (CASSANDRA-8616).

Option here is to do daemonInitialization in tool.

> compaction-stress AssertionError from getMemtableFor()
> --
>
> Key: CASSANDRA-13188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13188
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing, Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Exception:
> {noformat}
> ./compaction-stress compact -d /tmp/compaction -p 
> https://gist.githubusercontent.com/tjake/8995058fed11d9921e31/raw/a9334d1090017bf546d003e271747351a40692ea/blogpost.yaml
>  -t 4
> WARN  18:45:04,854 JNA link failure, one or more native method will be 
> unavailable.
> java.lang.AssertionError: []
> at 
> org.apache.cassandra.db.lifecycle.Tracker.getMemtableFor(Tracker.java:312)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1315)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:462)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:570)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:564)
> at 
> org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:356)
> at 
> org.apache.cassandra.schema.SchemaKeyspace.saveSystemKeyspacesSchema(SchemaKeyspace.java:265)
> at 
> org.apache.cassandra.db.SystemKeyspace.finishStartup(SystemKeyspace.java:495)
> at 
> org.apache.cassandra.stress.CompactionStress$Compaction.run(CompactionStress.java:209)
> at 
> org.apache.cassandra.stress.CompactionStress.main(CompactionStress.java:349)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13413:

   Resolution: Fixed
Fix Version/s: 4.0
   3.11.0
   3.0.13
   2.2.10
   2.1.18
   Status: Resolved  (was: Ready to Commit)

committed, thanks

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.1.18, 2.2.10, 3.0.13, 3.11.0, 4.0
>
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[09/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[marcuse]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e102b41
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e102b41
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e102b41

Branch: refs/heads/cassandra-3.11
Commit: 1e102b41e8ac03e1974a5eafacadc49809831f08
Parents: 377f611 b021a1b
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:09:09 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:09:09 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--

[14/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[marcuse]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ebdf7719
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ebdf7719
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ebdf7719

Branch: refs/heads/trunk
Commit: ebdf77194c2835a01134635e9b71ed338c01aac2
Parents: 980f32d 8d34076
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:11:19 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:11:29 2017 +0200

--
 circle.yml | 19 ++-
 doc/source/development/testing.rst |  3 +++
 2 files changed, 13 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebdf7719/doc/source/development/testing.rst
--
diff --cc doc/source/development/testing.rst
index eac336a,000..b8eea6b
mode 100644,00..100644
--- a/doc/source/development/testing.rst
+++ b/doc/source/development/testing.rst
@@@ -1,86 -1,0 +1,89 @@@
 +.. Licensed to the Apache Software Foundation (ASF) under one
 +.. or more contributor license agreements.  See the NOTICE file
 +.. distributed with this work for additional information
 +.. regarding copyright ownership.  The ASF licenses this file
 +.. to you under the Apache License, Version 2.0 (the
 +.. "License"); you may not use this file except in compliance
 +.. with the License.  You may obtain a copy of the License at
 +..
 +.. http://www.apache.org/licenses/LICENSE-2.0
 +..
 +.. Unless required by applicable law or agreed to in writing, software
 +.. distributed under the License is distributed on an "AS IS" BASIS,
 +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 +.. See the License for the specific language governing permissions and
 +.. limitations under the License.
 +
 +.. highlight:: none
 +
 +Testing
 +***
 +
 +Creating tests is one of the most important and also most difficult parts of 
developing Cassandra. There are different ways to test your code depending on 
what you're working on.
 +
 +
 +Unit Testing
 +
 +
 +The most simple way to test code in Cassandra is probably by writing a unit 
test. Cassandra uses JUnit as a testing framework and test cases can be found 
in the ``test/unit`` directory. Ideally you’d be able to create a unit test 
for your implementation that would exclusively cover the class you created (the 
unit under test). Unfortunately this is not always possible and Cassandra 
doesn’t have a very mock friendly code base. Often you’ll find yourself in 
a situation where you have to make use of an embedded Cassandra instance that 
you’ll be able to interact with in your test. If you want to make use of CQL 
in your test, you can simply extend CQLTester and use some of the convenient 
helper methods such as in the following example.
 +
 +.. code-block:: java
 +
 +  @Test
 +  public void testBatchAndList() throws Throwable
 +  {
 + createTable("CREATE TABLE %s (k int PRIMARY KEY, l list)");
 + execute("BEGIN BATCH " +
 + "UPDATE %1$s SET l = l +[ 1 ] WHERE k = 0; " +
 + "UPDATE %1$s SET l = l + [ 2 ] WHERE k = 0; " +
 + "UPDATE %1$s SET l = l + [ 3 ] WHERE k = 0; " +
 + "APPLY BATCH");
 +
 + assertRows(execute("SELECT l FROM %s WHERE k = 0"),
 +row(list(1, 2, 3)));
 +  }
 +
 +Unit tests can be run from the command line using the ``ant test`` command, 
``ant test -Dtest.name=`` to execute a test suite or ``ant 
testsome -Dtest.name= -Dtest.methods=[,testmethod2]`` for 
individual tests.  For example, to run all test methods in the 
``org.apache.cassandra.cql3.SimpleQueryTest`` class, you would run::
 +
 +ant test -Dtest.name=SimpleQueryTest
 +
 +To run only the ``testStaticCompactTables()`` test method from that class, 
you would run::
 +
 +ant testsome -Dtest.name=org.apache.cassandra.cql3.SimpleQueryTest 
-Dtest.methods=testStaticCompactTables
 +
 +Long running tests
 +--
 +
 +Test that consume a significant amount of time during execution can be found 
in the ``test/long`` directory and executed as a regular JUnit test or 
standalone program. Except for the execution time, there’s nothing really 
special about them. However, ant will execute tests under ``test/long`` only 
when using the ``ant long-test`` target.
 +
 +DTests
 +==
 +
 +One way of doing integration or system testing at larger scale is by using 
`dtest `_, which stands for 
“Cassandra Distributed Tests”. The idea is to automatically setup Cassandra 
clusters using various configurations and simulate certain use cases you want 
to test. This is done using Python scripts and ``ccm

[04/15] cassandra git commit: Make circleci run more test targets.

[marcuse]

Make circleci run more test targets.

Patch by marcuse; reviewed by Jeff Jirsa and Michael Shuler for CASSANDRA-13413


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b021a1ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b021a1ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b021a1ba

Branch: refs/heads/cassandra-3.0
Commit: b021a1ba55c862eb237a0944e9e1092cfc36f4ef
Parents: 010b5f3
Author: Marcus Eriksson 
Authored: Wed Apr 5 12:43:54 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:07:18 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b021a1ba/circle.yml
--
diff --git a/circle.yml b/circle.yml
index f0df312..9d31277 100644
--- a/circle.yml
+++ b/circle.yml
@@ -4,14 +4,15 @@ machine:
 
 test:
   pre:
-- sudo ifconfig lo:1 127.0.0.2 up
-- sudo ifconfig lo:2 127.0.0.3 up
-- sudo ifconfig lo:3 127.0.0.4 up
-- sudo ifconfig lo:4 127.0.0.5 up
-- ant build
-
+- sudo apt-get update; sudo apt-get install wamerican:
+parallel: true
   override:
-- ant test
+- case $CIRCLE_NODE_INDEX in 0) ant eclipse-warnings; ant test ;; 1) ant 
long-test ;; 2) ant test-compression ;; 3) ant stress-test ;;esac:
+parallel: true
+
   post:
-- mkdir -p $CIRCLE_TEST_REPORTS/junit/
-- cp ./build/test/output/*.xml $CIRCLE_TEST_REPORTS/junit/
+- mkdir -p $CIRCLE_TEST_REPORTS/junit/:
+parallel: true
+- find ./build/test/output/ -iname "*.xml" -exec cp {} 
$CIRCLE_TEST_REPORTS/junit/ \;:
+parallel: true
+

[05/15] cassandra git commit: Make circleci run more test targets.

[marcuse]

Make circleci run more test targets.

Patch by marcuse; reviewed by Jeff Jirsa and Michael Shuler for CASSANDRA-13413


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b021a1ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b021a1ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b021a1ba

Branch: refs/heads/cassandra-3.11
Commit: b021a1ba55c862eb237a0944e9e1092cfc36f4ef
Parents: 010b5f3
Author: Marcus Eriksson 
Authored: Wed Apr 5 12:43:54 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:07:18 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b021a1ba/circle.yml
--
diff --git a/circle.yml b/circle.yml
index f0df312..9d31277 100644
--- a/circle.yml
+++ b/circle.yml
@@ -4,14 +4,15 @@ machine:
 
 test:
   pre:
-- sudo ifconfig lo:1 127.0.0.2 up
-- sudo ifconfig lo:2 127.0.0.3 up
-- sudo ifconfig lo:3 127.0.0.4 up
-- sudo ifconfig lo:4 127.0.0.5 up
-- ant build
-
+- sudo apt-get update; sudo apt-get install wamerican:
+parallel: true
   override:
-- ant test
+- case $CIRCLE_NODE_INDEX in 0) ant eclipse-warnings; ant test ;; 1) ant 
long-test ;; 2) ant test-compression ;; 3) ant stress-test ;;esac:
+parallel: true
+
   post:
-- mkdir -p $CIRCLE_TEST_REPORTS/junit/
-- cp ./build/test/output/*.xml $CIRCLE_TEST_REPORTS/junit/
+- mkdir -p $CIRCLE_TEST_REPORTS/junit/:
+parallel: true
+- find ./build/test/output/ -iname "*.xml" -exec cp {} 
$CIRCLE_TEST_REPORTS/junit/ \;:
+parallel: true
+

[06/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[marcuse]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e102b41
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e102b41
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e102b41

Branch: refs/heads/trunk
Commit: 1e102b41e8ac03e1974a5eafacadc49809831f08
Parents: 377f611 b021a1b
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:09:09 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:09:09 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--

[13/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[marcuse]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ebdf7719
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ebdf7719
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ebdf7719

Branch: refs/heads/cassandra-3.11
Commit: ebdf77194c2835a01134635e9b71ed338c01aac2
Parents: 980f32d 8d34076
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:11:19 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:11:29 2017 +0200

--
 circle.yml | 19 ++-
 doc/source/development/testing.rst |  3 +++
 2 files changed, 13 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebdf7719/doc/source/development/testing.rst
--
diff --cc doc/source/development/testing.rst
index eac336a,000..b8eea6b
mode 100644,00..100644
--- a/doc/source/development/testing.rst
+++ b/doc/source/development/testing.rst
@@@ -1,86 -1,0 +1,89 @@@
 +.. Licensed to the Apache Software Foundation (ASF) under one
 +.. or more contributor license agreements.  See the NOTICE file
 +.. distributed with this work for additional information
 +.. regarding copyright ownership.  The ASF licenses this file
 +.. to you under the Apache License, Version 2.0 (the
 +.. "License"); you may not use this file except in compliance
 +.. with the License.  You may obtain a copy of the License at
 +..
 +.. http://www.apache.org/licenses/LICENSE-2.0
 +..
 +.. Unless required by applicable law or agreed to in writing, software
 +.. distributed under the License is distributed on an "AS IS" BASIS,
 +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 +.. See the License for the specific language governing permissions and
 +.. limitations under the License.
 +
 +.. highlight:: none
 +
 +Testing
 +***
 +
 +Creating tests is one of the most important and also most difficult parts of 
developing Cassandra. There are different ways to test your code depending on 
what you're working on.
 +
 +
 +Unit Testing
 +
 +
 +The most simple way to test code in Cassandra is probably by writing a unit 
test. Cassandra uses JUnit as a testing framework and test cases can be found 
in the ``test/unit`` directory. Ideally you’d be able to create a unit test 
for your implementation that would exclusively cover the class you created (the 
unit under test). Unfortunately this is not always possible and Cassandra 
doesn’t have a very mock friendly code base. Often you’ll find yourself in 
a situation where you have to make use of an embedded Cassandra instance that 
you’ll be able to interact with in your test. If you want to make use of CQL 
in your test, you can simply extend CQLTester and use some of the convenient 
helper methods such as in the following example.
 +
 +.. code-block:: java
 +
 +  @Test
 +  public void testBatchAndList() throws Throwable
 +  {
 + createTable("CREATE TABLE %s (k int PRIMARY KEY, l list)");
 + execute("BEGIN BATCH " +
 + "UPDATE %1$s SET l = l +[ 1 ] WHERE k = 0; " +
 + "UPDATE %1$s SET l = l + [ 2 ] WHERE k = 0; " +
 + "UPDATE %1$s SET l = l + [ 3 ] WHERE k = 0; " +
 + "APPLY BATCH");
 +
 + assertRows(execute("SELECT l FROM %s WHERE k = 0"),
 +row(list(1, 2, 3)));
 +  }
 +
 +Unit tests can be run from the command line using the ``ant test`` command, 
``ant test -Dtest.name=`` to execute a test suite or ``ant 
testsome -Dtest.name= -Dtest.methods=[,testmethod2]`` for 
individual tests.  For example, to run all test methods in the 
``org.apache.cassandra.cql3.SimpleQueryTest`` class, you would run::
 +
 +ant test -Dtest.name=SimpleQueryTest
 +
 +To run only the ``testStaticCompactTables()`` test method from that class, 
you would run::
 +
 +ant testsome -Dtest.name=org.apache.cassandra.cql3.SimpleQueryTest 
-Dtest.methods=testStaticCompactTables
 +
 +Long running tests
 +--
 +
 +Test that consume a significant amount of time during execution can be found 
in the ``test/long`` directory and executed as a regular JUnit test or 
standalone program. Except for the execution time, there’s nothing really 
special about them. However, ant will execute tests under ``test/long`` only 
when using the ``ant long-test`` target.
 +
 +DTests
 +==
 +
 +One way of doing integration or system testing at larger scale is by using 
`dtest `_, which stands for 
“Cassandra Distributed Tests”. The idea is to automatically setup Cassandra 
clusters using various configurations and simulate certain use cases you want 
to test. This is done using Python scripts 

[12/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d34076a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d34076a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d34076a

Branch: refs/heads/cassandra-3.0
Commit: 8d34076aac88a29d6d8a4f3050fdc7a75389298e
Parents: a8a3d4e 1e102b4
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:09:45 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:09:45 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--

[10/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d34076a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d34076a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d34076a

Branch: refs/heads/trunk
Commit: 8d34076aac88a29d6d8a4f3050fdc7a75389298e
Parents: a8a3d4e 1e102b4
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:09:45 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:09:45 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--

[02/15] cassandra git commit: Make circleci run more test targets.

[marcuse]

Make circleci run more test targets.

Patch by marcuse; reviewed by Jeff Jirsa and Michael Shuler for CASSANDRA-13413


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b021a1ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b021a1ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b021a1ba

Branch: refs/heads/cassandra-2.2
Commit: b021a1ba55c862eb237a0944e9e1092cfc36f4ef
Parents: 010b5f3
Author: Marcus Eriksson 
Authored: Wed Apr 5 12:43:54 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:07:18 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b021a1ba/circle.yml
--
diff --git a/circle.yml b/circle.yml
index f0df312..9d31277 100644
--- a/circle.yml
+++ b/circle.yml
@@ -4,14 +4,15 @@ machine:
 
 test:
   pre:
-- sudo ifconfig lo:1 127.0.0.2 up
-- sudo ifconfig lo:2 127.0.0.3 up
-- sudo ifconfig lo:3 127.0.0.4 up
-- sudo ifconfig lo:4 127.0.0.5 up
-- ant build
-
+- sudo apt-get update; sudo apt-get install wamerican:
+parallel: true
   override:
-- ant test
+- case $CIRCLE_NODE_INDEX in 0) ant eclipse-warnings; ant test ;; 1) ant 
long-test ;; 2) ant test-compression ;; 3) ant stress-test ;;esac:
+parallel: true
+
   post:
-- mkdir -p $CIRCLE_TEST_REPORTS/junit/
-- cp ./build/test/output/*.xml $CIRCLE_TEST_REPORTS/junit/
+- mkdir -p $CIRCLE_TEST_REPORTS/junit/:
+parallel: true
+- find ./build/test/output/ -iname "*.xml" -exec cp {} 
$CIRCLE_TEST_REPORTS/junit/ \;:
+parallel: true
+

[07/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[marcuse]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e102b41
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e102b41
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e102b41

Branch: refs/heads/cassandra-2.2
Commit: 1e102b41e8ac03e1974a5eafacadc49809831f08
Parents: 377f611 b021a1b
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:09:09 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:09:09 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--

[11/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d34076a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d34076a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d34076a

Branch: refs/heads/cassandra-3.11
Commit: 8d34076aac88a29d6d8a4f3050fdc7a75389298e
Parents: a8a3d4e 1e102b4
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:09:45 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:09:45 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--

[08/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[marcuse]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e102b41
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e102b41
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e102b41

Branch: refs/heads/cassandra-3.0
Commit: 1e102b41e8ac03e1974a5eafacadc49809831f08
Parents: 377f611 b021a1b
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:09:09 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:09:09 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--

[03/15] cassandra git commit: Make circleci run more test targets.

[marcuse]

Make circleci run more test targets.

Patch by marcuse; reviewed by Jeff Jirsa and Michael Shuler for CASSANDRA-13413


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b021a1ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b021a1ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b021a1ba

Branch: refs/heads/trunk
Commit: b021a1ba55c862eb237a0944e9e1092cfc36f4ef
Parents: 010b5f3
Author: Marcus Eriksson 
Authored: Wed Apr 5 12:43:54 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:07:18 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b021a1ba/circle.yml
--
diff --git a/circle.yml b/circle.yml
index f0df312..9d31277 100644
--- a/circle.yml
+++ b/circle.yml
@@ -4,14 +4,15 @@ machine:
 
 test:
   pre:
-- sudo ifconfig lo:1 127.0.0.2 up
-- sudo ifconfig lo:2 127.0.0.3 up
-- sudo ifconfig lo:3 127.0.0.4 up
-- sudo ifconfig lo:4 127.0.0.5 up
-- ant build
-
+- sudo apt-get update; sudo apt-get install wamerican:
+parallel: true
   override:
-- ant test
+- case $CIRCLE_NODE_INDEX in 0) ant eclipse-warnings; ant test ;; 1) ant 
long-test ;; 2) ant test-compression ;; 3) ant stress-test ;;esac:
+parallel: true
+
   post:
-- mkdir -p $CIRCLE_TEST_REPORTS/junit/
-- cp ./build/test/output/*.xml $CIRCLE_TEST_REPORTS/junit/
+- mkdir -p $CIRCLE_TEST_REPORTS/junit/:
+parallel: true
+- find ./build/test/output/ -iname "*.xml" -exec cp {} 
$CIRCLE_TEST_REPORTS/junit/ \;:
+parallel: true
+

[15/15] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[marcuse]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/393226de
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/393226de
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/393226de

Branch: refs/heads/trunk
Commit: 393226de62045b199140e1c3bea8d634272c0b3e
Parents: e0f3770 ebdf771
Author: Marcus Eriksson 
Authored: Fri Apr 7 08:11:44 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:11:44 2017 +0200

--
 circle.yml | 19 ++-
 doc/source/development/testing.rst |  3 +++
 2 files changed, 13 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/393226de/doc/source/development/testing.rst
--

[01/15] cassandra git commit: Make circleci run more test targets.

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 010b5f3a5 -> b021a1ba5
  refs/heads/cassandra-2.2 377f6117c -> 1e102b41e
  refs/heads/cassandra-3.0 a8a3d4e28 -> 8d34076aa
  refs/heads/cassandra-3.11 980f32d56 -> ebdf77194
  refs/heads/trunk e0f377000 -> 393226de6


Make circleci run more test targets.

Patch by marcuse; reviewed by Jeff Jirsa and Michael Shuler for CASSANDRA-13413


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b021a1ba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b021a1ba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b021a1ba

Branch: refs/heads/cassandra-2.1
Commit: b021a1ba55c862eb237a0944e9e1092cfc36f4ef
Parents: 010b5f3
Author: Marcus Eriksson 
Authored: Wed Apr 5 12:43:54 2017 +0200
Committer: Marcus Eriksson 
Committed: Fri Apr 7 08:07:18 2017 +0200

--
 circle.yml | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b021a1ba/circle.yml
--
diff --git a/circle.yml b/circle.yml
index f0df312..9d31277 100644
--- a/circle.yml
+++ b/circle.yml
@@ -4,14 +4,15 @@ machine:
 
 test:
   pre:
-- sudo ifconfig lo:1 127.0.0.2 up
-- sudo ifconfig lo:2 127.0.0.3 up
-- sudo ifconfig lo:3 127.0.0.4 up
-- sudo ifconfig lo:4 127.0.0.5 up
-- ant build
-
+- sudo apt-get update; sudo apt-get install wamerican:
+parallel: true
   override:
-- ant test
+- case $CIRCLE_NODE_INDEX in 0) ant eclipse-warnings; ant test ;; 1) ant 
long-test ;; 2) ant test-compression ;; 3) ant stress-test ;;esac:
+parallel: true
+
   post:
-- mkdir -p $CIRCLE_TEST_REPORTS/junit/
-- cp ./build/test/output/*.xml $CIRCLE_TEST_REPORTS/junit/
+- mkdir -p $CIRCLE_TEST_REPORTS/junit/:
+parallel: true
+- find ./build/test/output/ -iname "*.xml" -exec cp {} 
$CIRCLE_TEST_REPORTS/junit/ \;:
+parallel: true
+

[jira] [Commented] (CASSANDRA-13065) Skip building views during base table streams on range movements

[Benjamin Roth (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960301#comment-15960301
 ] 

Benjamin Roth commented on CASSANDRA-13065:
---

[~pauloricardomg] thanks for the review! Would you like to take a look at 
CASSANDRA-13066 for review? It depends on this patch and I have to touch it 
anyway. So if you have comments on the concept and namings (and probably you 
will) I can do that in one run.

> Skip building views during base table streams on range movements
> 
>
> Key: CASSANDRA-13065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13065
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 4.0
>
>
> Booting or decommisioning nodes with MVs is unbearably slow as all streams go 
> through the regular write paths. This causes read-before-writes for every 
> mutation and during bootstrap it causes them to be sent to batchlog.
> The makes it virtually impossible to boot a new node in an acceptable amount 
> of time.
> Using the regular streaming behaviour for consistent range movements works 
> much better in this case and does not break the MV local consistency contract.
> Already tested on own cluster.
> Bootstrap case is super easy to handle, decommission case requires 
> CASSANDRA-13064



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960270#comment-15960270
 ] 

Marcus Eriksson commented on CASSANDRA-13413:
-

[~aweisberg] just leave the parallelism at 1 and it should works just like 
before

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13188) compaction-stress AssertionError from getMemtableFor()

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-13188:
---
Attachment: (was: 13188-3.11-update.txt)

> compaction-stress AssertionError from getMemtableFor()
> --
>
> Key: CASSANDRA-13188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13188
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing, Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Exception:
> {noformat}
> ./compaction-stress compact -d /tmp/compaction -p 
> https://gist.githubusercontent.com/tjake/8995058fed11d9921e31/raw/a9334d1090017bf546d003e271747351a40692ea/blogpost.yaml
>  -t 4
> WARN  18:45:04,854 JNA link failure, one or more native method will be 
> unavailable.
> java.lang.AssertionError: []
> at 
> org.apache.cassandra.db.lifecycle.Tracker.getMemtableFor(Tracker.java:312)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1315)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:462)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:570)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:564)
> at 
> org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:356)
> at 
> org.apache.cassandra.schema.SchemaKeyspace.saveSystemKeyspacesSchema(SchemaKeyspace.java:265)
> at 
> org.apache.cassandra.db.SystemKeyspace.finishStartup(SystemKeyspace.java:495)
> at 
> org.apache.cassandra.stress.CompactionStress$Compaction.run(CompactionStress.java:209)
> at 
> org.apache.cassandra.stress.CompactionStress.main(CompactionStress.java:349)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13188) compaction-stress AssertionError from getMemtableFor()

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-13188:
---
Attachment: (was: 13188-3.11.txt)

> compaction-stress AssertionError from getMemtableFor()
> --
>
> Key: CASSANDRA-13188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13188
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing, Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Exception:
> {noformat}
> ./compaction-stress compact -d /tmp/compaction -p 
> https://gist.githubusercontent.com/tjake/8995058fed11d9921e31/raw/a9334d1090017bf546d003e271747351a40692ea/blogpost.yaml
>  -t 4
> WARN  18:45:04,854 JNA link failure, one or more native method will be 
> unavailable.
> java.lang.AssertionError: []
> at 
> org.apache.cassandra.db.lifecycle.Tracker.getMemtableFor(Tracker.java:312)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1315)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:462)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:570)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:564)
> at 
> org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:356)
> at 
> org.apache.cassandra.schema.SchemaKeyspace.saveSystemKeyspacesSchema(SchemaKeyspace.java:265)
> at 
> org.apache.cassandra.db.SystemKeyspace.finishStartup(SystemKeyspace.java:495)
> at 
> org.apache.cassandra.stress.CompactionStress$Compaction.run(CompactionStress.java:209)
> at 
> org.apache.cassandra.stress.CompactionStress.main(CompactionStress.java:349)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13188) compaction-stress AssertionError from getMemtableFor()

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960215#comment-15960215
 ] 

Jay Zhuang commented on CASSANDRA-13188:


Rebased the code:
|[3.11| 
https://github.com/cooldoger/cassandra/commit/2de68a8ca3d25fa030d8e1a65de68198ea244de2]
 | 
[trunk|https://github.com/cooldoger/cassandra/commit/64912267433c304b1a736df65014f54f177b2e21]
 |
Passed utest locally. Please review.

> compaction-stress AssertionError from getMemtableFor()
> --
>
> Key: CASSANDRA-13188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13188
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing, Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Attachments: 13188-3.11.txt, 13188-3.11-update.txt
>
>
> Exception:
> {noformat}
> ./compaction-stress compact -d /tmp/compaction -p 
> https://gist.githubusercontent.com/tjake/8995058fed11d9921e31/raw/a9334d1090017bf546d003e271747351a40692ea/blogpost.yaml
>  -t 4
> WARN  18:45:04,854 JNA link failure, one or more native method will be 
> unavailable.
> java.lang.AssertionError: []
> at 
> org.apache.cassandra.db.lifecycle.Tracker.getMemtableFor(Tracker.java:312)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1315)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:462)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:570)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:564)
> at 
> org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:356)
> at 
> org.apache.cassandra.schema.SchemaKeyspace.saveSystemKeyspacesSchema(SchemaKeyspace.java:265)
> at 
> org.apache.cassandra.db.SystemKeyspace.finishStartup(SystemKeyspace.java:495)
> at 
> org.apache.cassandra.stress.CompactionStress$Compaction.run(CompactionStress.java:209)
> at 
> org.apache.cassandra.stress.CompactionStress.main(CompactionStress.java:349)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13364) Cqlsh COPY fails importing Map>, ParseError unhashable type list

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-13364:
-
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   (was: 2.2.x)
   (was: 2.1.x)
   3.11.0
   3.0.13
   2.2.10
   2.1.18
   Status: Resolved  (was: Ready to Commit)

Thank you [~ifesdjeen] and [~nicolaen].

Committed to 2.1 as 010b5f3a567663a5ceb823932a0b430848d331e3 and merged upwards.

> Cqlsh COPY fails importing Map>, ParseError unhashable 
> type list
> 
>
> Key: CASSANDRA-13364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13364
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Nicolae Namolovan
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.18, 2.2.10, 3.0.13, 3.11.0
>
>
> When importing data with the _COPY_ command into a column family that has a 
> _map>>_ field, I get a _unhashable type: 'list'_ 
> error. Here is how to reproduce:
> {code}
> CREATE TABLE table1 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table1 (col1, col2map) values (1, {'key': ['value1']});
> cqlsh:ks> copy table1 to 'table1.csv';
> table1.csv file content:
> 1,{'key': ['value1']}
> cqlsh:ks> copy table1 from 'table1.csv';
> ...
> Failed to import 1 rows: ParseError - Failed to parse {'key': ['value1']} : 
> unhashable type: 'list',  given up without retries
> Failed to process 1 rows; failed rows written to kv_table1.err
> Processed: 1 rows; Rate:   2 rows/s; Avg. rate:   2 rows/s
> 1 rows imported from 1 files in 0.420 seconds (0 skipped).
> {code}
> But it works fine for Map>.
> {code}
> CREATE TABLE table2 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table2 (col1, col2map) values (1, {'key': {'value1'}});
> cqlsh:ks> copy table2 to 'table2.csv';
> table2.csv file content:
> 1,{'key': {'value1'}}
> cqlsh:ks> copy table2 from 'table2.csv';
> Processed: 1 rows; Rate:   2 rows/s; Avg. rate:   2 rows/s
> 1 rows imported from 1 files in 0.417 seconds (0 skipped).
> {code}
> The exception seems to arrive in _convert_map_ function in _ImportConversion_ 
> class inside _copyutil.py_.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[13/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/980f32d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/980f32d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/980f32d5

Branch: refs/heads/cassandra-3.11
Commit: 980f32d563aa33ef45f579f198e0cfd6938c3936
Parents: ab1310b a8a3d4e
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:05:53 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:05:53 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/980f32d5/CHANGES.txt
--
diff --cc CHANGES.txt
index 23e0225,5bcdf16..0761ca5
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -58,147 -64,6 +58,148 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 + * Nodetool upgradesstables/scrub/compact ignores system tables 
(CASSANDRA-13410)
 +Merged from 2.2:
 + * Fix JVM metric names (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
++ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 + * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 + * Use portable stderr for java error in startup (CASSANDRA-13211)
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA

[07/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[stefania]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/377f6117
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/377f6117
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/377f6117

Branch: refs/heads/cassandra-2.2
Commit: 377f6117ce0870758f1b207c3288121e784741e0
Parents: fbb061b 010b5f3
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:02:40 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:02:40 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/377f6117/CHANGES.txt
--
diff --cc CHANGES.txt
index c77576b,31c7b8a..e01a63c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,57 -1,9 +1,58 @@@
 -2.1.18
 +2.2.10
 + * Fix JVM metric paths (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
 + * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
 + * Coalescing strategy sleeps too much (CASSANDRA-13090)
 + * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 +Merged from 2.1:
+  * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 - * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
   * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 +
  
 -2.1.17
 +2.2.9
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough 

[01/15] cassandra git commit: Fix ParseError unhashable type list in cqlsh copy from

[stefania]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 d718d421f -> 010b5f3a5
  refs/heads/cassandra-2.2 fbb061be6 -> 377f6117c
  refs/heads/cassandra-3.0 6991556e4 -> a8a3d4e28
  refs/heads/cassandra-3.11 ab1310b78 -> 980f32d56
  refs/heads/trunk 9ed04da0c -> e0f377000


Fix ParseError unhashable type list in cqlsh copy from

patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13364


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/010b5f3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/010b5f3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/010b5f3a

Branch: refs/heads/cassandra-2.1
Commit: 010b5f3a567663a5ceb823932a0b430848d331e3
Parents: d718d42
Author: Stefania Alborghetti 
Authored: Wed Mar 29 11:22:08 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:57:54 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 111f3de..31c7b8a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.18
+ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
  * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
  * Remove unused repositories (CASSANDRA-13278)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 5565fed..3a45353 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1832,7 +1832,7 @@ class ImportConversion(object):
 return tuple(convert_mandatory(t, v) for t, v in zip(ct.subtypes, 
split(val)))
 
 def convert_list(val, ct=cql_type):
-return list(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
+return tuple(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
 
 def convert_set(val, ct=cql_type):
 return frozenset(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))

[08/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[stefania]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/377f6117
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/377f6117
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/377f6117

Branch: refs/heads/cassandra-3.0
Commit: 377f6117ce0870758f1b207c3288121e784741e0
Parents: fbb061b 010b5f3
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:02:40 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:02:40 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/377f6117/CHANGES.txt
--
diff --cc CHANGES.txt
index c77576b,31c7b8a..e01a63c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,57 -1,9 +1,58 @@@
 -2.1.18
 +2.2.10
 + * Fix JVM metric paths (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
 + * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
 + * Coalescing strategy sleeps too much (CASSANDRA-13090)
 + * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 +Merged from 2.1:
+  * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 - * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
   * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 +
  
 -2.1.17
 +2.2.9
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough 

[14/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/980f32d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/980f32d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/980f32d5

Branch: refs/heads/trunk
Commit: 980f32d563aa33ef45f579f198e0cfd6938c3936
Parents: ab1310b a8a3d4e
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:05:53 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:05:53 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/980f32d5/CHANGES.txt
--
diff --cc CHANGES.txt
index 23e0225,5bcdf16..0761ca5
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -58,147 -64,6 +58,148 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 + * Nodetool upgradesstables/scrub/compact ignores system tables 
(CASSANDRA-13410)
 +Merged from 2.2:
 + * Fix JVM metric names (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
++ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 + * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 + * Use portable stderr for java error in startup (CASSANDRA-13211)
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 

[03/15] cassandra git commit: Fix ParseError unhashable type list in cqlsh copy from

[stefania]

Fix ParseError unhashable type list in cqlsh copy from

patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13364


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/010b5f3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/010b5f3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/010b5f3a

Branch: refs/heads/trunk
Commit: 010b5f3a567663a5ceb823932a0b430848d331e3
Parents: d718d42
Author: Stefania Alborghetti 
Authored: Wed Mar 29 11:22:08 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:57:54 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 111f3de..31c7b8a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.18
+ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
  * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
  * Remove unused repositories (CASSANDRA-13278)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 5565fed..3a45353 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1832,7 +1832,7 @@ class ImportConversion(object):
 return tuple(convert_mandatory(t, v) for t, v in zip(ct.subtypes, 
split(val)))
 
 def convert_list(val, ct=cql_type):
-return list(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
+return tuple(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
 
 def convert_set(val, ct=cql_type):
 return frozenset(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))

[09/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[stefania]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/377f6117
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/377f6117
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/377f6117

Branch: refs/heads/cassandra-3.11
Commit: 377f6117ce0870758f1b207c3288121e784741e0
Parents: fbb061b 010b5f3
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:02:40 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:02:40 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/377f6117/CHANGES.txt
--
diff --cc CHANGES.txt
index c77576b,31c7b8a..e01a63c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,57 -1,9 +1,58 @@@
 -2.1.18
 +2.2.10
 + * Fix JVM metric paths (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
 + * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
 + * Coalescing strategy sleeps too much (CASSANDRA-13090)
 + * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 +Merged from 2.1:
+  * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 - * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
   * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 +
  
 -2.1.17
 +2.2.9
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough

[05/15] cassandra git commit: Fix ParseError unhashable type list in cqlsh copy from

[stefania]

Fix ParseError unhashable type list in cqlsh copy from

patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13364


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/010b5f3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/010b5f3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/010b5f3a

Branch: refs/heads/cassandra-3.11
Commit: 010b5f3a567663a5ceb823932a0b430848d331e3
Parents: d718d42
Author: Stefania Alborghetti 
Authored: Wed Mar 29 11:22:08 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:57:54 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 111f3de..31c7b8a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.18
+ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
  * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
  * Remove unused repositories (CASSANDRA-13278)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 5565fed..3a45353 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1832,7 +1832,7 @@ class ImportConversion(object):
 return tuple(convert_mandatory(t, v) for t, v in zip(ct.subtypes, 
split(val)))
 
 def convert_list(val, ct=cql_type):
-return list(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
+return tuple(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
 
 def convert_set(val, ct=cql_type):
 return frozenset(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))

[15/15] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[stefania]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e0f37700
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e0f37700
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e0f37700

Branch: refs/heads/trunk
Commit: e0f37700086a09e82e8209c0e3bebf8df4d2e830
Parents: 9ed04da 980f32d
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:06:13 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:06:13 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e0f37700/CHANGES.txt
--

[10/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8a3d4e2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8a3d4e2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8a3d4e2

Branch: refs/heads/trunk
Commit: a8a3d4e282334216a39ceeabea06b9f60f7b4dc1
Parents: 6991556 377f611
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:02:59 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:03:54 2017 +0800

--
 CHANGES.txt| 3 ++-
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8a3d4e2/CHANGES.txt
--
diff --cc CHANGES.txt
index 212c738,e01a63c..5bcdf16
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -22,21 -4,8 +22,22 @@@ Merged from 2.2
   * Discard in-flight shadow round responses (CASSANDRA-12653)
   * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
   * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
   * Fix queries updating multiple time the same list (CASSANDRA-13130)
   * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
- 
++Merged from 2.1:
++ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 +
 +3.0.12
 + * Prevent data loss on upgrade 2.1 - 3.0 by adding component separator to 
LogRecord absolute path (CASSANDRA-13294)
 + * Improve testing on macOS by eliminating sigar logging (CASSANDRA-13233)
 + * Cqlsh copy-from should error out when csv contains invalid data for 
collections (CASSANDRA-13071)
 + * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
 + * Faster StreamingHistogram (CASSANDRA-13038)
 + * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
 + * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
 + * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 +Merged from 2.2:
   * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
   * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
   * Coalescing strategy sleeps too much (CASSANDRA-13090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8a3d4e2/pylib/cqlshlib/copyutil.py
--

[02/15] cassandra git commit: Fix ParseError unhashable type list in cqlsh copy from

[stefania]

Fix ParseError unhashable type list in cqlsh copy from

patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13364


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/010b5f3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/010b5f3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/010b5f3a

Branch: refs/heads/cassandra-2.2
Commit: 010b5f3a567663a5ceb823932a0b430848d331e3
Parents: d718d42
Author: Stefania Alborghetti 
Authored: Wed Mar 29 11:22:08 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:57:54 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 111f3de..31c7b8a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.18
+ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
  * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
  * Remove unused repositories (CASSANDRA-13278)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 5565fed..3a45353 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1832,7 +1832,7 @@ class ImportConversion(object):
 return tuple(convert_mandatory(t, v) for t, v in zip(ct.subtypes, 
split(val)))
 
 def convert_list(val, ct=cql_type):
-return list(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
+return tuple(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
 
 def convert_set(val, ct=cql_type):
 return frozenset(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))

[04/15] cassandra git commit: Fix ParseError unhashable type list in cqlsh copy from

[stefania]

Fix ParseError unhashable type list in cqlsh copy from

patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13364


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/010b5f3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/010b5f3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/010b5f3a

Branch: refs/heads/cassandra-3.0
Commit: 010b5f3a567663a5ceb823932a0b430848d331e3
Parents: d718d42
Author: Stefania Alborghetti 
Authored: Wed Mar 29 11:22:08 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:57:54 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 111f3de..31c7b8a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.18
+ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
  * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
  * Remove unused repositories (CASSANDRA-13278)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/010b5f3a/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 5565fed..3a45353 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1832,7 +1832,7 @@ class ImportConversion(object):
 return tuple(convert_mandatory(t, v) for t, v in zip(ct.subtypes, 
split(val)))
 
 def convert_list(val, ct=cql_type):
-return list(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
+return tuple(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))
 
 def convert_set(val, ct=cql_type):
 return frozenset(convert_mandatory(ct.subtypes[0], v) for v in 
split(val))

[06/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[stefania]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/377f6117
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/377f6117
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/377f6117

Branch: refs/heads/trunk
Commit: 377f6117ce0870758f1b207c3288121e784741e0
Parents: fbb061b 010b5f3
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:02:40 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:02:40 2017 +0800

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/377f6117/CHANGES.txt
--
diff --cc CHANGES.txt
index c77576b,31c7b8a..e01a63c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,57 -1,9 +1,58 @@@
 -2.1.18
 +2.2.10
 + * Fix JVM metric paths (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
 + * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
 + * Coalescing strategy sleeps too much (CASSANDRA-13090)
 + * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 +Merged from 2.1:
+  * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 - * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
   * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 +
  
 -2.1.17
 +2.2.9
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk spa

[11/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8a3d4e2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8a3d4e2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8a3d4e2

Branch: refs/heads/cassandra-3.11
Commit: a8a3d4e282334216a39ceeabea06b9f60f7b4dc1
Parents: 6991556 377f611
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:02:59 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:03:54 2017 +0800

--
 CHANGES.txt| 3 ++-
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8a3d4e2/CHANGES.txt
--
diff --cc CHANGES.txt
index 212c738,e01a63c..5bcdf16
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -22,21 -4,8 +22,22 @@@ Merged from 2.2
   * Discard in-flight shadow round responses (CASSANDRA-12653)
   * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
   * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
   * Fix queries updating multiple time the same list (CASSANDRA-13130)
   * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
- 
++Merged from 2.1:
++ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 +
 +3.0.12
 + * Prevent data loss on upgrade 2.1 - 3.0 by adding component separator to 
LogRecord absolute path (CASSANDRA-13294)
 + * Improve testing on macOS by eliminating sigar logging (CASSANDRA-13233)
 + * Cqlsh copy-from should error out when csv contains invalid data for 
collections (CASSANDRA-13071)
 + * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
 + * Faster StreamingHistogram (CASSANDRA-13038)
 + * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
 + * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
 + * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 +Merged from 2.2:
   * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
   * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
   * Coalescing strategy sleeps too much (CASSANDRA-13090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8a3d4e2/pylib/cqlshlib/copyutil.py
--

[12/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8a3d4e2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8a3d4e2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8a3d4e2

Branch: refs/heads/cassandra-3.0
Commit: a8a3d4e282334216a39ceeabea06b9f60f7b4dc1
Parents: 6991556 377f611
Author: Stefania Alborghetti 
Authored: Fri Apr 7 09:02:59 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 09:03:54 2017 +0800

--
 CHANGES.txt| 3 ++-
 pylib/cqlshlib/copyutil.py | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8a3d4e2/CHANGES.txt
--
diff --cc CHANGES.txt
index 212c738,e01a63c..5bcdf16
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -22,21 -4,8 +22,22 @@@ Merged from 2.2
   * Discard in-flight shadow round responses (CASSANDRA-12653)
   * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
   * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
   * Fix queries updating multiple time the same list (CASSANDRA-13130)
   * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
- 
++Merged from 2.1:
++ * Fix ParseError unhashable type list in cqlsh copy from (CASSANDRA-13364)
 +
 +3.0.12
 + * Prevent data loss on upgrade 2.1 - 3.0 by adding component separator to 
LogRecord absolute path (CASSANDRA-13294)
 + * Improve testing on macOS by eliminating sigar logging (CASSANDRA-13233)
 + * Cqlsh copy-from should error out when csv contains invalid data for 
collections (CASSANDRA-13071)
 + * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
 + * Faster StreamingHistogram (CASSANDRA-13038)
 + * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
 + * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
 + * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 +Merged from 2.2:
   * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
   * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
   * Coalescing strategy sleeps too much (CASSANDRA-13090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8a3d4e2/pylib/cqlshlib/copyutil.py
--

[jira] [Updated] (CASSANDRA-12213) dtest failure in write_failures_test.TestWriteFailures.test_paxos_any

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12213:
-
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.11.0
   3.0.13
   Status: Resolved  (was: Ready to Commit)

Thank you, committed to 3.0 as 6991556e431a51575744248a4c484270c4f918c9 and 
merged upwards.

The failures on trunk are unrelated: I've verified that the same tests pass 
locally and that the cause cannot be related to this patch.

> dtest failure in write_failures_test.TestWriteFailures.test_paxos_any
> -
>
> Key: CASSANDRA-12213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Craig Kodman
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.0.13, 3.11.0
>
> Attachments: jenkins-stef1927-12014-dtest-2_logs.001.tar.gz, 
> node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, 
> node2.log, node3_debug.log, node3_gc.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_paxos_any
> and:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_mutation_v3/
> Failed on CassCI build cassandra-3.9_dtest #10



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[2/6] cassandra git commit: Fix startup problems due to schema tables not completely flushed

[stefania]

Fix startup problems due to schema tables not completely flushed

patch by Stefania Alborghetti; reviewed by Aleksey Yeschenko for 
CASSANDRA-12213.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6991556e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6991556e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6991556e

Branch: refs/heads/cassandra-3.11
Commit: 6991556e431a51575744248a4c484270c4f918c9
Parents: 6efb44b
Author: Stefania Alborghetti 
Authored: Wed Feb 22 11:33:03 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Apr 6 10:14:35 2017 +0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/schema/SchemaKeyspace.java | 22 +++-
 2 files changed, 18 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6991556e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8bd21bc..212c738 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Fix startup problems due to schema tables not completely flushed 
(CASSANDRA-12213)
  * Fix view builder bug that can filter out data on restart (CASSANDRA-13405)
  * Fix 2i page size calculation when there are no regular columns 
(CASSANDRA-13400)
  * Fix the conversion of 2.X expired rows without regular column data 
(CASSANDRA-13395)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6991556e/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java 
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index eee4fbb..e860c59 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -25,8 +25,7 @@ import java.util.*;
 import java.util.concurrent.TimeUnit;
 import java.util.stream.Collectors;
 
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.MapDifference;
+import com.google.common.collect.*;
 import com.google.common.collect.Maps;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -82,8 +81,21 @@ public final class SchemaKeyspace
 public static final String AGGREGATES = "aggregates";
 public static final String INDEXES = "indexes";
 
-public static final List ALL =
-ImmutableList.of(KEYSPACES, TABLES, COLUMNS, DROPPED_COLUMNS, 
TRIGGERS, VIEWS, TYPES, FUNCTIONS, AGGREGATES, INDEXES);
+/**
+ * The order in this list matters.
+ *
+ * When flushing schema tables, we want to flush them in a way that 
mitigates the effects of an abrupt shutdown whilst
+ * the tables are being flushed. On startup, we load the schema from disk 
before replaying the CL, so we need to
+ * try to avoid problems like reading a table without columns or types, 
for example. So columns and types should be
+ * flushed before tables, which should be flushed before keyspaces.
+ *
+ * When truncating, the order should be reversed. For immutable lists this 
is an efficient operation that simply
+ * iterates in reverse order.
+ *
+ * See CASSANDRA-12213 for more details.
+ */
+public static final ImmutableList ALL =
+ImmutableList.of(COLUMNS, DROPPED_COLUMNS, TRIGGERS, TYPES, FUNCTIONS, 
AGGREGATES, INDEXES, TABLES, VIEWS, KEYSPACES);
 
 private static final CFMetaData Keyspaces =
 compile(KEYSPACES,
@@ -271,7 +283,7 @@ public final class SchemaKeyspace
 
 public static void truncate()
 {
-ALL.forEach(table -> getSchemaCFS(table).truncateBlocking());
+ALL.reverse().forEach(table -> getSchemaCFS(table).truncateBlocking());
 }
 
 static void flush()

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab1310b7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab1310b7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab1310b7

Branch: refs/heads/trunk
Commit: ab1310b78a084b34ab9013344870433986eb960b
Parents: bfdc1e0 6991556
Author: Stefania Alborghetti 
Authored: Fri Apr 7 08:46:45 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:46:45 2017 +0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/schema/SchemaKeyspace.java | 23 +++-
 2 files changed, 18 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1310b7/CHANGES.txt
--
diff --cc CHANGES.txt
index d3460d8,212c738..23e0225
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,25 -1,5 +1,26 @@@
 -3.0.13
 +3.11.0
 + * Fix testLimitSSTables flake caused by concurrent flush (CASSANDRA-12820)
 + * cdc column addition strikes again (CASSANDRA-13382)
 + * Fix static column indexes (CASSANDRA-13277)
 + * DataOutputBuffer.asNewBuffer broken (CASSANDRA-13298)
 + * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370)
 + * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns 
(CASSANDRA-13247)
 + * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern 
(CASSANDRA-13317)
 + * Possible AssertionError in UnfilteredRowIteratorWithLowerBound 
(CASSANDRA-13366)
 + * Support unaligned memory access for AArch64 (CASSANDRA-13326)
 + * Improve SASI range iterator efficiency on intersection with an empty range 
(CASSANDRA-12915).
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 + * Address message coalescing regression (CASSANDRA-12676)
 + * Delete illegal character from StandardTokenizerImpl.jflex (CASSANDRA-13417)
 +Merged from 3.0:
+  * Fix startup problems due to schema tables not completely flushed 
(CASSANDRA-12213)
   * Fix view builder bug that can filter out data on restart (CASSANDRA-13405)
   * Fix 2i page size calculation when there are no regular columns 
(CASSANDRA-13400)
   * Fix the conversion of 2.X expired rows without regular column data 
(CASSANDRA-13395)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1310b7/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --cc src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index ff7cf04,e860c59..2c2416e
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@@ -82,15 -81,22 +80,28 @@@ public final class SchemaKeyspac
  public static final String AGGREGATES = "aggregates";
  public static final String INDEXES = "indexes";
  
- public static final List ALL =
- ImmutableList.of(KEYSPACES, TABLES, COLUMNS, DROPPED_COLUMNS, 
TRIGGERS, VIEWS, TYPES, FUNCTIONS, AGGREGATES, INDEXES);
+ /**
+  * The order in this list matters.
+  *
+  * When flushing schema tables, we want to flush them in a way that 
mitigates the effects of an abrupt shutdown whilst
+  * the tables are being flushed. On startup, we load the schema from disk 
before replaying the CL, so we need to
+  * try to avoid problems like reading a table without columns or types, 
for example. So columns and types should be
+  * flushed before tables, which should be flushed before keyspaces.
+  *
+  * When truncating, the order should be reversed. For immutable lists 
this is an efficient operation that simply
+  * iterates in reverse order.
+  *
+  * See CASSANDRA-12213 for more details.
+  */
+ public static final ImmutableList ALL =
+ ImmutableList.of(COLUMNS, DROPPED_COLUMNS, TRIGGERS, TYPES, 
FUNCTIONS, AGGREGATES, INDEXES, TABLES, VIEWS, KEYSPACES);
  
 +/**
 + * The tables to which we added the cdc column. This is used in {@link 
#makeUpdateForSchema} below to make sure we skip that
 + * column is cdc is disabled as the columns breaks pre-cdc to post-cdc 
upgrades (typically, 3.0 -> 3.X).
 + */
 +private static final Set TABLES_WITH_CDC_ADDED = 
Immutabl

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab1310b7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab1310b7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab1310b7

Branch: refs/heads/cassandra-3.11
Commit: ab1310b78a084b34ab9013344870433986eb960b
Parents: bfdc1e0 6991556
Author: Stefania Alborghetti 
Authored: Fri Apr 7 08:46:45 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:46:45 2017 +0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/schema/SchemaKeyspace.java | 23 +++-
 2 files changed, 18 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1310b7/CHANGES.txt
--
diff --cc CHANGES.txt
index d3460d8,212c738..23e0225
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,25 -1,5 +1,26 @@@
 -3.0.13
 +3.11.0
 + * Fix testLimitSSTables flake caused by concurrent flush (CASSANDRA-12820)
 + * cdc column addition strikes again (CASSANDRA-13382)
 + * Fix static column indexes (CASSANDRA-13277)
 + * DataOutputBuffer.asNewBuffer broken (CASSANDRA-13298)
 + * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370)
 + * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns 
(CASSANDRA-13247)
 + * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern 
(CASSANDRA-13317)
 + * Possible AssertionError in UnfilteredRowIteratorWithLowerBound 
(CASSANDRA-13366)
 + * Support unaligned memory access for AArch64 (CASSANDRA-13326)
 + * Improve SASI range iterator efficiency on intersection with an empty range 
(CASSANDRA-12915).
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 + * Address message coalescing regression (CASSANDRA-12676)
 + * Delete illegal character from StandardTokenizerImpl.jflex (CASSANDRA-13417)
 +Merged from 3.0:
+  * Fix startup problems due to schema tables not completely flushed 
(CASSANDRA-12213)
   * Fix view builder bug that can filter out data on restart (CASSANDRA-13405)
   * Fix 2i page size calculation when there are no regular columns 
(CASSANDRA-13400)
   * Fix the conversion of 2.X expired rows without regular column data 
(CASSANDRA-13395)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab1310b7/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --cc src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index ff7cf04,e860c59..2c2416e
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@@ -82,15 -81,22 +80,28 @@@ public final class SchemaKeyspac
  public static final String AGGREGATES = "aggregates";
  public static final String INDEXES = "indexes";
  
- public static final List ALL =
- ImmutableList.of(KEYSPACES, TABLES, COLUMNS, DROPPED_COLUMNS, 
TRIGGERS, VIEWS, TYPES, FUNCTIONS, AGGREGATES, INDEXES);
+ /**
+  * The order in this list matters.
+  *
+  * When flushing schema tables, we want to flush them in a way that 
mitigates the effects of an abrupt shutdown whilst
+  * the tables are being flushed. On startup, we load the schema from disk 
before replaying the CL, so we need to
+  * try to avoid problems like reading a table without columns or types, 
for example. So columns and types should be
+  * flushed before tables, which should be flushed before keyspaces.
+  *
+  * When truncating, the order should be reversed. For immutable lists 
this is an efficient operation that simply
+  * iterates in reverse order.
+  *
+  * See CASSANDRA-12213 for more details.
+  */
+ public static final ImmutableList ALL =
+ ImmutableList.of(COLUMNS, DROPPED_COLUMNS, TRIGGERS, TYPES, 
FUNCTIONS, AGGREGATES, INDEXES, TABLES, VIEWS, KEYSPACES);
  
 +/**
 + * The tables to which we added the cdc column. This is used in {@link 
#makeUpdateForSchema} below to make sure we skip that
 + * column is cdc is disabled as the columns breaks pre-cdc to post-cdc 
upgrades (typically, 3.0 -> 3.X).
 + */
 +private static final Set TABLES_WITH_CDC_ADDED = 

[3/6] cassandra git commit: Fix startup problems due to schema tables not completely flushed

[stefania]

Fix startup problems due to schema tables not completely flushed

patch by Stefania Alborghetti; reviewed by Aleksey Yeschenko for 
CASSANDRA-12213.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6991556e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6991556e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6991556e

Branch: refs/heads/trunk
Commit: 6991556e431a51575744248a4c484270c4f918c9
Parents: 6efb44b
Author: Stefania Alborghetti 
Authored: Wed Feb 22 11:33:03 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Apr 6 10:14:35 2017 +0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/schema/SchemaKeyspace.java | 22 +++-
 2 files changed, 18 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6991556e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8bd21bc..212c738 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Fix startup problems due to schema tables not completely flushed 
(CASSANDRA-12213)
  * Fix view builder bug that can filter out data on restart (CASSANDRA-13405)
  * Fix 2i page size calculation when there are no regular columns 
(CASSANDRA-13400)
  * Fix the conversion of 2.X expired rows without regular column data 
(CASSANDRA-13395)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6991556e/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java 
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index eee4fbb..e860c59 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -25,8 +25,7 @@ import java.util.*;
 import java.util.concurrent.TimeUnit;
 import java.util.stream.Collectors;
 
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.MapDifference;
+import com.google.common.collect.*;
 import com.google.common.collect.Maps;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -82,8 +81,21 @@ public final class SchemaKeyspace
 public static final String AGGREGATES = "aggregates";
 public static final String INDEXES = "indexes";
 
-public static final List ALL =
-ImmutableList.of(KEYSPACES, TABLES, COLUMNS, DROPPED_COLUMNS, 
TRIGGERS, VIEWS, TYPES, FUNCTIONS, AGGREGATES, INDEXES);
+/**
+ * The order in this list matters.
+ *
+ * When flushing schema tables, we want to flush them in a way that 
mitigates the effects of an abrupt shutdown whilst
+ * the tables are being flushed. On startup, we load the schema from disk 
before replaying the CL, so we need to
+ * try to avoid problems like reading a table without columns or types, 
for example. So columns and types should be
+ * flushed before tables, which should be flushed before keyspaces.
+ *
+ * When truncating, the order should be reversed. For immutable lists this 
is an efficient operation that simply
+ * iterates in reverse order.
+ *
+ * See CASSANDRA-12213 for more details.
+ */
+public static final ImmutableList ALL =
+ImmutableList.of(COLUMNS, DROPPED_COLUMNS, TRIGGERS, TYPES, FUNCTIONS, 
AGGREGATES, INDEXES, TABLES, VIEWS, KEYSPACES);
 
 private static final CFMetaData Keyspaces =
 compile(KEYSPACES,
@@ -271,7 +283,7 @@ public final class SchemaKeyspace
 
 public static void truncate()
 {
-ALL.forEach(table -> getSchemaCFS(table).truncateBlocking());
+ALL.reverse().forEach(table -> getSchemaCFS(table).truncateBlocking());
 }
 
 static void flush()

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[stefania]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ed04da0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ed04da0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ed04da0

Branch: refs/heads/trunk
Commit: 9ed04da0c606a17b5567c8655f34d8f73d22b18f
Parents: c794d2b ab1310b
Author: Stefania Alborghetti 
Authored: Fri Apr 7 08:48:43 2017 +0800
Committer: Stefania Alborghetti 
Committed: Fri Apr 7 08:48:43 2017 +0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/schema/SchemaKeyspace.java | 23 +++-
 2 files changed, 18 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ed04da0/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ed04da0/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --cc src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index 47684ba,2c2416e..d2aa7e1
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@@ -23,10 -23,9 +23,8 @@@ import java.security.MessageDigest
  import java.security.NoSuchAlgorithmException;
  import java.util.*;
  import java.util.concurrent.TimeUnit;
 -import java.util.stream.Collectors;
  
- import com.google.common.collect.ImmutableList;
- import com.google.common.collect.ImmutableSet;
- import com.google.common.collect.MapDifference;
+ import com.google.common.collect.*;
  import com.google.common.collect.Maps;
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;

[1/6] cassandra git commit: Fix startup problems due to schema tables not completely flushed

[stefania]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 6efb44b3a -> 6991556e4
  refs/heads/cassandra-3.11 bfdc1e0fd -> ab1310b78
  refs/heads/trunk c794d2bed -> 9ed04da0c


Fix startup problems due to schema tables not completely flushed

patch by Stefania Alborghetti; reviewed by Aleksey Yeschenko for 
CASSANDRA-12213.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6991556e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6991556e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6991556e

Branch: refs/heads/cassandra-3.0
Commit: 6991556e431a51575744248a4c484270c4f918c9
Parents: 6efb44b
Author: Stefania Alborghetti 
Authored: Wed Feb 22 11:33:03 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Apr 6 10:14:35 2017 +0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/schema/SchemaKeyspace.java | 22 +++-
 2 files changed, 18 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6991556e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8bd21bc..212c738 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Fix startup problems due to schema tables not completely flushed 
(CASSANDRA-12213)
  * Fix view builder bug that can filter out data on restart (CASSANDRA-13405)
  * Fix 2i page size calculation when there are no regular columns 
(CASSANDRA-13400)
  * Fix the conversion of 2.X expired rows without regular column data 
(CASSANDRA-13395)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6991556e/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java 
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index eee4fbb..e860c59 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -25,8 +25,7 @@ import java.util.*;
 import java.util.concurrent.TimeUnit;
 import java.util.stream.Collectors;
 
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.MapDifference;
+import com.google.common.collect.*;
 import com.google.common.collect.Maps;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -82,8 +81,21 @@ public final class SchemaKeyspace
 public static final String AGGREGATES = "aggregates";
 public static final String INDEXES = "indexes";
 
-public static final List ALL =
-ImmutableList.of(KEYSPACES, TABLES, COLUMNS, DROPPED_COLUMNS, 
TRIGGERS, VIEWS, TYPES, FUNCTIONS, AGGREGATES, INDEXES);
+/**
+ * The order in this list matters.
+ *
+ * When flushing schema tables, we want to flush them in a way that 
mitigates the effects of an abrupt shutdown whilst
+ * the tables are being flushed. On startup, we load the schema from disk 
before replaying the CL, so we need to
+ * try to avoid problems like reading a table without columns or types, 
for example. So columns and types should be
+ * flushed before tables, which should be flushed before keyspaces.
+ *
+ * When truncating, the order should be reversed. For immutable lists this 
is an efficient operation that simply
+ * iterates in reverse order.
+ *
+ * See CASSANDRA-12213 for more details.
+ */
+public static final ImmutableList ALL =
+ImmutableList.of(COLUMNS, DROPPED_COLUMNS, TRIGGERS, TYPES, FUNCTIONS, 
AGGREGATES, INDEXES, TABLES, VIEWS, KEYSPACES);
 
 private static final CFMetaData Keyspaces =
 compile(KEYSPACES,
@@ -271,7 +283,7 @@ public final class SchemaKeyspace
 
 public static void truncate()
 {
-ALL.forEach(table -> getSchemaCFS(table).truncateBlocking());
+ALL.reverse().forEach(table -> getSchemaCFS(table).truncateBlocking());
 }
 
 static void flush()

[jira] [Commented] (CASSANDRA-12229) Move streaming to non-blocking IO and netty (streaming 2.1)

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959944#comment-15959944
 ] 

Jason Brown commented on CASSANDRA-12229:
-

[~JoshuaMcKenzie] Ariel is gonna take a look, but for sure I'd love another set 
of eyes on this.

> Move streaming to non-blocking IO and netty (streaming 2.1)
> ---
>
> Key: CASSANDRA-12229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12229
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 4.0
>
>
> As followup work to CASSANDRA-8457, we need to move streaming to use netty.
> Streaming 2.0 (CASSANDRA-5286) brought many good improvements to how files 
> are transferred between nodes in a cluster. However, the low-level details of 
> the current streaming implementation does not line up nicely with a 
> non-blocking model, so I think this is a good time to review some of those 
> details and add in additional goodness. The current implementation assumes a 
> sequential or "single threaded" approach to the sending of stream messages as 
> well as the transfer of files. In short, after several iterative prototypes, 
> I propose the following:
> 1) use a single bi-diredtional connection (instead of requiring to two 
> sockets & two threads)
> 2) send the "non-file" {{StreamMessage}} s (basically anything not 
> {{OutboundFileMessage}}) via the normal internode messaging. This will 
> require a slight bit more management of the session (the ability to look up a 
> {{StreamSession}} from a static function on {{StreamManager}}, but we have 
> have most of the pieces we need for this already.
> 3) switch to a non-blocking IO model (facilitated via netty)
> 4) Allow files to be streamed in parallel (CASSANDRA-4663) - this should just 
> be a thing already
> 5) If the entire sstable is to streamed, in addition to the DATA component, 
> transfer all the components of the sstable (primary index, bloom filter, 
> stats, and so on). This way we can avoid the CPU and GC pressure from 
> deserializing the stream into objects. File streaming then amounts to a 
> block-level transfer.
> Note: The progress/results of CASSANDRA-11303 will need to be reflected here, 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-12229) Move streaming to non-blocking IO and netty (streaming 2.1)

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959944#comment-15959944
 ] 

Jason Brown edited comment on CASSANDRA-12229 at 4/6/17 11:12 PM:
--

[~JoshuaMcKenzie] [~aweisberg] is gonna take a look, but for sure I'd love 
another set of eyes on this.


was (Author: jasobrown):
[~JoshuaMcKenzie] Ariel is gonna take a look, but for sure I'd love another set 
of eyes on this.

> Move streaming to non-blocking IO and netty (streaming 2.1)
> ---
>
> Key: CASSANDRA-12229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12229
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 4.0
>
>
> As followup work to CASSANDRA-8457, we need to move streaming to use netty.
> Streaming 2.0 (CASSANDRA-5286) brought many good improvements to how files 
> are transferred between nodes in a cluster. However, the low-level details of 
> the current streaming implementation does not line up nicely with a 
> non-blocking model, so I think this is a good time to review some of those 
> details and add in additional goodness. The current implementation assumes a 
> sequential or "single threaded" approach to the sending of stream messages as 
> well as the transfer of files. In short, after several iterative prototypes, 
> I propose the following:
> 1) use a single bi-diredtional connection (instead of requiring to two 
> sockets & two threads)
> 2) send the "non-file" {{StreamMessage}} s (basically anything not 
> {{OutboundFileMessage}}) via the normal internode messaging. This will 
> require a slight bit more management of the session (the ability to look up a 
> {{StreamSession}} from a static function on {{StreamManager}}, but we have 
> have most of the pieces we need for this already.
> 3) switch to a non-blocking IO model (facilitated via netty)
> 4) Allow files to be streamed in parallel (CASSANDRA-4663) - this should just 
> be a thing already
> 5) If the entire sstable is to streamed, in addition to the DATA component, 
> transfer all the components of the sstable (primary index, bloom filter, 
> stats, and so on). This way we can avoid the CPU and GC pressure from 
> deserializing the stream into objects. File streaming then amounts to a 
> block-level transfer.
> Note: The progress/results of CASSANDRA-11303 will need to be reflected here, 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13151) Unicode unittest fail

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959851#comment-15959851
 ] 

Jay Zhuang commented on CASSANDRA-13151:


Looks good to me. +1

Could you please also push the change to 3.11?

> Unicode unittest fail
> -
>
> Key: CASSANDRA-13151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13151
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jay Zhuang
>Assignee: Alex Petrov
>
> Following unittests are failed in 3.11 and trunk(4.0) branch
> {noformat}
> SASIIndexTest.testUnicodeSupport
> StandardAnalyzerTest.testTokenizationJaJp1
> StandardAnalyzerTest.testTokenizationJaJp2
> StandardAnalyzerTest.testTokenizationRuRu1
> StandardAnalyzerTest.testTokenizationZnTw1
> {noformat}
> It works fine on my local mac, but not linux server. I guess it's related to 
> Unicode setting. Does anyone have any idea on that? (could it be related to 
> CASSANDRA-11077, CASSANDRA-11431?)
> Here are the failure details
> {noformat}
> $ ant testsome -Dtest.name=org.apache.cassandra.index.sasi.SASIIndexTest 
> -Dtest.methods=testUnicodeSupport
> ...
> [junit] Testcase: 
> testUnicodeSupport(org.apache.cassandra.index.sasi.SASIIndexTest):
> FAILED
> [junit] []
> [junit] junit.framework.AssertionFailedError: []
> [junit] at 
> org.apache.cassandra.index.sasi.SASIIndexTest.testUnicodeSupport(SASIIndexTest.java:1159)
> [junit] at 
> org.apache.cassandra.index.sasi.SASIIndexTest.testUnicodeSupport(SASIIndexTest.java:1122)
> {noformat}
> {noformat}
> $ ant test -Dtest.name=StandardAnalyzerTest
> ...
> [junit] Testcase: 
> testTokenizationJaJp1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<210> but was:<0>
> [junit] junit.framework.AssertionFailedError: expected:<210> but was:<0>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationJaJp1(StandardAnalyzerTest.java:85)
> [junit]
> [junit]
> [junit] Testcase: 
> testTokenizationJaJp2(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<57> but was:<9>
> [junit] junit.framework.AssertionFailedError: expected:<57> but was:<9>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationJaJp2(StandardAnalyzerTest.java:104)
> [junit]
> [junit]
> [junit] Testcase: 
> testTokenizationRuRu1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<456> but was:<0>
> [junit] junit.framework.AssertionFailedError: expected:<456> but was:<0>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationRuRu1(StandardAnalyzerTest.java:120)
> [junit]
> [junit]
> [junit] Testcase: 
> testTokenizationZnTw1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
>  FAILED
> [junit] expected:<963> but was:<0>
> [junit] junit.framework.AssertionFailedError: expected:<963> but was:<0>
> [junit] at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationZnTw1(StandardAnalyzerTest.java:136)
> [junit]
> [junit]
> [junit] Test 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest FAILED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13422) CompactionStrategyManager should take write not read lock when handling add/remove notifications

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959848#comment-15959848
 ] 

Ariel Weisberg commented on CASSANDRA-13422:


||Code|utests|dtests||
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...aweisberg:cassandra-13422-3.11?expand=1]|[utests|https://circleci.com/gh/aweisberg/cassandra/10]|
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13422-trunk?expand=1]|[utests|https://circleci.com/gh/aweisberg/cassandra/12]|

> CompactionStrategyManager should take write not read lock when handling 
> add/remove notifications
> 
>
> Key: CASSANDRA-13422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13422
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0, 3.11.x
>
>
> {{getNextBackgroundTask}} in various compaction strategies (definitely 
> {{LCS}}) rely on checking the result of {{DataTracker.getCompacting()}} to 
> avoid accessing data and metadata related to tables that have already head 
> their resources released.
> There is a race where this check is unreliable and will claim a table that 
> has its resources already released is not compacting resulting in use after 
> free.
> [{{LeveledCompactionStrategy.findDroppableSSTable}}|https://github.com/apache/cassandra/blob/c794d2bed7ca1d10e13c4da08a3d45f5c755c1d8/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java#L504]
>  for instance has this three part logical && condition where the first check 
> is against the compacting set before calling {{worthDroppingTombstones}} 
> which fails if the table has been released.
> The order of events is basically that CompactionStrategyManager acquires the 
> read lock in getNextBackgroundTask(), then proceeds eventually to 
> findDroppableSSTable and acquires a set of SSTables from the manifest. While 
> the manifest is thread safe it's not accessed atomically WRT to other 
> operations. Once it has acquire the set of tables it acquires the (not 
> atomically) the set of compacting SSTables and iterates checking the former 
> against the latter.
> Meanwhile other compaction threads are marking tables obsolete or compacted 
> and releasing their references. Doing this removes them from {{DataTracker}} 
> and publishes a notification to the strategies, but this notification only 
> requires the read lock. After the compaction thread has published the 
> notifications it eventually marks the table as not compacting in 
> {{DataTracker}} or removes it entirely.
> The race is then that the compaction thread generating a new background task 
> acquires the sstables from the manifest on the stack. Any table in that set 
> that was compacting at that time must remain compacting so that it can be 
> skipped. Another compaction thread finishes a compaction and is able to 
> remove the table from the manifest and then remove it from the compacting 
> set. The thread generating the background task then acquires the list of 
> compacting tables which doesn't include the table it is supposed to skip.
> The simple fix appears to be to require threads to acquire the write lock in 
> order to publish notifications of tables being added/removed from compaction 
> strategies. While holding the write lock it won't be possible for someone to 
> see a view of tables in the manifest where tables that are compacting aren't 
> compacting in the view



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13421) WriteResponseHandlerTest is sensitive to test execution order

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-13421:
---
Status: Patch Available  (was: Open)

||Code|utests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13421-trunk?expand=1]|[utests|https://circleci.com/gh/aweisberg/cassandra/9]|

> WriteResponseHandlerTest is sensitive to test execution order
> -
>
> Key: CASSANDRA-13421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13421
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> One of the tests checks statistics that aren't reset between tests. If the 
> test doesn't run first then the expected values are incorrect.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959840#comment-15959840
 ] 

Ariel Weisberg commented on CASSANDRA-13413:


These seems less useful for me in day to day testing where I would rather 
optimize for response time vs. daily test runs that are more thorough.

If I need to build several branches at once I would rather use my containers 
for that or better yet for running the same branch faster.

For me this would be a step backwards.

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-12643) Improve reporting of overflowing histograms

[Edward Capriolo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo resolved CASSANDRA-12643.
-
Resolution: Won't Fix

> Improve reporting of overflowing histograms
> ---
>
> Key: CASSANDRA-12643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Edward Capriolo
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12643) Improve reporting of overflowing histograms

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959832#comment-15959832
 ] 

Edward Capriolo commented on CASSANDRA-12643:
-

Im unassigning myself from Cassandra as just about every ticket i have is -1 or 
only gets a comment 2x a year.

> Improve reporting of overflowing histograms
> ---
>
> Key: CASSANDRA-12643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Edward Capriolo
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12643) Improve reporting of overflowing histograms

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959822#comment-15959822
 ] 

Paulo Motta commented on CASSANDRA-12643:
-

I'm just unassigning myself from tens of tickets I'm not currently active to 
clean my JIRA backlog, but happy to resume review if another patch is provided. 
Thanks!

> Improve reporting of overflowing histograms
> ---
>
> Key: CASSANDRA-12643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Edward Capriolo
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13422) CompactionStrategyManager should take write not read lock when handling add/remove notifications

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-13422:
---
Description: 
{{getNextBackgroundTask}} in various compaction strategies (definitely {{LCS}}) 
rely on checking the result of {{DataTracker.getCompacting()}} to avoid 
accessing data and metadata related to tables that have already head their 
resources released.

There is a race where this check is unreliable and will claim a table that has 
its resources already released is not compacting resulting in use after free.

[{{LeveledCompactionStrategy.findDroppableSSTable}}|https://github.com/apache/cassandra/blob/c794d2bed7ca1d10e13c4da08a3d45f5c755c1d8/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java#L504]
 for instance has this three part logical && condition where the first check is 
against the compacting set before calling {{worthDroppingTombstones}} which 
fails if the table has been released.

The order of events is basically that CompactionStrategyManager acquires the 
read lock in getNextBackgroundTask(), then proceeds eventually to 
findDroppableSSTable and acquires a set of SSTables from the manifest. While 
the manifest is thread safe it's not accessed atomically WRT to other 
operations. Once it has acquire the set of tables it acquires the (not 
atomically) the set of compacting SSTables and iterates checking the former 
against the latter.

Meanwhile other compaction threads are marking tables obsolete or compacted and 
releasing their references. Doing this removes them from {{DataTracker}} and 
publishes a notification to the strategies, but this notification only requires 
the read lock. After the compaction thread has published the notifications it 
eventually marks the table as not compacting in {{DataTracker}} or removes it 
entirely.

The race is then that the compaction thread generating a new background task 
acquires the sstables from the manifest on the stack. Any table in that set 
that was compacting at that time must remain compacting so that it can be 
skipped. Another compaction thread finishes a compaction and is able to remove 
the table from the manifest and then remove it from the compacting set. The 
thread generating the background task then acquires the list of compacting 
tables which doesn't include the table it is supposed to skip.

The simple fix appears to be to require threads to acquire the write lock in 
order to publish notifications of tables being added/removed from compaction 
strategies. While holding the write lock it won't be possible for someone to 
see a view of tables in the manifest where tables that are compacting aren't 
compacting in the view

> CompactionStrategyManager should take write not read lock when handling 
> add/remove notifications
> 
>
> Key: CASSANDRA-13422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13422
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0, 3.11.x
>
>
> {{getNextBackgroundTask}} in various compaction strategies (definitely 
> {{LCS}}) rely on checking the result of {{DataTracker.getCompacting()}} to 
> avoid accessing data and metadata related to tables that have already head 
> their resources released.
> There is a race where this check is unreliable and will claim a table that 
> has its resources already released is not compacting resulting in use after 
> free.
> [{{LeveledCompactionStrategy.findDroppableSSTable}}|https://github.com/apache/cassandra/blob/c794d2bed7ca1d10e13c4da08a3d45f5c755c1d8/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java#L504]
>  for instance has this three part logical && condition where the first check 
> is against the compacting set before calling {{worthDroppingTombstones}} 
> which fails if the table has been released.
> The order of events is basically that CompactionStrategyManager acquires the 
> read lock in getNextBackgroundTask(), then proceeds eventually to 
> findDroppableSSTable and acquires a set of SSTables from the manifest. While 
> the manifest is thread safe it's not accessed atomically WRT to other 
> operations. Once it has acquire the set of tables it acquires the (not 
> atomically) the set of compacting SSTables and iterates checking the former 
> against the latter.
> Meanwhile other compaction threads are marking tables obsolete or compacted 
> and releasing their references. Doing this removes them from {{DataTracker}} 
> and publishes a notification to the strategies, but this notification only 
> requires the read lock. After the compaction thread has published the 
> notifications it eventually marks the t

[jira] [Commented] (CASSANDRA-12643) Improve reporting of overflowing histograms

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959817#comment-15959817
 ] 

Edward Capriolo commented on CASSANDRA-12643:
-

Still seeing this happen... putting blinders on Maybe the next person will 
have more give-a-crap then me.

> Improve reporting of overflowing histograms
> ---
>
> Key: CASSANDRA-12643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Edward Capriolo
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-12643) Improve reporting of overflowing histograms

[Edward Capriolo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo reassigned CASSANDRA-12643:
---

Assignee: (was: Edward Capriolo)

> Improve reporting of overflowing histograms
> ---
>
> Key: CASSANDRA-12643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Edward Capriolo
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13385) Delegate utests index name creation to CQLTester.createIndex

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13385:

Reviewer: Benjamin Lerer

> Delegate utests index name creation to CQLTester.createIndex
> 
>
> Key: CASSANDRA-13385
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13385
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>  Labels: cql, unit-test
>
> Currently, many unit tests rely on {{CQLTester.createIndex}} to create 
> indexes. The index name should be specified by the test itself, for example:
> {code}
> createIndex("CREATE CUSTOM INDEX myindex ON %s(c) USING 
> 'org.apache.cassandra.index.internal.CustomCassandraIndex'");
> {code}
> Two different tests using the same index name can produce racy {{Index 
> myindex already exists}} errors due to the asynchronicity of 
> {{CQLTester.afterTest}} cleanup methods. 
> It would be nice to modify {{CQLTester.createIndex}} to make it generate its 
> own index names, as it is done by {{CQLTester.createTable}}:
> {code}
> createIndex("CREATE CUSTOM INDEX %s ON %s(c) USING 
> 'org.apache.cassandra.index.internal.CustomCassandraIndex'");
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12870) Calculate pending ranges for identical KS settings just once

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12870:

Reviewer: Joel Knighton

> Calculate pending ranges for identical KS settings just once
> 
>
> Key: CASSANDRA-12870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12870
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 4.x
>
> Attachments: 12870-trunk.patch
>
>
> The {{TokenMetadata.calculatePendingRanges()}} operation can be very 
> expensive and already has been subject to micro optimization in 
> CASSANDRA-9258. Instead of further optimizing the calculation itself, I'd 
> like to prevent it from being executed as often by only calling it just once 
> for all keyspaces sharing identical replication settings. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12229) Move streaming to non-blocking IO and netty (streaming 2.1)

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959798#comment-15959798
 ] 

Joshua McKenzie commented on CASSANDRA-12229:
-

[~jasobrown]: this ticket's awaiting feedback w/out a reviewer - you in the 
market for someone to work with you on review or do you guys have resources on 
that side to allocate to the effort? Honestly, worth it to have dual-review on 
something as big / critical as this effort IMO.

> Move streaming to non-blocking IO and netty (streaming 2.1)
> ---
>
> Key: CASSANDRA-12229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12229
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 4.0
>
>
> As followup work to CASSANDRA-8457, we need to move streaming to use netty.
> Streaming 2.0 (CASSANDRA-5286) brought many good improvements to how files 
> are transferred between nodes in a cluster. However, the low-level details of 
> the current streaming implementation does not line up nicely with a 
> non-blocking model, so I think this is a good time to review some of those 
> details and add in additional goodness. The current implementation assumes a 
> sequential or "single threaded" approach to the sending of stream messages as 
> well as the transfer of files. In short, after several iterative prototypes, 
> I propose the following:
> 1) use a single bi-diredtional connection (instead of requiring to two 
> sockets & two threads)
> 2) send the "non-file" {{StreamMessage}} s (basically anything not 
> {{OutboundFileMessage}}) via the normal internode messaging. This will 
> require a slight bit more management of the session (the ability to look up a 
> {{StreamSession}} from a static function on {{StreamManager}}, but we have 
> have most of the pieces we need for this already.
> 3) switch to a non-blocking IO model (facilitated via netty)
> 4) Allow files to be streamed in parallel (CASSANDRA-4663) - this should just 
> be a thing already
> 5) If the entire sstable is to streamed, in addition to the DATA component, 
> transfer all the components of the sstable (primary index, bloom filter, 
> stats, and so on). This way we can avoid the CPU and GC pressure from 
> deserializing the stream into objects. File streaming then amounts to a 
> block-level transfer.
> Note: The progress/results of CASSANDRA-11303 will need to be reflected here, 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13422) CompactionStrategyManager should take write not read lock when handling add/remove notifications

[Ariel Weisberg (JIRA)]

Ariel Weisberg created CASSANDRA-13422:
--

 Summary: CompactionStrategyManager should take write not read lock 
when handling add/remove notifications
 Key: CASSANDRA-13422
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13422
 Project: Cassandra
  Issue Type: Bug
  Components: Local Write-Read Paths
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 4.0, 3.11.x






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12485) Always require replace_address to replace existing token

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12485:

Reviewer:   (was: Paulo Motta)

> Always require replace_address to replace existing token
> 
>
> Key: CASSANDRA-12485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12485
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Paulo Motta
>Priority: Minor
>  Labels: lhf
>
> CASSANDRA-10134 prevented replace an existing node unless 
> {{\-Dcassandra.replace_address}} or 
> {{\-Dcassandra.allow_unsafe_replace=true}} is specified.
> We should extend this behavior to tokens, preventing a node from joining the 
> ring if another node with the same token already existing in the ring, unless 
> {{\-Dcassandra.replace_address}} or 
> {{\-Dcassandra.allow_unsafe_replace=true}} is specified in order to avoid 
> catastrophic scenarios.
> One scenario where this can easily happen is if you replace a node with 
> another node with a different IP, and after some time you restart the 
> original node by mistake. The original node will then take over the tokens of 
> the replaced node (since it has a newer gossip generation).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-11303) New inbound throughput parameters for streaming

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11303:

Status: Open  (was: Patch Available)

> New inbound throughput parameters for streaming
> ---
>
> Key: CASSANDRA-11303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11303
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Configuration
>Reporter: Satoshi Konno
>Assignee: Satoshi Konno
>Priority: Minor
> Attachments: 11303_inbound_limit_debug_20160419.log, 
> 11303_inbound_nolimit_debug_20160419.log, 
> 11303_inbound_patch_for_trunk_20160419.diff, 
> 11303_inbound_patch_for_trunk_20160525.diff, 
> 11303_inbound_patch_for_trunk_20160704.diff, 
> 200vs40inboundstreamthroughput.png, cassandra_inbound_stream.diff
>
>
> Hi,
> To specify stream throughputs of a node more clearly, I would like to add the 
> following new inbound parameters like existing outbound parameters in the 
> cassandra.yaml.
> - stream_throughput_inbound_megabits_per_sec
> - inter_dc_stream_throughput_outbound_megabits_per_sec  
> We use only the existing outbound parameters now, but it is difficult to 
> control the total throughputs of a node.　In our production network, some 
> critical alerts occurs when a node exceed the specified total throughput 
> which is the sum of the input and output throughputs.
> In our operation of Cassandra, the alerts occurs during the bootstrap or 
> repair processing when a new node is added. In the worst case, we have to 
> stop the operation of the exceed node.
> I have attached the patch under consideration. I would like to add a new 
> limiter class, StreamInboundRateLimiter, and use the limiter class in 
> StreamDeserializer class. I use Row::dataSize( ）to get the input throughput 
> in StreamDeserializer::newPartition(), but I am not sure whether the 
> dataSize() returns the correct data size.
> Can someone please tell me how to do it ?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-10446) Run repair with down replicas

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10446:

Reviewer:   (was: Paulo Motta)

> Run repair with down replicas
> -
>
> Key: CASSANDRA-10446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10446
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> We should have an option of running repair when replicas are down. We can 
> call it -force.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-11258) Repair scheduling - Resource locking API

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11258:

Reviewer:   (was: Paulo Motta)

> Repair scheduling - Resource locking API
> 
>
> Key: CASSANDRA-11258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11258
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Minor
> Attachments: newDuration.patch
>
>
> Create a resource locking API & implementation that is able to lock a 
> resource in a specified data center. It should handle priorities to avoid 
> node starvation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10862:

Reviewer:   (was: Paulo Motta)

> LCS repair: compact tables before making available in L0
> 
>
> Key: CASSANDRA-10862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction, Streaming and Messaging
>Reporter: Jeff Ferland
>Assignee: Chen Shen
>  Labels: lcs
>
> When doing repair on a system with lots of mismatched ranges, the number of 
> tables in L0 goes up dramatically, as correspondingly goes the number of 
> tables referenced for a query. Latency increases dramatically in tandem.
> Eventually all the copied tables are compacted down in L0, then copied into 
> L1 (which may be a very large copy), finally reducing the number of SSTables 
> per query into the manageable range.
> It seems to me that the cleanest answer is to compact after streaming, then 
> mark tables available rather than marking available when the file itself is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13065) Skip building views during base table streams on range movements

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-13065.
-
Resolution: Fixed

Committed as {{c794d2bed7ca1d10e13c4da08a3d45f5c755c1d8}} to trunk. Good job! 
:-)

> Skip building views during base table streams on range movements
> 
>
> Key: CASSANDRA-13065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13065
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 4.0
>
>
> Booting or decommisioning nodes with MVs is unbearably slow as all streams go 
> through the regular write paths. This causes read-before-writes for every 
> mutation and during bootstrap it causes them to be sent to batchlog.
> The makes it virtually impossible to boot a new node in an acceptable amount 
> of time.
> Using the regular streaming behaviour for consistent range movements works 
> much better in this case and does not break the MV local consistency contract.
> Already tested on own cluster.
> Bootstrap case is super easy to handle, decommission case requires 
> CASSANDRA-13064



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13065) Skip building views during base table streams on range movements

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-13065:

Summary: Skip building views during base table streams on range movements  
(was: Consistent range movements to not require MV updates to go through write 
paths )

> Skip building views during base table streams on range movements
> 
>
> Key: CASSANDRA-13065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13065
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 4.0
>
>
> Booting or decommisioning nodes with MVs is unbearably slow as all streams go 
> through the regular write paths. This causes read-before-writes for every 
> mutation and during bootstrap it causes them to be sent to batchlog.
> The makes it virtually impossible to boot a new node in an acceptable amount 
> of time.
> Using the regular streaming behaviour for consistent range movements works 
> much better in this case and does not break the MV local consistency contract.
> Already tested on own cluster.
> Bootstrap case is super easy to handle, decommission case requires 
> CASSANDRA-13064



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

cassandra git commit: Skip building views during base table streams on range movements

[paulo]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 30820eacb -> c794d2bed


Skip building views during base table streams on range movements

patch by Benjamin Roth; reviewed by Paulo Motta for CASSANDRA-13065


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c794d2be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c794d2be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c794d2be

Branch: refs/heads/trunk
Commit: c794d2bed7ca1d10e13c4da08a3d45f5c755c1d8
Parents: 30820ea
Author: brstgt 
Authored: Tue Feb 28 13:36:16 2017 +0100
Committer: Paulo Motta 
Committed: Thu Apr 6 18:18:55 2017 -0300

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/db/SystemKeyspace.java |  5 +-
 .../org/apache/cassandra/dht/BootStrapper.java  |  2 +-
 .../org/apache/cassandra/dht/RangeStreamer.java |  7 +-
 .../apache/cassandra/dht/StreamStateStore.java  |  2 +-
 .../cassandra/io/sstable/SSTableLoader.java |  2 +-
 .../net/IncomingStreamingConnection.java|  2 +-
 .../apache/cassandra/repair/LocalSyncTask.java  |  9 +-
 .../cassandra/repair/StreamingRepairTask.java   |  3 +-
 .../cassandra/service/StorageService.java   |  8 +-
 .../cassandra/streaming/ConnectionHandler.java  |  2 +-
 .../apache/cassandra/streaming/StreamEvent.java |  4 +-
 .../cassandra/streaming/StreamOperation.java| 69 +++
 .../apache/cassandra/streaming/StreamPlan.java  | 18 ++--
 .../cassandra/streaming/StreamReceiveTask.java  | 93 
 .../cassandra/streaming/StreamResultFuture.java | 34 +++
 .../cassandra/streaming/StreamSession.java  |  4 +-
 .../apache/cassandra/streaming/StreamState.java |  6 +-
 .../management/StreamStateCompositeData.java|  7 +-
 .../streaming/messages/StreamInitMessage.java   | 13 +--
 .../cassandra/tools/nodetool/NetStats.java  |  2 +-
 .../apache/cassandra/dht/BootStrapperTest.java  |  3 +-
 .../cassandra/io/sstable/LegacySSTableTest.java |  5 +-
 .../streaming/StreamOperationTest.java  | 47 ++
 .../streaming/StreamTransferTaskTest.java   |  2 +-
 .../streaming/StreamingTransferTest.java| 12 +--
 26 files changed, 254 insertions(+), 108 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c794d2be/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0e3da3f..b7b2cc9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Skip building views during base table streams on range movements 
(CASSANDRA-13065)
  * Improve error messages for +/- operations on maps and tuples 
(CASSANDRA-13197)
  * Remove deprecated repair JMX APIs (CASSANDRA-11530)
  * Fix version check to enable streaming keep-alive (CASSANDRA-12929)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c794d2be/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java 
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 058b378..0d64a94 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -59,6 +59,7 @@ import org.apache.cassandra.schema.SchemaConstants;
 import org.apache.cassandra.service.StorageService;
 import org.apache.cassandra.service.paxos.Commit;
 import org.apache.cassandra.service.paxos.PaxosState;
+import org.apache.cassandra.streaming.StreamOperation;
 import org.apache.cassandra.transport.ProtocolVersion;
 import org.apache.cassandra.utils.*;
 
@@ -1196,7 +1197,7 @@ public final class SystemKeyspace
 availableRanges.truncateBlocking();
 }
 
-public static synchronized void updateTransferredRanges(String description,
+public static synchronized void updateTransferredRanges(StreamOperation 
streamOperation,
  InetAddress peer,
  String keyspace,
  
Collection> streamedRanges)
@@ -1207,7 +1208,7 @@ public final class SystemKeyspace
 {
 rangesToUpdate.add(rangeToBytes(range));
 }
-executeInternal(format(cql, TRANSFERRED_RANGES), rangesToUpdate, 
description, peer, keyspace);
+executeInternal(format(cql, TRANSFERRED_RANGES), rangesToUpdate, 
streamOperation.getDescription(), peer, keyspace);
 }
 
 public static synchronized Map>> 
getTransferredRanges(String description, String keyspace, IPartitioner 
partitioner)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c794d2be/src/java/org/apache/cassandra/dht/BootStrapper.java

[jira] [Updated] (CASSANDRA-12643) Improve reporting of overflowing histograms

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12643:

Reviewer:   (was: Paulo Motta)

> Improve reporting of overflowing histograms
> ---
>
> Key: CASSANDRA-12643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12643
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-7839) Support standard EC2 naming conventions in Ec2Snitch

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-7839:
---
Reviewer:   (was: Paulo Motta)

> Support standard EC2 naming conventions in Ec2Snitch
> 
>
> Key: CASSANDRA-7839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7839
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Gregory Ramsperger
>Assignee: Gregory Ramsperger
>  Labels: docs-impacting
> Attachments: CASSANDRA-7839-aws-naming-conventions.patch
>
>
> The EC2 snitches use datacenter and rack naming conventions inconsistent with 
> those presented in Amazon EC2 APIs as region and availability zone. A 
> discussion of this is found in CASSANDRA-4026. This has not been changed for 
> valid backwards compatibility reasons. Using SnitchProperties, it is possible 
> to switch between the legacy naming and the full, AWS-style naming. 
> Proposal:
> * introduce a property (ec2_naming_scheme) to switch naming schemes.
> * default to current/legacy naming scheme
> * add support for a new scheme ("standard") which is consistent AWS 
> conventions
> ** data centers will be the region name, including the number
> ** racks will be the availability zone name, including the region name
> Examples:
> * * legacy* : datacenter is the part of the availability zone name preceding 
> the last "\-" when the zone ends in \-1 and includes the number if not \-1. 
> Rack is the portion of the availability zone name following  the last "\-".
> ** us-west-1a => dc: us-west, rack: 1a
> ** us-west-2b => dc: us-west-2, rack: 2b; 
> * *standard* : datacenter is the part of the availability zone name preceding 
> zone letter. rack is the entire availability zone name.
> ** us-west-1a => dc: us-west-1, rack: us-west-1a
> ** us-west-2b => dc: us-west-2, rack: us-west-2b; 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-10111) reconnecting snitch can bypass cluster name check

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10111:

Reviewer:   (was: Paulo Motta)

> reconnecting snitch can bypass cluster name check
> -
>
> Key: CASSANDRA-10111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10111
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Chris Burroughs
>Assignee: Joel Knighton
>  Labels: gossip, messaging-service-bump-required
> Fix For: 3.11.x
>
>
> Setup:
>  * Two clusters: A & B
>  * Both are two DC cluster
>  * Both use GossipingPropertyFileSnitch with different 
> listen_address/broadcast_address
> A new node was added to cluster A with a broadcast_address of an existing 
> node in cluster B (due to an out of data DNS entry).  Cluster B  added all of 
> the nodes from cluster A, somehow bypassing the cluster name mismatch check 
> for this nodes.  The first reference to cluster A nodes in cluster B logs is 
> when then were added:
> {noformat}
>  INFO [GossipStage:1] 2015-08-17 15:08:33,858 Gossiper.java (line 983) Node 
> /8.37.70.168 is now part of the cluster
> {noformat}
> Cluster B nodes then tried to gossip to cluster A nodes, but cluster A kept 
> them out with 'ClusterName mismatch'.  Cluster B however tried to send to 
> send reads/writes to cluster A and general mayhem ensued.
> Obviously this is a Bad (TM) config that Should Not Be Done.  However, since 
> the consequence of crazy merged clusters are really bad (the reason there is 
> the name mismatch check in the first place) I think the hole is reasonable to 
> plug.  I'm not sure exactly what the code path is that skips the check in 
> GossipDigestSynVerbHandler.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-10070) Automatic repair scheduling

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10070:

Reviewer:   (was: Paulo Motta)

> Automatic repair scheduling
> ---
>
> Key: CASSANDRA-10070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10070
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: Distributed Repair Scheduling.doc, Distributed Repair 
> Scheduling_V2.doc
>
>
> Scheduling and running repairs in a Cassandra cluster is most often a 
> required task, but this can both be hard for new users and it also requires a 
> bit of manual configuration. There are good tools out there that can be used 
> to simplify things, but wouldn't this be a good feature to have inside of 
> Cassandra? To automatically schedule and run repairs, so that when you start 
> up your cluster it basically maintains itself in terms of normal 
> anti-entropy, with the possibility for manual configuration.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-11875) Create sstableconvert tool with support to ma format

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-11875.
-
Resolution: Later

> Create sstableconvert tool with support to ma format
> 
>
> Key: CASSANDRA-11875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11875
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Kaide Mu
>Priority: Minor
> Attachments: trunk-11875-WIP-V1.patch
>
>
> Currently {{Upgrader}} receives an sstable in any readable format, and writes 
> into {{BigFormat.getLatestVersion()}}. We should generalize it by making it 
> receive a {{target}} version and probably also rename it to 
> {{SSTableConverter}}. 
> Based on this we can create an {{StandaloneDowngrader}} tool which will 
> perform downgrade of specified sstables to a target version. To start with, 
> we should support only downgrading to {{ma}} format (from current format 
> {{mb}}), downgrade to any other version should be forbidden. Since we already 
> support serializing to "ma" we will not need to do any data conversion.
> We should also create a test suite that creates an sstable with data in the 
> current format, perform the downgrade, and verify data in the new format is 
> correct. This will be the base tests suite for more advanced conversions in 
> the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-11877) Add initial support to pre-3.0 sstable serialization

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-11877.
-
Resolution: Later

> Add initial support to pre-3.0 sstable serialization
> 
>
> Key: CASSANDRA-11877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11877
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Kaide Mu
>Priority: Minor
>
> In order to support writing pre-3.0 sstables, the easiest route is to port 
> pre-3.0 {{BigTableWriter}} to {{LegacyLayout}}, so it can be easily removed 
> when this support is removed in the next majorversion.
> For simplicity, we should consider only the simple case in this ticket: no 
> range tombstones and index sampling. We can add support to those in follow-up 
> tickets.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-11876) Add serialization support to legacy index

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-11876.
-
Resolution: Later

> Add serialization support to legacy index
> -
>
> Key: CASSANDRA-11876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11876
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Kaide Mu
>Priority: Minor
>
> Currently {{LegacyShallowIndexedEntry}} only supports reading from legacy 
> format. We should add support to writing in old format 
> ({{!format.storeRows()}}).
> There are two cases to consider: {{RowIndexEntry}} (when the index is smaller 
> than {{column_index_size_in_kb}}) and {{IndexedEntry}} (otherwise).
> We should add tests to verify that the index is being properly converted from 
> the current version to the legacy version.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-10526) Add dtest for CASSANDRA-10406

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-10526.
-
Resolution: Fixed

> Add dtest for CASSANDRA-10406
> -
>
> Key: CASSANDRA-10526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10526
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 3.11.x
>
>
> CASSANDRA-10406 adds new function to nodetool rebuild, so it needs to be 
> tested with cassandra-dtest.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-9305) Test display of all types in cqlsh

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-9305:
---
Reviewer:   (was: Paulo Motta)

> Test display of all types in cqlsh
> --
>
> Key: CASSANDRA-9305
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9305
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Tyler Hobbs
>Assignee: Jim Witschey
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.1.x, 3.11.x
>
>
> Although we have cqlsh tests for displaying a lot of types, we don't have 
> full coverage.  It looks like we need to add tests for the following types:
> * map, set, list
> * frozen (and nested) map, set, list
> * tuples (including nested)
> * time and date types (CASSANDRA-7523)
> * timeuuid
> * CompositeType (needs to be created as a custom type, like {{mycol 
> 'org.apache.cassandra.db.marshal.CompositeType(AsciiType, Int32Type)'}}, see 
> CASSANDRA-8919)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-8911) Consider Mutation-based Repairs

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-8911:
---
Reviewer:   (was: Paulo Motta)

> Consider Mutation-based Repairs
> ---
>
> Key: CASSANDRA-8911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8911
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tyler Hobbs
> Fix For: 3.11.x
>
>
> We should consider a mutation-based repair to replace the existing streaming 
> repair.  While we're at it, we could do away with a lot of the complexity 
> around merkle trees.
> I have not planned this out in detail, but here's roughly what I'm thinking:
>  * Instead of building an entire merkle tree up front, just send the "leaves" 
> one-by-one.  Instead of dealing with token ranges, make the leaves primary 
> key ranges.  The PK ranges would need to be contiguous, so that the start of 
> each range would match the end of the previous range. (The first and last 
> leaves would need to be open-ended on one end of the PK range.) This would be 
> similar to doing a read with paging.
>  * Once one page of data is read, compute a hash of it and send it to the 
> other replicas along with the PK range that it covers and a row count.
>  * When the replicas receive the hash, the perform a read over the same PK 
> range (using a LIMIT of the row count + 1) and compare hashes (unless the row 
> counts don't match, in which case this can be skipped).
>  * If there is a mismatch, the replica will send a mutation covering that 
> page's worth of data (ignoring the row count this time) to the source node.
> Here are the advantages that I can think of:
>  * With the current repair behavior of streaming, vnode-enabled clusters may 
> need to stream hundreds of small SSTables.  This results in increased compact
> ion load on the receiving node.  With the mutation-based approach, memtables 
> would naturally merge these.
>  * It's simple to throttle.  For example, you could give a number of rows/sec 
> that should be repaired.
>  * It's easy to see what PK range has been repaired so far.  This could make 
> it simpler to resume a repair that fails midway.
>  * Inconsistencies start to be repaired almost right away.
>  * Less special code \(?\)
>  * Wide partitions are no longer a problem.
> There are a few problems I can think of:
>  * Counters.  I don't know if this can be made safe, or if they need to be 
> skipped.
>  * To support incremental repair, we need to be able to read from only 
> repaired sstables.  Probably not too difficult to do.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-8312:
---
Reviewer:   (was: Paulo Motta)

> Use live sstables in snapshot repair if possible
> 
>
> Key: CASSANDRA-8312
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8312
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jimmy Mårdell
>Priority: Minor
>  Labels: lcs
> Fix For: 2.1.x
>
> Attachments: cassandra-2.0-8312-1.txt
>
>
> Snapshot repair can be very much slower than parallel repairs because of the 
> overhead of opening the SSTables in the snapshot. This is particular true 
> when using LCS, as you typically have many smaller SSTables then.
> I compared parallel and sequential repair on a small range on one of our 
> clusters (2*3 replicas). With parallel repair, this took 22 seconds. With 
> sequential repair (default in 2.0), the same range took 330 seconds! This is 
> an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were 
> 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely 
> causes lots of memory churning.
> The idea would be to list the sstables in the snapshot, but use the 
> corresponding sstables in the live set if it's still available. For almost 
> all sstables, the original one should still exist.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-7276) Include keyspace and table names in logs where possible

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-7276:
---
Reviewer:   (was: Paulo Motta)

> Include keyspace and table names in logs where possible
> ---
>
> Key: CASSANDRA-7276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7276
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tyler Hobbs
>Priority: Minor
>  Labels: bootcamp, lhf
> Fix For: 2.1.x
>
> Attachments: 0001-Better-Logging-for-KS-and-CF.patch, 
> 0001-Consistent-KS-and-Table-Logging.patch, 
> 0001-Logging-for-Keyspace-and-Tables.patch, 
> 0001-Logging-KS-and-CF-consistently.patch, 2.1-CASSANDRA-7276-v1.txt, 
> cassandra-2.1-7276-compaction.txt, cassandra-2.1-7276.txt, 
> cassandra-2.1.9-7276.txt, cassandra-2.1.9-7276-v2.txt
>
>
> Most error messages and stacktraces give you no clue as to what keyspace or 
> table was causing the problem.  For example:
> {noformat}
> ERROR [MutationStage:61648] 2014-05-20 12:05:45,145 CassandraDaemon.java 
> (line 198) Exception in thread Thread[MutationStage:61648,5,main]
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:63)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:98)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059)
> at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985)
> at 
> org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:328)
> at 
> org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:200)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:173)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:893)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:333)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:206)
> at 
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:56)
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
> We should try to include info on the keyspace and column family in the error 
> messages or logs whenever possible.  This includes reads, writes, 
> compactions, flushes, repairs, and probably more.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-6559) cqlsh should warn about ALLOW FILTERING

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-6559:
---
Reviewer:   (was: Paulo Motta)

> cqlsh should warn about ALLOW FILTERING
> ---
>
> Key: CASSANDRA-6559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6559
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Tupshin Harper
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-6559.txt
>
>
> ALLOW FILTERING can be a convenience for preliminary exploration of your 
> data, and can be useful for batch jobs, but it is such an anti-pattern for 
> regular production queries, that cqlsh should provie an explicit warn 
> ingwhenever such a query is performed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13421) WriteResponseHandlerTest is sensitive to test execution order

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-13421:
---
Reviewer: Jeff Jirsa

> WriteResponseHandlerTest is sensitive to test execution order
> -
>
> Key: CASSANDRA-13421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13421
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> One of the tests checks statistics that aren't reset between tests. If the 
> test doesn't run first then the expected values are incorrect.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13421) WriteResponseHandlerTest is sensitive to test execution order

[Ariel Weisberg (JIRA)]

Ariel Weisberg created CASSANDRA-13421:
--

 Summary: WriteResponseHandlerTest is sensitive to test execution 
order
 Key: CASSANDRA-13421
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13421
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 4.0


One of the tests checks statistics that aren't reset between tests. If the test 
doesn't run first then the expected values are incorrect.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13397) Return value of CountDownLatch.await() not being checked in Repair

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13397:

Reviewer: Paulo Motta

> Return value of CountDownLatch.await() not being checked in Repair
> --
>
> Key: CASSANDRA-13397
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13397
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Fix For: 3.0.13
>
> Attachments: CASSANDRA-13397-v1.patch
>
>
> While looking into repair code, I realize that we should check return value 
> of CountDownLatch.await(). Most of the places that we don't check the return 
> value, nothing bad would happen due to other protection. However, 
> ActiveRepairService#prepareForRepair should have the check. Code to reproduce:
> {code}
> public static void testLatch() throws InterruptedException {
> CountDownLatch latch = new CountDownLatch(2);
> latch.countDown();
> new Thread(() -> {
> try {
> Thread.sleep(1200);
> } catch (InterruptedException e) {
> System.err.println("interrupted");
> }
> latch.countDown();
> System.out.println("counted down");
> }).start();
> latch.await(1, TimeUnit.SECONDS);
> if (latch.getCount() > 0) {
> System.err.println("failed");
> } else {
> System.out.println("success");
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959691#comment-15959691
 ] 

Joshua McKenzie commented on CASSANDRA-13354:
-

[~krummas]: have bandwidth for review on this one?

> LCS estimated compaction tasks does not take number of files into account
> -
>
> Key: CASSANDRA-13354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.2.9
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Attachments: 13354-trunk.txt, patchedTest.png, unpatchedTest.png
>
>
> In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
> taking the size of a SSTable and multiply it by four. This would give 4*160mb 
> with default settings. This calculation is used to determine whether repaired 
> or repaired data is being compacted.
> Now this works well until you take repair into account. Repair streams over 
> many many sstables which could be smaller than the configured SSTable size 
> depending on your use case. In our case we are talking about many thousands 
> of tiny SSTables. As number of files increases one can run into any number of 
> problems, including GC issues, too many open files or plain increase in read 
> latency.
> With the current algorithm we will choose repaired or unrepaired depending on 
> whichever side has more data in it. Even if the repaired files outnumber the 
> unrepaired files by a large margin.
> Similarily, our algorithm that selects compaction candidates takes up to 32 
> SSTables at a time in L0, however our estimated task calculation does not 
> take this number into account. These two mechanisms should be aligned with 
> each other.
> I propose that we take the number of files in L0 into account when estimating 
> remaining tasks. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13308) Gossip breaks, Hint files not being deleted on nodetool decommission

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13308:

Reviewer: Aleksey Yeschenko

> Gossip breaks, Hint files not being deleted on nodetool decommission
> 
>
> Key: CASSANDRA-13308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13308
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Using Cassandra version 3.0.9
>Reporter: Arijit
>Assignee: Jeff Jirsa
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: 28207.stack, logs, logs_decommissioned_node
>
>
> How to reproduce the issue I'm seeing:
> Shut down Cassandra on one node of the cluster and wait until we accumulate a 
> ton of hints. Start Cassandra on the node and immediately run "nodetool 
> decommission" on it.
> The node streams its replicas and marks itself as DECOMMISSIONED, but other 
> nodes do not seem to see this message. "nodetool status" shows the 
> decommissioned node in state "UL" on all other nodes (it is also present in 
> system.peers), and Cassandra logs show that gossip tasks on nodes are not 
> proceeding (number of pending tasks keeps increasing). Jstack suggests that a 
> gossip task is blocked on hints dispatch (I can provide traces if this is not 
> obvious). Because the cluster is large and there are a lot of hints, this is 
> taking a while. 
> On inspecting "/var/lib/cassandra/hints" on the nodes, I see a bunch of hint 
> files for the decommissioned node. Documentation seems to suggest that these 
> hints should be deleted during "nodetool decommission", but it does not seem 
> to be the case here. This is the bug being reported.
> To recover from this scenario, if I manually delete hint files on the nodes, 
> the hints dispatcher threads throw a bunch of exceptions and the 
> decommissioned node is now in state "DL" (perhaps it missed some gossip 
> messages?). The node is still in my "system.peers" table
> Restarting Cassandra on all nodes after this step does not fix the issue (the 
> node remains in the peers table). In fact, after this point the 
> decommissioned node is in state "DN"



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13263) Incorrect ComplexColumnData hashCode implementation

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13263:

Reviewer: Sylvain Lebresne

> Incorrect ComplexColumnData hashCode implementation
> ---
>
> Key: CASSANDRA-13263
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13263
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>
> I went through some of the logs from CASSANDRA-13175 and one of the more 
> serious issues that we should address seem to be the 
> {{ComplexColumnData.hashCode()}} implementation. As Objects.hashCode is not 
> using deep hashing for array arguments, hashed will be based on the identity 
> instead of the array's content. See patch for simple fix.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13302) last row of previous page == first row of next page while querying data using SASI index

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13302:

Reviewer: Andrés de la Peña

> last row of previous page == first row of next page while querying data using 
> SASI index
> 
>
> Key: CASSANDRA-13302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13302
> Project: Cassandra
>  Issue Type: Bug
> Environment: Tested with C* 3.9 and 3.10.
>Reporter: Andy Tolbert
>Assignee: Alex Petrov
>
> Apologies if this is a duplicate (couldn't track down an existing bug).
> Similarly to [CASSANDRA-11208], it appears it is possible to retrieve 
> duplicate rows when paging using a SASI index as documented in 
> [JAVA-1413|https://datastax-oss.atlassian.net/browse/JAVA-1413], the 
> following test demonstrates that data is repeated while querying using a SASI 
> index:
> {code:java}
> public class TestPagingBug
> {
>   public static void main(String[] args)
>   {
>   Cluster.Builder builder = Cluster.builder();
>   Cluster c = builder.addContactPoints("192.168.98.190").build(); 
> 
>   Session s = c.connect();
>   
>   s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication 
> = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
>   s.execute("CREATE TABLE IF NOT EXISTS test.test_table_sec(sec 
> BIGINT PRIMARY KEY, id INT)");
> //create secondary index on ID column, used for select 
> statement
> String index = "CREATE CUSTOM INDEX test_table_sec_idx ON 
> test.test_table_sec (id) USING 'org.apache.cassandra.index.sasi.SASIIndex' "
> + "WITH OPTIONS = { 'mode': 'PREFIX' }";
> s.execute(index);
>   
>   PreparedStatement insert = s.prepare("INSERT INTO 
> test.test_table_sec (id, sec) VALUES (1, ?)");
>   for (int i = 0; i < 1000; i++)
>   s.execute(insert.bind((long) i));
>   
>   PreparedStatement select = s.prepare("SELECT sec FROM 
> test.test_table_sec WHERE id = 1");
>   
>   long lastSec = -1;  
>   for (Row row : s.execute(select.bind().setFetchSize(300)))
>   {
>   long sec = row.getLong("sec");
>   if (sec == lastSec)
>   System.out.println(String.format("Duplicated id 
> %d", sec));
>   
>   lastSec = sec;
>   }
>   System.exit(0);
>   }
> }
> {code}
> The program outputs the following:
> {noformat}
> Duplicated id 23
> Duplicated id 192
> Duplicated id 684
> {noformat}
> Note that the simple primary key is required to reproduce this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13263) Incorrect ComplexColumnData hashCode implementation

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie reassigned CASSANDRA-13263:
---

Assignee: Stefan Podkowinski

> Incorrect ComplexColumnData hashCode implementation
> ---
>
> Key: CASSANDRA-13263
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13263
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>
> I went through some of the logs from CASSANDRA-13175 and one of the more 
> serious issues that we should address seem to be the 
> {{ComplexColumnData.hashCode()}} implementation. As Objects.hashCode is not 
> using deep hashing for array arguments, hashed will be based on the identity 
> instead of the array's content. See patch for simple fix.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13196) test failure in snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13196:

Status: Open  (was: Patch Available)

> test failure in 
> snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address
> -
>
> Key: CASSANDRA-13196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13196
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Aleksandr Sorokoumov
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1487/testReport/snitch_test/TestGossipingPropertyFileSnitch/test_prefer_local_reconnect_on_listen_address
> {code}
> {novnode}
> Error Message
> Error from server: code=2200 [Invalid query] message="keyspace keyspace1 does 
> not exist"
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-k6b0iF
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> cassandra.policies: INFO: Using datacenter 'dc1' for DCAwareRoundRobinPolicy 
> (via host '127.0.0.1'); if incorrect, please specify a local_dc to the 
> constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  discovered
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/snitch_test.py", line 87, in 
> test_prefer_local_reconnect_on_listen_address
> new_rows = list(session.execute("SELECT * FROM {}".format(stress_table)))
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 1998, in execute
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile, paging_state).result()
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 3784, in result
> raise self._final_exception
> 'Error from server: code=2200 [Invalid query] message="keyspace keyspace1 
> does not exist"\n >> begin captured logging << 
> \ndtest: DEBUG: cluster ccm directory: 
> /tmp/dtest-k6b0iF\ndtest: DEBUG: Done setting configuration options:\n{   
> \'initial_token\': None,\n\'num_tokens\': \'32\',\n
> \'phi_convict_threshold\': 5,\n\'range_request_timeout_in_ms\': 1,\n  
>   \'read_request_timeout_in_ms\': 1,\n\'request_timeout_in_ms\': 
> 1,\n\'truncate_request_timeout_in_ms\': 1,\n
> \'write_request_timeout_in_ms\': 1}\ncassandra.policies: INFO: Using 
> datacenter \'dc1\' for DCAwareRoundRobinPolicy (via host \'127.0.0.1\'); if 
> incorrect, please specify a local_dc to the constructor, or limit contact 
> points to local cluster nodes\ncassandra.cluster: INFO: New Cassandra host 
>  discovered\n- >> end captured 
> logging << -'
> {novnode}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13011) heap exhaustion when cleaning table with wide partitions and a secondary index attached to it

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13011:

Reviewer: Andrés de la Peña

> heap exhaustion when cleaning table with wide partitions and a secondary 
> index attached to it
> -
>
> Key: CASSANDRA-13011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13011
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Milan Majercik
> Fix For: 2.2.x
>
>
> We have a table with rather wide partitions and a secondary index attached to 
> it. When tried to clean unused data on a node after expansion of our cluster 
> via issuing {{nodetool cleanup}} command we observed a heap exhaustion issue. 
> The culprit appears to be in method 
> {{org.apache.cassandra.db.compaction.CompactionManager.CleanupStrategy.Full.cleanup}}
>  as it tries to remove related secondary index entries. The method first 
> populates a list will all cells belonging to the given partition...
> {code:java}
> while (row.hasNext())
> {
> OnDiskAtom column = row.next();
> if (column instanceof Cell && 
> cfs.indexManager.indexes((Cell) column))
> {
> if (indexedColumnsInRow == null)
> indexedColumnsInRow = new ArrayList<>();
> indexedColumnsInRow.add((Cell) column);
> }
> }
> {code}
> ... and then submits it to the index manager for removal.
> {code:java}
> // acquire memtable lock here because secondary index 
> deletion may cause a race. See CASSANDRA-3712
> try (OpOrder.Group opGroup = 
> cfs.keyspace.writeOrder.start())
> {
> cfs.indexManager.deleteFromIndexes(row.getKey(), 
> indexedColumnsInRow, opGroup);
> }
> {code}
> After imposing a limit on array size and implementing some sort of pagination 
> the cleanup worked fine.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13188) compaction-stress AssertionError from getMemtableFor()

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13188:

Reviewer: T Jake Luciani

> compaction-stress AssertionError from getMemtableFor()
> --
>
> Key: CASSANDRA-13188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13188
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing, Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Attachments: 13188-3.11.txt, 13188-3.11-update.txt
>
>
> Exception:
> {noformat}
> ./compaction-stress compact -d /tmp/compaction -p 
> https://gist.githubusercontent.com/tjake/8995058fed11d9921e31/raw/a9334d1090017bf546d003e271747351a40692ea/blogpost.yaml
>  -t 4
> WARN  18:45:04,854 JNA link failure, one or more native method will be 
> unavailable.
> java.lang.AssertionError: []
> at 
> org.apache.cassandra.db.lifecycle.Tracker.getMemtableFor(Tracker.java:312)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1315)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:462)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:232)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:570)
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:564)
> at 
> org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:356)
> at 
> org.apache.cassandra.schema.SchemaKeyspace.saveSystemKeyspacesSchema(SchemaKeyspace.java:265)
> at 
> org.apache.cassandra.db.SystemKeyspace.finishStartup(SystemKeyspace.java:495)
> at 
> org.apache.cassandra.stress.CompactionStress$Compaction.run(CompactionStress.java:209)
> at 
> org.apache.cassandra.stress.CompactionStress.main(CompactionStress.java:349)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-10130) Node failure during 2i update after streaming can have incomplete 2i when restarted

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie reassigned CASSANDRA-10130:
---

Assignee: Andrés de la Peña  (was: T Jake Luciani)

> Node failure during 2i update after streaming can have incomplete 2i when 
> restarted
> ---
>
> Key: CASSANDRA-10130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10130
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Yuki Morishita
>Assignee: Andrés de la Peña
>Priority: Minor
>
> Since MV/2i update happens after SSTables are received, node failure during 
> MV/2i update can leave received SSTables live when restarted while MV/2i are 
> partially up to date.
> We can add some kind of tracking mechanism to automatically rebuild at the 
> startup, or at least warn user when the node restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13147) Secondary index query on partition key columns might not return all the rows.

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959587#comment-15959587
 ] 

Andrés de la Peña commented on CASSANDRA-13147:
---

I see, thanks for the comments. I have just updated the 2.1 and 2.2 patches to 
use {{hasNoRegularColumnsRestriction()}}/{{!hasRegularColumnsRestriction()}} (I 
guess you meant this instead of {{hasClusteringColumnsRestriction()}}). 

Do you think we should add the aforementioned 
{{testIndexOnRegularColumnWithPartitionWithoutRows}}?

> Secondary index query on partition key columns might not return all the rows.
> -
>
> Key: CASSANDRA-13147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13147
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Andrés de la Peña
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> A secondary index query on a partition key column will, apparently, not 
> return the empty partitions with static data.
> The following unit test can be used to reproduce the problem.
> {code}
> public void testIndexOnPartitionKeyWithStaticColumnAndNoRows() throws 
> Throwable
> {
> createTable("CREATE TABLE %s (pk1 int, pk2 int, c int, s int static, 
> v int, PRIMARY KEY((pk1, pk2), c))");
> createIndex("CREATE INDEX ON %s (pk2)");
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)", 
> 1, 1, 1, 9, 1);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)", 
> 1, 1, 2, 9, 2);
> execute("INSERT INTO %s (pk1, pk2, s) VALUES (?, ?, ?)", 2, 1, 9);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)", 
> 3, 1, 1, 9, 1);
> assertRows(execute("SELECT * FROM %s WHERE pk2 = ?", 1),
>row(1, 1, 1, 9, 1),
>row(1, 1, 2, 9, 2),
>row(2, 1, null, 9, null), <-- is not returned
>row(3, 1, 1, 9, 1));
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959585#comment-15959585
 ] 

Michael Shuler commented on CASSANDRA-13413:


ship it! :)

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959561#comment-15959561
 ] 

Jeff Jirsa commented on CASSANDRA-13413:


lgtm



> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)