[3/6] cassandra git commit: Increment version to 3.0.13
Increment version to 3.0.13 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25f10655 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25f10655 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25f10655 Branch: refs/heads/trunk Commit: 25f106554a1a3cfd16437c2a9a08f627cd103f4b Parents: 849f8cd Author: Michael Shuler Authored: Wed Mar 29 21:59:11 2017 -0500 Committer: Michael Shuler Committed: Wed Mar 29 21:59:11 2017 -0500 -- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/25f10655/build.xml -- diff --git a/build.xml b/build.xml index cffaa17..e483265 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0b1675d4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0b1675d4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0b1675d4 Branch: refs/heads/trunk Commit: 0b1675d43d1cf141c5a2f891fd93cbf1bfcf1586 Parents: a0d5ad9 25f1065 Author: Michael Shuler Authored: Wed Mar 29 21:59:23 2017 -0500 Committer: Michael Shuler Committed: Wed Mar 29 21:59:23 2017 -0500 -- --
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0b1675d4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0b1675d4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0b1675d4 Branch: refs/heads/cassandra-3.11 Commit: 0b1675d43d1cf141c5a2f891fd93cbf1bfcf1586 Parents: a0d5ad9 25f1065 Author: Michael Shuler Authored: Wed Mar 29 21:59:23 2017 -0500 Committer: Michael Shuler Committed: Wed Mar 29 21:59:23 2017 -0500 -- --
[2/6] cassandra git commit: Increment version to 3.0.13
Increment version to 3.0.13 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25f10655 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25f10655 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25f10655 Branch: refs/heads/cassandra-3.11 Commit: 25f106554a1a3cfd16437c2a9a08f627cd103f4b Parents: 849f8cd Author: Michael Shuler Authored: Wed Mar 29 21:59:11 2017 -0500 Committer: Michael Shuler Committed: Wed Mar 29 21:59:11 2017 -0500 -- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/25f10655/build.xml -- diff --git a/build.xml b/build.xml index cffaa17..e483265 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b3465f93 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b3465f93 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b3465f93 Branch: refs/heads/trunk Commit: b3465f93735c3c4417f65f2590f469b2ad3a26c4 Parents: cd238a5 0b1675d Author: Michael Shuler Authored: Wed Mar 29 21:59:33 2017 -0500 Committer: Michael Shuler Committed: Wed Mar 29 21:59:33 2017 -0500 -- --
[1/6] cassandra git commit: Increment version to 3.0.13
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 849f8cd61 -> 25f106554 refs/heads/cassandra-3.11 a0d5ad98b -> 0b1675d43 refs/heads/trunk cd238a516 -> b3465f937 Increment version to 3.0.13 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25f10655 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25f10655 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25f10655 Branch: refs/heads/cassandra-3.0 Commit: 25f106554a1a3cfd16437c2a9a08f627cd103f4b Parents: 849f8cd Author: Michael Shuler Authored: Wed Mar 29 21:59:11 2017 -0500 Committer: Michael Shuler Committed: Wed Mar 29 21:59:11 2017 -0500 -- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/25f10655/build.xml -- diff --git a/build.xml b/build.xml index cffaa17..e483265 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>
[jira] [Commented] (CASSANDRA-13389) Possible NPE on upgrade to 3.0/3.X in case of IO errors
[
https://issues.apache.org/jira/browse/CASSANDRA-13389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948205#comment-15948205
]
Stefania commented on CASSANDRA-13389:
--
Thanks for the review. Committed to 3.0 as
849f8cd6162c4850d64581a2c4a542c677e43e0a and merged into 3.11, then merged into
trunk with {{-s ours}}.
> Possible NPE on upgrade to 3.0/3.X in case of IO errors
>
>
> Key: CASSANDRA-13389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13389
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.13, 3.11.0
>
>
> There is a NPE on upgrade to 3.0/3.X if a data directory contains directories
> that generate IO errors, for example if the cassandra process does not have
> permission to read them.
> Here is the exception:
> {code}
> ERROR [main] 2017-03-06 16:41:30,678 CassandraDaemon.java:710 - Exception
> encountered during startup
> java.lang.NullPointerException: null
> at org.apache.cassandra.io.util.FileUtils.delete(FileUtils.java:372)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.db.SystemKeyspace.migrateDataDirs(SystemKeyspace.java:1359)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:190)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> {code}
> This is caused by {{File.listFiles()}}, which returns null in case of an IO
> error.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13389) Possible NPE on upgrade to 3.0/3.X in case of IO errors
[
https://issues.apache.org/jira/browse/CASSANDRA-13389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania updated CASSANDRA-13389:
-
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 3.0.x)
3.11.0
3.0.13
Status: Resolved (was: Ready to Commit)
> Possible NPE on upgrade to 3.0/3.X in case of IO errors
>
>
> Key: CASSANDRA-13389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13389
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.13, 3.11.0
>
>
> There is a NPE on upgrade to 3.0/3.X if a data directory contains directories
> that generate IO errors, for example if the cassandra process does not have
> permission to read them.
> Here is the exception:
> {code}
> ERROR [main] 2017-03-06 16:41:30,678 CassandraDaemon.java:710 - Exception
> encountered during startup
> java.lang.NullPointerException: null
> at org.apache.cassandra.io.util.FileUtils.delete(FileUtils.java:372)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.db.SystemKeyspace.migrateDataDirs(SystemKeyspace.java:1359)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:190)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> {code}
> This is caused by {{File.listFiles()}}, which returns null in case of an IO
> error.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cd238a51 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cd238a51 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cd238a51 Branch: refs/heads/trunk Commit: cd238a5168c605caac78b0d41d24c40d016f58b7 Parents: b953f9e a0d5ad9 Author: Stefania Alborghetti Authored: Thu Mar 30 09:14:37 2017 +0800 Committer: Stefania Alborghetti Committed: Thu Mar 30 09:14:37 2017 +0800 -- --
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a0d5ad98 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a0d5ad98 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a0d5ad98 Branch: refs/heads/cassandra-3.11 Commit: a0d5ad98b2dc94cabd89035320b9ea744bf5cd21 Parents: 580963a 849f8cd Author: Stefania Alborghetti Authored: Thu Mar 30 09:09:16 2017 +0800 Committer: Stefania Alborghetti Committed: Thu Mar 30 09:09:16 2017 +0800 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/SystemKeyspace.java | 44 .../org/apache/cassandra/io/util/FileUtils.java | 26 + .../apache/cassandra/db/SystemKeyspaceTest.java | 106 ++- 4 files changed, 154 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/CHANGES.txt -- diff --cc CHANGES.txt index 81f4688,5d7b267..9cde2d8 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,25 -1,6 +1,26 @@@ -3.0.13 +3.11.0 + * cdc column addition strikes again (CASSANDRA-13382) + * Fix static column indexes (CASSANDRA-13277) + * DataOutputBuffer.asNewBuffer broken (CASSANDRA-13298) + * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370) + * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns (CASSANDRA-13247) + * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern (CASSANDRA-13317) + * Possible AssertionError in UnfilteredRowIteratorWithLowerBound (CASSANDRA-13366) + * Support unaligned memory access for AArch64 (CASSANDRA-13326) + * Improve SASI range iterator efficiency on intersection with an empty range (CASSANDRA-12915). + * Fix equality comparisons of columns using the duration type (CASSANDRA-13174) + * Obfuscate password in stress-graphs (CASSANDRA-12233) + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034) + * nodetool stopdaemon errors out (CASSANDRA-13030) + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954) + * Fix primary index calculation for SASI (CASSANDRA-12910) + * More fixes to the TokenAllocator (CASSANDRA-12990) + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983) + * Address message coalescing regression (CASSANDRA-12676) +Merged from 3.0: + * Fix possible NPE on upgrade to 3.0/3.X in case of IO errors (CASSANDRA-13389) * Legacy deserializer can create empty range tombstones (CASSANDRA-13341) + * Legacy caching options can prevent 3.0 upgrade (CASSANDRA-13384) * Use the Kernel32 library to retrieve the PID on Windows and fix startup checks (CASSANDRA-1) * Fix code to not exchange schema across major versions (CASSANDRA-13274) * Dropping column results in "corrupt" SSTable (CASSANDRA-13337) http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/src/java/org/apache/cassandra/db/SystemKeyspace.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/src/java/org/apache/cassandra/io/util/FileUtils.java -- diff --cc src/java/org/apache/cassandra/io/util/FileUtils.java index 662dd4e,0bfbbb1..c6e4e63 --- a/src/java/org/apache/cassandra/io/util/FileUtils.java +++ b/src/java/org/apache/cassandra/io/util/FileUtils.java @@@ -28,8 -27,10 +28,11 @@@ import java.text.DecimalFormat import java.util.Arrays; import java.util.Collections; import java.util.List; +import java.util.Optional; import java.util.concurrent.atomic.AtomicReference; + import java.util.function.Consumer; + import java.util.function.Predicate; + import java.util.stream.StreamSupport; import org.slf4j.Logger; import org.slf4j.LoggerFactory; http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java --
[2/6] cassandra git commit: Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13389
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/849f8cd6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/849f8cd6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/849f8cd6
Branch: refs/heads/cassandra-3.11
Commit: 849f8cd6162c4850d64581a2c4a542c677e43e0a
Parents: 451fe9d
Author: Stefania Alborghetti
Authored: Wed Mar 29 10:29:26 2017 +0800
Committer: Stefania Alborghetti
Committed: Thu Mar 30 09:05:34 2017 +0800
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/SystemKeyspace.java | 44
.../org/apache/cassandra/io/util/FileUtils.java | 26 +
.../apache/cassandra/db/SystemKeyspaceTest.java | 106 ++-
4 files changed, 154 insertions(+), 23 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/849f8cd6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c4293de..5d7b267 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.13
+ * Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
(CASSANDRA-13389)
* Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/849f8cd6/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index da96b38..cc21435 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -64,6 +64,7 @@ import static java.util.Collections.emptyMap;
import static java.util.Collections.singletonMap;
import static org.apache.cassandra.cql3.QueryProcessor.executeInternal;
import static org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal;
+import static org.apache.cassandra.io.util.FileUtils.visitDirectory;
public final class SystemKeyspace
{
@@ -1338,28 +1339,33 @@ public final class SystemKeyspace
Iterable dirs =
Arrays.asList(DatabaseDescriptor.getAllDataFileLocations());
for (String dataDir : dirs)
{
-logger.trace("Checking {} for old files", dataDir);
+logger.debug("Checking {} for legacy files", dataDir);
File dir = new File(dataDir);
assert dir.exists() : dir + " should have been created by startup
checks";
-for (File ksdir : dir.listFiles((d, n) -> new File(d,
n).isDirectory()))
-{
-logger.trace("Checking {} for old files", ksdir);
-
-for (File cfdir : ksdir.listFiles((d, n) -> new File(d,
n).isDirectory()))
-{
-logger.trace("Checking {} for old files", cfdir);
-
-if (Descriptor.isLegacyFile(cfdir))
-{
-FileUtils.deleteRecursive(cfdir);
-}
-else
-{
-FileUtils.delete(cfdir.listFiles((d, n) ->
Descriptor.isLegacyFile(new File(d, n;
-}
-}
-}
+visitDirectory(dir.toPath(),
+ File::isDirectory,
+ ksdir ->
+ {
+ logger.trace("Checking {} for legacy files",
ksdir);
+ visitDirectory(ksdir.toPath(),
+ File::isDirectory,
+ cfdir ->
+ {
+ logger.trace("Checking {}
for legacy files", cfdir);
+
+ if
(Descriptor.isLegacyFile(cfdir))
+ {
+
FileUtils.deleteRecursive(cfdir);
+ }
+ else
+ {
+
visitDirectory(cfdir.toPath(),
+
Descriptor::isLegacyFile,
+
FileUtils::de
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a0d5ad98 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a0d5ad98 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a0d5ad98 Branch: refs/heads/trunk Commit: a0d5ad98b2dc94cabd89035320b9ea744bf5cd21 Parents: 580963a 849f8cd Author: Stefania Alborghetti Authored: Thu Mar 30 09:09:16 2017 +0800 Committer: Stefania Alborghetti Committed: Thu Mar 30 09:09:16 2017 +0800 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/SystemKeyspace.java | 44 .../org/apache/cassandra/io/util/FileUtils.java | 26 + .../apache/cassandra/db/SystemKeyspaceTest.java | 106 ++- 4 files changed, 154 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/CHANGES.txt -- diff --cc CHANGES.txt index 81f4688,5d7b267..9cde2d8 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,25 -1,6 +1,26 @@@ -3.0.13 +3.11.0 + * cdc column addition strikes again (CASSANDRA-13382) + * Fix static column indexes (CASSANDRA-13277) + * DataOutputBuffer.asNewBuffer broken (CASSANDRA-13298) + * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370) + * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns (CASSANDRA-13247) + * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern (CASSANDRA-13317) + * Possible AssertionError in UnfilteredRowIteratorWithLowerBound (CASSANDRA-13366) + * Support unaligned memory access for AArch64 (CASSANDRA-13326) + * Improve SASI range iterator efficiency on intersection with an empty range (CASSANDRA-12915). + * Fix equality comparisons of columns using the duration type (CASSANDRA-13174) + * Obfuscate password in stress-graphs (CASSANDRA-12233) + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034) + * nodetool stopdaemon errors out (CASSANDRA-13030) + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954) + * Fix primary index calculation for SASI (CASSANDRA-12910) + * More fixes to the TokenAllocator (CASSANDRA-12990) + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983) + * Address message coalescing regression (CASSANDRA-12676) +Merged from 3.0: + * Fix possible NPE on upgrade to 3.0/3.X in case of IO errors (CASSANDRA-13389) * Legacy deserializer can create empty range tombstones (CASSANDRA-13341) + * Legacy caching options can prevent 3.0 upgrade (CASSANDRA-13384) * Use the Kernel32 library to retrieve the PID on Windows and fix startup checks (CASSANDRA-1) * Fix code to not exchange schema across major versions (CASSANDRA-13274) * Dropping column results in "corrupt" SSTable (CASSANDRA-13337) http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/src/java/org/apache/cassandra/db/SystemKeyspace.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/src/java/org/apache/cassandra/io/util/FileUtils.java -- diff --cc src/java/org/apache/cassandra/io/util/FileUtils.java index 662dd4e,0bfbbb1..c6e4e63 --- a/src/java/org/apache/cassandra/io/util/FileUtils.java +++ b/src/java/org/apache/cassandra/io/util/FileUtils.java @@@ -28,8 -27,10 +28,11 @@@ import java.text.DecimalFormat import java.util.Arrays; import java.util.Collections; import java.util.List; +import java.util.Optional; import java.util.concurrent.atomic.AtomicReference; + import java.util.function.Consumer; + import java.util.function.Predicate; + import java.util.stream.StreamSupport; import org.slf4j.Logger; import org.slf4j.LoggerFactory; http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0d5ad98/test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java --
[1/6] cassandra git commit: Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 451fe9d8a -> 849f8cd61
refs/heads/cassandra-3.11 580963a21 -> a0d5ad98b
refs/heads/trunk b953f9eb2 -> cd238a516
Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13389
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/849f8cd6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/849f8cd6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/849f8cd6
Branch: refs/heads/cassandra-3.0
Commit: 849f8cd6162c4850d64581a2c4a542c677e43e0a
Parents: 451fe9d
Author: Stefania Alborghetti
Authored: Wed Mar 29 10:29:26 2017 +0800
Committer: Stefania Alborghetti
Committed: Thu Mar 30 09:05:34 2017 +0800
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/SystemKeyspace.java | 44
.../org/apache/cassandra/io/util/FileUtils.java | 26 +
.../apache/cassandra/db/SystemKeyspaceTest.java | 106 ++-
4 files changed, 154 insertions(+), 23 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/849f8cd6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c4293de..5d7b267 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.13
+ * Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
(CASSANDRA-13389)
* Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/849f8cd6/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index da96b38..cc21435 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -64,6 +64,7 @@ import static java.util.Collections.emptyMap;
import static java.util.Collections.singletonMap;
import static org.apache.cassandra.cql3.QueryProcessor.executeInternal;
import static org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal;
+import static org.apache.cassandra.io.util.FileUtils.visitDirectory;
public final class SystemKeyspace
{
@@ -1338,28 +1339,33 @@ public final class SystemKeyspace
Iterable dirs =
Arrays.asList(DatabaseDescriptor.getAllDataFileLocations());
for (String dataDir : dirs)
{
-logger.trace("Checking {} for old files", dataDir);
+logger.debug("Checking {} for legacy files", dataDir);
File dir = new File(dataDir);
assert dir.exists() : dir + " should have been created by startup
checks";
-for (File ksdir : dir.listFiles((d, n) -> new File(d,
n).isDirectory()))
-{
-logger.trace("Checking {} for old files", ksdir);
-
-for (File cfdir : ksdir.listFiles((d, n) -> new File(d,
n).isDirectory()))
-{
-logger.trace("Checking {} for old files", cfdir);
-
-if (Descriptor.isLegacyFile(cfdir))
-{
-FileUtils.deleteRecursive(cfdir);
-}
-else
-{
-FileUtils.delete(cfdir.listFiles((d, n) ->
Descriptor.isLegacyFile(new File(d, n;
-}
-}
-}
+visitDirectory(dir.toPath(),
+ File::isDirectory,
+ ksdir ->
+ {
+ logger.trace("Checking {} for legacy files",
ksdir);
+ visitDirectory(ksdir.toPath(),
+ File::isDirectory,
+ cfdir ->
+ {
+ logger.trace("Checking {}
for legacy files", cfdir);
+
+ if
(Descriptor.isLegacyFile(cfdir))
+ {
+
FileUtils.deleteRecursive(cfdir);
+ }
+ else
+ {
+
visitDirectory(cfdir.toPath(
[3/6] cassandra git commit: Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
patch by Stefania Alborghetti; reviewed by Alex Petrov for CASSANDRA-13389
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/849f8cd6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/849f8cd6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/849f8cd6
Branch: refs/heads/trunk
Commit: 849f8cd6162c4850d64581a2c4a542c677e43e0a
Parents: 451fe9d
Author: Stefania Alborghetti
Authored: Wed Mar 29 10:29:26 2017 +0800
Committer: Stefania Alborghetti
Committed: Thu Mar 30 09:05:34 2017 +0800
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/SystemKeyspace.java | 44
.../org/apache/cassandra/io/util/FileUtils.java | 26 +
.../apache/cassandra/db/SystemKeyspaceTest.java | 106 ++-
4 files changed, 154 insertions(+), 23 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/849f8cd6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c4293de..5d7b267 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.13
+ * Fix possible NPE on upgrade to 3.0/3.X in case of IO errors
(CASSANDRA-13389)
* Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/849f8cd6/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index da96b38..cc21435 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -64,6 +64,7 @@ import static java.util.Collections.emptyMap;
import static java.util.Collections.singletonMap;
import static org.apache.cassandra.cql3.QueryProcessor.executeInternal;
import static org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal;
+import static org.apache.cassandra.io.util.FileUtils.visitDirectory;
public final class SystemKeyspace
{
@@ -1338,28 +1339,33 @@ public final class SystemKeyspace
Iterable dirs =
Arrays.asList(DatabaseDescriptor.getAllDataFileLocations());
for (String dataDir : dirs)
{
-logger.trace("Checking {} for old files", dataDir);
+logger.debug("Checking {} for legacy files", dataDir);
File dir = new File(dataDir);
assert dir.exists() : dir + " should have been created by startup
checks";
-for (File ksdir : dir.listFiles((d, n) -> new File(d,
n).isDirectory()))
-{
-logger.trace("Checking {} for old files", ksdir);
-
-for (File cfdir : ksdir.listFiles((d, n) -> new File(d,
n).isDirectory()))
-{
-logger.trace("Checking {} for old files", cfdir);
-
-if (Descriptor.isLegacyFile(cfdir))
-{
-FileUtils.deleteRecursive(cfdir);
-}
-else
-{
-FileUtils.delete(cfdir.listFiles((d, n) ->
Descriptor.isLegacyFile(new File(d, n;
-}
-}
-}
+visitDirectory(dir.toPath(),
+ File::isDirectory,
+ ksdir ->
+ {
+ logger.trace("Checking {} for legacy files",
ksdir);
+ visitDirectory(ksdir.toPath(),
+ File::isDirectory,
+ cfdir ->
+ {
+ logger.trace("Checking {}
for legacy files", cfdir);
+
+ if
(Descriptor.isLegacyFile(cfdir))
+ {
+
FileUtils.deleteRecursive(cfdir);
+ }
+ else
+ {
+
visitDirectory(cfdir.toPath(),
+
Descriptor::isLegacyFile,
+
FileUtils::delete);
+
[jira] [Commented] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948179#comment-15948179
]
Yuki Morishita commented on CASSANDRA-13390:
bq. Could not create snapshot at /10.22.150.204
Did you look at log in /10.22.150.204? There should be something there.
> Nodetool repair fails with snapshot error message
> -
>
> Key: CASSANDRA-13390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13390
> Project: Cassandra
> Issue Type: Bug
> Environment: * CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
> * Cassandra 3.10 (setup from vanilla tar.gz)
>Reporter: Barthelemy Vessemont
>
> I'm trying to run a repair with following options :
> {{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...}}
> `some_keyspace` definition :
> {quote}
> CREATE KEYSPACE some_keyspace WITH replication = <'class':
> 'NetworkTopologyStrategy', 'AMST': '3'> AND durable_writes = true;
> {quote}
> Tables can be both of DTCS and LCS
> Cluster :
> {quote}
> Datacenter: AMST
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
>Rack
> UN 10.22.150.236 205.22 GiB 256 ? B04
> UN 10.22.150.204 200.87 GiB 256 ? B05
> UN 10.22.140.77 203.12 GiB 256 ? D22
> {quote}
> Repair starts well but seems to fails at creating remote snapshots on nodes
> holding replicas:
> *info* : no other repair have been started on others nodes during this time,
> snapshots were also cleaned before.
> {quote}
> # nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...
> 11:42:03 [2017-03-29 09:42:03,399] Starting repair command #10
> (f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
> repair options (parallelism: sequential, primary range: true, incremental:
> false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
> some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
> pull repair: false)
> 11:47:43 [2017-03-29 09:47:43,578] Repair session
> f6e74940-1463-11e7-aa9f-01f10058e068 for range
> [(4978050141600810389,4982072488404840895],
> (-1340498249852980466,-1311689893502261125],
> (-5302264476874517513,-5291739032085711936],
> (-5981241605827420506,-5966850479287809973],
> (1857121221013279321,1899365815561615863],
> (942740326159033054,946603639333506869],
> (382316032181285431,397235785699549982],
> [...]
> (-6731897432582288959,-6728305970724193972],
> (-3193765198824884162,-3152653432337268817],
> (6879878057460360708,6898924573938960263],
> (7238964282930318864,7282255874655871690],
> (-4737709921934606628,-4734841018997895217]] failed with error Could not
> create snapshot at /10.22.150.204 (progress: 1%)
> 11:47:43 [2017-03-29 09:47:43,579] Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> 11:47:43 [2017-03-29 09:47:43,580] Repair command #10 finished in 5
> minutes 40 seconds
> {quote}
> errors logs on replica side :
> {quote}
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
> RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> at
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_91]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.10.jar:3.10]
> at java.lang.Thread.run(Thread.java:745) ~[na:1
[jira] [Commented] (CASSANDRA-7069) Prevent operator mistakes due to simultaneous bootstrap
[
https://issues.apache.org/jira/browse/CASSANDRA-7069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948129#comment-15948129
]
Anthony Grasso commented on CASSANDRA-7069:
---
{quote} Disccusing this offline with Jake, we decided it's still possible to
violate consistent range movement even following the 2 minute rule, so leaving
this as-is. If people don't care, they can simply disable consistent range
movement. {quote}
[~brandon.williams] and [~tjake] just wondering how you both worked out it was
possible to violate consistent range movement even after following the 2 minute
rule? Is it possible for tokens assigned to a first bootstrapping node to then
be reassigned to a second bootstrapping node? In which case does the second
bootstrapping node stream from the first. If so, can see how bootstrapping in
parallel is an issue.
> Prevent operator mistakes due to simultaneous bootstrap
> ---
>
> Key: CASSANDRA-7069
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7069
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Minor
> Fix For: 2.1.1, 2.2.0 beta 1
>
> Attachments: 7069.txt
>
>
> Cassandra has always had the '2 minute rule' between beginning topology
> changes to ensure the range announcement is known to all nodes before the
> next one begins. Trying to bootstrap a bunch of nodes simultaneously is a
> common mistake and seems to be on the rise as of late.
> We can prevent users from shooting themselves in the foot this way by looking
> for other joining nodes in the shadow round, then comparing their generation
> against our own and if there isn't a large enough difference, bail out or
> sleep until it is large enough.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-12213) dtest failure in write_failures_test.TestWriteFailures.test_paxos_any
[ https://issues.apache.org/jira/browse/CASSANDRA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-12213: Reviewer: Aleksey Yeschenko > dtest failure in write_failures_test.TestWriteFailures.test_paxos_any > - > > Key: CASSANDRA-12213 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12213 > Project: Cassandra > Issue Type: Bug > Components: Distributed Metadata >Reporter: Craig Kodman >Assignee: Stefania > Labels: dtest > Fix For: 3.0.x, 3.11.x > > Attachments: jenkins-stef1927-12014-dtest-2_logs.001.tar.gz, > node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, > node2.log, node3_debug.log, node3_gc.log, node3.log > > > example failure: > http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_paxos_any > and: > http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_mutation_v3/ > Failed on CassCI build cassandra-3.9_dtest #10 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13364) Cqlsh COPY fails importing Map>, ParseError unhashable type list
[
https://issues.apache.org/jira/browse/CASSANDRA-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13364:
Reviewer: Alex Petrov
> Cqlsh COPY fails importing Map>, ParseError unhashable
> type list
>
>
> Key: CASSANDRA-13364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13364
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Nicolae Namolovan
>Assignee: Stefania
> Labels: cqlsh
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
>
> When importing data with the _COPY_ command into a column family that has a
> _map>>_ field, I get a _unhashable type: 'list'_
> error. Here is how to reproduce:
> {code}
> CREATE TABLE table1 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table1 (col1, col2map) values (1, {'key': ['value1']});
> cqlsh:ks> copy table1 to 'table1.csv';
> table1.csv file content:
> 1,{'key': ['value1']}
> cqlsh:ks> copy table1 from 'table1.csv';
> ...
> Failed to import 1 rows: ParseError - Failed to parse {'key': ['value1']} :
> unhashable type: 'list', given up without retries
> Failed to process 1 rows; failed rows written to kv_table1.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.420 seconds (0 skipped).
> {code}
> But it works fine for Map>.
> {code}
> CREATE TABLE table2 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table2 (col1, col2map) values (1, {'key': {'value1'}});
> cqlsh:ks> copy table2 to 'table2.csv';
> table2.csv file content:
> 1,{'key': {'value1'}}
> cqlsh:ks> copy table2 from 'table2.csv';
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.417 seconds (0 skipped).
> {code}
> The exception seems to arrive in _convert_map_ function in _ImportConversion_
> class inside _copyutil.py_.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13388) Add hosted CI config files (such as Circle and TravisCI) for easier developer testing
[ https://issues.apache.org/jira/browse/CASSANDRA-13388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-13388: --- Status: Awaiting Feedback (was: Open) > Add hosted CI config files (such as Circle and TravisCI) for easier developer > testing > - > > Key: CASSANDRA-13388 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13388 > Project: Cassandra > Issue Type: Improvement > Components: Testing >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > Attachments: circle.yml, travis.yml > > > We currently require unit tests and dtests to accept a patch, but it's not > easy for most contributors to actually execute these tests in a way that's > visible for the reviewer / other JIRA users. > We should push some standard config files into the various branches to > facilitate easier testing. > I propose we start with TravisCI and CircleCI, because: > - Travis has limited free support for developers working on OSS projects, and > is already an accepted vendor at the ASF (apparently the ASF pays for 30 > concurrent tests already), and > - CircleCI is also free for developers working on OSS projects, and has > slightly more flexibility than Travis in terms of the number of free workers > and durations. > Both are enabled by pushing a single YAML file into each branch to configure > jobs, and require each individual developer who WANTS to run tests to link > their github account to the vendor. Developers who don't want to use this > functionality can simply ignore it - they're effectively no-ops for anyone > who's not already using those services. > Are there any others we should consider including? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13388) Add hosted CI config files (such as Circle and TravisCI) for easier developer testing
[ https://issues.apache.org/jira/browse/CASSANDRA-13388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-13388: --- Attachment: travis.yml circle.yml Proposing these two to start - they're minimal, unit-test only, but we can expand on them later (adding dtests, parallelizing, etc). > Add hosted CI config files (such as Circle and TravisCI) for easier developer > testing > - > > Key: CASSANDRA-13388 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13388 > Project: Cassandra > Issue Type: Improvement > Components: Testing >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > Attachments: circle.yml, travis.yml > > > We currently require unit tests and dtests to accept a patch, but it's not > easy for most contributors to actually execute these tests in a way that's > visible for the reviewer / other JIRA users. > We should push some standard config files into the various branches to > facilitate easier testing. > I propose we start with TravisCI and CircleCI, because: > - Travis has limited free support for developers working on OSS projects, and > is already an accepted vendor at the ASF (apparently the ASF pays for 30 > concurrent tests already), and > - CircleCI is also free for developers working on OSS projects, and has > slightly more flexibility than Travis in terms of the number of free workers > and durations. > Both are enabled by pushing a single YAML file into each branch to configure > jobs, and require each individual developer who WANTS to run tests to link > their github account to the vendor. Developers who don't want to use this > functionality can simply ignore it - they're effectively no-ops for anyone > who's not already using those services. > Are there any others we should consider including? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13391) nodetool clearsnapshot should require --all to clear all snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-13391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947896#comment-15947896 ] sankalp kohli commented on CASSANDRA-13391: --- +1 on the idea. > nodetool clearsnapshot should require --all to clear all snapshots > -- > > Key: CASSANDRA-13391 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13391 > Project: Cassandra > Issue Type: Bug >Reporter: Jon Haddad > > Deleting all snapshots by default is insanely dangerous. It would be like if > rm's default was -rf /. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13373) Provide additional speculative retry statistics
[ https://issues.apache.org/jira/browse/CASSANDRA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947876#comment-15947876 ] Ariel Weisberg commented on CASSANDRA-13373: OK ready for review again. I changed things up quite a bit renaming the metrics and adding more. I also added a unit test. I don't really like overriding methods in ReadCallback since it's not really intended to be an extension point so I went with catching and rethrowing the exception. > Provide additional speculative retry statistics > --- > > Key: CASSANDRA-13373 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13373 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 4.x > > > Right now there is a single metric for speculative retry on reads that is the > number of speculative retries attempted. You can't tell how many of those > actually succeeded in salvaging the read. > The metric is also per table and there is no keyspace level rollup as there > is for several other metrics. > Add a metric that counts reads that attempt to speculate but fail to complete > before the timeout (ignoring read errors). > Add a rollup metric for the current count of speculation attempts as well as > the count of failed speculations. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13373) Provide additional speculative retry statistics
[ https://issues.apache.org/jira/browse/CASSANDRA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939432#comment-15939432 ] Ariel Weisberg edited comment on CASSANDRA-13373 at 3/29/17 8:59 PM: - ||Code|utests|dtests|| |[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13373-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-testall/3/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-dtest/3/]| was (Author: aweisberg): ||Code|utests|dtests|| |[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13373-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-testall/2/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-dtest/2/]| > Provide additional speculative retry statistics > --- > > Key: CASSANDRA-13373 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13373 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 4.x > > > Right now there is a single metric for speculative retry on reads that is the > number of speculative retries attempted. You can't tell how many of those > actually succeeded in salvaging the read. > The metric is also per table and there is no keyspace level rollup as there > is for several other metrics. > Add a metric that counts reads that attempt to speculate but fail to complete > before the timeout (ignoring read errors). > Add a rollup metric for the current count of speculation attempts as well as > the count of failed speculations. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CASSANDRA-13391) nodetool clearsnapshot should require --all to clear all snapshots
Jon Haddad created CASSANDRA-13391: -- Summary: nodetool clearsnapshot should require --all to clear all snapshots Key: CASSANDRA-13391 URL: https://issues.apache.org/jira/browse/CASSANDRA-13391 Project: Cassandra Issue Type: Bug Reporter: Jon Haddad Deleting all snapshots by default is insanely dangerous. It would be like if rm's default was -rf /. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13373) Provide additional speculative retry statistics
[ https://issues.apache.org/jira/browse/CASSANDRA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939432#comment-15939432 ] Ariel Weisberg edited comment on CASSANDRA-13373 at 3/29/17 7:49 PM: - ||Code|utests|dtests|| |[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13373-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-testall/2/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-dtest/2/]| was (Author: aweisberg): ||Code|utests|dtests|| |[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13373-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-testall/1/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13373-trunk-dtest/1/]| > Provide additional speculative retry statistics > --- > > Key: CASSANDRA-13373 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13373 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 4.x > > > Right now there is a single metric for speculative retry on reads that is the > number of speculative retries attempted. You can't tell how many of those > actually succeeded in salvaging the read. > The metric is also per table and there is no keyspace level rollup as there > is for several other metrics. > Add a metric that counts reads that attempt to speculate but fail to complete > before the timeout (ignoring read errors). > Add a rollup metric for the current count of speculation attempts as well as > the count of failed speculations. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947725#comment-15947725 ] Paulo Motta commented on CASSANDRA-13327: - bq. To answer your question. What I believe happens is that while streaming is occurring the replacing node remains in the joining state. This is designed behavior per CASSANDRA-8523. "JOINING" is just a display name on nodetool so we must probably fix that to show REPLACING instead, but internally it means the node is trying to join the ring with the same tokens of the node it's trying to replace (only IFF the operation completes successfully, that's why it's on pending state). bq. If the down node is not coming back and you are replacing it why should there be unavailables? The unavailable only happened because there were 2 pending nodes in the requested range (the joining node AND the replacing node), and the current CAS design forbids more than 1 pending endpoint in the requested range (CASSANDRA-8346). bq. The question for me is whether the replacing node is really pending? What is the definition of pending and why should it include a replacing node? The replacing node is pending because we cannot count that node as an ordinary node towards the consistency level, otherwise if the replace operation fails the operations that used the replacement node as a member of the quorum will become inconsistent, that's why CASSANDRA-833 added pending/joining nodes as additional members of the cohort. > Pending endpoints size check for CAS doesn't play nicely with > writes-on-replacement > --- > > Key: CASSANDRA-13327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13327 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > Consider this ring: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.4 MR DOWN NORMAL -7148113328562451251 > where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to > the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and > making no progress. > Then the down node was replaced so we had: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.5 MR UP JOINING -7148113328562451251 > It’s confusing in the ring - the first JOINING is a genuine bootstrap, the > second is a replacement. We now had CAS unavailables (but no non-CAS > unvailables). I think it’s because the pending endpoints check thinks that > 127.0.0.5 is gaining a range when it’s just replacing. > The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t > unnecessarily fail these requests. > It also appears like required participants is bumped by 1 during a host > replacement so if the replacing host fails you will get unavailables and > timeouts. > This is related to the check added in CASSANDRA-8346 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13307) The specification of protocol version in cqlsh means the python driver doesn't automatically downgrade protocol version.
[ https://issues.apache.org/jira/browse/CASSANDRA-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947667#comment-15947667 ] Matt Byrd commented on CASSANDRA-13307: --- Hey [~tjake] are you at all keen to review? or shall I see if someone else can? Thanks > The specification of protocol version in cqlsh means the python driver > doesn't automatically downgrade protocol version. > > > Key: CASSANDRA-13307 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13307 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Matt Byrd >Assignee: Matt Byrd >Priority: Minor > Fix For: 3.11.x > > > Hi, > Looks like we've regressed on the issue described in: > https://issues.apache.org/jira/browse/CASSANDRA-9467 > In that we're no longer able to connect from newer cqlsh versions > (e.g trunk) to older versions of Cassandra with a lower version of the > protocol (e.g 2.1 with protocol version 3) > The problem seems to be that we're relying on the ability for the client to > automatically downgrade protocol version implemented in Cassandra here: > https://issues.apache.org/jira/browse/CASSANDRA-12838 > and utilised in the python client here: > https://datastax-oss.atlassian.net/browse/PYTHON-240 > The problem however comes when we implemented: > https://datastax-oss.atlassian.net/browse/PYTHON-537 > "Don't downgrade protocol version if explicitly set" > (included when we bumped from 3.5.0 to 3.7.0 of the python driver as part of > fixing: https://issues.apache.org/jira/browse/CASSANDRA-11534) > Since we do explicitly specify the protocol version in the bin/cqlsh.py. > I've got a patch which just adds an option to explicitly specify the protocol > version (for those who want to do that) and then otherwise defaults to not > setting the protocol version, i.e using the protocol version from the client > which we ship, which should by default be the same protocol as the server. > Then it should downgrade gracefully as was intended. > Let me know if that seems reasonable. > Thanks, > Matt -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947636#comment-15947636
]
Jason Brown edited comment on CASSANDRA-13289 at 3/29/17 6:19 PM:
--
{{StorageProxyMBean#setIdealConsistencyLevel}} already [returns a
{{String}}|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13289?expand=1#diff-097eb77c5d9d80a48e7547fbb81822caR70].
The implementation in {{StorageProxy}} simply performs {{return
original.toString();}} as the method return value.
was (Author: jasobrown):
{{StorageProxyMBean#setIdealConsistencyLevel}} already [returns a
{{String}}|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13289?expand=1#diff-097eb77c5d9d80a48e7547fbb81822caR70].
The implementation in {{StorageProxy}} simply performs {[return
original.toString();}} as the method return value.
> Make it possible to monitor an ideal consistency level separate from actual
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter
> replication and consistency you may want to have more information on from
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those
> writes failing to achieve EACH_QUORUM at other data centers. If you failed
> your application over to one of those data centers roughly how inconsistent
> might it be given the number of writes that didn't propagate since the last
> incremental repair?
> You might also want to know roughly what the latency of writes would be if
> you switched to a different consistency level. For instance you are writing
> at LOCAL_QUORUM and want to know what would happen if you switched to
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is
> specified no additional tracking is done.
> if an ideal consistency level is specified then the
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler
> that tracks whether the ideal consistency level is met before a write times
> out. It also tracks the latency for achieving the ideal CL of successful
> writes.
> These two metrics would be reported on a per keyspace basis.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947636#comment-15947636
]
Jason Brown commented on CASSANDRA-13289:
-
{{StorageProxyMBean#setIdealConsistencyLevel}} already [returns a
{{String}}|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13289?expand=1#diff-097eb77c5d9d80a48e7547fbb81822caR70].
The implementation in {{StorageProxy}} simply performs {[return
original.toString();}} as the method return value.
> Make it possible to monitor an ideal consistency level separate from actual
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter
> replication and consistency you may want to have more information on from
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those
> writes failing to achieve EACH_QUORUM at other data centers. If you failed
> your application over to one of those data centers roughly how inconsistent
> might it be given the number of writes that didn't propagate since the last
> incremental repair?
> You might also want to know roughly what the latency of writes would be if
> you switched to a different consistency level. For instance you are writing
> at LOCAL_QUORUM and want to know what would happen if you switched to
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is
> specified no additional tracking is done.
> if an ideal consistency level is specified then the
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler
> that tracks whether the ideal consistency level is met before a write times
> out. It also tracks the latency for achieving the ideal CL of successful
> writes.
> These two metrics would be reported on a per keyspace basis.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947607#comment-15947607 ] Ariel Weisberg edited comment on CASSANDRA-13327 at 3/29/17 5:59 PM: - Give me a bit to package up my dtest demonstrating this. To answer your question. What I believe happens is that while streaming is occurring the replacing node remains in the joining state. If the down node is not coming back and you are replacing it why should there be unavailables? As far as the cluster is concerned the operator has already announced the node is being replaced and it's not in an up/alive state otherwise the host replacement wouldn't start because it checks gossip to make sure the node it is replacing isn't up. The question for me is whether the replacing node is really pending? What is the definition of pending and why should it include a replacing node? was (Author: aweisberg): Give me a bit to package up my dtest demonstrating this. > Pending endpoints size check for CAS doesn't play nicely with > writes-on-replacement > --- > > Key: CASSANDRA-13327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13327 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > Consider this ring: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.4 MR DOWN NORMAL -7148113328562451251 > where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to > the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and > making no progress. > Then the down node was replaced so we had: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.5 MR UP JOINING -7148113328562451251 > It’s confusing in the ring - the first JOINING is a genuine bootstrap, the > second is a replacement. We now had CAS unavailables (but no non-CAS > unvailables). I think it’s because the pending endpoints check thinks that > 127.0.0.5 is gaining a range when it’s just replacing. > The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t > unnecessarily fail these requests. > It also appears like required participants is bumped by 1 during a host > replacement so if the replacing host fails you will get unavailables and > timeouts. > This is related to the check added in CASSANDRA-8346 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947607#comment-15947607 ] Ariel Weisberg commented on CASSANDRA-13327: Give me a bit to package up my dtest demonstrating this. > Pending endpoints size check for CAS doesn't play nicely with > writes-on-replacement > --- > > Key: CASSANDRA-13327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13327 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > Consider this ring: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.4 MR DOWN NORMAL -7148113328562451251 > where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to > the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and > making no progress. > Then the down node was replaced so we had: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.5 MR UP JOINING -7148113328562451251 > It’s confusing in the ring - the first JOINING is a genuine bootstrap, the > second is a replacement. We now had CAS unavailables (but no non-CAS > unvailables). I think it’s because the pending endpoints check thinks that > 127.0.0.5 is gaining a range when it’s just replacing. > The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t > unnecessarily fail these requests. > It also appears like required participants is bumped by 1 during a host > replacement so if the replacing host fails you will get unavailables and > timeouts. > This is related to the check added in CASSANDRA-8346 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947564#comment-15947564 ] Paulo Motta edited comment on CASSANDRA-13327 at 3/29/17 5:29 PM: -- I'm a bit confused here, did CAS unavailable happened while 127.0.0.4 was being replaced, or did 127.0.0.5 remained in JOINING/replacing state after replace was finished? If the former, than this is expected behavior I guess? Because you had 3 normal endpoints (where 1 was down) and 2 pending endpoints (the bootstrapping and the replacing node) for the requested key so the CAS should not allowed due to CASSANDRA-8346. If the latter than this is a bug with replace and must be fixed since the node must bump to NORMAL state after replacement is completed and the CAS should succeed. Can you reproduce this easily or have logs to understand why the replacement node did not go into NORMAL state after replacement was finished? was (Author: pauloricardomg): I'm a bit confused here, did CAS unavailable happened while 127.0.0.5 was being replaced, or did 127.0.0.5 remained in JOINING/replacing state after replace was finished? If the former, than this is expected behavior I guess? Because you had 3 normal endpoints (where 1 was down) and 2 pending endpoints (the bootstrapping and the replacing node) for the requested key so the CAS should not allowed due to CASSANDRA-8346. If the latter than this is a bug with replace and must be fixed since the node must bump to NORMAL state after replacement is completed and the CAS should succeed. Can you reproduce this easily or have logs to understand why the replacement node did not go into NORMAL state after replacement was finished? > Pending endpoints size check for CAS doesn't play nicely with > writes-on-replacement > --- > > Key: CASSANDRA-13327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13327 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > Consider this ring: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.4 MR DOWN NORMAL -7148113328562451251 > where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to > the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and > making no progress. > Then the down node was replaced so we had: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.5 MR UP JOINING -7148113328562451251 > It’s confusing in the ring - the first JOINING is a genuine bootstrap, the > second is a replacement. We now had CAS unavailables (but no non-CAS > unvailables). I think it’s because the pending endpoints check thinks that > 127.0.0.5 is gaining a range when it’s just replacing. > The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t > unnecessarily fail these requests. > It also appears like required participants is bumped by 1 during a host > replacement so if the replacing host fails you will get unavailables and > timeouts. > This is related to the check added in CASSANDRA-8346 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947564#comment-15947564 ] Paulo Motta commented on CASSANDRA-13327: - I'm a bit confused here, did CAS unavailable happened while 127.0.0.5 was being replaced, or did 127.0.0.5 remained in JOINING/replacing state after replace was finished? If the former, than this is expected behavior I guess? Because you had 3 normal endpoints (where 1 was down) and 2 pending endpoints (the bootstrapping and the replacing node) for the requested key so the CAS should not allowed due to CASSANDRA-8346. If the latter than this is a bug with replace and must be fixed since the node must bump to NORMAL state after replacement is completed and the CAS should succeed. Can you reproduce this easily or have logs to understand why the replacement node did not go into NORMAL state after replacement was finished? > Pending endpoints size check for CAS doesn't play nicely with > writes-on-replacement > --- > > Key: CASSANDRA-13327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13327 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > Consider this ring: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.4 MR DOWN NORMAL -7148113328562451251 > where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to > the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and > making no progress. > Then the down node was replaced so we had: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.5 MR UP JOINING -7148113328562451251 > It’s confusing in the ring - the first JOINING is a genuine bootstrap, the > second is a replacement. We now had CAS unavailables (but no non-CAS > unvailables). I think it’s because the pending endpoints check thinks that > 127.0.0.5 is gaining a range when it’s just replacing. > The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t > unnecessarily fail these requests. > It also appears like required participants is bumped by 1 during a host > replacement so if the replacing host fails you will get unavailables and > timeouts. > This is related to the check added in CASSANDRA-8346 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947370#comment-15947370
]
Ariel Weisberg commented on CASSANDRA-13289:
bq. StorageProxy#setIdealConsistencyLevel returns the previous value of the
ideal_consistency_level. Would that be confusing to operators who execute the
JMX command? maybe return "updating setIdealConsistencyLevel, previous value
was "?
How do I return "updating setIdealConsistencyLevel, previous value was
"? You mean change the return value to a string?
> Make it possible to monitor an ideal consistency level separate from actual
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter
> replication and consistency you may want to have more information on from
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those
> writes failing to achieve EACH_QUORUM at other data centers. If you failed
> your application over to one of those data centers roughly how inconsistent
> might it be given the number of writes that didn't propagate since the last
> incremental repair?
> You might also want to know roughly what the latency of writes would be if
> you switched to a different consistency level. For instance you are writing
> at LOCAL_QUORUM and want to know what would happen if you switched to
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is
> specified no additional tracking is done.
> if an ideal consistency level is specified then the
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler
> that tracks whether the ideal consistency level is met before a write times
> out. It also tracks the latency for achieving the ideal CL of successful
> writes.
> These two metrics would be reported on a per keyspace basis.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947306#comment-15947306 ] Sylvain Lebresne commented on CASSANDRA-13327: -- [~pauloricardomg], you worked on replacement, any comments? Is there any reason to not remove replacement from pending as soon as we replace them? > Pending endpoints size check for CAS doesn't play nicely with > writes-on-replacement > --- > > Key: CASSANDRA-13327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13327 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > Consider this ring: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.4 MR DOWN NORMAL -7148113328562451251 > where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to > the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and > making no progress. > Then the down node was replaced so we had: > 127.0.0.1 MR UP JOINING -7301836195843364181 > 127.0.0.2MR UP NORMAL -7263405479023135948 > 127.0.0.3MR UP NORMAL -7205759403792793599 > 127.0.0.5 MR UP JOINING -7148113328562451251 > It’s confusing in the ring - the first JOINING is a genuine bootstrap, the > second is a replacement. We now had CAS unavailables (but no non-CAS > unvailables). I think it’s because the pending endpoints check thinks that > 127.0.0.5 is gaining a range when it’s just replacing. > The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t > unnecessarily fail these requests. > It also appears like required participants is bumped by 1 during a host > replacement so if the replacing host fails you will get unavailables and > timeouts. > This is related to the check added in CASSANDRA-8346 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13386) Can't start Cassandra with powershell script when connections are already established to other Cassandra nodes on the same host
[
https://issues.apache.org/jira/browse/CASSANDRA-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947142#comment-15947142
]
Jacques-Henri Berthemet commented on CASSANDRA-13386:
-
done :)
> Can't start Cassandra with powershell script when connections are already
> established to other Cassandra nodes on the same host
> ---
>
> Key: CASSANDRA-13386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13386
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows with powershell
>Reporter: Jacques-Henri Berthemet
> Labels: easyfix, windows
> Attachments: cassandra.ps1.patch
>
>
> In our test env we run our client application on the same host as Cassandra
> nodes. When we restart the Cassandra node when the application is still
> running it fails in VerifyPortsAreAvailable function of bin\cassandra.ps1:98
> with the below error:
> {code}
> VerifyPortsAreAvailable : Found a port already in use. Aborting startup
> At C:\apache-cassandra-2.2.7\bin\cassandra.ps1:98 char:9
> + VerifyPortsAreAvailable
> + ~~~
> + CategoryInfo : NotSpecified: (:) [Write-Error],
> WriteErrorException
> + FullyQualifiedErrorId :
> Microsoft.PowerShell.Commands.WriteErrorException,VerifyPortsAreAvailable
> VerifyPortsAreAvailable : TCPxxx.xx.xx.171:61936xxx.xx.xx.24:9042
>ESTABLISHED
> {code}
> It looks like VerifyPortsAreAvailable is picking a remote 9042 port and
> refuses to start Cassandra.
> The VerifyPortsAreAvailable function should be fixed so that it only looks
> for LISTENING ports.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13386) Can't start Cassandra with powershell script when connections are already established to other Cassandra nodes on the same host
[
https://issues.apache.org/jira/browse/CASSANDRA-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques-Henri Berthemet updated CASSANDRA-13386:
Attachment: cassandra.ps1.patch
> Can't start Cassandra with powershell script when connections are already
> established to other Cassandra nodes on the same host
> ---
>
> Key: CASSANDRA-13386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13386
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows with powershell
>Reporter: Jacques-Henri Berthemet
> Labels: easyfix, windows
> Attachments: cassandra.ps1.patch
>
>
> In our test env we run our client application on the same host as Cassandra
> nodes. When we restart the Cassandra node when the application is still
> running it fails in VerifyPortsAreAvailable function of bin\cassandra.ps1:98
> with the below error:
> {code}
> VerifyPortsAreAvailable : Found a port already in use. Aborting startup
> At C:\apache-cassandra-2.2.7\bin\cassandra.ps1:98 char:9
> + VerifyPortsAreAvailable
> + ~~~
> + CategoryInfo : NotSpecified: (:) [Write-Error],
> WriteErrorException
> + FullyQualifiedErrorId :
> Microsoft.PowerShell.Commands.WriteErrorException,VerifyPortsAreAvailable
> VerifyPortsAreAvailable : TCPxxx.xx.xx.171:61936xxx.xx.xx.24:9042
>ESTABLISHED
> {code}
> It looks like VerifyPortsAreAvailable is picking a remote 9042 port and
> refuses to start Cassandra.
> The VerifyPortsAreAvailable function should be fixed so that it only looks
> for LISTENING ports.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13386) Can't start Cassandra with powershell script when connections are already established to other Cassandra nodes on the same host
[
https://issues.apache.org/jira/browse/CASSANDRA-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques-Henri Berthemet updated CASSANDRA-13386:
Attachment: cassandra.ps1.patch
> Can't start Cassandra with powershell script when connections are already
> established to other Cassandra nodes on the same host
> ---
>
> Key: CASSANDRA-13386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13386
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows with powershell
>Reporter: Jacques-Henri Berthemet
> Labels: easyfix, windows
>
> In our test env we run our client application on the same host as Cassandra
> nodes. When we restart the Cassandra node when the application is still
> running it fails in VerifyPortsAreAvailable function of bin\cassandra.ps1:98
> with the below error:
> {code}
> VerifyPortsAreAvailable : Found a port already in use. Aborting startup
> At C:\apache-cassandra-2.2.7\bin\cassandra.ps1:98 char:9
> + VerifyPortsAreAvailable
> + ~~~
> + CategoryInfo : NotSpecified: (:) [Write-Error],
> WriteErrorException
> + FullyQualifiedErrorId :
> Microsoft.PowerShell.Commands.WriteErrorException,VerifyPortsAreAvailable
> VerifyPortsAreAvailable : TCPxxx.xx.xx.171:61936xxx.xx.xx.24:9042
>ESTABLISHED
> {code}
> It looks like VerifyPortsAreAvailable is picking a remote 9042 port and
> refuses to start Cassandra.
> The VerifyPortsAreAvailable function should be fixed so that it only looks
> for LISTENING ports.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13386) Can't start Cassandra with powershell script when connections are already established to other Cassandra nodes on the same host
[
https://issues.apache.org/jira/browse/CASSANDRA-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques-Henri Berthemet updated CASSANDRA-13386:
Attachment: (was: cassandra.ps1.patch)
> Can't start Cassandra with powershell script when connections are already
> established to other Cassandra nodes on the same host
> ---
>
> Key: CASSANDRA-13386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13386
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows with powershell
>Reporter: Jacques-Henri Berthemet
> Labels: easyfix, windows
>
> In our test env we run our client application on the same host as Cassandra
> nodes. When we restart the Cassandra node when the application is still
> running it fails in VerifyPortsAreAvailable function of bin\cassandra.ps1:98
> with the below error:
> {code}
> VerifyPortsAreAvailable : Found a port already in use. Aborting startup
> At C:\apache-cassandra-2.2.7\bin\cassandra.ps1:98 char:9
> + VerifyPortsAreAvailable
> + ~~~
> + CategoryInfo : NotSpecified: (:) [Write-Error],
> WriteErrorException
> + FullyQualifiedErrorId :
> Microsoft.PowerShell.Commands.WriteErrorException,VerifyPortsAreAvailable
> VerifyPortsAreAvailable : TCPxxx.xx.xx.171:61936xxx.xx.xx.24:9042
>ESTABLISHED
> {code}
> It looks like VerifyPortsAreAvailable is picking a remote 9042 port and
> refuses to start Cassandra.
> The VerifyPortsAreAvailable function should be fixed so that it only looks
> for LISTENING ports.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13386) Can't start Cassandra with powershell script when connections are already established to other Cassandra nodes on the same host
[
https://issues.apache.org/jira/browse/CASSANDRA-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947075#comment-15947075
]
Joshua McKenzie commented on CASSANDRA-13386:
-
Feel free to attach a patch! :)
> Can't start Cassandra with powershell script when connections are already
> established to other Cassandra nodes on the same host
> ---
>
> Key: CASSANDRA-13386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13386
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows with powershell
>Reporter: Jacques-Henri Berthemet
> Labels: easyfix, windows
>
> In our test env we run our client application on the same host as Cassandra
> nodes. When we restart the Cassandra node when the application is still
> running it fails in VerifyPortsAreAvailable function of bin\cassandra.ps1:98
> with the below error:
> {code}
> VerifyPortsAreAvailable : Found a port already in use. Aborting startup
> At C:\apache-cassandra-2.2.7\bin\cassandra.ps1:98 char:9
> + VerifyPortsAreAvailable
> + ~~~
> + CategoryInfo : NotSpecified: (:) [Write-Error],
> WriteErrorException
> + FullyQualifiedErrorId :
> Microsoft.PowerShell.Commands.WriteErrorException,VerifyPortsAreAvailable
> VerifyPortsAreAvailable : TCPxxx.xx.xx.171:61936xxx.xx.xx.24:9042
>ESTABLISHED
> {code}
> It looks like VerifyPortsAreAvailable is picking a remote 9042 port and
> refuses to start Cassandra.
> The VerifyPortsAreAvailable function should be fixed so that it only looks
> for LISTENING ports.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947063#comment-15947063
]
Jason Brown commented on CASSANDRA-13289:
-
I'm pretty much +1 on the patch, except for the
{{AbstractWriteResponseHandler#responsesAndExpirations}} extra garbage and two
trivial nits, which you can feel free to completely ignore:
- maybe use the javacdoc comment style on {{Config#ideal_consistency_level}}
rather than the line comment style?
- {{StorageProxy#setIdealConsistencyLevel}} returns the previous value of the
{{ideal_consistency_level}}. Would that be confusing to operators who execute
the JMX command? maybe return "updating setIdealConsistencyLevel, previous
value was "?
The last thing is testing: i think it's possible to add unit tests to confirm
most of the behaviors here. Can you take a look into that?
> Make it possible to monitor an ideal consistency level separate from actual
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter
> replication and consistency you may want to have more information on from
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those
> writes failing to achieve EACH_QUORUM at other data centers. If you failed
> your application over to one of those data centers roughly how inconsistent
> might it be given the number of writes that didn't propagate since the last
> incremental repair?
> You might also want to know roughly what the latency of writes would be if
> you switched to a different consistency level. For instance you are writing
> at LOCAL_QUORUM and want to know what would happen if you switched to
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is
> specified no additional tracking is done.
> if an ideal consistency level is specified then the
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler
> that tracks whether the ideal consistency level is met before a write times
> out. It also tracks the latency for achieving the ideal CL of successful
> writes.
> These two metrics would be reported on a per keyspace basis.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Brown updated CASSANDRA-13289:
Reviewer: Jason Brown (was: Blake Eggleston)
> Make it possible to monitor an ideal consistency level separate from actual
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter
> replication and consistency you may want to have more information on from
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those
> writes failing to achieve EACH_QUORUM at other data centers. If you failed
> your application over to one of those data centers roughly how inconsistent
> might it be given the number of writes that didn't propagate since the last
> incremental repair?
> You might also want to know roughly what the latency of writes would be if
> you switched to a different consistency level. For instance you are writing
> at LOCAL_QUORUM and want to know what would happen if you switched to
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is
> specified no additional tracking is done.
> if an ideal consistency level is specified then the
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler
> that tracks whether the ideal consistency level is met before a write times
> out. It also tracks the latency for achieving the ideal CL of successful
> writes.
> These two metrics would be reported on a per keyspace basis.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13216) testall failure in org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
[
https://issues.apache.org/jira/browse/CASSANDRA-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-13216:
Status: Patch Available (was: Reopened)
> testall failure in
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
>
>
> Key: CASSANDRA-13216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13216
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
> Labels: test-failure, testall
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: TEST-org.apache.cassandra.net.MessagingServiceTest.log,
> TEST-org.apache.cassandra.net.MessagingServiceTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.11_testall/81/testReport/org.apache.cassandra.net/MessagingServiceTest/testDroppedMessages
> {code}
> Error Message
> expected:<... dropped latency: 27[30 ms and Mean cross-node dropped latency:
> 2731] ms> but was:<... dropped latency: 27[28 ms and Mean cross-node dropped
> latency: 2730] ms>
> {code}{code}
> Stacktrace
> junit.framework.AssertionFailedError: expected:<... dropped latency: 27[30 ms
> and Mean cross-node dropped latency: 2731] ms> but was:<... dropped latency:
> 27[28 ms and Mean cross-node dropped latency: 2730] ms>
> at
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages(MessagingServiceTest.java:83)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13216) testall failure in org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
[
https://issues.apache.org/jira/browse/CASSANDRA-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947038#comment-15947038
]
Alex Petrov edited comment on CASSANDRA-13216 at 3/29/17 12:42 PM:
---
Found the problem. I didn't anticipate initially that this test is
time-dependent. The initial fix is still applicable. It's reproducible quite
easily by adding a {{sleep}} of as few as 100 milliseconds around
[here|https://github.com/apache/cassandra/blob/732d1af866b91e5ba63e7e2a467d99d4cb90e11f/test/unit/org/apache/cassandra/net/MessagingServiceTest.java#L112].
YMMV with an exact sleep number.
However, I do not think there's any way we can reliably fetch latency numbers,
since dropwizard metrics reservoirs (used within
[timers|https://github.com/dropwizard/metrics/blob/15dde825de1843927898a7ad3c3bb11b2913a931/metrics-core/src/main/java/com/codahale/metrics/Timer.java#L64]
are tracking real time, and snapshots we're doing (however precise) won't ever
be perfect. I've mocked the clock:
||[3.11|https://github.com/ifesdjeen/cassandra/tree/13216-followup-3.11]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-followup-3.11-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-followup-3.11-dtest/]|
||[trunk|https://github.com/ifesdjeen/cassandra/tree/13216-followup-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-followup-trunk-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-followup-trunk-dtest/]|
3.0 branch is not susceptible to this problem, since we use time-independent
[Meter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/metrics/DroppedMessageMetrics.java#L31]
instead of timer there.
Let's wait for 24 hours, I've put the utest on retry.
was (Author: ifesdjeen):
Found the problem. I didn't anticipate initially that this test is
time-dependent. The initial fix is still applicable. It's reproducible quite
easily by adding a {{sleep}} of as few as 100 milliseconds around
[here|https://github.com/apache/cassandra/blob/732d1af866b91e5ba63e7e2a467d99d4cb90e11f/test/unit/org/apache/cassandra/net/MessagingServiceTest.java#L112].
YMMV with an exact sleep number.
However, I do not think there's any way we can reliably fetch latency numbers,
since dropwizard metrics reservoirs (used within
[timers|https://github.com/dropwizard/metrics/blob/15dde825de1843927898a7ad3c3bb11b2913a931/metrics-core/src/main/java/com/codahale/metrics/Timer.java#L64]
are tracking real time, and snapshots we're doing (however precise) won't ever
be perfect. I've mocked the clock:
|[3.11|https://github.com/ifesdjeen/cassandra/tree/13216-3.11-followup]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-3.11-followup-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-3.11-followup-dtest/]|
|[trunk|https://github.com/ifesdjeen/cassandra/tree/13216-followup-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-trunk-followup-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-trunk-followup-dtest/]|
3.0 branch is not susceptible to this problem, since we use time-independent
[Meter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/metrics/DroppedMessageMetrics.java#L31]
instead of timer there.
Let's wait for 24 hours, I've put the utest on retry.
> testall failure in
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
>
>
> Key: CASSANDRA-13216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13216
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
> Labels: test-failure, testall
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: TEST-org.apache.cassandra.net.MessagingServiceTest.log,
> TEST-org.apache.cassandra.net.MessagingServiceTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.11_testall/81/testReport/org.apache.cassandra.net/MessagingServiceTest/testDroppedMessages
> {code}
> Error Message
> expected:<... dropped latency: 27[30 ms and Mean cross-node dropped latency:
> 2731] ms> but was:<... dropped latency: 27[28 ms and Mean cross-node dropped
> latency: 2730] ms>
> {code}{code}
> Stacktrace
> junit.framework.AssertionFailedError: expected:<... dropped latency: 27[30 ms
> and Mean cross-node dropped latency: 2731] ms> but was:<... dropped latency:
> 27[28 ms and Mean cross-node dropped latency: 2730] ms>
> at
> org.apache.cassandra.net.Messa
[jira] [Commented] (CASSANDRA-13216) testall failure in org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
[
https://issues.apache.org/jira/browse/CASSANDRA-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947038#comment-15947038
]
Alex Petrov commented on CASSANDRA-13216:
-
Found the problem. I didn't anticipate initially that this test is
time-dependent. The initial fix is still applicable. It's reproducible quite
easily by adding a {{sleep}} of as few as 100 milliseconds around
[here|https://github.com/apache/cassandra/blob/732d1af866b91e5ba63e7e2a467d99d4cb90e11f/test/unit/org/apache/cassandra/net/MessagingServiceTest.java#L112].
YMMV with an exact sleep number.
However, I do not think there's any way we can reliably fetch latency numbers,
since dropwizard metrics reservoirs (used within
[timers|https://github.com/dropwizard/metrics/blob/15dde825de1843927898a7ad3c3bb11b2913a931/metrics-core/src/main/java/com/codahale/metrics/Timer.java#L64]
are tracking real time, and snapshots we're doing (however precise) won't ever
be perfect. I've mocked the clock:
|[3.11|https://github.com/ifesdjeen/cassandra/tree/13216-3.11-followup]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-3.11-followup-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-3.11-followup-dtest/]|
|[trunk|https://github.com/ifesdjeen/cassandra/tree/13216-followup-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-trunk-followup-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-trunk-followup-dtest/]|
3.0 branch is not susceptible to this problem, since we use time-independent
[Meter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/metrics/DroppedMessageMetrics.java#L31]
instead of timer there.
Let's wait for 24 hours, I've put the utest on retry.
> testall failure in
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
>
>
> Key: CASSANDRA-13216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13216
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
> Labels: test-failure, testall
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: TEST-org.apache.cassandra.net.MessagingServiceTest.log,
> TEST-org.apache.cassandra.net.MessagingServiceTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.11_testall/81/testReport/org.apache.cassandra.net/MessagingServiceTest/testDroppedMessages
> {code}
> Error Message
> expected:<... dropped latency: 27[30 ms and Mean cross-node dropped latency:
> 2731] ms> but was:<... dropped latency: 27[28 ms and Mean cross-node dropped
> latency: 2730] ms>
> {code}{code}
> Stacktrace
> junit.framework.AssertionFailedError: expected:<... dropped latency: 27[30 ms
> and Mean cross-node dropped latency: 2731] ms> but was:<... dropped latency:
> 27[28 ms and Mean cross-node dropped latency: 2730] ms>
> at
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages(MessagingServiceTest.java:83)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level
[
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946991#comment-15946991
]
Jason Brown commented on CASSANDRA-13289:
-
bq. There is a part of me that doesn't like having a null field
Maybe set the type of the field to {{Optional}}, and set it to
{{Optional.empty}} when not being used? Not sure I love this but it does avoid
an NPE.
bq. I don't want to throw an error at the request level. I can't validate the
configuration. And I don't want to silently not increment the counters.
I agree with these points, especially the last. I suspect not counting would be
confusing to an operator who went out of their way to enable this feature, and
are expecting an output of some kind. I don't think logging is necessary (or
helpful).
wrt timeouts, you are correct. I think I was forgetting that this patch is only
affecting the write path, and I must have been thinking the read path was
pulled along, as well. Either way, this concern is not a real issue.
I'll rereview the ticket now with these things in mind.
> Make it possible to monitor an ideal consistency level separate from actual
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter
> replication and consistency you may want to have more information on from
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those
> writes failing to achieve EACH_QUORUM at other data centers. If you failed
> your application over to one of those data centers roughly how inconsistent
> might it be given the number of writes that didn't propagate since the last
> incremental repair?
> You might also want to know roughly what the latency of writes would be if
> you switched to a different consistency level. For instance you are writing
> at LOCAL_QUORUM and want to know what would happen if you switched to
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is
> specified no additional tracking is done.
> if an ideal consistency level is specified then the
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler
> that tracks whether the ideal consistency level is met before a write times
> out. It also tracks the latency for achieving the ideal CL of successful
> writes.
> These two metrics would be reported on a per keyspace basis.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (CASSANDRA-13216) testall failure in org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
[
https://issues.apache.org/jira/browse/CASSANDRA-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov reassigned CASSANDRA-13216:
---
Assignee: Alex Petrov
> testall failure in
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
>
>
> Key: CASSANDRA-13216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13216
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
> Labels: test-failure, testall
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: TEST-org.apache.cassandra.net.MessagingServiceTest.log,
> TEST-org.apache.cassandra.net.MessagingServiceTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.11_testall/81/testReport/org.apache.cassandra.net/MessagingServiceTest/testDroppedMessages
> {code}
> Error Message
> expected:<... dropped latency: 27[30 ms and Mean cross-node dropped latency:
> 2731] ms> but was:<... dropped latency: 27[28 ms and Mean cross-node dropped
> latency: 2730] ms>
> {code}{code}
> Stacktrace
> junit.framework.AssertionFailedError: expected:<... dropped latency: 27[30 ms
> and Mean cross-node dropped latency: 2731] ms> but was:<... dropped latency:
> 27[28 ms and Mean cross-node dropped latency: 2730] ms>
> at
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages(MessagingServiceTest.java:83)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-10520) Compressed writer and reader should support non-compressed data.
[
https://issues.apache.org/jira/browse/CASSANDRA-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946982#comment-15946982
]
Branimir Lambov commented on CASSANDRA-10520:
-
Committed follow-up patch as
[b953f9eb2a3a1ba992fbb561b1e35531607739f3|https://github.com/apache/cassandra/commit/b953f9eb2a3a1ba992fbb561b1e35531607739f3].
> Compressed writer and reader should support non-compressed data.
>
>
> Key: CASSANDRA-10520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10520
> Project: Cassandra
> Issue Type: Improvement
> Components: Local Write-Read Paths
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
> Labels: messaging-service-bump-required
> Fix For: 4.x
>
> Attachments: ReadWriteTestCompression.java
>
>
> Compressing uncompressible data, as done, for instance, to write SSTables
> during stress-tests, results in chunks larger than 64k which are a problem
> for the buffer pooling mechanisms employed by the
> {{CompressedRandomAccessReader}}. This results in non-negligible performance
> issues due to excessive memory allocation.
> To solve this problem and avoid decompression delays in the cases where it
> does not provide benefits, I think we should allow compressed files to store
> uncompressed chunks as alternative to compressed data. Such a chunk could be
> written after compression returns a buffer larger than, for example, 90% of
> the input, and would not result in additional delays in writing. On reads it
> could be recognized by size (using a single global threshold constant in the
> compression metadata) and data could be directly transferred into the
> decompressed buffer, skipping the decompression step and ensuring a 64k
> buffer for compressed data always suffices.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
cassandra git commit: Change default min_compress_ratio to 0 to avoid sending parameter to old nodes
Repository: cassandra
Updated Branches:
refs/heads/trunk 07795f101 -> b953f9eb2
Change default min_compress_ratio to 0 to avoid sending parameter to old
nodes
Follow-up commit to CASSANDRA-10520
patch by Branimir Lambov; reviewed by Robert Stupp for CASSANDRA-10520
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b953f9eb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b953f9eb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b953f9eb
Branch: refs/heads/trunk
Commit: b953f9eb2a3a1ba992fbb561b1e35531607739f3
Parents: 07795f1
Author: Branimir Lambov
Authored: Fri Feb 24 11:02:08 2017 +0200
Committer: Branimir Lambov
Committed: Wed Mar 29 14:42:29 2017 +0300
--
pylib/cqlshlib/test/test_cqlsh_output.py | 2 +-
.../org/apache/cassandra/schema/CompressionParams.java| 7 +--
.../cassandra/cql3/validation/operations/AlterTest.java | 8
.../cassandra/cql3/validation/operations/CreateTest.java | 10 +-
4 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b953f9eb/pylib/cqlshlib/test/test_cqlsh_output.py
--
diff --git a/pylib/cqlshlib/test/test_cqlsh_output.py
b/pylib/cqlshlib/test/test_cqlsh_output.py
index f4aaa57..b7240f1 100644
--- a/pylib/cqlshlib/test/test_cqlsh_output.py
+++ b/pylib/cqlshlib/test/test_cqlsh_output.py
@@ -615,7 +615,7 @@ class TestCqlshOutput(BaseTestCase):
AND cdc = false
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
-AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor', 'min_compress_ratio': '1.1'}
+AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b953f9eb/src/java/org/apache/cassandra/schema/CompressionParams.java
--
diff --git a/src/java/org/apache/cassandra/schema/CompressionParams.java
b/src/java/org/apache/cassandra/schema/CompressionParams.java
index a319311..c4b52df 100644
--- a/src/java/org/apache/cassandra/schema/CompressionParams.java
+++ b/src/java/org/apache/cassandra/schema/CompressionParams.java
@@ -54,7 +54,9 @@ public final class CompressionParams
private static volatile boolean hasLoggedCrcCheckChanceWarning;
public static final int DEFAULT_CHUNK_LENGTH = 65536;
-public static final double DEFAULT_MIN_COMPRESS_RATIO = 1.1;
+public static final double DEFAULT_MIN_COMPRESS_RATIO = 0.0;//
Since pre-4.0 versions do not understand the
+// new
compression parameter we can't use a
+//
different default value.
public static final IVersionedSerializer serializer =
new Serializer();
public static final String CLASS = "class";
@@ -499,7 +501,8 @@ public final class CompressionParams
Map options = new HashMap<>(otherOptions);
options.put(CLASS, sstableCompressor.getClass().getName());
options.put(CHUNK_LENGTH_IN_KB, chunkLengthInKB());
-options.put(MIN_COMPRESS_RATIO, String.valueOf(minCompressRatio));
+if (minCompressRatio != DEFAULT_MIN_COMPRESS_RATIO)
+options.put(MIN_COMPRESS_RATIO, String.valueOf(minCompressRatio));
return options;
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b953f9eb/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index 3e6f0db..dce72f9 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -328,7 +328,7 @@ public class AlterTest extends CQLTester
SchemaKeyspace.TABLES),
KEYSPACE,
currentTable()),
- row(map("chunk_length_in_kb", "64", "class",
"org.apache.cassandra.io.compress.LZ4Compressor", "min_compress_ratio",
"1.1")));
+ row
[jira] [Commented] (CASSANDRA-13216) testall failure in org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
[
https://issues.apache.org/jira/browse/CASSANDRA-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946966#comment-15946966
]
Alex Petrov commented on CASSANDRA-13216:
-
This is extremely weird. I've ran the test 100 times
[here|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-13216-trunk-testall/lastCompletedBuild/consoleFull]
and it passed every time. Checked for the interference (by ensuring that each
test class is running in it's own forked JVM), and confirmed it (by outputting
and comparing PIDs of the processes in the log files). I've checked for
possible races / reordering and could not find any problem so far. Looks like
we can only reproduce this issue when running a full test suite, not an
individual test.
> testall failure in
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages
>
>
> Key: CASSANDRA-13216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13216
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Sean McCarthy
> Labels: test-failure, testall
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: TEST-org.apache.cassandra.net.MessagingServiceTest.log,
> TEST-org.apache.cassandra.net.MessagingServiceTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.11_testall/81/testReport/org.apache.cassandra.net/MessagingServiceTest/testDroppedMessages
> {code}
> Error Message
> expected:<... dropped latency: 27[30 ms and Mean cross-node dropped latency:
> 2731] ms> but was:<... dropped latency: 27[28 ms and Mean cross-node dropped
> latency: 2730] ms>
> {code}{code}
> Stacktrace
> junit.framework.AssertionFailedError: expected:<... dropped latency: 27[30 ms
> and Mean cross-node dropped latency: 2731] ms> but was:<... dropped latency:
> 27[28 ms and Mean cross-node dropped latency: 2730] ms>
> at
> org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages(MessagingServiceTest.java:83)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13389) Possible NPE on upgrade to 3.0/3.X in case of IO errors
[
https://issues.apache.org/jira/browse/CASSANDRA-13389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-13389:
Status: Ready to Commit (was: Patch Available)
> Possible NPE on upgrade to 3.0/3.X in case of IO errors
>
>
> Key: CASSANDRA-13389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13389
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x, 3.11.x
>
>
> There is a NPE on upgrade to 3.0/3.X if a data directory contains directories
> that generate IO errors, for example if the cassandra process does not have
> permission to read them.
> Here is the exception:
> {code}
> ERROR [main] 2017-03-06 16:41:30,678 CassandraDaemon.java:710 - Exception
> encountered during startup
> java.lang.NullPointerException: null
> at org.apache.cassandra.io.util.FileUtils.delete(FileUtils.java:372)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.db.SystemKeyspace.migrateDataDirs(SystemKeyspace.java:1359)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:190)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> {code}
> This is caused by {{File.listFiles()}}, which returns null in case of an IO
> error.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13389) Possible NPE on upgrade to 3.0/3.X in case of IO errors
[
https://issues.apache.org/jira/browse/CASSANDRA-13389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946960#comment-15946960
]
Alex Petrov commented on CASSANDRA-13389:
-
LGTM, +1
Thank you for the patch!
For history purposes, I'll note that this patch isn't applicable to trunk as
it's related to legacy table migration.
> Possible NPE on upgrade to 3.0/3.X in case of IO errors
>
>
> Key: CASSANDRA-13389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13389
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x, 3.11.x
>
>
> There is a NPE on upgrade to 3.0/3.X if a data directory contains directories
> that generate IO errors, for example if the cassandra process does not have
> permission to read them.
> Here is the exception:
> {code}
> ERROR [main] 2017-03-06 16:41:30,678 CassandraDaemon.java:710 - Exception
> encountered during startup
> java.lang.NullPointerException: null
> at org.apache.cassandra.io.util.FileUtils.delete(FileUtils.java:372)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.db.SystemKeyspace.migrateDataDirs(SystemKeyspace.java:1359)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:190)
> ~[cassandra-all-3.0.11.1564.jar:3.0.11.1564]
> {code}
> This is caused by {{File.listFiles()}}, which returns null in case of an IO
> error.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13341) Legacy deserializer can create empty range tombstones
[
https://issues.apache.org/jira/browse/CASSANDRA-13341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-13341:
-
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 3.0.x)
3.11.0
3.0.13
Status: Resolved (was: Ready to Commit)
Tests are good, committed, thanks.
> Legacy deserializer can create empty range tombstones
> -
>
> Key: CASSANDRA-13341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13341
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.13, 3.11.0
>
>
> Range tombstones in the 2.x file format is a bit far-westy so you can
> actually get sequences of range tombstones like {{\[1, 4\]@3 \[1, 10\]@5}}.
> But the current legacy deserializer doesn't handle this correctly. On the
> first range, it will generate a {{INCL_START(1)@3}} open marker, but upon
> seeing the next tombstone it will decide to close the previously opened range
> and re-open with deletion time 5, so will generate
> {{EXCL_END_INCL_START(1)@3-5}}. That result in the first range being empty,
> which break future assertions in the code.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk
* cassandra-3.11:
Legacy deserializer can create empty range tombstones
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/07795f10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/07795f10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/07795f10
Branch: refs/heads/trunk
Commit: 07795f1016970ac519503293883c1f7013f3f48a
Parents: ea708c4 580963a
Author: Sylvain Lebresne
Authored: Wed Mar 29 13:24:39 2017 +0200
Committer: Sylvain Lebresne
Committed: Wed Mar 29 13:24:39 2017 +0200
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/UnfilteredDeserializer.java | 5 -
.../apache/cassandra/db/rows/RangeTombstoneBoundMarker.java | 8
.../cassandra/db/rows/RangeTombstoneBoundaryMarker.java | 5 +
.../org/apache/cassandra/db/rows/RangeTombstoneMarker.java | 2 ++
5 files changed, 16 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/07795f10/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/07795f10/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --cc src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index d361bac,05e8148..84ff691
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@@ -18,13 -18,19 +18,10 @@@
package org.apache.cassandra.db;
import java.io.IOException;
-import java.io.IOError;
-import java.util.*;
-import java.util.function.Supplier;
- import org.slf4j.Logger;
- import org.slf4j.LoggerFactory;
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.collect.Iterables;
-import com.google.common.collect.PeekingIterator;
--
-import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.db.rows.*;
import org.apache.cassandra.io.util.DataInputPlus;
-import org.apache.cassandra.io.util.FileDataInput;
-import org.apache.cassandra.net.MessagingService;
/**
* Helper class to deserialize Unfiltered object from disk efficiently.
@@@ -33,11 -39,9 +30,9 @@@
* we don't do more work than necessary (i.e. we don't allocate/deserialize
* objects for things we don't care about).
*/
-public abstract class UnfilteredDeserializer
+public class UnfilteredDeserializer
{
- private static final Logger logger =
LoggerFactory.getLogger(UnfilteredDeserializer.class);
-
-protected final CFMetaData metadata;
+protected final TableMetadata metadata;
protected final DataInputPlus in;
protected final SerializationHelper helper;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/07795f10/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundMarker.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/07795f10/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
--
[3/6] cassandra git commit: Legacy deserializer can create empty range tombstones
Legacy deserializer can create empty range tombstones
patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13341
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/451fe9d8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/451fe9d8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/451fe9d8
Branch: refs/heads/trunk
Commit: 451fe9d8ac567942f62852f542d28d7d1116f1a1
Parents: 6edc268
Author: Sylvain Lebresne
Authored: Thu Mar 16 17:25:39 2017 +0100
Committer: Sylvain Lebresne
Committed: Wed Mar 29 13:17:58 2017 +0200
--
CHANGES.txt | 1 +
.../cassandra/db/UnfilteredDeserializer.java| 124 ++-
.../db/rows/RangeTombstoneBoundMarker.java | 8 ++
.../db/rows/RangeTombstoneBoundaryMarker.java | 5 +
.../cassandra/db/rows/RangeTombstoneMarker.java | 2 +
.../cassandra/db/OldFormatDeserializerTest.java | 54
6 files changed, 162 insertions(+), 32 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/451fe9d8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b46eb50..c4293de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.13
+ * Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
* Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/451fe9d8/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 7bbbfdb..92690e1 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -26,8 +26,6 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Iterables;
import com.google.common.collect.PeekingIterator;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.rows.*;
import org.apache.cassandra.io.util.DataInputPlus;
@@ -43,8 +41,6 @@ import org.apache.cassandra.net.MessagingService;
*/
public abstract class UnfilteredDeserializer
{
-private static final Logger logger =
LoggerFactory.getLogger(UnfilteredDeserializer.class);
-
protected final CFMetaData metadata;
protected final DataInputPlus in;
protected final SerializationHelper helper;
@@ -433,21 +429,31 @@ public abstract class UnfilteredDeserializer
{
if (atoms.hasNext())
{
+// If there is a range tombstone to open strictly
before the next row/RT, we need to return that open (or boundary) marker first.
+if
(tombstoneTracker.hasOpeningMarkerBefore(atoms.peek()))
+{
+next = tombstoneTracker.popOpeningMarker();
+}
// If a range tombstone closes strictly before the
next row/RT, we need to return that close (or boundary) marker first.
-if
(tombstoneTracker.hasClosingMarkerBefore(atoms.peek()))
+else if
(tombstoneTracker.hasClosingMarkerBefore(atoms.peek()))
{
next = tombstoneTracker.popClosingMarker();
}
else
{
LegacyLayout.LegacyAtom atom = atoms.next();
-if (!tombstoneTracker.isShadowed(atom))
-next = isRow(atom) ? readRow(atom) :
tombstoneTracker.openNew(atom.asRangeTombstone());
+if (tombstoneTracker.isShadowed(atom))
+continue;
+
+if (isRow(atom))
+next = readRow(atom);
+else
+
tombstoneTracker.openNew(atom.asRangeTombstone());
}
}
else if (tombstoneTracker.hasOpenTombstones())
{
-next = tombstoneTracker.popClosingMarker();
+next = tombstoneTracker.popMarker();
}
else
{
@@ -562,11 +5
[2/6] cassandra git commit: Legacy deserializer can create empty range tombstones
Legacy deserializer can create empty range tombstones
patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13341
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/451fe9d8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/451fe9d8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/451fe9d8
Branch: refs/heads/cassandra-3.11
Commit: 451fe9d8ac567942f62852f542d28d7d1116f1a1
Parents: 6edc268
Author: Sylvain Lebresne
Authored: Thu Mar 16 17:25:39 2017 +0100
Committer: Sylvain Lebresne
Committed: Wed Mar 29 13:17:58 2017 +0200
--
CHANGES.txt | 1 +
.../cassandra/db/UnfilteredDeserializer.java| 124 ++-
.../db/rows/RangeTombstoneBoundMarker.java | 8 ++
.../db/rows/RangeTombstoneBoundaryMarker.java | 5 +
.../cassandra/db/rows/RangeTombstoneMarker.java | 2 +
.../cassandra/db/OldFormatDeserializerTest.java | 54
6 files changed, 162 insertions(+), 32 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/451fe9d8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b46eb50..c4293de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.13
+ * Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
* Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/451fe9d8/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 7bbbfdb..92690e1 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -26,8 +26,6 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Iterables;
import com.google.common.collect.PeekingIterator;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.rows.*;
import org.apache.cassandra.io.util.DataInputPlus;
@@ -43,8 +41,6 @@ import org.apache.cassandra.net.MessagingService;
*/
public abstract class UnfilteredDeserializer
{
-private static final Logger logger =
LoggerFactory.getLogger(UnfilteredDeserializer.class);
-
protected final CFMetaData metadata;
protected final DataInputPlus in;
protected final SerializationHelper helper;
@@ -433,21 +429,31 @@ public abstract class UnfilteredDeserializer
{
if (atoms.hasNext())
{
+// If there is a range tombstone to open strictly
before the next row/RT, we need to return that open (or boundary) marker first.
+if
(tombstoneTracker.hasOpeningMarkerBefore(atoms.peek()))
+{
+next = tombstoneTracker.popOpeningMarker();
+}
// If a range tombstone closes strictly before the
next row/RT, we need to return that close (or boundary) marker first.
-if
(tombstoneTracker.hasClosingMarkerBefore(atoms.peek()))
+else if
(tombstoneTracker.hasClosingMarkerBefore(atoms.peek()))
{
next = tombstoneTracker.popClosingMarker();
}
else
{
LegacyLayout.LegacyAtom atom = atoms.next();
-if (!tombstoneTracker.isShadowed(atom))
-next = isRow(atom) ? readRow(atom) :
tombstoneTracker.openNew(atom.asRangeTombstone());
+if (tombstoneTracker.isShadowed(atom))
+continue;
+
+if (isRow(atom))
+next = readRow(atom);
+else
+
tombstoneTracker.openNew(atom.asRangeTombstone());
}
}
else if (tombstoneTracker.hasOpenTombstones())
{
-next = tombstoneTracker.popClosingMarker();
+next = tombstoneTracker.popMarker();
}
else
{
@@ -
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
* cassandra-3.0:
Legacy deserializer can create empty range tombstones
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/580963a2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/580963a2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/580963a2
Branch: refs/heads/trunk
Commit: 580963a2184b8f70f319ba59353fd258de239e36
Parents: bd77081 451fe9d
Author: Sylvain Lebresne
Authored: Wed Mar 29 13:22:29 2017 +0200
Committer: Sylvain Lebresne
Committed: Wed Mar 29 13:22:29 2017 +0200
--
CHANGES.txt | 1 +
.../cassandra/db/UnfilteredDeserializer.java| 119 ++-
.../db/rows/RangeTombstoneBoundMarker.java | 8 ++
.../db/rows/RangeTombstoneBoundaryMarker.java | 5 +
.../cassandra/db/rows/RangeTombstoneMarker.java | 2 +
.../cassandra/db/OldFormatDeserializerTest.java | 54 +
6 files changed, 162 insertions(+), 27 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/CHANGES.txt
--
diff --cc CHANGES.txt
index 9cd1ca7,c4293de..81f4688
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,24 -1,5 +1,25 @@@
-3.0.13
+3.11.0
+ * cdc column addition strikes again (CASSANDRA-13382)
+ * Fix static column indexes (CASSANDRA-13277)
+ * DataOutputBuffer.asNewBuffer broken (CASSANDRA-13298)
+ * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370)
+ * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns
(CASSANDRA-13247)
+ * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern
(CASSANDRA-13317)
+ * Possible AssertionError in UnfilteredRowIteratorWithLowerBound
(CASSANDRA-13366)
+ * Support unaligned memory access for AArch64 (CASSANDRA-13326)
+ * Improve SASI range iterator efficiency on intersection with an empty range
(CASSANDRA-12915).
+ * Fix equality comparisons of columns using the duration type
(CASSANDRA-13174)
+ * Obfuscate password in stress-graphs (CASSANDRA-12233)
+ * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
+ * nodetool stopdaemon errors out (CASSANDRA-13030)
+ * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
+ * Fix primary index calculation for SASI (CASSANDRA-12910)
+ * More fixes to the TokenAllocator (CASSANDRA-12990)
+ * NoReplicationTokenAllocator should work with zero replication factor
(CASSANDRA-12983)
+ * Address message coalescing regression (CASSANDRA-12676)
+Merged from 3.0:
+ * Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
+ * Legacy caching options can prevent 3.0 upgrade (CASSANDRA-13384)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
* Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundMarker.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
--
diff --cc
src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
index fd41bea,0683d76..70d6a9d
--- a/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
+++ b/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
@@@ -120,9 -120,14 +120,14 @@@ public class RangeTombstoneBoundaryMark
return new RangeTombstoneBoundaryMarker(clustering().copy(allocator),
endDeletion, startDeletion);
}
+ public RangeTombstoneBoundaryMarker withNewOpeningDeletionTime(boolean
reversed, DeletionTime newDeletionTime)
+ {
+ return new RangeTombstoneBoundaryMarker(clustering(), reversed ?
newDeletionTime : endDeletion, reversed ? startDeletion : newDeletionTime);
+ }
+
-public static RangeTombstoneBoundaryMarker makeBoundary(boolean reversed,
Slice.Bound close, Slice.Bound open, DeletionTime closeDeletion, DeletionTime
openDeletion)
+public static RangeTombstoneBoundaryMarker makeBoundary(boolean reversed,
ClusteringBound close, ClusteringBound open, DeletionTime closeDeletion,
DeletionTime openDeletion)
{
-assert RangeTombstone.Bound.Kind.compare(close.kind(), open.kind())
== 0
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
* cassandra-3.0:
Legacy deserializer can create empty range tombstones
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/580963a2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/580963a2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/580963a2
Branch: refs/heads/cassandra-3.11
Commit: 580963a2184b8f70f319ba59353fd258de239e36
Parents: bd77081 451fe9d
Author: Sylvain Lebresne
Authored: Wed Mar 29 13:22:29 2017 +0200
Committer: Sylvain Lebresne
Committed: Wed Mar 29 13:22:29 2017 +0200
--
CHANGES.txt | 1 +
.../cassandra/db/UnfilteredDeserializer.java| 119 ++-
.../db/rows/RangeTombstoneBoundMarker.java | 8 ++
.../db/rows/RangeTombstoneBoundaryMarker.java | 5 +
.../cassandra/db/rows/RangeTombstoneMarker.java | 2 +
.../cassandra/db/OldFormatDeserializerTest.java | 54 +
6 files changed, 162 insertions(+), 27 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/CHANGES.txt
--
diff --cc CHANGES.txt
index 9cd1ca7,c4293de..81f4688
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,24 -1,5 +1,25 @@@
-3.0.13
+3.11.0
+ * cdc column addition strikes again (CASSANDRA-13382)
+ * Fix static column indexes (CASSANDRA-13277)
+ * DataOutputBuffer.asNewBuffer broken (CASSANDRA-13298)
+ * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370)
+ * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns
(CASSANDRA-13247)
+ * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern
(CASSANDRA-13317)
+ * Possible AssertionError in UnfilteredRowIteratorWithLowerBound
(CASSANDRA-13366)
+ * Support unaligned memory access for AArch64 (CASSANDRA-13326)
+ * Improve SASI range iterator efficiency on intersection with an empty range
(CASSANDRA-12915).
+ * Fix equality comparisons of columns using the duration type
(CASSANDRA-13174)
+ * Obfuscate password in stress-graphs (CASSANDRA-12233)
+ * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
+ * nodetool stopdaemon errors out (CASSANDRA-13030)
+ * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
+ * Fix primary index calculation for SASI (CASSANDRA-12910)
+ * More fixes to the TokenAllocator (CASSANDRA-12990)
+ * NoReplicationTokenAllocator should work with zero replication factor
(CASSANDRA-12983)
+ * Address message coalescing regression (CASSANDRA-12676)
+Merged from 3.0:
+ * Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
+ * Legacy caching options can prevent 3.0 upgrade (CASSANDRA-13384)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
* Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundMarker.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/580963a2/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
--
diff --cc
src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
index fd41bea,0683d76..70d6a9d
--- a/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
+++ b/src/java/org/apache/cassandra/db/rows/RangeTombstoneBoundaryMarker.java
@@@ -120,9 -120,14 +120,14 @@@ public class RangeTombstoneBoundaryMark
return new RangeTombstoneBoundaryMarker(clustering().copy(allocator),
endDeletion, startDeletion);
}
+ public RangeTombstoneBoundaryMarker withNewOpeningDeletionTime(boolean
reversed, DeletionTime newDeletionTime)
+ {
+ return new RangeTombstoneBoundaryMarker(clustering(), reversed ?
newDeletionTime : endDeletion, reversed ? startDeletion : newDeletionTime);
+ }
+
-public static RangeTombstoneBoundaryMarker makeBoundary(boolean reversed,
Slice.Bound close, Slice.Bound open, DeletionTime closeDeletion, DeletionTime
openDeletion)
+public static RangeTombstoneBoundaryMarker makeBoundary(boolean reversed,
ClusteringBound close, ClusteringBound open, DeletionTime closeDeletion,
DeletionTime openDeletion)
{
-assert RangeTombstone.Bound.Kind.compare(close.kind(), open.kind
[1/6] cassandra git commit: Legacy deserializer can create empty range tombstones
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 6edc26824 -> 451fe9d8a
refs/heads/cassandra-3.11 bd7708192 -> 580963a21
refs/heads/trunk ea708c47e -> 07795f101
Legacy deserializer can create empty range tombstones
patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13341
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/451fe9d8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/451fe9d8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/451fe9d8
Branch: refs/heads/cassandra-3.0
Commit: 451fe9d8ac567942f62852f542d28d7d1116f1a1
Parents: 6edc268
Author: Sylvain Lebresne
Authored: Thu Mar 16 17:25:39 2017 +0100
Committer: Sylvain Lebresne
Committed: Wed Mar 29 13:17:58 2017 +0200
--
CHANGES.txt | 1 +
.../cassandra/db/UnfilteredDeserializer.java| 124 ++-
.../db/rows/RangeTombstoneBoundMarker.java | 8 ++
.../db/rows/RangeTombstoneBoundaryMarker.java | 5 +
.../cassandra/db/rows/RangeTombstoneMarker.java | 2 +
.../cassandra/db/OldFormatDeserializerTest.java | 54
6 files changed, 162 insertions(+), 32 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/451fe9d8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b46eb50..c4293de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.13
+ * Legacy deserializer can create empty range tombstones (CASSANDRA-13341)
* Use the Kernel32 library to retrieve the PID on Windows and fix startup
checks (CASSANDRA-1)
* Fix code to not exchange schema across major versions (CASSANDRA-13274)
* Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/451fe9d8/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 7bbbfdb..92690e1 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -26,8 +26,6 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Iterables;
import com.google.common.collect.PeekingIterator;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.rows.*;
import org.apache.cassandra.io.util.DataInputPlus;
@@ -43,8 +41,6 @@ import org.apache.cassandra.net.MessagingService;
*/
public abstract class UnfilteredDeserializer
{
-private static final Logger logger =
LoggerFactory.getLogger(UnfilteredDeserializer.class);
-
protected final CFMetaData metadata;
protected final DataInputPlus in;
protected final SerializationHelper helper;
@@ -433,21 +429,31 @@ public abstract class UnfilteredDeserializer
{
if (atoms.hasNext())
{
+// If there is a range tombstone to open strictly
before the next row/RT, we need to return that open (or boundary) marker first.
+if
(tombstoneTracker.hasOpeningMarkerBefore(atoms.peek()))
+{
+next = tombstoneTracker.popOpeningMarker();
+}
// If a range tombstone closes strictly before the
next row/RT, we need to return that close (or boundary) marker first.
-if
(tombstoneTracker.hasClosingMarkerBefore(atoms.peek()))
+else if
(tombstoneTracker.hasClosingMarkerBefore(atoms.peek()))
{
next = tombstoneTracker.popClosingMarker();
}
else
{
LegacyLayout.LegacyAtom atom = atoms.next();
-if (!tombstoneTracker.isShadowed(atom))
-next = isRow(atom) ? readRow(atom) :
tombstoneTracker.openNew(atom.asRangeTombstone());
+if (tombstoneTracker.isShadowed(atom))
+continue;
+
+if (isRow(atom))
+next = readRow(atom);
+else
+
tombstoneTracker.openNew(atom.asRangeTombstone());
}
}
else if (tombstoneTracker.hasOpenTombstones())
{
-
[jira] [Updated] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barthelemy Vessemont updated CASSANDRA-13390:
-
Description:
I'm trying to run a repair with following options :
{{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...}}
`some_keyspace` definition :
{quote}
CREATE KEYSPACE some_keyspace WITH replication = <'class':
'NetworkTopologyStrategy', 'AMST': '3'> AND durable_writes = true;
{quote}
Tables can be both of DTCS and LCS
Cluster :
{quote}
Datacenter: AMST
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoad Tokens OwnsHost ID
Rack
UN 10.22.150.236 205.22 GiB 256 ? B04
UN 10.22.150.204 200.87 GiB 256 ? B05
UN 10.22.140.77 203.12 GiB 256 ? D22
{quote}
Repair starts well but seems to fails at creating remote snapshots on nodes
holding replicas:
*info* : no other repair have been started on others nodes during this time,
snapshots were also cleaned before.
{quote}
# nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...
11:42:03[2017-03-29 09:42:03,399] Starting repair command #10
(f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
repair options (parallelism: sequential, primary range: true, incremental:
false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
pull repair: false)
11:47:43[2017-03-29 09:47:43,578] Repair session
f6e74940-1463-11e7-aa9f-01f10058e068 for range
[(4978050141600810389,4982072488404840895],
(-1340498249852980466,-1311689893502261125],
(-5302264476874517513,-5291739032085711936],
(-5981241605827420506,-5966850479287809973],
(1857121221013279321,1899365815561615863],
(942740326159033054,946603639333506869],
(382316032181285431,397235785699549982],
[...]
(-6731897432582288959,-6728305970724193972],
(-3193765198824884162,-3152653432337268817],
(6879878057460360708,6898924573938960263],
(7238964282930318864,7282255874655871690],
(-4737709921934606628,-4734841018997895217]] failed with error Could not create
snapshot at /10.22.150.204 (progress: 1%)
11:47:43[2017-03-29 09:47:43,579] Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
11:47:43[2017-03-29 09:47:43,580] Repair command #10 finished in 5
minutes 40 seconds
{quote}
errors logs on replica side :
{quote}
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[AntiEntropyStage:1,5,main]
java.lang.RuntimeException: Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
at
org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
[apache-cassandra-3.10.jar:3.10]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
#f6e74940-1463-11e7-aa9f-01f10058e068] span_names sync failed
ERROR [Repair#10:4] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[Repair#10:4,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Could not create snapshot at /10.22.150.204
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
~[guava-18.0.jar:na]
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
~[guava-18.0.jar:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
~[apache-cassandra-3.10.jar:3.10]
at
[jira] [Updated] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barthelemy Vessemont updated CASSANDRA-13390:
-
Environment:
* CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
* Cassandra 3.10 (setup from vanilla tar.gz)
was:
* CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
* Cassandra 3.10 (setup from tar.gz)
> Nodetool repair fails with snapshot error message
> -
>
> Key: CASSANDRA-13390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13390
> Project: Cassandra
> Issue Type: Bug
> Environment: * CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
> * Cassandra 3.10 (setup from vanilla tar.gz)
>Reporter: Barthelemy Vessemont
>
> I'm trying to run a repair with following options :
> {{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...}}
> `some_keyspace` definition :
> {quote}
> CREATE KEYSPACE some_keyspace WITH replication = <'class':
> 'NetworkTopologyStrategy', 'AMST': '3'> AND durable_writes = true;
> {quote}
> Tables can be both of DTCS and LCS
> Cluster :
> {quote}
> Datacenter: AMST
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
>Rack
> UN 10.22.150.236 205.22 GiB 256 ? B04
> UN 10.22.150.204 200.87 GiB 256 ? B05
> UN 10.22.140.77 203.12 GiB 256 ? D22
> {quote}
> Repair starts well but seems to fails at creating remote snapshots on nodes
> holding replicas:
> {quote}
> # nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...
> 11:42:03 [2017-03-29 09:42:03,399] Starting repair command #10
> (f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
> repair options (parallelism: sequential, primary range: true, incremental:
> false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
> some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
> pull repair: false)
> 11:47:43 [2017-03-29 09:47:43,578] Repair session
> f6e74940-1463-11e7-aa9f-01f10058e068 for range
> [(4978050141600810389,4982072488404840895],
> (-1340498249852980466,-1311689893502261125],
> (-5302264476874517513,-5291739032085711936],
> (-5981241605827420506,-5966850479287809973],
> (1857121221013279321,1899365815561615863],
> (942740326159033054,946603639333506869],
> (382316032181285431,397235785699549982],
> [...]
> (-6731897432582288959,-6728305970724193972],
> (-3193765198824884162,-3152653432337268817],
> (6879878057460360708,6898924573938960263],
> (7238964282930318864,7282255874655871690],
> (-4737709921934606628,-4734841018997895217]] failed with error Could not
> create snapshot at /10.22.150.204 (progress: 1%)
> 11:47:43 [2017-03-29 09:47:43,579] Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> 11:47:43 [2017-03-29 09:47:43,580] Repair command #10 finished in 5
> minutes 40 seconds
> {quote}
> errors logs on replica side :
> {quote}
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
> RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> at
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_91]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.10.jar:3.10]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
> WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
> #f6
[jira] [Updated] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barthelemy Vessemont updated CASSANDRA-13390:
-
Description:
I'm trying to run a repair with following options :
{{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...}}
`some_keyspace` definition :
{quote}
CREATE KEYSPACE some_keyspace WITH replication = <'class':
'NetworkTopologyStrategy', 'AMST': '3'> AND durable_writes = true;
{quote}
Tables can be both of DTCS and LCS
Cluster :
{quote}
Datacenter: AMST
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoad Tokens OwnsHost ID
Rack
UN 10.22.150.236 205.22 GiB 256 ? B04
UN 10.22.150.204 200.87 GiB 256 ? B05
UN 10.22.140.77 203.12 GiB 256 ? D22
{quote}
Repair starts well but seems to fails at creating remote snapshots on nodes
holding replicas:
{quote}
# nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...
11:42:03[2017-03-29 09:42:03,399] Starting repair command #10
(f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
repair options (parallelism: sequential, primary range: true, incremental:
false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
pull repair: false)
11:47:43[2017-03-29 09:47:43,578] Repair session
f6e74940-1463-11e7-aa9f-01f10058e068 for range
[(4978050141600810389,4982072488404840895],
(-1340498249852980466,-1311689893502261125],
(-5302264476874517513,-5291739032085711936],
(-5981241605827420506,-5966850479287809973],
(1857121221013279321,1899365815561615863],
(942740326159033054,946603639333506869],
(382316032181285431,397235785699549982],
[...]
(-6731897432582288959,-6728305970724193972],
(-3193765198824884162,-3152653432337268817],
(6879878057460360708,6898924573938960263],
(7238964282930318864,7282255874655871690],
(-4737709921934606628,-4734841018997895217]] failed with error Could not create
snapshot at /10.22.150.204 (progress: 1%)
11:47:43[2017-03-29 09:47:43,579] Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
11:47:43[2017-03-29 09:47:43,580] Repair command #10 finished in 5
minutes 40 seconds
{quote}
errors logs on replica side :
{quote}
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[AntiEntropyStage:1,5,main]
java.lang.RuntimeException: Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
at
org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
[apache-cassandra-3.10.jar:3.10]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
#f6e74940-1463-11e7-aa9f-01f10058e068] span_names sync failed
ERROR [Repair#10:4] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[Repair#10:4,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Could not create snapshot at /10.22.150.204
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
~[guava-18.0.jar:na]
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
~[guava-18.0.jar:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java
[jira] [Updated] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barthelemy Vessemont updated CASSANDRA-13390:
-
Description:
I'm trying to run a repair with following options :
{{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...}}
`some_keyspace` definition :
{quote}
CREATE KEYSPACE some_keyspace WITH replication = <'class':
'NetworkTopologyStrategy', 'AMST': '3'> AND durable_writes = true;
{quote}
Tables can be both of DTCS and LCS
Cluster :
{quote}
Datacenter: AMST
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoad Tokens OwnsHost ID
Rack
UN 10.22.150.236 205.22 GiB 256 ? B04
UN 10.22.150.204 200.87 GiB 256 ? B05
UN 10.22.140.77 203.12 GiB 256 ? D22
{quote}
Repair starts well but seems to fails at creating remote snapshots on nodes
holding replicas:
{quote}
# nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...
11:42:03[2017-03-29 09:42:03,399] Starting repair command #10
(f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
repair options (parallelism: sequential, primary range: true, incremental:
false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
pull repair: false)
11:47:43[2017-03-29 09:47:43,578] Repair session
f6e74940-1463-11e7-aa9f-01f10058e068 for range
[(4978050141600810389,4982072488404840895],
(-1340498249852980466,-1311689893502261125],
(-5302264476874517513,-5291739032085711936],
(-5981241605827420506,-5966850479287809973],
(1857121221013279321,1899365815561615863],
(942740326159033054,946603639333506869],
(382316032181285431,397235785699549982],
[...]
(-6731897432582288959,-6728305970724193972],
(-3193765198824884162,-3152653432337268817],
(6879878057460360708,6898924573938960263],
(7238964282930318864,7282255874655871690],
(-4737709921934606628,-4734841018997895217]] failed with error Could not create
snapshot at /10.22.150.204 (progress: 1%)
11:47:43[2017-03-29 09:47:43,579] Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
11:47:43[2017-03-29 09:47:43,580] Repair command #10 finished in 5
minutes 40 seconds
{quote}
errors logs on replica side :
{quote}
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[AntiEntropyStage:1,5,main]
java.lang.RuntimeException: Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
at
org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
[apache-cassandra-3.10.jar:3.10]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
#f6e74940-1463-11e7-aa9f-01f10058e068] span_names sync failed
ERROR [Repair#10:4] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[Repair#10:4,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Could not create snapshot at /10.22.150.204
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
~[guava-18.0.jar:na]
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
~[guava-18.0.jar:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java
[jira] [Comment Edited] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946889#comment-15946889
]
Barthelemy Vessemont edited comment on CASSANDRA-13390 at 3/29/17 10:26 AM:
I've seen this kind of errors in bugtracker for 2.X versions but those should
have been fixed for a long time.
+ I'm in 3.10 so it shouldn't be related to this old issues ?
Can you help me a bit ?
was (Author: frnunurs):
I've seen this kind of errors in bugtracker for 2.X version & those should have
been fixed.
+ I'm in 3.10 so it shouldn't be related to this old issues ?
Can you help me a bit ?
> Nodetool repair fails with snapshot error message
> -
>
> Key: CASSANDRA-13390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13390
> Project: Cassandra
> Issue Type: Bug
> Environment: * CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
> * Cassandra 3.10 (setup from tar.gz)
>Reporter: Barthelemy Vessemont
>
> I'm trying to run a repair with following options :
> {{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...}}
> `some_keyspace` definition :
> {quote}
> CREATE KEYSPACE some_keyspace WITH replication = {'class':
> 'NetworkTopologyStrategy', 'AMST': '3'} AND durable_writes = true;
> {quote}
> Tables can be both of DTCS and LCS
> Cluster :
> {quote}
> Datacenter: AMST
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
>Rack
> UN 10.22.150.236 205.22 GiB 256 ? B04
> UN 10.22.150.204 200.87 GiB 256 ? B05
> UN 10.22.140.77 203.12 GiB 256 ? D22
> {quote}
> Repair starts well but seems to fails at creating remote snapshots on nodes
> holding replicas:
> {quote}
> # nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...
> 11:42:03 [2017-03-29 09:42:03,399] Starting repair command #10
> (f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
> repair options (parallelism: sequential, primary range: true, incremental:
> false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
> some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
> pull repair: false)
> 11:47:43 [2017-03-29 09:47:43,578] Repair session
> f6e74940-1463-11e7-aa9f-01f10058e068 for range
> [(4978050141600810389,4982072488404840895],
> (-1340498249852980466,-1311689893502261125],
> (-5302264476874517513,-5291739032085711936],
> (-5981241605827420506,-5966850479287809973],
> (1857121221013279321,1899365815561615863],
> (942740326159033054,946603639333506869],
> (382316032181285431,397235785699549982],
> [...]
> (-6731897432582288959,-6728305970724193972],
> (-3193765198824884162,-3152653432337268817],
> (6879878057460360708,6898924573938960263],
> (7238964282930318864,7282255874655871690],
> (-4737709921934606628,-4734841018997895217]] failed with error Could not
> create snapshot at /10.22.150.204 (progress: 1%)
> 11:47:43 [2017-03-29 09:47:43,579] Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> 11:47:43 [2017-03-29 09:47:43,580] Repair command #10 finished in 5
> minutes 40 seconds
> {quote}
> errors logs on replica side :
> {quote}
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
> RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> at
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_91]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
[jira] [Updated] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barthelemy Vessemont updated CASSANDRA-13390:
-
Description:
I'm trying to run a repair with following options :
{{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...}}
`some_keyspace` definition :
{quote}
CREATE KEYSPACE some_keyspace WITH replication = {'class':
'NetworkTopologyStrategy', 'AMST': '3'} AND durable_writes = true;
{quote}
Tables can be both of DTCS and LCS
Cluster :
{quote}
Datacenter: AMST
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoad Tokens OwnsHost ID
Rack
UN 10.22.150.236 205.22 GiB 256 ? B04
UN 10.22.150.204 200.87 GiB 256 ? B05
UN 10.22.140.77 203.12 GiB 256 ? D22
{quote}
Repair starts well but seems to fails at creating remote snapshots on nodes
holding replicas:
{quote}
# nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...
11:42:03[2017-03-29 09:42:03,399] Starting repair command #10
(f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
repair options (parallelism: sequential, primary range: true, incremental:
false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
pull repair: false)
11:47:43[2017-03-29 09:47:43,578] Repair session
f6e74940-1463-11e7-aa9f-01f10058e068 for range
[(4978050141600810389,4982072488404840895],
(-1340498249852980466,-1311689893502261125],
(-5302264476874517513,-5291739032085711936],
(-5981241605827420506,-5966850479287809973],
(1857121221013279321,1899365815561615863],
(942740326159033054,946603639333506869],
(382316032181285431,397235785699549982],
[...]
(-6731897432582288959,-6728305970724193972],
(-3193765198824884162,-3152653432337268817],
(6879878057460360708,6898924573938960263],
(7238964282930318864,7282255874655871690],
(-4737709921934606628,-4734841018997895217]] failed with error Could not create
snapshot at /10.22.150.204 (progress: 1%)
11:47:43[2017-03-29 09:47:43,579] Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
11:47:43[2017-03-29 09:47:43,580] Repair command #10 finished in 5
minutes 40 seconds
{quote}
errors logs on replica side :
{quote}
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[AntiEntropyStage:1,5,main]
java.lang.RuntimeException: Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
at
org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
[apache-cassandra-3.10.jar:3.10]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
#f6e74940-1463-11e7-aa9f-01f10058e068] span_names sync failed
ERROR [Repair#10:4] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[Repair#10:4,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Could not create snapshot at /10.22.150.204
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
~[guava-18.0.jar:na]
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
~[guava-18.0.jar:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java
[jira] [Comment Edited] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946889#comment-15946889
]
Barthelemy Vessemont edited comment on CASSANDRA-13390 at 3/29/17 10:21 AM:
I've seen this kind of errors in bugtracker for 2.X version & those should have
been fixed.
+ I'm in 3.10 so it shouldn't be related to this old issues ?
Can you help me a bit ?
was (Author: frnunurs):
I've seen this kind of errors in bugtracker for 2.X version , but I'm in 3.10.
Can you help me a bit ?
> Nodetool repair fails with snapshot error message
> -
>
> Key: CASSANDRA-13390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13390
> Project: Cassandra
> Issue Type: Bug
> Environment: * CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
> * Cassandra 3.10 (setup from tar.gz)
>Reporter: Barthelemy Vessemont
>
> I'm trying to run a repair with following options :
> {{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...}}
> `some_keyspace` definition :
> {quote}
> CREATE KEYSPACE some_keyspace WITH replication = {'class':
> 'NetworkTopologyStrategy', 'AMST': '3'} AND durable_writes = true;
> {quote}
> Tables can be both of DTCS and LCS
> Cluster :
> {quote}
> Datacenter: AMST
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
>Rack
> UN 10.22.150.236 205.22 GiB 256 ? B04
> UN 10.22.150.204 200.87 GiB 256 ? B05
> UN 10.22.140.77 203.12 GiB 256 ? D22
> {quote}
> Repair starts well but seems to fails at creating remote snapshots on nodes
> holding replicas:
> {quote}
> nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...
> 11:42:03 [2017-03-29 09:42:03,399] Starting repair command #10
> (f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
> repair options (parallelism: sequential, primary range: true, incremental:
> false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
> some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
> pull repair: false)
> 11:47:43 [2017-03-29 09:47:43,578] Repair session
> f6e74940-1463-11e7-aa9f-01f10058e068 for range
> [(4978050141600810389,4982072488404840895],
> (-1340498249852980466,-1311689893502261125],
> (-5302264476874517513,-5291739032085711936],
> (-5981241605827420506,-5966850479287809973],
> (1857121221013279321,1899365815561615863],
> (942740326159033054,946603639333506869],
> (382316032181285431,397235785699549982],
> [...]
> (-6731897432582288959,-6728305970724193972],
> (-3193765198824884162,-3152653432337268817],
> (6879878057460360708,6898924573938960263],
> (7238964282930318864,7282255874655871690],
> (-4737709921934606628,-4734841018997895217]] failed with error Could not
> create snapshot at /10.22.150.204 (progress: 1%)
> 11:47:43 [2017-03-29 09:47:43,579] Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> 11:47:43 [2017-03-29 09:47:43,580] Repair command #10 finished in 5
> minutes 40 seconds
> {quote}
> errors logs on replica side :
> {quote}
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
> RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> at
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_91]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0
[jira] [Commented] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946889#comment-15946889
]
Barthelemy Vessemont commented on CASSANDRA-13390:
--
I've seen this kind of errors in bugtracker for 2.X version , but I'm in 3.10.
Can you help me a bit ?
> Nodetool repair fails with snapshot error message
> -
>
> Key: CASSANDRA-13390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13390
> Project: Cassandra
> Issue Type: Bug
> Environment: * CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
> * Cassandra 3.10 (setup from tar.gz)
>Reporter: Barthelemy Vessemont
>
> I'm trying to run a repair with following options :
> {{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...}}
> `some_keyspace` definition :
> {quote}
> CREATE KEYSPACE some_keyspace WITH replication = {'class':
> 'NetworkTopologyStrategy', 'AMST': '3'} AND durable_writes = true;
> {quote}
> Tables can be both of DTCS and LCS
> Cluster :
> {quote}
> Datacenter: AMST
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
>Rack
> UN 10.22.150.236 205.22 GiB 256 ? B04
> UN 10.22.150.204 200.87 GiB 256 ? B05
> UN 10.22.140.77 203.12 GiB 256 ? D22
> {quote}
> Repair starts well but seems to fails at creating remote snapshots on nodes
> holding replicas:
> {quote}
> nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...
> 11:42:03 [2017-03-29 09:42:03,399] Starting repair command #10
> (f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
> repair options (parallelism: sequential, primary range: true, incremental:
> false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
> some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
> pull repair: false)
> 11:47:43 [2017-03-29 09:47:43,578] Repair session
> f6e74940-1463-11e7-aa9f-01f10058e068 for range
> [(4978050141600810389,4982072488404840895],
> (-1340498249852980466,-1311689893502261125],
> (-5302264476874517513,-5291739032085711936],
> (-5981241605827420506,-5966850479287809973],
> (1857121221013279321,1899365815561615863],
> (942740326159033054,946603639333506869],
> (382316032181285431,397235785699549982],
> [...]
> (-6731897432582288959,-6728305970724193972],
> (-3193765198824884162,-3152653432337268817],
> (6879878057460360708,6898924573938960263],
> (7238964282930318864,7282255874655871690],
> (-4737709921934606628,-4734841018997895217]] failed with error Could not
> create snapshot at /10.22.150.204 (progress: 1%)
> 11:47:43 [2017-03-29 09:47:43,579] Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> 11:47:43 [2017-03-29 09:47:43,580] Repair command #10 finished in 5
> minutes 40 seconds
> {quote}
> errors logs on replica side :
> {quote}
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
> RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> at
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_91]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.10.jar:3.10]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
> WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
> #f6e74940-1463-11e7-aa9f-01f10058e068] span
[jira] [Updated] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barthelemy Vessemont updated CASSANDRA-13390:
-
Description:
I'm trying to run a repair with following options :
{{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...}}
`some_keyspace` definition :
{quote}
CREATE KEYSPACE some_keyspace WITH replication = {'class':
'NetworkTopologyStrategy', 'AMST': '3'} AND durable_writes = true;
{quote}
Tables can be both of DTCS and LCS
Cluster :
{quote}
Datacenter: AMST
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoad Tokens OwnsHost ID
Rack
UN 10.22.150.236 205.22 GiB 256 ? B04
UN 10.22.150.204 200.87 GiB 256 ? B05
UN 10.22.140.77 203.12 GiB 256 ? D22
{quote}
Repair starts well but seems to fails at creating remote snapshots on nodes
holding replicas:
{quote}
nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...
11:42:03[2017-03-29 09:42:03,399] Starting repair command #10
(f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
repair options (parallelism: sequential, primary range: true, incremental:
false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
pull repair: false)
11:47:43[2017-03-29 09:47:43,578] Repair session
f6e74940-1463-11e7-aa9f-01f10058e068 for range
[(4978050141600810389,4982072488404840895],
(-1340498249852980466,-1311689893502261125],
(-5302264476874517513,-5291739032085711936],
(-5981241605827420506,-5966850479287809973],
(1857121221013279321,1899365815561615863],
(942740326159033054,946603639333506869],
(382316032181285431,397235785699549982],
[...]
(-6731897432582288959,-6728305970724193972],
(-3193765198824884162,-3152653432337268817],
(6879878057460360708,6898924573938960263],
(7238964282930318864,7282255874655871690],
(-4737709921934606628,-4734841018997895217]] failed with error Could not create
snapshot at /10.22.150.204 (progress: 1%)
11:47:43[2017-03-29 09:47:43,579] Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
11:47:43[2017-03-29 09:47:43,580] Repair command #10 finished in 5
minutes 40 seconds
{quote}
errors logs on replica side :
{quote}
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[AntiEntropyStage:1,5,main]
java.lang.RuntimeException: Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
at
org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
[apache-cassandra-3.10.jar:3.10]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
#f6e74940-1463-11e7-aa9f-01f10058e068] span_names sync failed
ERROR [Repair#10:4] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[Repair#10:4,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Could not create snapshot at /10.22.150.204
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
~[guava-18.0.jar:na]
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
~[guava-18.0.jar:na]
at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.ut
[jira] [Updated] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
[
https://issues.apache.org/jira/browse/CASSANDRA-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barthelemy Vessemont updated CASSANDRA-13390:
-
Environment:
* CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
* Cassandra 3.10 (setup from tar.gz)
was:
* CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
* Cassandra 3.10
> Nodetool repair fails with snapshot error message
> -
>
> Key: CASSANDRA-13390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13390
> Project: Cassandra
> Issue Type: Bug
> Environment: * CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
> * Cassandra 3.10 (setup from tar.gz)
>Reporter: Barthelemy Vessemont
>
> I'm trying to run a repair with following options :
> {{nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...}}
> `some_keyspace` definition :
> {quote}
> CREATE KEYSPACE some_keyspace WITH replication = {'class':
> 'NetworkTopologyStrategy', 'AMST': '3'} AND durable_writes = true;
> {quote}
> Tables can be both of DTCS and LCS
> Cluster :
> {quote}
> Datacenter: AMST
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
>Rack
> UN 10.22.150.236 205.22 GiB 256 ? B04
> UN 10.22.150.204 200.87 GiB 256 ? B05
> UN 10.22.140.77 203.12 GiB 256 ? D22
> {quote}
> Repair starts well but seems to fails at creating remote snapshots on nodes
> holding replicas:
> {quote}
> nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
> some_other_table ...
> 11:42:03 [2017-03-29 09:42:03,399] Starting repair command #10
> (f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
> repair options (parallelism: sequential, primary range: true, incremental:
> false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
> some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
> pull repair: false)
> 11:47:43 [2017-03-29 09:47:43,578] Repair session
> f6e74940-1463-11e7-aa9f-01f10058e068 for range
> [(4978050141600810389,4982072488404840895],
> (-1340498249852980466,-1311689893502261125],
> (-5302264476874517513,-5291739032085711936],
> (-5981241605827420506,-5966850479287809973],
> (1857121221013279321,1899365815561615863],
> (942740326159033054,946603639333506869],
> (382316032181285431,397235785699549982],
> [...]
> (-6731897432582288959,-6728305970724193972],
> (-3193765198824884162,-3152653432337268817],
> (6879878057460360708,6898924573938960263],
> (7238964282930318864,7282255874655871690],
> (-4737709921934606628,-4734841018997895217]] failed with error Could not
> create snapshot at /10.22.150.204 (progress: 1%)
> 11:47:43 [2017-03-29 09:47:43,579] Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> 11:47:43 [2017-03-29 09:47:43,580] Repair command #10 finished in 5
> minutes 40 seconds
> {quote}
> errors logs on replica side :
> {quote}
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
> RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id =
> f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
> at
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-3.10.jar:3.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_91]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.10.jar:3.10]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
> WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
> #f6e74940-1463-11e7-aa9f-01f10058e068] sp
[jira] [Created] (CASSANDRA-13390) Nodetool repair fails with snapshot error message
Barthelemy Vessemont created CASSANDRA-13390:
Summary: Nodetool repair fails with snapshot error message
Key: CASSANDRA-13390
URL: https://issues.apache.org/jira/browse/CASSANDRA-13390
Project: Cassandra
Issue Type: Bug
Environment: * CentOS 7 : 4.4.21-1.el7.elrepo.x86_64
* Cassandra 3.10
Reporter: Barthelemy Vessemont
I'm trying to run a repair with following options :
`nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...`
`some_keyspace` definition :
```
CREATE KEYSPACE some_keyspace WITH replication = {'class':
'NetworkTopologyStrategy', 'AMST': '3'} AND durable_writes = true;
```
Tables can be both of DTCS and LCS
Cluster :
```
Datacenter: AMST
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoad Tokens OwnsHost ID
Rack
UN 10.22.150.236 205.22 GiB 256 ? B04
UN 10.22.150.204 200.87 GiB 256 ? B05
UN 10.22.140.77 203.12 GiB 256 ? D22
```
Repair starts well but seems to fails at creating remote snapshots on nodes
holding replicas:
```
nodetool repair -seq -full -pr some_keyspace some_table1 some_table2
some_other_table ...
11:42:03[2017-03-29 09:42:03,399] Starting repair command #10
(f6dac620-1463-11e7-aa9f-01f10058e068), repairing keyspace some_keyspace with
repair options (parallelism: sequential, primary range: true, incremental:
false, job threads: 1, ColumnFamilies: [some_keyspace, some_table1,
some_table2, some_other_table], dataCenters: [], hosts: [], # of ranges: 256,
pull repair: false)
11:47:43[2017-03-29 09:47:43,578] Repair session
f6e74940-1463-11e7-aa9f-01f10058e068 for range
[(4978050141600810389,4982072488404840895],
(-1340498249852980466,-1311689893502261125],
(-5302264476874517513,-5291739032085711936],
(-5981241605827420506,-5966850479287809973],
(1857121221013279321,1899365815561615863],
(942740326159033054,946603639333506869],
(382316032181285431,397235785699549982],
[...]
(-6731897432582288959,-6728305970724193972],
(-3193765198824884162,-3152653432337268817],
(6879878057460360708,6898924573938960263],
(7238964282930318864,7282255874655871690],
(-4737709921934606628,-4734841018997895217]] failed with error Could not create
snapshot at /10.22.150.204 (progress: 1%)
11:47:43[2017-03-29 09:47:43,579] Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
11:47:43[2017-03-29 09:47:43,580] Repair command #10 finished in 5
minutes 40 seconds
```
errors logs on replica side :
```
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,577
RepairMessageVerbHandler.java:168 - Got error, removing parent repair session
ERROR [AntiEntropyStage:1] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[AntiEntropyStage:1,5,main]
java.lang.RuntimeException: Parent repair session with id =
f6dac620-1463-11e7-aa9f-01f10058e068 has failed.
at
org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:400)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:416)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:170)
~[apache-cassandra-3.10.jar:3.10]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[apache-cassandra-3.10.jar:3.10]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_91]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
[apache-cassandra-3.10.jar:3.10]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
WARN [RepairJobTask:13] 2017-03-29 09:47:43,578 RepairJob.java:153 - [repair
#f6e74940-1463-11e7-aa9f-01f10058e068] span_names sync failed
ERROR [Repair#10:4] 2017-03-29 09:47:43,578 CassandraDaemon.java:229 -
Exception in thread Thread[Repair#10:4,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Could not create snapshot at /10.22.150.204
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
~[guava-18.0.jar:na]
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
~[guava-18.0.jar:na]
at org.apache.cassandra.repair.Repa
[jira] [Commented] (CASSANDRA-13386) Can't start Cassandra with powershell script when connections are already established to other Cassandra nodes on the same host
[
https://issues.apache.org/jira/browse/CASSANDRA-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946873#comment-15946873
]
Jacques-Henri Berthemet commented on CASSANDRA-13386:
-
Thanks for the tip!
Maybe on line 321:
{code}
if ($line -match "TCP" -and $line -match $portRegex)
{code}
You should also match on "LISTENING"
> Can't start Cassandra with powershell script when connections are already
> established to other Cassandra nodes on the same host
> ---
>
> Key: CASSANDRA-13386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13386
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows with powershell
>Reporter: Jacques-Henri Berthemet
> Labels: easyfix, windows
>
> In our test env we run our client application on the same host as Cassandra
> nodes. When we restart the Cassandra node when the application is still
> running it fails in VerifyPortsAreAvailable function of bin\cassandra.ps1:98
> with the below error:
> {code}
> VerifyPortsAreAvailable : Found a port already in use. Aborting startup
> At C:\apache-cassandra-2.2.7\bin\cassandra.ps1:98 char:9
> + VerifyPortsAreAvailable
> + ~~~
> + CategoryInfo : NotSpecified: (:) [Write-Error],
> WriteErrorException
> + FullyQualifiedErrorId :
> Microsoft.PowerShell.Commands.WriteErrorException,VerifyPortsAreAvailable
> VerifyPortsAreAvailable : TCPxxx.xx.xx.171:61936xxx.xx.xx.24:9042
>ESTABLISHED
> {code}
> It looks like VerifyPortsAreAvailable is picking a remote 9042 port and
> refuses to start Cassandra.
> The VerifyPortsAreAvailable function should be fixed so that it only looks
> for LISTENING ports.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13339) java.nio.BufferOverflowException: null
[
https://issues.apache.org/jira/browse/CASSANDRA-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946784#comment-15946784
]
Chris Richards commented on CASSANDRA-13339:
This seems to be very sensitive to timing : adding code to convert the mutation
to text before computing the size of the serialized data (for subsequent
reporting of the mutation that was seen by at the time serializedSize() runs)
seems to reduce the frequency that this occurs significantly (I've not managed
to catch an instance with this in place).
In all cases the serialized mutation that has overflowed the allocated buffer
contains a single modification : I have seen it fire on various tables /
operations : even plain INSERTS with no paxos constraints.
> java.nio.BufferOverflowException: null
> --
>
> Key: CASSANDRA-13339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13339
> Project: Cassandra
> Issue Type: Bug
>Reporter: Chris Richards
>
> I'm seeing the following exception running Cassandra 3.9 (with Netty updated
> to 4.1.8.Final) running on a 2 node cluster. It would have been processing
> around 50 queries/second at the time (mixture of
> inserts/updates/selects/deletes) : there's a collection of tables (some with
> counters some without) and a single materialized view.
> ERROR [MutationStage-4] 2017-03-15 22:50:33,052 StorageProxy.java:1353 -
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
> at
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.SerializationHeader$Serializer.serializeForMessaging(SerializationHeader.java:380)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:122)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serialize(PartitionUpdate.java:790)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.serialize(Mutation.java:393)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:279)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:241)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1347)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2539)
> [apache-cassandra-3.9.jar:3.9]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
> [apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> [apache-cassandra-3.9.jar:3.9]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> and then again shortly afterwards
> ERROR [MutationStage-3] 2017-03-15 23:27:36,198 StorageProxy.java:1353 -
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
> at
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:
[jira] [Updated] (CASSANDRA-12962) SASI: Index are rebuilt on restart
[
https://issues.apache.org/jira/browse/CASSANDRA-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-12962:
Status: Patch Available (was: Open)
> SASI: Index are rebuilt on restart
> --
>
> Key: CASSANDRA-12962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12962
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
>Reporter: Corentin Chary
>Assignee: Alex Petrov
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: screenshot-1.png
>
>
> Apparently when cassandra any index that does not index a value in *every*
> live SSTable gets rebuild. The offending code can be found in the constructor
> of SASIIndex.
> You can easilly reproduce it:
> {code}
> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test.test (
> a text PRIMARY KEY,
> b text,
> c text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX test_b_idx ON test.test (b) USING
> 'org.apache.cassandra.index.sasi.SASIIndex';
> CREATE CUSTOM INDEX test_c_idx ON test.test (c) USING
> 'org.apache.cassandra.index.sasi.SASIIndex';
> INSERT INTO test.test (a, b) VALUES ('a', 'b');
> {code}
> Log (I added additional traces):
> {code}
> INFO [main] 2016-11-28 15:32:21,191 ColumnFamilyStore.java:406 -
> Initializing test.test
> DEBUG [SSTableBatchOpen:1] 2016-11-28 15:32:21,192 SSTableReader.java:505 -
> Opening
> /mnt/ssd/tmp/data/data/test/test-229e6380b57711e68407158fde22e121/mc-1-big
> (0.034KiB)
> DEBUG [main] 2016-11-28 15:32:21,194 SASIIndex.java:118 - index:
> org.apache.cassandra.schema.IndexMetadata@2f661b1a[id=6b00489b-7010-396e-9348-9f32f5167f88,name=test_b_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=b}], base CFS(Keyspace='test',
> ColumnFamily='test'), tracker
> org.apache.cassandra.db.lifecycle.Tracker@15900b83
> INFO [main] 2016-11-28 15:32:21,194 DataTracker.java:152 -
> SSTableIndex.open(column: b, minTerm: value, maxTerm: value, minKey: key,
> maxKey: key, sstable: BigTableReader(path='/mnt/ssd/tmp/data/data/test/test\
> -229e6380b57711e68407158fde22e121/mc-1-big-Data.db'))
> DEBUG [main] 2016-11-28 15:32:21,195 SASIIndex.java:129 - Rebuilding SASI
> Indexes: {}
> DEBUG [main] 2016-11-28 15:32:21,195 ColumnFamilyStore.java:895 - Enqueuing
> flush of IndexInfo: 0.386KiB (0%) on-heap, 0.000KiB (0%) off-heap
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204
> Memtable.java:465 - Writing Memtable-IndexInfo@748981977(0.054KiB serialized
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range = (min(-9223\
> 372036854775808), max(9223372036854775807)]
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204
> Memtable.java:494 - Completed flushing
> /mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db
> (0.035KiB) for\
> commitlog position CommitLogPosition(segmentId=1480343535479, position=15652)
> DEBUG [MemtableFlushWriter:1] 2016-11-28 15:32:21,224
> ColumnFamilyStore.java:1200 - Flushed to
> [BigTableReader(path='/mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db\
> ')] (1 sstables, 4.838KiB), biggest 4.838KiB, smallest 4.838KiB
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:118 - index:
> org.apache.cassandra.schema.IndexMetadata@12f3d291[id=45fcb286-b87a-3d18-a04b-b899a9880c91,name=test_c_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=c}], base CFS(Keyspace='test',
> ColumnFamily='test'), tracker
> org.apache.cassandra.db.lifecycle.Tracker@15900b83
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:121 - to rebuild: index:
> BigTableReader(path='/mnt/ssd/tmp/data/data/test/test-229e6380b57711e68407158fde22e121/mc-1-big-Data.db'),
> sstable: org.apache.cassa\
> ndra.index.sasi.conf.ColumnIndex@6cbb6b0e
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:129 - Rebuilding SASI
> Indexes:
> {BigTableReader(path='/mnt/ssd/tmp/data/data/test/test-229e6380b5
[jira] [Comment Edited] (CASSANDRA-13364) Cqlsh COPY fails importing Map>, ParseError unhashable type list
[
https://issues.apache.org/jira/browse/CASSANDRA-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946698#comment-15946698
]
Nicolae Namolovan edited comment on CASSANDRA-13364 at 3/29/17 8:11 AM:
I see. Thanks for the fix 👍 I've just checked that the tuple type in python
preserves the order, and it does.
was (Author: nicolaen):
I see. Thanks for the fix 👍 I just checked that the tuple type in python
preserves the order, and it does.
> Cqlsh COPY fails importing Map>, ParseError unhashable
> type list
>
>
> Key: CASSANDRA-13364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13364
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Nicolae Namolovan
>Assignee: Stefania
> Labels: cqlsh
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
>
> When importing data with the _COPY_ command into a column family that has a
> _map>>_ field, I get a _unhashable type: 'list'_
> error. Here is how to reproduce:
> {code}
> CREATE TABLE table1 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table1 (col1, col2map) values (1, {'key': ['value1']});
> cqlsh:ks> copy table1 to 'table1.csv';
> table1.csv file content:
> 1,{'key': ['value1']}
> cqlsh:ks> copy table1 from 'table1.csv';
> ...
> Failed to import 1 rows: ParseError - Failed to parse {'key': ['value1']} :
> unhashable type: 'list', given up without retries
> Failed to process 1 rows; failed rows written to kv_table1.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.420 seconds (0 skipped).
> {code}
> But it works fine for Map>.
> {code}
> CREATE TABLE table2 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table2 (col1, col2map) values (1, {'key': {'value1'}});
> cqlsh:ks> copy table2 to 'table2.csv';
> table2.csv file content:
> 1,{'key': {'value1'}}
> cqlsh:ks> copy table2 from 'table2.csv';
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.417 seconds (0 skipped).
> {code}
> The exception seems to arrive in _convert_map_ function in _ImportConversion_
> class inside _copyutil.py_.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13364) Cqlsh COPY fails importing Map>, ParseError unhashable type list
[
https://issues.apache.org/jira/browse/CASSANDRA-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946698#comment-15946698
]
Nicolae Namolovan edited comment on CASSANDRA-13364 at 3/29/17 8:11 AM:
I see. Thanks for the fix 👍 I just checked that the tuple type in python
preserves the order, and it does.
was (Author: nicolaen):
I see. Thanks for the fix 👍
> Cqlsh COPY fails importing Map>, ParseError unhashable
> type list
>
>
> Key: CASSANDRA-13364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13364
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Nicolae Namolovan
>Assignee: Stefania
> Labels: cqlsh
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
>
> When importing data with the _COPY_ command into a column family that has a
> _map>>_ field, I get a _unhashable type: 'list'_
> error. Here is how to reproduce:
> {code}
> CREATE TABLE table1 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table1 (col1, col2map) values (1, {'key': ['value1']});
> cqlsh:ks> copy table1 to 'table1.csv';
> table1.csv file content:
> 1,{'key': ['value1']}
> cqlsh:ks> copy table1 from 'table1.csv';
> ...
> Failed to import 1 rows: ParseError - Failed to parse {'key': ['value1']} :
> unhashable type: 'list', given up without retries
> Failed to process 1 rows; failed rows written to kv_table1.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.420 seconds (0 skipped).
> {code}
> But it works fine for Map>.
> {code}
> CREATE TABLE table2 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table2 (col1, col2map) values (1, {'key': {'value1'}});
> cqlsh:ks> copy table2 to 'table2.csv';
> table2.csv file content:
> 1,{'key': {'value1'}}
> cqlsh:ks> copy table2 from 'table2.csv';
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.417 seconds (0 skipped).
> {code}
> The exception seems to arrive in _convert_map_ function in _ImportConversion_
> class inside _copyutil.py_.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13364) Cqlsh COPY fails importing Map>, ParseError unhashable type list
[
https://issues.apache.org/jira/browse/CASSANDRA-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946698#comment-15946698
]
Nicolae Namolovan commented on CASSANDRA-13364:
---
I see. Thanks for the fix 👍
> Cqlsh COPY fails importing Map>, ParseError unhashable
> type list
>
>
> Key: CASSANDRA-13364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13364
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Nicolae Namolovan
>Assignee: Stefania
> Labels: cqlsh
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
>
> When importing data with the _COPY_ command into a column family that has a
> _map>>_ field, I get a _unhashable type: 'list'_
> error. Here is how to reproduce:
> {code}
> CREATE TABLE table1 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table1 (col1, col2map) values (1, {'key': ['value1']});
> cqlsh:ks> copy table1 to 'table1.csv';
> table1.csv file content:
> 1,{'key': ['value1']}
> cqlsh:ks> copy table1 from 'table1.csv';
> ...
> Failed to import 1 rows: ParseError - Failed to parse {'key': ['value1']} :
> unhashable type: 'list', given up without retries
> Failed to process 1 rows; failed rows written to kv_table1.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.420 seconds (0 skipped).
> {code}
> But it works fine for Map>.
> {code}
> CREATE TABLE table2 (
> col1 int PRIMARY KEY,
> col2map map>>
> );
> insert into table2 (col1, col2map) values (1, {'key': {'value1'}});
> cqlsh:ks> copy table2 to 'table2.csv';
> table2.csv file content:
> 1,{'key': {'value1'}}
> cqlsh:ks> copy table2 from 'table2.csv';
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 1 rows imported from 1 files in 0.417 seconds (0 skipped).
> {code}
> The exception seems to arrive in _convert_map_ function in _ImportConversion_
> class inside _copyutil.py_.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
