[1/3] cassandra git commit: Forbid compact clustering column type changes in ALTER TABLE
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 9a7860819 -> aa21bd295
Forbid compact clustering column type changes in ALTER TABLE
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-8879
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5414950d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5414950d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5414950d
Branch: refs/heads/cassandra-3.0
Commit: 5414950d6a42a2f7d41dec6427cbf210dd7d99af
Parents: a8e8a67
Author: Aleksey Yeschenko
Authored: Wed Nov 11 18:45:50 2015 +
Committer: Aleksey Yeschenko
Committed: Tue Nov 17 16:44:37 2015 +
--
CHANGES.txt | 1 +
.../cassandra/cql3/statements/AlterTableStatement.java | 3 +++
.../cassandra/cql3/validation/operations/AlterTest.java | 8
3 files changed, 12 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b6b394a..eea14c8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Forbid compact clustering column type changes in ALTER TABLE
(CASSANDRA-8879)
* Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
b/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
index 9d07459..44bc2bc 100644
--- a/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
@@ -182,6 +182,9 @@ public class AlterTableStatement extends
SchemaAlteringStatement
}
break;
case CLUSTERING_COLUMN:
+if (!cfm.isCQL3Table())
+throw new
InvalidRequestException(String.format("Cannot alter clustering column %s in a
non-CQL3 table", columnName));
+
AbstractType oldType =
cfm.comparator.subtype(def.position());
// Note that CFMetaData.validateCompatibility already
validate the change we're about to do. However, the error message it
// sends is a bit cryptic for a CQL3 user, so
validating here for a sake of returning a better error message
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index cc7d2a4..a6aad87 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -21,6 +21,7 @@ import org.apache.cassandra.cql3.CQLTester;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.exceptions.SyntaxException;
import org.junit.Test;
@@ -198,4 +199,11 @@ public class AlterTest extends CQLTester
assertInvalidSyntaxMessage("no viable alternative at input
'WITH'", stmt);
}
}
+
+@Test // tests CASSANDRA-8879
+public void testAlterClusteringColumnTypeInCompactTable() throws Throwable
+{
+createTable("CREATE TABLE %s (key blob, column1 blob, value blob,
PRIMARY KEY ((key), column1)) WITH COMPACT STORAGE");
+assertInvalidThrow(InvalidRequestException.class, "ALTER TABLE %s
ALTER column1 TYPE ascii");
+}
}
[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aa21bd29
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aa21bd29
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aa21bd29
Branch: refs/heads/cassandra-3.0
Commit: aa21bd295adcaeb4899d0800a184508ab2997bfd
Parents: 9a78608 a68f8bd
Author: Aleksey Yeschenko
Authored: Tue Nov 17 16:52:16 2015 +
Committer: Aleksey Yeschenko
Committed: Tue Nov 17 16:52:16 2015 +
--
CHANGES.txt | 1 +
.../cassandra/cql3/statements/AlterTableStatement.java | 3 +++
.../cassandra/cql3/validation/operations/AlterTest.java | 8
3 files changed, 12 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/aa21bd29/CHANGES.txt
--
diff --cc CHANGES.txt
index 13647cc,fb8f89a..b68cf0d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -9,9 -3,19 +9,10 @@@ Merged from 2.2
* Fix SimpleDateType type compatibility (CASSANDRA-10027)
* (Hadoop) fix splits calculation (CASSANDRA-10640)
* (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
- * (cqlsh) show partial trace if incomplete after max_trace_wait
(CASSANDRA-7645)
- * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
- * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
- * Expose phi values from failure detector via JMX and tweak debug
- and trace logging (CASSANDRA-9526)
- * Fix RangeNamesQueryPager (CASSANDRA-10509)
- * Deprecate Pig support (CASSANDRA-10542)
- * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
Merged from 2.1:
+ * Forbid compact clustering column type changes in ALTER TABLE
(CASSANDRA-8879)
* Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
- * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
* Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
* Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/aa21bd29/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
index 879f618,7f2fe3a..a9b9d37
--- a/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
@@@ -332,50 -286,6 +332,53 @@@ public class AlterTableStatement extend
return true;
}
+private static void validateAlter(CFMetaData cfm, ColumnDefinition def,
AbstractType validatorType)
+{
+switch (def.kind)
+{
+case PARTITION_KEY:
+if (validatorType instanceof CounterColumnType)
+throw new InvalidRequestException(String.format("counter
type is not supported for PRIMARY KEY part %s", def.name));
+
+AbstractType currentType =
cfm.getKeyValidatorAsClusteringComparator().subtype(def.position());
+if (!validatorType.isValueCompatibleWith(currentType))
+throw new ConfigurationException(String.format("Cannot
change %s from type %s to type %s: types are incompatible.",
+ def.name,
+
currentType.asCQL3Type(),
+
validatorType.asCQL3Type()));
+break;
+case CLUSTERING:
++if (!cfm.isCQLTable())
++throw new InvalidRequestException(String.format("Cannot
alter clustering column %s in a non-CQL3 table", def.name));
++
+AbstractType oldType =
cfm.comparator.subtype(def.position());
+// Note that CFMetaData.validateCompatibility already
validate the change we're about to do. However, the error message it
+// sends is a bit cryptic for a CQL3 user, so validating here
for a sake of returning a better error message
+// Do note that we need isCompatibleWith here, not just
isValueCompatibleWith.
+if (!validatorType.isCompatibleWith(oldType))
+{
+throw new ConfigurationException(String.format("Cannot
change %s from type %s to type %s: types are not order-compatible.",
+
[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a68f8bd7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a68f8bd7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a68f8bd7
Branch: refs/heads/cassandra-3.0
Commit: a68f8bd700666e87877993107ad1b20ed957c8a8
Parents: 99b82db 5414950
Author: Aleksey Yeschenko
Authored: Tue Nov 17 16:47:41 2015 +
Committer: Aleksey Yeschenko
Committed: Tue Nov 17 16:47:41 2015 +
--
CHANGES.txt | 1 +
.../cassandra/cql3/statements/AlterTableStatement.java | 3 +++
.../cassandra/cql3/validation/operations/AlterTest.java | 8
3 files changed, 12 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a68f8bd7/CHANGES.txt
--
diff --cc CHANGES.txt
index f5d3416,eea14c8..fb8f89a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,5 +1,18 @@@
-2.1.12
+2.2.4
+ * Don't do anticompaction after subrange repair (CASSANDRA-10422)
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
+ * (Hadoop) fix splits calculation (CASSANDRA-10640)
+ * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
+ * (cqlsh) show partial trace if incomplete after max_trace_wait
(CASSANDRA-7645)
+ * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
+ * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
+ * Expose phi values from failure detector via JMX and tweak debug
+ and trace logging (CASSANDRA-9526)
+ * Fix RangeNamesQueryPager (CASSANDRA-10509)
+ * Deprecate Pig support (CASSANDRA-10542)
+ * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
+Merged from 2.1:
+ * Forbid compact clustering column type changes in ALTER TABLE
(CASSANDRA-8879)
* Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a68f8bd7/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a68f8bd7/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --cc
test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index 566c0ea,a6aad87..2cc8a18
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@@ -201,20 -200,10 +202,27 @@@ public class AlterTest extends CQLTeste
}
}
+/**
+ * tests CASSANDRA-10027
+ */
+@Test
+public void testAlterColumnTypeToDate() throws Throwable
+{
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 int);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+execute("ALTER TABLE %s ALTER c1 TYPE date;");
+assertRows(execute("SELECT * FROM %s"), row(1, 1));
+
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 varint);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+assertInvalidMessage("Cannot change c1 from type varint to type date:
types are incompatible.",
+ "ALTER TABLE %s ALTER c1 TYPE date;");
+}
++
+ @Test // tests CASSANDRA-8879
+ public void testAlterClusteringColumnTypeInCompactTable() throws Throwable
+ {
+ createTable("CREATE TABLE %s (key blob, column1 blob, value blob,
PRIMARY KEY ((key), column1)) WITH COMPACT STORAGE");
+ assertInvalidThrow(InvalidRequestException.class, "ALTER TABLE %s
ALTER column1 TYPE ascii");
+ }
}
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a68f8bd7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a68f8bd7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a68f8bd7
Branch: refs/heads/cassandra-2.2
Commit: a68f8bd700666e87877993107ad1b20ed957c8a8
Parents: 99b82db 5414950
Author: Aleksey Yeschenko
Authored: Tue Nov 17 16:47:41 2015 +
Committer: Aleksey Yeschenko
Committed: Tue Nov 17 16:47:41 2015 +
--
CHANGES.txt | 1 +
.../cassandra/cql3/statements/AlterTableStatement.java | 3 +++
.../cassandra/cql3/validation/operations/AlterTest.java | 8
3 files changed, 12 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a68f8bd7/CHANGES.txt
--
diff --cc CHANGES.txt
index f5d3416,eea14c8..fb8f89a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,5 +1,18 @@@
-2.1.12
+2.2.4
+ * Don't do anticompaction after subrange repair (CASSANDRA-10422)
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
+ * (Hadoop) fix splits calculation (CASSANDRA-10640)
+ * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
+ * (cqlsh) show partial trace if incomplete after max_trace_wait
(CASSANDRA-7645)
+ * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
+ * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
+ * Expose phi values from failure detector via JMX and tweak debug
+ and trace logging (CASSANDRA-9526)
+ * Fix RangeNamesQueryPager (CASSANDRA-10509)
+ * Deprecate Pig support (CASSANDRA-10542)
+ * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
+Merged from 2.1:
+ * Forbid compact clustering column type changes in ALTER TABLE
(CASSANDRA-8879)
* Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a68f8bd7/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a68f8bd7/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --cc
test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index 566c0ea,a6aad87..2cc8a18
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@@ -201,20 -200,10 +202,27 @@@ public class AlterTest extends CQLTeste
}
}
+/**
+ * tests CASSANDRA-10027
+ */
+@Test
+public void testAlterColumnTypeToDate() throws Throwable
+{
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 int);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+execute("ALTER TABLE %s ALTER c1 TYPE date;");
+assertRows(execute("SELECT * FROM %s"), row(1, 1));
+
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 varint);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+assertInvalidMessage("Cannot change c1 from type varint to type date:
types are incompatible.",
+ "ALTER TABLE %s ALTER c1 TYPE date;");
+}
++
+ @Test // tests CASSANDRA-8879
+ public void testAlterClusteringColumnTypeInCompactTable() throws Throwable
+ {
+ createTable("CREATE TABLE %s (key blob, column1 blob, value blob,
PRIMARY KEY ((key), column1)) WITH COMPACT STORAGE");
+ assertInvalidThrow(InvalidRequestException.class, "ALTER TABLE %s
ALTER column1 TYPE ascii");
+ }
}
[1/2] cassandra git commit: Forbid compact clustering column type changes in ALTER TABLE
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 99b82dbb4 -> a68f8bd70
Forbid compact clustering column type changes in ALTER TABLE
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-8879
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5414950d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5414950d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5414950d
Branch: refs/heads/cassandra-2.2
Commit: 5414950d6a42a2f7d41dec6427cbf210dd7d99af
Parents: a8e8a67
Author: Aleksey Yeschenko
Authored: Wed Nov 11 18:45:50 2015 +
Committer: Aleksey Yeschenko
Committed: Tue Nov 17 16:44:37 2015 +
--
CHANGES.txt | 1 +
.../cassandra/cql3/statements/AlterTableStatement.java | 3 +++
.../cassandra/cql3/validation/operations/AlterTest.java | 8
3 files changed, 12 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b6b394a..eea14c8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Forbid compact clustering column type changes in ALTER TABLE
(CASSANDRA-8879)
* Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
b/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
index 9d07459..44bc2bc 100644
--- a/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
@@ -182,6 +182,9 @@ public class AlterTableStatement extends
SchemaAlteringStatement
}
break;
case CLUSTERING_COLUMN:
+if (!cfm.isCQL3Table())
+throw new
InvalidRequestException(String.format("Cannot alter clustering column %s in a
non-CQL3 table", columnName));
+
AbstractType oldType =
cfm.comparator.subtype(def.position());
// Note that CFMetaData.validateCompatibility already
validate the change we're about to do. However, the error message it
// sends is a bit cryptic for a CQL3 user, so
validating here for a sake of returning a better error message
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index cc7d2a4..a6aad87 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -21,6 +21,7 @@ import org.apache.cassandra.cql3.CQLTester;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.exceptions.SyntaxException;
import org.junit.Test;
@@ -198,4 +199,11 @@ public class AlterTest extends CQLTester
assertInvalidSyntaxMessage("no viable alternative at input
'WITH'", stmt);
}
}
+
+@Test // tests CASSANDRA-8879
+public void testAlterClusteringColumnTypeInCompactTable() throws Throwable
+{
+createTable("CREATE TABLE %s (key blob, column1 blob, value blob,
PRIMARY KEY ((key), column1)) WITH COMPACT STORAGE");
+assertInvalidThrow(InvalidRequestException.class, "ALTER TABLE %s
ALTER column1 TYPE ascii");
+}
}
cassandra git commit: Forbid compact clustering column type changes in ALTER TABLE
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 a8e8a6730 -> 5414950d6
Forbid compact clustering column type changes in ALTER TABLE
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-8879
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5414950d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5414950d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5414950d
Branch: refs/heads/cassandra-2.1
Commit: 5414950d6a42a2f7d41dec6427cbf210dd7d99af
Parents: a8e8a67
Author: Aleksey Yeschenko
Authored: Wed Nov 11 18:45:50 2015 +
Committer: Aleksey Yeschenko
Committed: Tue Nov 17 16:44:37 2015 +
--
CHANGES.txt | 1 +
.../cassandra/cql3/statements/AlterTableStatement.java | 3 +++
.../cassandra/cql3/validation/operations/AlterTest.java | 8
3 files changed, 12 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b6b394a..eea14c8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Forbid compact clustering column type changes in ALTER TABLE
(CASSANDRA-8879)
* Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
b/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
index 9d07459..44bc2bc 100644
--- a/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/AlterTableStatement.java
@@ -182,6 +182,9 @@ public class AlterTableStatement extends
SchemaAlteringStatement
}
break;
case CLUSTERING_COLUMN:
+if (!cfm.isCQL3Table())
+throw new
InvalidRequestException(String.format("Cannot alter clustering column %s in a
non-CQL3 table", columnName));
+
AbstractType oldType =
cfm.comparator.subtype(def.position());
// Note that CFMetaData.validateCompatibility already
validate the change we're about to do. However, the error message it
// sends is a bit cryptic for a CQL3 user, so
validating here for a sake of returning a better error message
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5414950d/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index cc7d2a4..a6aad87 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -21,6 +21,7 @@ import org.apache.cassandra.cql3.CQLTester;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.exceptions.SyntaxException;
import org.junit.Test;
@@ -198,4 +199,11 @@ public class AlterTest extends CQLTester
assertInvalidSyntaxMessage("no viable alternative at input
'WITH'", stmt);
}
}
+
+@Test // tests CASSANDRA-8879
+public void testAlterClusteringColumnTypeInCompactTable() throws Throwable
+{
+createTable("CREATE TABLE %s (key blob, column1 blob, value blob,
PRIMARY KEY ((key), column1)) WITH COMPACT STORAGE");
+assertInvalidThrow(InvalidRequestException.class, "ALTER TABLE %s
ALTER column1 TYPE ascii");
+}
}
[jira] [Updated] (CASSANDRA-9043) Improve COPY command to work with Counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-9043: --- Reviewer: Stefania [~Stefania] can you take review on this? > Improve COPY command to work with Counter columns > - > > Key: CASSANDRA-9043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9043 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Assignee: ZhaoYang >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: CASSANDRA-9043(2.1.8).patch, CASSANDRA-9043-trunk.patch > > > Noticed today that the copy command doesn't work with counter column tables. > This makes sense given that we need to use UPDATE instead of INSERT with > counters. > Given that we're making improvements in the COPY command in 3.0 with > CASSANDRA-7405, can we also tweak it to work with counters? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10692) Don't remove level info when doing upgradesstables
[ https://issues.apache.org/jira/browse/CASSANDRA-10692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008962#comment-15008962 ] Yuki Morishita commented on CASSANDRA-10692: Thanks. I see newly added test is failing in 2.2-testall, and I can reproduce it on my dev machine. Would you mind take a look? > Don't remove level info when doing upgradesstables > -- > > Key: CASSANDRA-10692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10692 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.x, 2.2.x > > > Seems we blow away the level info when doing upgradesstables. Introduced in > CASSANDRA-8004 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10541) cqlshlib tests cannot run on Windows
[
https://issues.apache.org/jira/browse/CASSANDRA-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Witschey updated CASSANDRA-10541:
-
Reviewer: Jim Witschey
> cqlshlib tests cannot run on Windows
>
>
> Key: CASSANDRA-10541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10541
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Benjamin Lerer
>Assignee: Paulo Motta
>Priority: Minor
> Labels: cqlsh, windows
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> If I try to run the {{cqlshlib}} tests on Windows, I got the following error:
> {quote}
> ==
> ERROR: Failure: AttributeError ('module' object has no attribute 'symlink')
> --
> Traceback (most recent call last):
> File "C:\Python27\lib\site-packages\nose\loader.py", line 414, in
> loadTestsFromName
> addr.filename, addr.module)
> File "C:\Python27\lib\site-packages\nose\importer.py", line 47, in
> importFromPath
> return self.importFromDir(dir_path, fqname)
> File "C:\Python27\lib\site-packages\nose\importer.py", line 94, in
> importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
> File "[...]\pylib\cqlshlib\test\__init__.py", line 17, in
> from .cassconnect import create_test_db, remove_test_db
> File "[...]\pylib\cqlshlib\test\cassconnect.py", line 22, in
> from .basecase import cql, cqlsh, cqlshlog, TEST_HOST, TEST_PORT, rundir
> File "[...]\pylib\cqlshlib\test\basecase.py", line 43, in
> os.symlink(path_to_cqlsh, modulepath)
> AttributeError: 'module' object has no attribute 'symlink'
> --
> Ran 1 test in 0.002s
> FAILED (errors=1)
> {quote}
> The problem comes from the fact tha Windows has no support for symlinks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9644) DTCS configuration proposals for handling consequences of repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008953#comment-15008953 ] Philip Thompson commented on CASSANDRA-9644: I wish I could, but cstar perf failed to preserve them. If it happens again, I'll send you them. > DTCS configuration proposals for handling consequences of repairs > - > > Key: CASSANDRA-9644 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9644 > Project: Cassandra > Issue Type: Improvement >Reporter: Antti Nissinen >Assignee: Marcus Eriksson > Labels: compaction, dtcs > Fix For: 2.1.x, 3.x > > Attachments: node0_20150621_1646_time_graph.txt, > node0_20150621_2320_time_graph.txt, node0_20150623_1526_time_graph.txt, > node1_20150621_1646_time_graph.txt, node1_20150621_2320_time_graph.txt, > node1_20150623_1526_time_graph.txt, node2_20150621_1646_time_graph.txt, > node2_20150621_2320_time_graph.txt, node2_20150623_1526_time_graph.txt, > nodetool status infos.txt, sstable_compaction_trace.txt, > sstable_compaction_trace_snipped.txt, sstable_counts.jpg > > > This is a document bringing up some issues when DTCS is used to compact time > series data in a three node cluster. The DTCS is currently configured with a > few parameters that are making the configuration fairly simple, but might > cause problems in certain special cases like recovering from the flood of > small SSTables due to repair operation. We are suggesting some ideas that > might be a starting point for further discussions. Following sections are > containing: > - Description of the cassandra setup > - Feeding process of the data > - Failure testing > - Issues caused by the repair operations for the DTCS > - Proposal for the DTCS configuration parameters > Attachments are included to support the discussion and there is a separate > section giving explanation for those. > Cassandra setup and data model > - Cluster is composed from three nodes running Cassandra 2.1.2. Replication > factor is two and read and write consistency levels are ONE. > - Data is time series data. Data is saved so that one row contains a certain > time span of data for a given metric ( 20 days in this case). The row key > contains information about the start time of the time span and metrix name. > Column name gives the offset from the beginning of time span. Column time > stamp is set to correspond time stamp when adding together the timestamp from > the row key and the offset (the actual time stamp of data point). Data model > is analog to KairosDB implementation. > - Average sampling rate is 10 seconds varying significantly from metric to > metric. > - 100 000 metrics are fed to the Cassandra. > - max_sstable_age_days is set to 5 days (objective is to keep SStable files > in manageable size, around 50 GB) > - TTL is not in use in the test. > Procedure for the failure test. > - Data is first dumped to Cassandra for 11 days and the data dumping is > stopped so that DTCS will have a change to finish all compactions. Data is > dumped with "fake timestamps" so that column time stamp is set when data is > written to Cassandra. > - One of the nodes is taken down and new data is dumped on top of the earlier > data covering couple of hours worth of data (faked time stamps). > - Dumping is stopped and the node is kept down for few hours. > - Node is taken up and the "nodetool repair" is applied on the node that was > down. > Consequences > - Repair operation will lead to massive amount of new SStables far back in > the history. New SStables are covering similar time spans than the files that > were created by DTCS before the shutdown of one of the nodes. > - To be able to compact the small files the max_sstable_age_days should be > increased to allow compaction to handle the files. However, the in a > practical case the time window will increase so large that generated files > will be huge that is not desirable. The compaction also combines together one > very large file with a bunch of small files in several phases that is not > effective. Generating really large files may also lead to out of disc space > problems. > - See the list of time graphs later in the document. > Improvement proposals for the DTCS configuration > Below is a list of desired properties for the configuration. Current > parameters are mentioned if available. > - Initial window size (currently:base_time_seconds) > - The amount of similar size windows for the bucketing (currently: > min_threshold) > - The multiplier for the window size when increased (currently: > min_threshold). This we would like to be independent from the min_threshold > parameter so that you could actually control the rate how fast the window > size is incre
[jira] [Commented] (CASSANDRA-10195) TWCS experiments and improvement proposals
[ https://issues.apache.org/jira/browse/CASSANDRA-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008944#comment-15008944 ] Philip Thompson commented on CASSANDRA-10195: - [~krummas], results are finished for the first two tests. These are only at ~275GB of data, on one node, using cstar perf. [TWCS with jeffjirsa/twcs-2.1|http://cstar.datastax.com/tests/id/8aee77a2-888e-11e5-88b6-0256e416528f] [DTCS with marcuse/9644|http://cstar.datastax.com/tests/id/8e17c1c2-888e-11e5-88b6-0256e416528f] [DTCS with marcuse/9644_2|http://cstar.datastax.com/tests/id/ac416862-8c92-11e5-b5e7-0256e416528f] is still running As you can see from the first two graphs, load and read performance were roughly equivalent. These lower density tests take about 36-48 hours to run each, if you need anymore, after the last finishes. > TWCS experiments and improvement proposals > -- > > Key: CASSANDRA-10195 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10195 > Project: Cassandra > Issue Type: Improvement >Reporter: Antti Nissinen > Fix For: 2.1.x, 2.2.x > > Attachments: 20150814_1027_compaction_hierarchy.txt, > node0_20150727_1250_time_graph.txt, node0_20150810_1017_time_graph.txt, > node0_20150812_1531_time_graph.txt, node0_20150813_0835_time_graph.txt, > node0_20150814_1054_time_graph.txt, node1_20150727_1250_time_graph.txt, > node1_20150810_1017_time_graph.txt, node1_20150812_1531_time_graph.txt, > node1_20150813_0835_time_graph.txt, node1_20150814_1054_time_graph.txt, > node2_20150727_1250_time_graph.txt, node2_20150810_1017_time_graph.txt, > node2_20150812_1531_time_graph.txt, node2_20150813_0835_time_graph.txt, > node2_20150814_1054_time_graph.txt, sstable_count_figure1.png, > sstable_count_figure2.png > > > This JIRA item describes experiments with DateTieredCompactionStartegy (DTCS) > and TimeWindowCompactionStrategy (TWCS) and proposes modifications to the > TWCS. In a test system several crashes were caused intentionally (and > unintentionally) and repair operations were executed leading to flood of > small SSTables. Target was to be able compact those files are release disk > space reserved by duplicate data. Setup is following: > - Three nodes > - DateTieredCompactionStrategy, max_sstable_age_days = 5 > Cassandra 2.1.2 > The setup and data format has been documented in detailed here > https://issues.apache.org/jira/browse/CASSANDRA-9644. > The test was started by dumping few days worth of data to the database for > 100 000 signals. Time graphs of SStables from different nodes indicates that > the DTCS has been working as expected and SStables are nicely ordered in time > wise. > See files: > node0_20150727_1250_time_graph.txt > node1_20150727_1250_time_graph.txt > node2_20150727_1250_time_graph.txt > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- AddressLoad Tokens OwnsHost ID > Rack > UN 139.66.43.170 188.87 GB 256 ? > dfc29863-c935-4909-9d7f-c59a47eda03d rack1 > UN 139.66.43.169 198.37 GB 256 ? > 12e7628b-7f05-48f6-b7e4-35a82010021a rack1 > UN 139.66.43.168 191.88 GB 256 ? > 26088392-f803-4d59-9073-c75f857fb332 rack1 > All nodes crashed due to power failure (know beforehand) and repair > operations were started for each node one at the time. Below is the behavior > of SSTable count on different nodes. New data was dumped simultaneously with > repair operation. > SEE FIGURE: sstable_count_figure1.png > Vertical lines indicate following events. > 1) Cluster was down due to power shutdown and was restarted. At the first > vertical line the repair operation (nodetool repair -pr) was started for the > first node > 2) Repair for the second repair operation was started after the first node > was successfully repaired. > 3) Repair for the third repair operation was started > 4) Third repair operation was finished > 5) One of the nodes crashed (unknown reason in OS level) > 6) Repair operation (nodetool repair -pr) was started for the first node > 7) Repair operation for the second node was started > 8) Repair operation for the third node was started > 9) Repair operations finished > These repair operations are leading to huge amount of small SSTables covering > the whole time span of the data. The compaction horizon of DTCS was limited > to 5 days (max_sstable_age_days) due to the size of the SStables on the disc. > Therefore, small SStables won't be compacted. Below are the time graphs from > SSTables after the second round of repairs. > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- AddressLoad Tokens OwnsHost ID > Rack > UN xx.xx.xx.170 663.61 GB 256 ? > dfc29863-c935-49
[jira] [Commented] (CASSANDRA-10585) SSTablesPerReadHistogram seems wrong when row cache hit happend
[
https://issues.apache.org/jira/browse/CASSANDRA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008933#comment-15008933
]
Carl Yeksigian commented on CASSANDRA-10585:
While the version for 2.1 is fine, for 2.2 and 3.0, I don't think we should
switch to {{ExponentiallyDecayingReservoir}}; the reasoning behind us using
{{EstimatedHistogram}} is discussed in CASSANDRA-5657.
I'd like to either come up with a way to either estimate the statistic trying
to be calculated here, or extend {{EstimatedHistogram}} to properly capture a 0
value. I think we can approximate this by using the
{{RowCacheHit}}/{{RowCacheMiss}} metrics; that would mean
{{SSTablesPerReadHistogram}} is only captured on {{RowCacheMiss}}, so the new
metric might be
{noformat}
(RowCacheMiss/(RowCacheHit + RowCacheMiss)) * SSTablesPerRead
{noformat}
Do either of those seem feasible, [~isburmistrov]?
> SSTablesPerReadHistogram seems wrong when row cache hit happend
> ---
>
> Key: CASSANDRA-10585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ivan Burmistrov
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: SSTablePerReadHistogram_RowCache-cassandra-2_1.patch,
> SSTablePerReadHistogram_RowCache-cassandra-2_2.patch,
> SSTablePerReadHistogram_RowCache-cassandra-3_0.patch
>
>
> SSTablePerReadHistogram metric now not considers case when row has been read
> from row cache.
> And so, this metric will have big values even almost all requests processed
> by row cache (and without touching SSTables, of course).
> So, it seems that correct behavior is to consider that if we read row from
> row cache then we read zero SSTables by this request.
> The patch at the attachment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8884) Opening a non-system keyspace before first accessing the system keyspace results in deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Kołaczkowski updated CASSANDRA-8884:
--
Priority: Minor (was: Major)
> Opening a non-system keyspace before first accessing the system keyspace
> results in deadlock
>
>
> Key: CASSANDRA-8884
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8884
> Project: Cassandra
> Issue Type: Bug
>Reporter: Piotr Kołaczkowski
>Assignee: Benjamin Lerer
>Priority: Minor
> Attachments: bulk.jstack
>
>
> I created a writer like this:
> {code}
> val writer = CQLSSTableWriter.builder()
> .forTable(tableDef.cql)
> .using(insertStatement)
> .withPartitioner(partitioner)
> .inDirectory(outputDirectory)
> .withBufferSizeInMB(bufferSizeInMB)
> .build()
> {code}
> Then I'm trying to write a row with {{addRow}} and it blocks forever.
> Everything related to {{CQLSSTableWriter}}, including its creation, is
> happening in only one thread.
> {noformat}
> "SSTableBatchOpen:3" daemon prio=10 tid=0x7f4b399d7000 nid=0x4778 waiting
> for monitor entry [0x7f4b240a7000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:118)
> - waiting to lock <0xe35fd6d0> (a java.lang.Class for
> org.apache.cassandra.db.Keyspace)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:99)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:1464)
> at
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:517)
> at
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:265)
> at
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:306)
> at
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:316)
> at
> org.apache.cassandra.db.SystemKeyspace.getSSTableReadMeter(SystemKeyspace.java:910)
> at
> org.apache.cassandra.io.sstable.SSTableReader.(SSTableReader.java:561)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:433)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:343)
> at
> org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:480)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "SSTableBatchOpen:2" daemon prio=10 tid=0x7f4b399e7800 nid=0x4777 waiting
> for monitor entry [0x7f4b23ca3000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:118)
> - waiting to lock <0xe35fd6d0> (a java.lang.Class for
> org.apache.cassandra.db.Keyspace)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:99)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:1464)
> at
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:517)
> at
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:265)
> at
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:306)
> at
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:316)
> at
> org.apache.cassandra.db.SystemKeyspace.getSSTableReadMeter(SystemKeyspace.java:910)
> at
> org.apache.cassandra.io.sstable.SSTableReader.(SSTableReader.java:561)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:433)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:343)
> at
> org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:480)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "SSTableBatchOpen:1" daemon prio=10 tid=0x7f4b399e7000 nid=0x4776 waiting
> for monitor entry [0x7f4b2359d000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.cassandra.db.Keyspace.op
[jira] [Commented] (CASSANDRA-10632) sstableutil tests failing
[
https://issues.apache.org/jira/browse/CASSANDRA-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008899#comment-15008899
]
Jim Witschey commented on CASSANDRA-10632:
--
Good catch, thanks. [dtest PR
filed|https://github.com/riptano/cassandra-dtest/pull/665].
> sstableutil tests failing
> -
>
> Key: CASSANDRA-10632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10632
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
> Fix For: 3.0.1, 3.1
>
>
> {{sstableutil_test.py:SSTableUtilTest.abortedcompaction_test}} and
> {{sstableutil_test.py:SSTableUtilTest.compaction_test}} fail on Windows:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/sstableutil_test/SSTableUtilTest/abortedcompaction_test/
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/sstableutil_test/SSTableUtilTest/compaction_test/
> This is a pretty simple failure -- looks like the underlying behavior is ok,
> but string comparison fails when the leading {{d}} in the filename is
> lowercase as returned by {{sstableutil}} (see the [{{_invoke_sstableutil}}
> test
> function|https://github.com/riptano/cassandra-dtest/blob/master/sstableutil_test.py#L128]),
> but uppercase as returned by {{glob.glob}} (see the [{{_get_sstable_files}}
> test
> function|https://github.com/riptano/cassandra-dtest/blob/master/sstableutil_test.py#L160]).
> Do I understand correctly that Windows filenames are case-insensitive,
> including the drive portion? If that's the case, then we can just lowercase
> the file names in the test helper functions above when the tests are run on
> Windows. [~JoshuaMcKenzie] can you confirm? I'll fix this in the tests if so.
> If I'm wrong, and something in {{sstableutil}} needs to be fixed, could you
> find an assignee?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10721) Altering a UDT might break UDA deserialisation
[
https://issues.apache.org/jira/browse/CASSANDRA-10721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008851#comment-15008851
]
Aleksey Yeschenko commented on CASSANDRA-10721:
---
I'm fine with simply preventing it for 3.0.1/3.1, to fix the bug itself, but
I'd prefer that we at least eventually address this properly.
> Altering a UDT might break UDA deserialisation
> --
>
> Key: CASSANDRA-10721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10721
> Project: Cassandra
> Issue Type: Bug
> Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Robert Stupp
> Fix For: 3.0.x
>
>
> CASSANDRA-10650 switched UDA's {{initcond}} serialisation in schema to its
> CQL literal. This means that if any particular field is renamed in the UDT,
> or of its type gets changes, we will not be able to parse initcond back.
> We should either:
> 1) Forbid renames and type switches in UDTs that are being used in UDAs, or
> 2) Make sure we alter the UDAs in schema alongside the new UDT at all times
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10721) Altering a UDT might break UDA deserialisation
[
https://issues.apache.org/jira/browse/CASSANDRA-10721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008844#comment-15008844
]
Robert Stupp commented on CASSANDRA-10721:
--
I think preventing such a change is the better option. Otherwise we'd also have
check permissions on the aggregates. Additionally changing nested
tuple/collection/type structures might become a nightmare.
Will also see whether we need a check for "garbaged" initcond values into
LegacySchemaMigrator.
> Altering a UDT might break UDA deserialisation
> --
>
> Key: CASSANDRA-10721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10721
> Project: Cassandra
> Issue Type: Bug
> Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Robert Stupp
> Fix For: 3.0.x
>
>
> CASSANDRA-10650 switched UDA's {{initcond}} serialisation in schema to its
> CQL literal. This means that if any particular field is renamed in the UDT,
> or of its type gets changes, we will not be able to parse initcond back.
> We should either:
> 1) Forbid renames and type switches in UDTs that are being used in UDAs, or
> 2) Make sure we alter the UDAs in schema alongside the new UDT at all times
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9556) Add newer data types to cassandra stress (e.g. decimal, dates, UDTs)
[
https://issues.apache.org/jira/browse/CASSANDRA-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008824#comment-15008824
]
Benjamin Lerer commented on CASSANDRA-9556:
---
[~jasonstack] If your problem is caused by the fact that the
{{Generator.generate}} method must return an object of the same type as the
type returned by the {{AbstractType}}, I think that you can just change the
{{Generator.generate}} return type to {{Object}}.
> Add newer data types to cassandra stress (e.g. decimal, dates, UDTs)
>
>
> Key: CASSANDRA-9556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9556
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: ZhaoYang
>Priority: Minor
> Labels: stress
>
> Currently you can't define a data model with decimal types and use Cassandra
> stress with it. Also, I imagine that holds true with other newer data types
> such as the new date and time types. Besides that, now that data models are
> including user defined types, we should allow users to create those
> structures with stress as well. Perhaps we could split out the UDTs into a
> different ticket if it holds the other types up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10383) Disable auto snapshot on selected tables.
[
https://issues.apache.org/jira/browse/CASSANDRA-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008820#comment-15008820
]
Robert Stupp commented on CASSANDRA-10383:
--
Although the patch looks good, we have to delay it until 4.0.
Reason for that is that schema tables must stay unchanged within all 3.x
versions - otherwise the node that does not know about the new column will fail
do deserialize mutations. What we would need is a messaging-version bump, which
will not happen before 4.0.
There will be a new schema-change protocol without these limitations - but that
would require a messaging-version bump, too.
{code}
java.lang.RuntimeException: Unknown column allow_auto_snapshot during
deserialization
at
org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:432)
~[main/:na]
at
org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:397)
~[main/:na]
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:190)
~[main/:na]
at
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:635)
~[main/:na]
at
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:623)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:323)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:332)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:589)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:572)
~[main/:na]
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114)
~[main/:na]
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:188)
~[main/:na]
{code}
> Disable auto snapshot on selected tables.
> -
>
> Key: CASSANDRA-10383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10383
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
> Labels: doc-impacting
> Attachments: 10383.txt
>
>
> I have a use case where I would like to turn off auto snapshot for selected
> tables, I don't want to turn it off completely since its a good feature.
> Looking at the code I think it would be relatively easy to fix.
> My plan is to create a new table property named something like
> "disable_auto_snapshot". If set to false it will prevent auto snapshot on the
> table, if set to true auto snapshot will be controlled by the "auto_snapshot"
> property in the cassandra.yaml. Default would be true.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10383) Disable auto snapshot on selected tables.
[ https://issues.apache.org/jira/browse/CASSANDRA-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-10383: - Fix Version/s: (was: 3.x) > Disable auto snapshot on selected tables. > - > > Key: CASSANDRA-10383 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10383 > Project: Cassandra > Issue Type: Improvement >Reporter: Tommy Stendahl >Assignee: Tommy Stendahl > Labels: doc-impacting > Attachments: 10383.txt > > > I have a use case where I would like to turn off auto snapshot for selected > tables, I don't want to turn it off completely since its a good feature. > Looking at the code I think it would be relatively easy to fix. > My plan is to create a new table property named something like > "disable_auto_snapshot". If set to false it will prevent auto snapshot on the > table, if set to true auto snapshot will be controlled by the "auto_snapshot" > property in the cassandra.yaml. Default would be true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10721) Altering a UDT might break UDA deserialisation
Aleksey Yeschenko created CASSANDRA-10721:
-
Summary: Altering a UDT might break UDA deserialisation
Key: CASSANDRA-10721
URL: https://issues.apache.org/jira/browse/CASSANDRA-10721
Project: Cassandra
Issue Type: Bug
Components: CQL, Distributed Metadata
Reporter: Aleksey Yeschenko
Assignee: Robert Stupp
Fix For: 3.0.x
CASSANDRA-10650 switched UDA's {{initcond}} serialisation in schema to its CQL
literal. This means that if any particular field is renamed in the UDT, or of
its type gets changes, we will not be able to parse initcond back.
We should either:
1) Forbid renames and type switches in UDTs that are being used in UDAs, or
2) Make sure we alter the UDAs in schema alongside the new UDT at all times
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10200) NetworkTopologyStrategy.calculateNaturalEndpoints is rather inefficient
[ https://issues.apache.org/jira/browse/CASSANDRA-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008781#comment-15008781 ] Branimir Lambov commented on CASSANDRA-10200: - [testall|http://cassci.datastax.com/job/blambov-10200-calculate-endpoints-testall/] [dtest|http://cassci.datastax.com/job/blambov-10200-calculate-endpoints-dtest/] > NetworkTopologyStrategy.calculateNaturalEndpoints is rather inefficient > --- > > Key: CASSANDRA-10200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10200 > Project: Cassandra > Issue Type: Improvement >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Minor > Fix For: 3.0.1, 3.1 > > > The method is much more complicated than it needs to be and creates too many > maps and sets. The code is easy to simplify if we use the known number of > racks and nodes per datacentre to choose what to do in advance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10541) cqlshlib tests cannot run on Windows
[
https://issues.apache.org/jira/browse/CASSANDRA-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008770#comment-15008770
]
Paulo Motta commented on CASSANDRA-10541:
-
Most of the tests (except the auto-completion suite) were fixed on Windows with
{{subprocess}}. Luckily the code was well modularized and allowed to replace
the posix-only pty terminal-access module with a custom WinPty emulation based
on {{subprocess}}.
The problem of accessing a subprocess interactively with {{subprocess.PIPE}} is
that {{subproc.stdout.read}} blocks until new data is present, so I used a
trick (described in [this blog
post|http://eyalarubas.com/python-subproc-nonblock.html]) that uses another
thread to read characters in the background from stdout and put them in a queue
which is consumed by the testing thread without blocking. This trick is not
necessary in dtests because all input is piped to the process, which processes
it and exits, and then all the output is verified at once. The cqlshlib tests
executes each command interactively and checks the output after each command.
Some other fixes:
* Replace the symlink trick to load cqlsh as a module with creating an empty
{{__init__.py}} file during tests on {{cassandra/bin}}, so python will allow
loading {{cqlsh.py}} as a module on both windows and Linux.
* Replaced the signal-based timeout handling on Windows with the eventlet
library, which is now necessary to run cqlshlib tests on Windows.
Although this was sufficient to fix the {{test_cqlsh_output.py}} and
{{test_cql_parsing.py}} suites, it did not fix the command completion tests of
{{test_cqlsh_completion.py}}, probably because readline disables
auto-completing from stdin piped input. For now, I disabled these tests on
Windows and created CASSANDRA-10720 to create platform-independent
auto-completion tests.
[~mambocab] [~philipthompson] any volunteer to review? :-) Also, could we set
up a Windows/Linux cqlshlib cassci run with this branch?
Tests will appear shortly below:
||2.2||3.0||trunk||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-10541]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-10541]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-10541]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10541-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10541-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10541-testall/lastCompletedBuild/testReport/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10541-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10541-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10541-dtest/lastCompletedBuild/testReport/]|
> cqlshlib tests cannot run on Windows
>
>
> Key: CASSANDRA-10541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10541
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Benjamin Lerer
>Assignee: Paulo Motta
>Priority: Minor
> Labels: cqlsh, windows
>
> If I try to run the {{cqlshlib}} tests on Windows, I got the following error:
> {quote}
> ==
> ERROR: Failure: AttributeError ('module' object has no attribute 'symlink')
> --
> Traceback (most recent call last):
> File "C:\Python27\lib\site-packages\nose\loader.py", line 414, in
> loadTestsFromName
> addr.filename, addr.module)
> File "C:\Python27\lib\site-packages\nose\importer.py", line 47, in
> importFromPath
> return self.importFromDir(dir_path, fqname)
> File "C:\Python27\lib\site-packages\nose\importer.py", line 94, in
> importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
> File "[...]\pylib\cqlshlib\test\__init__.py", line 17, in
> from .cassconnect import create_test_db, remove_test_db
> File "[...]\pylib\cqlshlib\test\cassconnect.py", line 22, in
> from .basecase import cql, cqlsh, cqlshlog, TEST_HOST, TEST_PORT, rundir
> File "[...]\pylib\cqlshlib\test\basecase.py", line 43, in
> os.symlink(path_to_cqlsh, modulepath)
> AttributeError: 'module' object has no attribute 'symlink'
> --
> Ran 1 test in 0.002s
> FAILED (errors=1)
> {quote}
> The problem comes from the fact tha Windows has no
[jira] [Updated] (CASSANDRA-10719) Inconsistencies within CQL 'describe', and CQL docs/'help describe'
[ https://issues.apache.org/jira/browse/CASSANDRA-10719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-10719: - Assignee: Michael Edge > Inconsistencies within CQL 'describe', and CQL docs/'help describe' > --- > > Key: CASSANDRA-10719 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10719 > Project: Cassandra > Issue Type: Improvement > Components: CQL, Documentation and Website >Reporter: Michael Edge >Assignee: Michael Edge >Priority: Minor > Fix For: 3.x > > > While investigating the issue CASSANDRA-9678 I noticed a number of > inconsistencies in the way 'describe' operates, so I'm opening a new issue to > address these. This issue will also address CASSANDRA-9678. > I'd be happy to work on this. > There are a number of inconsistencies in the way 'describe' operates within > cqlsh, and also in the 'help describe' description within cqlsh compared to > the CQL documentation (at > http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) > For example, 'desc functions' will list all functions for all keyspaces > regardless of whether there is a current keyspace or not, whereas 'desc > tables' or 'desc types' will list only the tables or types for the current > keyspace. > Some commands exist in cqlsh that are not in the CQL documentation, nor in > the 'help describe' description. For example, 'desc functions' is a valid > CQLSH command but does not appear in either the CQL docs or 'help describe'. > I suggest we align the way the 'describe' command works so that it works > consistently regardless of whether it is describing a table, type, function > or any other database object, and also update the CQL and 'help describe' > docs to match. Since 'describe tables' and it's variants has been around the > longest we should probably align other 'describe' commands to 'describe > tables'. > My preliminary analysis has shown at least the following inconsistencies: > - 'desc functions' (with current keyspace), differs from 'desc tables' and > 'desc types'. > - a number of commands are missing from the CQL docs or 'help describe, such > as: desc table ., desc functions (no current keyspace), > desc function , desc type ., etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10249) Make buffered read size configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-10249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-10249: -- Fix Version/s: 2.2.x > Make buffered read size configurable > > > Key: CASSANDRA-10249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10249 > Project: Cassandra > Issue Type: Improvement >Reporter: Albert P Tobey >Assignee: Albert P Tobey > Fix For: 2.1.x, 2.2.x > > Attachments: Screenshot 2015-09-11 09.32.04.png, Screenshot > 2015-09-11 09.34.10.png, patched-2.1.9-dstat-lvn10.png, > stock-2.1.9-dstat-lvn10.png, yourkit-screenshot.png > > > On read workloads, Cassandra 2.1 reads drastically more data than it emits > over the network. This causes problems throughput the system by wasting disk > IO and causing unnecessary GC. > I have reproduce the issue on clusters and locally with a single instance. > The only requirement to reproduce the issue is enough data to blow through > the page cache. The default schema and data size with cassandra-stress is > sufficient for exposing the issue. > With stock 2.1.9 I regularly observed anywhere from 300:1 to 500 > disk:network ratio. That is to say, for 1MB/s of network IO, Cassandra was > doing 300-500MB/s of disk reads, saturating the drive. > After applying this patch for standard IO mode > https://gist.github.com/tobert/10c307cf3709a585a7cf the ratio fell to around > 100:1 on my local test rig. Latency improved considerably and GC became a lot > less frequent. > I tested with 512 byte reads as well, but got the same performance, which > makes sense since all HDD and SSD made in the last few years have a 4K block > size (many of them lie and say 512). > I'm re-running the numbers now and will post them tomorrow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10249) Make buffered read size configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-10249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-10249: -- Flags: (was: Patch) > Make buffered read size configurable > > > Key: CASSANDRA-10249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10249 > Project: Cassandra > Issue Type: Improvement >Reporter: Albert P Tobey >Assignee: Albert P Tobey > Fix For: 2.1.x, 2.2.x > > Attachments: Screenshot 2015-09-11 09.32.04.png, Screenshot > 2015-09-11 09.34.10.png, patched-2.1.9-dstat-lvn10.png, > stock-2.1.9-dstat-lvn10.png, yourkit-screenshot.png > > > On read workloads, Cassandra 2.1 reads drastically more data than it emits > over the network. This causes problems throughput the system by wasting disk > IO and causing unnecessary GC. > I have reproduce the issue on clusters and locally with a single instance. > The only requirement to reproduce the issue is enough data to blow through > the page cache. The default schema and data size with cassandra-stress is > sufficient for exposing the issue. > With stock 2.1.9 I regularly observed anywhere from 300:1 to 500 > disk:network ratio. That is to say, for 1MB/s of network IO, Cassandra was > doing 300-500MB/s of disk reads, saturating the drive. > After applying this patch for standard IO mode > https://gist.github.com/tobert/10c307cf3709a585a7cf the ratio fell to around > 100:1 on my local test rig. Latency improved considerably and GC became a lot > less frequent. > I tested with 512 byte reads as well, but got the same performance, which > makes sense since all HDD and SSD made in the last few years have a 4K block > size (many of them lie and say 512). > I'm re-running the numbers now and will post them tomorrow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10249) Make buffered read size configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-10249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008756#comment-15008756 ] Aleksey Yeschenko edited comment on CASSANDRA-10249 at 11/17/15 2:52 PM: - I don't know where you got the 4K number for 2.2. So a patch for 2.2 is actually required. Anyway, pushed a 2.1 and a 2.2 branch (with some minor changes to the former to make it idiomatic), now awaiting CI results. ||branch||testall||dtest|| |[10249-2.1|https://github.com/iamaleksey/cassandra/tree/10249-2.1]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.1-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.1-dtest]| |[10249-2.2|https://github.com/iamaleksey/cassandra/tree/10249-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.2-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.2-dtest]| was (Author: iamaleksey): I'm don't know where you got the 4K number for 2.2. So a patch for 2.2 is actually required. Anyway, pushed a 2.1 and a 2.2 branch (with some minor changes to the former to make it idiomatic), now awaiting CI results. ||branch||testall||dtest|| |[10249-2.1|https://github.com/iamaleksey/cassandra/tree/10249-2.1]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.1-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.1-dtest]| |[10249-2.2|https://github.com/iamaleksey/cassandra/tree/10249-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.2-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.2-dtest]| > Make buffered read size configurable > > > Key: CASSANDRA-10249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10249 > Project: Cassandra > Issue Type: Improvement >Reporter: Albert P Tobey >Assignee: Albert P Tobey > Fix For: 2.1.x, 2.2.x > > Attachments: Screenshot 2015-09-11 09.32.04.png, Screenshot > 2015-09-11 09.34.10.png, patched-2.1.9-dstat-lvn10.png, > stock-2.1.9-dstat-lvn10.png, yourkit-screenshot.png > > > On read workloads, Cassandra 2.1 reads drastically more data than it emits > over the network. This causes problems throughput the system by wasting disk > IO and causing unnecessary GC. > I have reproduce the issue on clusters and locally with a single instance. > The only requirement to reproduce the issue is enough data to blow through > the page cache. The default schema and data size with cassandra-stress is > sufficient for exposing the issue. > With stock 2.1.9 I regularly observed anywhere from 300:1 to 500 > disk:network ratio. That is to say, for 1MB/s of network IO, Cassandra was > doing 300-500MB/s of disk reads, saturating the drive. > After applying this patch for standard IO mode > https://gist.github.com/tobert/10c307cf3709a585a7cf the ratio fell to around > 100:1 on my local test rig. Latency improved considerably and GC became a lot > less frequent. > I tested with 512 byte reads as well, but got the same performance, which > makes sense since all HDD and SSD made in the last few years have a 4K block > size (many of them lie and say 512). > I'm re-running the numbers now and will post them tomorrow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10249) Make buffered read size configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-10249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008756#comment-15008756 ] Aleksey Yeschenko commented on CASSANDRA-10249: --- I'm don't know where you got the 4K number for 2.2. So a patch for 2.2 is actually required. Anyway, pushed a 2.1 and a 2.2 branch (with some minor changes to the former to make it idiomatic), now awaiting CI results. ||branch||testall||dtest|| |[10249-2.1|https://github.com/iamaleksey/cassandra/tree/10249-2.1]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.1-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.1-dtest]| |[10249-2.2|https://github.com/iamaleksey/cassandra/tree/10249-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.2-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10249-2.2-dtest]| > Make buffered read size configurable > > > Key: CASSANDRA-10249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10249 > Project: Cassandra > Issue Type: Improvement >Reporter: Albert P Tobey >Assignee: Albert P Tobey > Fix For: 2.1.x > > Attachments: Screenshot 2015-09-11 09.32.04.png, Screenshot > 2015-09-11 09.34.10.png, patched-2.1.9-dstat-lvn10.png, > stock-2.1.9-dstat-lvn10.png, yourkit-screenshot.png > > > On read workloads, Cassandra 2.1 reads drastically more data than it emits > over the network. This causes problems throughput the system by wasting disk > IO and causing unnecessary GC. > I have reproduce the issue on clusters and locally with a single instance. > The only requirement to reproduce the issue is enough data to blow through > the page cache. The default schema and data size with cassandra-stress is > sufficient for exposing the issue. > With stock 2.1.9 I regularly observed anywhere from 300:1 to 500 > disk:network ratio. That is to say, for 1MB/s of network IO, Cassandra was > doing 300-500MB/s of disk reads, saturating the drive. > After applying this patch for standard IO mode > https://gist.github.com/tobert/10c307cf3709a585a7cf the ratio fell to around > 100:1 on my local test rig. Latency improved considerably and GC became a lot > less frequent. > I tested with 512 byte reads as well, but got the same performance, which > makes sense since all HDD and SSD made in the last few years have a 4K block > size (many of them lie and say 512). > I'm re-running the numbers now and will post them tomorrow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9644) DTCS configuration proposals for handling consequences of repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008734#comment-15008734 ] Marcus Eriksson commented on CASSANDRA-9644: [~philipthompson] can't reproduce locally, could you send me the logs? > DTCS configuration proposals for handling consequences of repairs > - > > Key: CASSANDRA-9644 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9644 > Project: Cassandra > Issue Type: Improvement >Reporter: Antti Nissinen >Assignee: Marcus Eriksson > Labels: compaction, dtcs > Fix For: 2.1.x, 3.x > > Attachments: node0_20150621_1646_time_graph.txt, > node0_20150621_2320_time_graph.txt, node0_20150623_1526_time_graph.txt, > node1_20150621_1646_time_graph.txt, node1_20150621_2320_time_graph.txt, > node1_20150623_1526_time_graph.txt, node2_20150621_1646_time_graph.txt, > node2_20150621_2320_time_graph.txt, node2_20150623_1526_time_graph.txt, > nodetool status infos.txt, sstable_compaction_trace.txt, > sstable_compaction_trace_snipped.txt, sstable_counts.jpg > > > This is a document bringing up some issues when DTCS is used to compact time > series data in a three node cluster. The DTCS is currently configured with a > few parameters that are making the configuration fairly simple, but might > cause problems in certain special cases like recovering from the flood of > small SSTables due to repair operation. We are suggesting some ideas that > might be a starting point for further discussions. Following sections are > containing: > - Description of the cassandra setup > - Feeding process of the data > - Failure testing > - Issues caused by the repair operations for the DTCS > - Proposal for the DTCS configuration parameters > Attachments are included to support the discussion and there is a separate > section giving explanation for those. > Cassandra setup and data model > - Cluster is composed from three nodes running Cassandra 2.1.2. Replication > factor is two and read and write consistency levels are ONE. > - Data is time series data. Data is saved so that one row contains a certain > time span of data for a given metric ( 20 days in this case). The row key > contains information about the start time of the time span and metrix name. > Column name gives the offset from the beginning of time span. Column time > stamp is set to correspond time stamp when adding together the timestamp from > the row key and the offset (the actual time stamp of data point). Data model > is analog to KairosDB implementation. > - Average sampling rate is 10 seconds varying significantly from metric to > metric. > - 100 000 metrics are fed to the Cassandra. > - max_sstable_age_days is set to 5 days (objective is to keep SStable files > in manageable size, around 50 GB) > - TTL is not in use in the test. > Procedure for the failure test. > - Data is first dumped to Cassandra for 11 days and the data dumping is > stopped so that DTCS will have a change to finish all compactions. Data is > dumped with "fake timestamps" so that column time stamp is set when data is > written to Cassandra. > - One of the nodes is taken down and new data is dumped on top of the earlier > data covering couple of hours worth of data (faked time stamps). > - Dumping is stopped and the node is kept down for few hours. > - Node is taken up and the "nodetool repair" is applied on the node that was > down. > Consequences > - Repair operation will lead to massive amount of new SStables far back in > the history. New SStables are covering similar time spans than the files that > were created by DTCS before the shutdown of one of the nodes. > - To be able to compact the small files the max_sstable_age_days should be > increased to allow compaction to handle the files. However, the in a > practical case the time window will increase so large that generated files > will be huge that is not desirable. The compaction also combines together one > very large file with a bunch of small files in several phases that is not > effective. Generating really large files may also lead to out of disc space > problems. > - See the list of time graphs later in the document. > Improvement proposals for the DTCS configuration > Below is a list of desired properties for the configuration. Current > parameters are mentioned if available. > - Initial window size (currently:base_time_seconds) > - The amount of similar size windows for the bucketing (currently: > min_threshold) > - The multiplier for the window size when increased (currently: > min_threshold). This we would like to be independent from the min_threshold > parameter so that you could actually control the rate how fast the window > size is increased. > - Maximum length of
[jira] [Updated] (CASSANDRA-9043) Improve COPY command to work with Counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-9043: - Issue Type: Improvement (was: New Feature) > Improve COPY command to work with Counter columns > - > > Key: CASSANDRA-9043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9043 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Assignee: ZhaoYang >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: CASSANDRA-9043(2.1.8).patch, CASSANDRA-9043-trunk.patch > > > Noticed today that the copy command doesn't work with counter column tables. > This makes sense given that we need to use UPDATE instead of INSERT with > counters. > Given that we're making improvements in the COPY command in 3.0 with > CASSANDRA-7405, can we also tweak it to work with counters? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10720) Create platform-independent cqlsh auto-completion tests
Paulo Motta created CASSANDRA-10720:
---
Summary: Create platform-independent cqlsh auto-completion tests
Key: CASSANDRA-10720
URL: https://issues.apache.org/jira/browse/CASSANDRA-10720
Project: Cassandra
Issue Type: Test
Components: Tools
Reporter: Paulo Motta
Priority: Minor
Currently the cqlsh auto-completion tests are based on the {{pty}} package,
only available on Linux.
We should be able to make platform-independent tests by testing the
{{cqlsh.complete}} or {{cqlsh.find_completions}} methods directly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9043) Improve COPY command to work with Counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-9043: - Priority: Minor (was: Major) > Improve COPY command to work with Counter columns > - > > Key: CASSANDRA-9043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9043 > Project: Cassandra > Issue Type: New Feature >Reporter: Sebastian Estevez >Assignee: ZhaoYang >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: CASSANDRA-9043(2.1.8).patch, CASSANDRA-9043-trunk.patch > > > Noticed today that the copy command doesn't work with counter column tables. > This makes sense given that we need to use UPDATE instead of INSERT with > counters. > Given that we're making improvements in the COPY command in 3.0 with > CASSANDRA-7405, can we also tweak it to work with counters? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9043) Improve COPY command to work with Counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-9043: - Fix Version/s: (was: 2.1.8) 3.x > Improve COPY command to work with Counter columns > - > > Key: CASSANDRA-9043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9043 > Project: Cassandra > Issue Type: New Feature >Reporter: Sebastian Estevez >Assignee: ZhaoYang > Labels: lhf > Fix For: 3.x > > Attachments: CASSANDRA-9043(2.1.8).patch, CASSANDRA-9043-trunk.patch > > > Noticed today that the copy command doesn't work with counter column tables. > This makes sense given that we need to use UPDATE instead of INSERT with > counters. > Given that we're making improvements in the COPY command in 3.0 with > CASSANDRA-7405, can we also tweak it to work with counters? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10653) Remove dependency on jgrapht for UDT resolution
[ https://issues.apache.org/jira/browse/CASSANDRA-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-10653: -- Issue Type: Bug (was: Improvement) > Remove dependency on jgrapht for UDT resolution > --- > > Key: CASSANDRA-10653 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10653 > Project: Cassandra > Issue Type: Bug > Components: Distributed Metadata >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 3.0.1, 3.1 > > > Now that the java-driver no longer pulls it as a dependency, it is silly to > pull a whole library for resolving UDTs dependencies. > Should rewrite the resolution code without jgrapht (maybe reuse whatever code > java-driver ended up writing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10653) Remove dependency on jgrapht for UDT resolution
[
https://issues.apache.org/jira/browse/CASSANDRA-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008669#comment-15008669
]
Aleksey Yeschenko commented on CASSANDRA-10653:
---
Pushed a branch to my repo, tests running:
||branch||testall||dtest||
|[10653|https://github.com/iamaleksey/cassandra/tree/10653]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10653-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-10653-dtest]|
It's not even taking that much more code, when hand-rolled. Now that the driver
doesn't use jgrapht, it's an utter overkill to keep it (though I still maintain
that v1 was ugly as hell).
Test coverage piggybacks on existing {{UserTypesTest}}, {{LegacyMigratorTest}}
and dtests. CASSANDRA-10364 will be adding more, including more complex UDT
graphs.
> Remove dependency on jgrapht for UDT resolution
> ---
>
> Key: CASSANDRA-10653
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10653
> Project: Cassandra
> Issue Type: Improvement
> Components: Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.0.x
>
>
> Now that the java-driver no longer pulls it as a dependency, it is silly to
> pull a whole library for resolving UDTs dependencies.
> Should rewrite the resolution code without jgrapht (maybe reuse whatever code
> java-driver ended up writing).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10653) Remove dependency on jgrapht for UDT resolution
[ https://issues.apache.org/jira/browse/CASSANDRA-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-10653: -- Reviewer: Sylvain Lebresne > Remove dependency on jgrapht for UDT resolution > --- > > Key: CASSANDRA-10653 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10653 > Project: Cassandra > Issue Type: Improvement > Components: Distributed Metadata >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 3.0.x > > > Now that the java-driver no longer pulls it as a dependency, it is silly to > pull a whole library for resolving UDTs dependencies. > Should rewrite the resolution code without jgrapht (maybe reuse whatever code > java-driver ended up writing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10719) Inconsistencies within CQL 'describe', and CQL docs/'help describe'
[ https://issues.apache.org/jira/browse/CASSANDRA-10719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Edge updated CASSANDRA-10719: - Description: While investigating the issue CASSANDRA-9678 I noticed a number of inconsistencies in the way 'describe' operates, so I'm opening a new issue to address these. This issue will also address CASSANDRA-9678. I'd be happy to work on this. There are a number of inconsistencies in the way 'describe' operates within cqlsh, and also in the 'help describe' description within cqlsh compared to the CQL documentation (at http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) For example, 'desc functions' will list all functions for all keyspaces regardless of whether there is a current keyspace or not, whereas 'desc tables' or 'desc types' will list only the tables or types for the current keyspace. Some commands exist in cqlsh that are not in the CQL documentation, nor in the 'help describe' description. For example, 'desc functions' is a valid CQLSH command but does not appear in either the CQL docs or 'help describe'. I suggest we align the way the 'describe' command works so that it works consistently regardless of whether it is describing a table, type, function or any other database object, and also update the CQL and 'help describe' docs to match. Since 'describe tables' and it's variants has been around the longest we should probably align other 'describe' commands to 'describe tables'. My preliminary analysis has shown at least the following inconsistencies: - 'desc functions' (with current keyspace), differs from 'desc tables' and 'desc types'. - a number of commands are missing from the CQL docs or 'help describe, such as: desc table ., desc functions (no current keyspace), desc function , desc type ., etc. was: While investigating the issue CASSANDRA-9678 I noticed a number of inconsistencies in the way 'describe' operates, so I'm opening a new issue to address these. This issue will also address Cassandra-9678. I'd be happy to work on this. There are a number of inconsistencies in the way 'describe' operates within cqlsh, and also in the 'help describe' description within cqlsh compared to the CQL documentation (at http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) For example, 'desc functions' will list all functions for all keyspaces regardless of whether there is a current keyspace or not, whereas 'desc tables' or 'desc types' will list only the tables or types for the current keyspace. Some commands exist in cqlsh that are not in the CQL documentation, nor in the 'help describe' description. For example, 'desc functions' is a valid CQLSH command but does not appear in either the CQL docs or 'help describe'. I suggest we align the way the 'describe' command works so that it works consistently regardless of whether it is describing a table, type, function or any other database object, and also update the CQL and 'help describe' docs to match. Since 'describe tables' and it's variants has been around the longest we should probably align other 'describe' commands to 'describe tables'. My preliminary analysis has shown at least the following inconsistencies: - 'desc functions' (with current keyspace), differs from 'desc tables' and 'desc types'. - a number of commands are missing from the CQL docs or 'help describe, such as: desc table ., desc functions (no current keyspace), desc function , desc type ., etc. > Inconsistencies within CQL 'describe', and CQL docs/'help describe' > --- > > Key: CASSANDRA-10719 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10719 > Project: Cassandra > Issue Type: Improvement > Components: CQL, Documentation and Website >Reporter: Michael Edge >Priority: Minor > Fix For: 3.x > > > While investigating the issue CASSANDRA-9678 I noticed a number of > inconsistencies in the way 'describe' operates, so I'm opening a new issue to > address these. This issue will also address CASSANDRA-9678. > I'd be happy to work on this. > There are a number of inconsistencies in the way 'describe' operates within > cqlsh, and also in the 'help describe' description within cqlsh compared to > the CQL documentation (at > http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) > For example, 'desc functions' will list all functions for all keyspaces > regardless of whether there is a current keyspace or not, whereas 'desc > tables' or 'desc types' will list only the tables or types for the current > keyspace. > Some commands exist in cqlsh that are not in the CQL documentation, nor in > the 'help describe' description. For example, 'desc functions' is a valid > CQLSH command but does not appear
[jira] [Updated] (CASSANDRA-10719) Inconsistencies within CQL 'describe', and CQL docs/'help describe'
[ https://issues.apache.org/jira/browse/CASSANDRA-10719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Edge updated CASSANDRA-10719: - Description: While investigating the issue CASSANDRA-9678 I noticed a number of inconsistencies in the way 'describe' operates, so I'm opening a new issue to address these. This issue will also address Cassandra-9678. I'd be happy to work on this. There are a number of inconsistencies in the way 'describe' operates within cqlsh, and also in the 'help describe' description within cqlsh compared to the CQL documentation (at http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) For example, 'desc functions' will list all functions for all keyspaces regardless of whether there is a current keyspace or not, whereas 'desc tables' or 'desc types' will list only the tables or types for the current keyspace. Some commands exist in cqlsh that are not in the CQL documentation, nor in the 'help describe' description. For example, 'desc functions' is a valid CQLSH command but does not appear in either the CQL docs or 'help describe'. I suggest we align the way the 'describe' command works so that it works consistently regardless of whether it is describing a table, type, function or any other database object, and also update the CQL and 'help describe' docs to match. Since 'describe tables' and it's variants has been around the longest we should probably align other 'describe' commands to 'describe tables'. My preliminary analysis has shown at least the following inconsistencies: - 'desc functions' (with current keyspace), differs from 'desc tables' and 'desc types'. - a number of commands are missing from the CQL docs or 'help describe, such as: desc table ., desc functions (no current keyspace), desc function , desc type ., etc. was: While investigating the issue Cassandra-9678 I noticed a number of inconsistencies in the way 'describe' operates, so I'm opening a new issue to address these. This issue will also address Cassandra-9678. I'd be happy to work on this. There are a number of inconsistencies in the way 'describe' operates within cqlsh, and also in the 'help describe' description within cqlsh compared to the CQL documentation (at http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) For example, 'desc functions' will list all functions for all keyspaces regardless of whether there is a current keyspace or not, whereas 'desc tables' or 'desc types' will list only the tables or types for the current keyspace. Some commands exist in cqlsh that are not in the CQL documentation, nor in the 'help describe' description. For example, 'desc functions' is a valid CQLSH command but does not appear in either the CQL docs or 'help describe'. I suggest we align the way the 'describe' command works so that it works consistently regardless of whether it is describing a table, type, function or any other database object, and also update the CQL and 'help describe' docs to match. Since 'describe tables' and it's variants has been around the longest we should probably align other 'describe' commands to 'describe tables'. My preliminary analysis has shown at least the following inconsistencies: - 'desc functions' (with current keyspace), differs from 'desc tables' and 'desc types'. - a number of commands are missing from the CQL docs or 'help describe, such as: desc table ., desc functions (no current keyspace), desc function , desc type ., etc. > Inconsistencies within CQL 'describe', and CQL docs/'help describe' > --- > > Key: CASSANDRA-10719 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10719 > Project: Cassandra > Issue Type: Improvement > Components: CQL, Documentation and Website >Reporter: Michael Edge >Priority: Minor > Fix For: 3.x > > > While investigating the issue CASSANDRA-9678 I noticed a number of > inconsistencies in the way 'describe' operates, so I'm opening a new issue to > address these. This issue will also address Cassandra-9678. > I'd be happy to work on this. > There are a number of inconsistencies in the way 'describe' operates within > cqlsh, and also in the 'help describe' description within cqlsh compared to > the CQL documentation (at > http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) > For example, 'desc functions' will list all functions for all keyspaces > regardless of whether there is a current keyspace or not, whereas 'desc > tables' or 'desc types' will list only the tables or types for the current > keyspace. > Some commands exist in cqlsh that are not in the CQL documentation, nor in > the 'help describe' description. For example, 'desc functions' is a valid > CQLSH command but does not appear
[jira] [Commented] (CASSANDRA-9678) help describe is missing the documentation for UDTs
[ https://issues.apache.org/jira/browse/CASSANDRA-9678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008651#comment-15008651 ] Michael Edge commented on CASSANDRA-9678: - While investigating this issue I noticed quite a few other inconsistencies between the way 'describe' operates and what is stated in the CQL docs and 'help describe', so I raised another issue (CASSANDRA-10719) to encompass all of these. I'd be happy to fix this issue, CASSANDRA-9678, along with the wider issues listed in CASSANDRA-10719. > help describe is missing the documentation for UDTs > --- > > Key: CASSANDRA-9678 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9678 > Project: Cassandra > Issue Type: Improvement >Reporter: Peter Halliday >Priority: Minor > Labels: cqlsh > Fix For: 2.1.x > > > On 2.1 when you type help describe, the documentation for describe types is > missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9474) DC/Rack property changed on live system
[ https://issues.apache.org/jira/browse/CASSANDRA-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008640#comment-15008640 ] Marcus Olsson commented on CASSANDRA-9474: -- [~pauloricardomg] In the 2.2 [NEWS.txt|https://github.com/apache/cassandra/commit/7cab3272455bdd16b639c510416ae339a8613414#diff-4302f2407249672d7845cd58027ff6e9R16] it seems like the part with cassandra.ignore_rack was removed as well, so it might have been done on purpose? Ping [~iamaleksey] I'll add the flags and checks to CassandraDaemon, will probably have a patch set up soon. :) > DC/Rack property changed on live system > --- > > Key: CASSANDRA-9474 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9474 > Project: Cassandra > Issue Type: Improvement > Environment: Cassandra 2.1.5 >Reporter: Marcus Olsson >Assignee: Marcus Olsson > Fix For: 2.1.x > > Attachments: cassandra-2.1-9474.patch, > cassandra-2.1-dc_rack_healthcheck.patch > > > When using GossipingPropertyFileSnitch it is possible to change the data > center and rack of a live node by changing the cassandra-rackdc.properties > file. Should this really be possible? In the documentation at > http://docs.datastax.com/en/cassandra/2.1/cassandra/initialize/initializeMultipleDS.html > it's stated that you should ??Choose the name carefully; renaming a data > center is not possible??, but with this functionality it doesn't seem > impossible(maybe a bit hard with changing replication etc.). > This functionality was introduced by CASSANDRA-5897 so I'm guessing there is > some use case for this? > Personally I would want the DC/rack settings to be as restricted as the > cluster name, otherwise if a node could just join another data center without > removing it's local information couldn't it mess up the token ranges? And > suddenly the old data center/rack would loose 1 replica of all the data that > the node contains. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10719) Inconsistencies within CQL 'describe', and CQL docs/'help describe'
Michael Edge created CASSANDRA-10719: Summary: Inconsistencies within CQL 'describe', and CQL docs/'help describe' Key: CASSANDRA-10719 URL: https://issues.apache.org/jira/browse/CASSANDRA-10719 Project: Cassandra Issue Type: Improvement Components: CQL, Documentation and Website Reporter: Michael Edge Priority: Minor Fix For: 3.x While investigating the issue Cassandra-9678 I noticed a number of inconsistencies in the way 'describe' operates, so I'm opening a new issue to address these. This issue will also address Cassandra-9678. I'd be happy to work on this. There are a number of inconsistencies in the way 'describe' operates within cqlsh, and also in the 'help describe' description within cqlsh compared to the CQL documentation (at http://docs.datastax.com/en/cql/3.3/cql/cql_reference/describe_r1.html) For example, 'desc functions' will list all functions for all keyspaces regardless of whether there is a current keyspace or not, whereas 'desc tables' or 'desc types' will list only the tables or types for the current keyspace. Some commands exist in cqlsh that are not in the CQL documentation, nor in the 'help describe' description. For example, 'desc functions' is a valid CQLSH command but does not appear in either the CQL docs or 'help describe'. I suggest we align the way the 'describe' command works so that it works consistently regardless of whether it is describing a table, type, function or any other database object, and also update the CQL and 'help describe' docs to match. Since 'describe tables' and it's variants has been around the longest we should probably align other 'describe' commands to 'describe tables'. My preliminary analysis has shown at least the following inconsistencies: - 'desc functions' (with current keyspace), differs from 'desc tables' and 'desc types'. - a number of commands are missing from the CQL docs or 'help describe, such as: desc table ., desc functions (no current keyspace), desc function , desc type ., etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9474) DC/Rack property changed on live system
[
https://issues.apache.org/jira/browse/CASSANDRA-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008596#comment-15008596
]
Paulo Motta commented on CASSANDRA-9474:
[~molsson] Thanks for spotting this. It seems the flag was removed during the
[up-merge|https://github.com/apache/cassandra/commit/7cab3272455bdd16b639c510416ae339a8613414#diff-ce3f6856b405c96859d9a50d9977e0b9]
of CASSANDRA-10242 to 2.2, since there was also a minor change in the
implementation, where the check was moved from SystemKeyspace to
CassandraDaemon (which is a more adequate place for this check anyway, so you
should also place the dc check there).
I believe it was an oversight, so could you please add
{{cassandra.ignore_rack}} flag back (in addition to the new
{{cassandra.ignore_dc}} flag)? since it's already documented on 2.1 NEWS.txt,
we should still keep both flags. Also could you mention these flags in the
error messages? So if somebody Knows What Is Doing©, then it will find out
about the flags.
> DC/Rack property changed on live system
> ---
>
> Key: CASSANDRA-9474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9474
> Project: Cassandra
> Issue Type: Improvement
> Environment: Cassandra 2.1.5
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
> Fix For: 2.1.x
>
> Attachments: cassandra-2.1-9474.patch,
> cassandra-2.1-dc_rack_healthcheck.patch
>
>
> When using GossipingPropertyFileSnitch it is possible to change the data
> center and rack of a live node by changing the cassandra-rackdc.properties
> file. Should this really be possible? In the documentation at
> http://docs.datastax.com/en/cassandra/2.1/cassandra/initialize/initializeMultipleDS.html
> it's stated that you should ??Choose the name carefully; renaming a data
> center is not possible??, but with this functionality it doesn't seem
> impossible(maybe a bit hard with changing replication etc.).
> This functionality was introduced by CASSANDRA-5897 so I'm guessing there is
> some use case for this?
> Personally I would want the DC/rack settings to be as restricted as the
> cluster name, otherwise if a node could just join another data center without
> removing it's local information couldn't it mess up the token ranges? And
> suddenly the old data center/rack would loose 1 replica of all the data that
> the node contains.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8879) Alter table on compact storage broken
[
https://issues.apache.org/jira/browse/CASSANDRA-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008576#comment-15008576
]
Aleksey Yeschenko commented on CASSANDRA-8879:
--
Thanks. I'll force-run dtests alone on 2.2 and 3.0 and commit once certain we
are cool there.
> Alter table on compact storage broken
> -
>
> Key: CASSANDRA-8879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8879
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nick Bailey
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: 8879-2.0.txt
>
>
> In 2.0 HEAD, alter table on compact storage tables seems to be broken. With
> the following table definition, altering the column breaks cqlsh and
> generates a stack trace in the log.
> {noformat}
> CREATE TABLE settings (
> key blob,
> column1 blob,
> value blob,
> PRIMARY KEY ((key), column1)
> ) WITH COMPACT STORAGE
> {noformat}
> {noformat}
> cqlsh:OpsCenter> alter table settings ALTER column1 TYPE ascii ;
> TSocket read 0 bytes
> cqlsh:OpsCenter> DESC TABLE settings;
> {noformat}
> {noformat}
> ERROR [Thrift:7] 2015-02-26 17:20:24,640 CassandraDaemon.java (line 199)
> Exception in thread Thread[Thrift:7,5,main]
> java.lang.AssertionError
> >...at
> >org.apache.cassandra.cql3.statements.AlterTableStatement.announceMigration(AlterTableStatement.java:198)
> >...at
> >org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:79)
> >...at
> >org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:158)
> >...at
> >org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:175)
> >...at
> >org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1958)
> >...at
> >org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4486)
> >...at
> >org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4470)
> >...at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> >...at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> >...at
> >org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
> >...at
> >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >...at
> >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >...at java.lang.Thread.run(Thread.java:724)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
[
https://issues.apache.org/jira/browse/CASSANDRA-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008567#comment-15008567
]
Mike Adamson commented on CASSANDRA-10717:
--
[~jlewandowski] I'm not that fussed because in your patch, if the
{{AuthProvider}} is set it will overwrite the {{PlainTextAuthProvider}} anyway.
The change I was proposing was to make this clearer in the code, that if the
user provided a username, password and {{AuthProvider}} then the provided
{{AuthProvider}} took precedence over the {{PlainTextAuthProvider}}.
> Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
> --
>
> Key: CASSANDRA-10717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
> Attachments: CqlConfigHelper_authentication_patch.patch
>
>
> Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
> anything but providing username and password in order to work with password
> protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code
> from Thrift to Java Driver, the user needs to explicitly select
> {{AuthProvider}} to make it work.
> I propose a simple change in {{CqlConfigHelper}} which make
> {{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}}
> if they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
[
https://issues.apache.org/jira/browse/CASSANDRA-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008552#comment-15008552
]
Jacek Lewandowski commented on CASSANDRA-10717:
---
[~mikea] - i'm afraid that it won't be very useful because of the following
reasons:
* the {{AuthProvider}} instance obtained few lines before, if it is
{{PlainTextAuthProvider}}, it is already initialised with username and
password, so the credentials are set in this case
* {{withCredentials}} method implicitly set the auth provider to be
{{PlainTextAuthProvider}}, so if there is already some other auth provider set,
it will be overridden by {{PlainTextAuthProvider}}; that method doesn't set
credentials in a generic way on the currently selected authenticator
> Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
> --
>
> Key: CASSANDRA-10717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
> Attachments: CqlConfigHelper_authentication_patch.patch
>
>
> Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
> anything but providing username and password in order to work with password
> protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code
> from Thrift to Java Driver, the user needs to explicitly select
> {{AuthProvider}} to make it work.
> I propose a simple change in {{CqlConfigHelper}} which make
> {{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}}
> if they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9355) RecoveryManagerTruncateTest fails in test-compression
[
https://issues.apache.org/jira/browse/CASSANDRA-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008547#comment-15008547
]
Branimir Lambov commented on CASSANDRA-9355:
Both options look good to me. I personally prefer the latter as that's what we
do in 2.2+, but it's up to you to choose the one to go with.
> RecoveryManagerTruncateTest fails in test-compression
> -
>
> Key: CASSANDRA-9355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9355
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
> Environment: 2.1 commit: ac70e37
>Reporter: Michael Shuler
>Assignee: Ariel Weisberg
> Fix For: 2.1.x
>
> Attachments: system.log
>
>
> {noformat}
> $ ant test-compression -Dtest.name=RecoveryManagerTruncateTest
> ...
> [junit] Testsuite: org.apache.cassandra.db.RecoveryManagerTruncateTest
> [junit] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 9.221 sec
> [junit]
> [junit] Testcase:
> testTruncatePointInTimeReplayList(org.apache.cassandra.db.RecoveryManagerTruncateTest):
>FAILED
> [junit]
> [junit] junit.framework.AssertionFailedError:
> [junit] at
> org.apache.cassandra.db.RecoveryManagerTruncateTest.testTruncatePointInTimeReplayList(RecoveryManagerTruncateTest.java:159)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.db.RecoveryManagerTruncateTest FAILED
> {noformat}
> system.log from just this failed test attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9474) DC/Rack property changed on live system
[ https://issues.apache.org/jira/browse/CASSANDRA-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008522#comment-15008522 ] Marcus Olsson commented on CASSANDRA-9474: -- I've started the implementation towards 2.2 and done a dtest for it. I have just one question regarding the cassandra.ignore_dc flag, in the 2.2 branch I can't seem to find the cassandra.ignore_rack flag [here|https://github.com/apache/cassandra/commit/7cab3272455bdd16b639c510416ae339a8613414], should I be consistent and not add it? > DC/Rack property changed on live system > --- > > Key: CASSANDRA-9474 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9474 > Project: Cassandra > Issue Type: Improvement > Environment: Cassandra 2.1.5 >Reporter: Marcus Olsson >Assignee: Marcus Olsson > Fix For: 2.1.x > > Attachments: cassandra-2.1-9474.patch, > cassandra-2.1-dc_rack_healthcheck.patch > > > When using GossipingPropertyFileSnitch it is possible to change the data > center and rack of a live node by changing the cassandra-rackdc.properties > file. Should this really be possible? In the documentation at > http://docs.datastax.com/en/cassandra/2.1/cassandra/initialize/initializeMultipleDS.html > it's stated that you should ??Choose the name carefully; renaming a data > center is not possible??, but with this functionality it doesn't seem > impossible(maybe a bit hard with changing replication etc.). > This functionality was introduced by CASSANDRA-5897 so I'm guessing there is > some use case for this? > Personally I would want the DC/rack settings to be as restricted as the > cluster name, otherwise if a node could just join another data center without > removing it's local information couldn't it mess up the token ranges? And > suddenly the old data center/rack would loose 1 replica of all the data that > the node contains. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Improve performance of the folderSize function
Repository: cassandra
Updated Branches:
refs/heads/trunk b59b26ba5 -> 6312f3355
Improve performance of the folderSize function
patch by briareus; reviewed by Stefania for CASSANDRA-10677
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6312f335
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6312f335
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6312f335
Branch: refs/heads/trunk
Commit: 6312f33559b3e66b5e4e2573a776eda0fc8e67dc
Parents: b59b26b
Author: Stefania Alborghetti
Authored: Tue Nov 17 10:40:19 2015 +0800
Committer: Sylvain Lebresne
Committed: Tue Nov 17 12:09:21 2015 +0100
--
CHANGES.txt | 1 +
.../org/apache/cassandra/io/util/FileUtils.java | 30 +++
.../apache/cassandra/io/util/FileUtilsTest.java | 52
3 files changed, 73 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6312f335/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0a7fb2d..b466e70 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.2
+ * Improve performance of the folderSize function (CASSANDRA-10677)
* Add support for type casting in selection clause (CASSANDRA-10310)
* Added graphing option to cassandra-stress (CASSANDRA-7918)
* Abort in-progress queries that time out (CASSANDRA-7392)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6312f335/src/java/org/apache/cassandra/io/util/FileUtils.java
--
diff --git a/src/java/org/apache/cassandra/io/util/FileUtils.java
b/src/java/org/apache/cassandra/io/util/FileUtils.java
index 46f2de5..d982e15 100644
--- a/src/java/org/apache/cassandra/io/util/FileUtils.java
+++ b/src/java/org/apache/cassandra/io/util/FileUtils.java
@@ -23,6 +23,7 @@ import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
+import java.nio.file.attribute.BasicFileAttributes;
import java.text.DecimalFormat;
import java.util.Arrays;
import java.util.Collections;
@@ -518,25 +519,34 @@ public class FileUtils
break;
}
}
+
/**
* Get the size of a directory in bytes
- * @param directory The directory for which we need size.
+ * @param folder The directory for which we need size.
* @return The size of the directory
*/
-public static long folderSize(File directory)
+public static long folderSize(File folder)
{
-long length = 0;
-for (File file : directory.listFiles())
+final long [] sizeArr = {0L};
+try
{
-if (file.isFile())
-length += file.length();
-else
-length += folderSize(file);
+Files.walkFileTree(folder.toPath(), new SimpleFileVisitor()
+{
+@Override
+public FileVisitResult visitFile(Path file,
BasicFileAttributes attrs)
+{
+sizeArr[0] += attrs.size();
+return FileVisitResult.CONTINUE;
+}
+});
}
-return length;
+catch (IOException e)
+{
+logger.error("Error while getting {} folder size. {}", folder, e);
+}
+return sizeArr[0];
}
-
public static void copyTo(DataInput in, OutputStream out, int length)
throws IOException
{
byte[] buffer = new byte[64 * 1024];
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6312f335/test/unit/org/apache/cassandra/io/util/FileUtilsTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/util/FileUtilsTest.java
b/test/unit/org/apache/cassandra/io/util/FileUtilsTest.java
index 7110504..ee33107 100644
--- a/test/unit/org/apache/cassandra/io/util/FileUtilsTest.java
+++ b/test/unit/org/apache/cassandra/io/util/FileUtilsTest.java
@@ -20,11 +20,20 @@ package org.apache.cassandra.io.util;
import java.io.File;
import java.io.IOException;
+import java.io.RandomAccessFile;
import java.nio.charset.Charset;
import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Arrays;
import org.junit.Test;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.cql3.CQLTester;
+import org.apache.cassandra.schema.SchemaKeyspace;
+import org.apache.cassandra.utils.FBUtilities;
+
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
@@ -52,4 +61,47 @@ public class FileUtilsTest
assertEquals(0, b.length);
}
+
[3/3] cassandra git commit: Merge branch 'cassandra-3.1' into trunk
Merge branch 'cassandra-3.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b59b26ba Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b59b26ba Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b59b26ba Branch: refs/heads/trunk Commit: b59b26ba5bf413b02ebde50f005b73cc148c31e2 Parents: 6645062 51a8711 Author: Sylvain Lebresne Authored: Tue Nov 17 11:52:57 2015 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 17 11:52:57 2015 +0100 -- test/unit/org/apache/cassandra/SchemaLoader.java | 2 +- test/unit/org/apache/cassandra/cql3/CQLTester.java | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) --
[1/3] cassandra git commit: Don't clean test folder twice in unit tests
Repository: cassandra
Updated Branches:
refs/heads/trunk 6645062ca -> b59b26ba5
Don't clean test folder twice in unit tests
patch by Stefania; reviewed by slebresne for CASSANDRA-10682
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a786081
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a786081
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a786081
Branch: refs/heads/trunk
Commit: 9a78608193f766af76cacfafc4f9a6635b1db4ee
Parents: eb8afb7
Author: Stefania Alborghetti
Authored: Fri Nov 13 13:22:13 2015 +0800
Committer: Sylvain Lebresne
Committed: Tue Nov 17 11:51:13 2015 +0100
--
test/unit/org/apache/cassandra/SchemaLoader.java | 2 +-
test/unit/org/apache/cassandra/cql3/CQLTester.java | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a786081/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java
b/test/unit/org/apache/cassandra/SchemaLoader.java
index f2c01e4..5d720c4 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -63,7 +63,7 @@ public class SchemaLoader
public static void prepareServer()
{
- CQLTester.prepareServer(false);
+ CQLTester.prepareServer();
}
public static void startGossiper()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a786081/test/unit/org/apache/cassandra/cql3/CQLTester.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/CQLTester.java
b/test/unit/org/apache/cassandra/cql3/CQLTester.java
index e92563b..fd0b086 100644
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@ -109,7 +109,7 @@ public abstract class CQLTester
PROTOCOL_VERSIONS = builder.build();
// Once per-JVM is enough
-prepareServer(true);
+prepareServer();
nativeAddr = InetAddress.getLoopbackAddress();
@@ -138,9 +138,9 @@ public abstract class CQLTester
// is not expected to be the same without preparation)
private boolean usePrepared = USE_PREPARED_VALUES;
-public static void prepareServer(boolean checkInit)
+public static void prepareServer()
{
-if (checkInit && isServerPrepared)
+if (isServerPrepared)
return;
// Cleanup first
[2/3] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1
Merge branch 'cassandra-3.0' into cassandra-3.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/51a87113 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/51a87113 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/51a87113 Branch: refs/heads/trunk Commit: 51a87113311e78e536ab3632499a186d455f59b8 Parents: 798a7a8 9a78608 Author: Sylvain Lebresne Authored: Tue Nov 17 11:52:41 2015 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 17 11:52:41 2015 +0100 -- test/unit/org/apache/cassandra/SchemaLoader.java | 2 +- test/unit/org/apache/cassandra/cql3/CQLTester.java | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) --
[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1
Merge branch 'cassandra-3.0' into cassandra-3.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/51a87113 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/51a87113 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/51a87113 Branch: refs/heads/cassandra-3.1 Commit: 51a87113311e78e536ab3632499a186d455f59b8 Parents: 798a7a8 9a78608 Author: Sylvain Lebresne Authored: Tue Nov 17 11:52:41 2015 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 17 11:52:41 2015 +0100 -- test/unit/org/apache/cassandra/SchemaLoader.java | 2 +- test/unit/org/apache/cassandra/cql3/CQLTester.java | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) --
cassandra git commit: Don't clean test folder twice in unit tests
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 eb8afb71f -> 9a7860819
Don't clean test folder twice in unit tests
patch by Stefania; reviewed by slebresne for CASSANDRA-10682
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a786081
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a786081
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a786081
Branch: refs/heads/cassandra-3.0
Commit: 9a78608193f766af76cacfafc4f9a6635b1db4ee
Parents: eb8afb7
Author: Stefania Alborghetti
Authored: Fri Nov 13 13:22:13 2015 +0800
Committer: Sylvain Lebresne
Committed: Tue Nov 17 11:51:13 2015 +0100
--
test/unit/org/apache/cassandra/SchemaLoader.java | 2 +-
test/unit/org/apache/cassandra/cql3/CQLTester.java | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a786081/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java
b/test/unit/org/apache/cassandra/SchemaLoader.java
index f2c01e4..5d720c4 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -63,7 +63,7 @@ public class SchemaLoader
public static void prepareServer()
{
- CQLTester.prepareServer(false);
+ CQLTester.prepareServer();
}
public static void startGossiper()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a786081/test/unit/org/apache/cassandra/cql3/CQLTester.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/CQLTester.java
b/test/unit/org/apache/cassandra/cql3/CQLTester.java
index e92563b..fd0b086 100644
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@ -109,7 +109,7 @@ public abstract class CQLTester
PROTOCOL_VERSIONS = builder.build();
// Once per-JVM is enough
-prepareServer(true);
+prepareServer();
nativeAddr = InetAddress.getLoopbackAddress();
@@ -138,9 +138,9 @@ public abstract class CQLTester
// is not expected to be the same without preparation)
private boolean usePrepared = USE_PREPARED_VALUES;
-public static void prepareServer(boolean checkInit)
+public static void prepareServer()
{
-if (checkInit && isServerPrepared)
+if (isServerPrepared)
return;
// Cleanup first
[1/2] cassandra git commit: Don't clean test folder twice in unit tests
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.1 798a7a8fe -> 51a871133
Don't clean test folder twice in unit tests
patch by Stefania; reviewed by slebresne for CASSANDRA-10682
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a786081
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a786081
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a786081
Branch: refs/heads/cassandra-3.1
Commit: 9a78608193f766af76cacfafc4f9a6635b1db4ee
Parents: eb8afb7
Author: Stefania Alborghetti
Authored: Fri Nov 13 13:22:13 2015 +0800
Committer: Sylvain Lebresne
Committed: Tue Nov 17 11:51:13 2015 +0100
--
test/unit/org/apache/cassandra/SchemaLoader.java | 2 +-
test/unit/org/apache/cassandra/cql3/CQLTester.java | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a786081/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java
b/test/unit/org/apache/cassandra/SchemaLoader.java
index f2c01e4..5d720c4 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -63,7 +63,7 @@ public class SchemaLoader
public static void prepareServer()
{
- CQLTester.prepareServer(false);
+ CQLTester.prepareServer();
}
public static void startGossiper()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a786081/test/unit/org/apache/cassandra/cql3/CQLTester.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/CQLTester.java
b/test/unit/org/apache/cassandra/cql3/CQLTester.java
index e92563b..fd0b086 100644
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@ -109,7 +109,7 @@ public abstract class CQLTester
PROTOCOL_VERSIONS = builder.build();
// Once per-JVM is enough
-prepareServer(true);
+prepareServer();
nativeAddr = InetAddress.getLoopbackAddress();
@@ -138,9 +138,9 @@ public abstract class CQLTester
// is not expected to be the same without preparation)
private boolean usePrepared = USE_PREPARED_VALUES;
-public static void prepareServer(boolean checkInit)
+public static void prepareServer()
{
-if (checkInit && isServerPrepared)
+if (isServerPrepared)
return;
// Cleanup first
[jira] [Updated] (CASSANDRA-10682) Fix timeouts in BeforeFirstTest
[
https://issues.apache.org/jira/browse/CASSANDRA-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-10682:
-
Reviewer: Sylvain Lebresne
> Fix timeouts in BeforeFirstTest
> ---
>
> Key: CASSANDRA-10682
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10682
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x
>
> Attachments:
> TEST-org.apache.cassandra.db.SinglePartitionSliceCommandTest.log,
> TEST-org.apache.cassandra.db.commitlog.CommitLogFailurePolicyTest.log
>
>
> Some unit tests fail with a timeout in {{BeforeFirstTest}}, see for example
> [here|http://cassci.datastax.com/job/cassandra-3.0_testall/242/testReport/org.apache.cassandra.db/SinglePartitionSliceCommandTest/BeforeFirstTest/].
>
> In the corresponding log file, attached, there is a {{NoSuchFileException}}
> which might be the cause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9043) Improve COPY command to work with Counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008393#comment-15008393 ] ZhaoYang commented on CASSANDRA-9043: - [~thobbs] Hi hobbs, can you check? thank you > Improve COPY command to work with Counter columns > - > > Key: CASSANDRA-9043 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9043 > Project: Cassandra > Issue Type: New Feature >Reporter: Sebastian Estevez >Assignee: ZhaoYang > Labels: lhf > Fix For: 2.1.8 > > Attachments: CASSANDRA-9043(2.1.8).patch, CASSANDRA-9043-trunk.patch > > > Noticed today that the copy command doesn't work with counter column tables. > This makes sense given that we need to use UPDATE instead of INSERT with > counters. > Given that we're making improvements in the COPY command in 3.0 with > CASSANDRA-7405, can we also tweak it to work with counters? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10718) Group pending compactions based on column family
Marcus Eriksson created CASSANDRA-10718:
---
Summary: Group pending compactions based on column family
Key: CASSANDRA-10718
URL: https://issues.apache.org/jira/browse/CASSANDRA-10718
Project: Cassandra
Issue Type: Improvement
Reporter: Marcus Eriksson
Priority: Minor
Fix For: 3.x
Currently we only give a global number on how many compactions are pending, we
should group this on a per-table basis, example:
{code}
$ nodetool compactionstats
pending tasks:
- keyspace1.standard1: 10
- other_ks.table: 2
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10718) Group pending compactions based on table
[
https://issues.apache.org/jira/browse/CASSANDRA-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-10718:
Summary: Group pending compactions based on table (was: Group pending
compactions based on column family)
> Group pending compactions based on table
>
>
> Key: CASSANDRA-10718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10718
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
>
> Currently we only give a global number on how many compactions are pending,
> we should group this on a per-table basis, example:
> {code}
> $ nodetool compactionstats
> pending tasks:
> - keyspace1.standard1: 10
> - other_ks.table: 2
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10682) Fix timeouts in BeforeFirstTest
[
https://issues.apache.org/jira/browse/CASSANDRA-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008368#comment-15008368
]
Stefania edited comment on CASSANDRA-10682 at 11/17/15 9:36 AM:
CI for 3.0 is here:
http://cassci.datastax.com/job/stef1927-10682-3.0-testall/11/testReport/
This is ready for review. I only fixed the problem with the unit test code. I
did not address the fact that we do not catch exceptions in the
{{FlushRunnable}} thread, which causes other threads to hang on a blocking
flush.
was (Author: stefania):
CI for 3.0 is here:
http://cassci.datastax.com/job/stef1927-10682-3.0-testall/11/testReport/
This is ready for review.
> Fix timeouts in BeforeFirstTest
> ---
>
> Key: CASSANDRA-10682
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10682
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x
>
> Attachments:
> TEST-org.apache.cassandra.db.SinglePartitionSliceCommandTest.log,
> TEST-org.apache.cassandra.db.commitlog.CommitLogFailurePolicyTest.log
>
>
> Some unit tests fail with a timeout in {{BeforeFirstTest}}, see for example
> [here|http://cassci.datastax.com/job/cassandra-3.0_testall/242/testReport/org.apache.cassandra.db/SinglePartitionSliceCommandTest/BeforeFirstTest/].
>
> In the corresponding log file, attached, there is a {{NoSuchFileException}}
> which might be the cause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10682) Fix timeouts in BeforeFirstTest
[
https://issues.apache.org/jira/browse/CASSANDRA-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008368#comment-15008368
]
Stefania commented on CASSANDRA-10682:
--
CI for 3.0 is here:
http://cassci.datastax.com/job/stef1927-10682-3.0-testall/11/testReport/
This is ready for review.
> Fix timeouts in BeforeFirstTest
> ---
>
> Key: CASSANDRA-10682
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10682
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x
>
> Attachments:
> TEST-org.apache.cassandra.db.SinglePartitionSliceCommandTest.log,
> TEST-org.apache.cassandra.db.commitlog.CommitLogFailurePolicyTest.log
>
>
> Some unit tests fail with a timeout in {{BeforeFirstTest}}, see for example
> [here|http://cassci.datastax.com/job/cassandra-3.0_testall/242/testReport/org.apache.cassandra.db/SinglePartitionSliceCommandTest/BeforeFirstTest/].
>
> In the corresponding log file, attached, there is a {{NoSuchFileException}}
> which might be the cause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10422) Avoid anticompaction when doing subrange repair
[ https://issues.apache.org/jira/browse/CASSANDRA-10422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson resolved CASSANDRA-10422. - Resolution: Fixed Fix Version/s: (was: 2.2.x) (was: 2.1.x) 2.2.4 2.1.12 +1, committed also added a few dtests: https://github.com/riptano/cassandra-dtest/pull/664 > Avoid anticompaction when doing subrange repair > --- > > Key: CASSANDRA-10422 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10422 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Marcus Eriksson >Assignee: Ariel Weisberg > Fix For: 2.1.12, 2.2.4, 3.0.1, 3.1 > > > If we do split the owned range in say 1000 parts, and then do one repair > each, we could potentially anticompact every sstable 1000 times (ie, we > anticompact the repaired range out 1000 times). We should avoid > anticompacting at all in these cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[6/6] cassandra git commit: Merge branch 'cassandra-3.1' into trunk
Merge branch 'cassandra-3.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6645062c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6645062c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6645062c Branch: refs/heads/trunk Commit: 6645062caaebfa7d77ae0c77db9e33cd1be44005 Parents: 0010fce 798a7a8 Author: Marcus Eriksson Authored: Tue Nov 17 10:15:58 2015 +0100 Committer: Marcus Eriksson Committed: Tue Nov 17 10:15:58 2015 +0100 -- CHANGES.txt | 2 ++ .../cassandra/repair/messages/RepairOption.java | 18 +++--- .../cassandra/service/ActiveRepairService.java| 2 ++ .../apache/cassandra/service/StorageService.java | 9 +++-- .../repair/messages/RepairOptionTest.java | 17 +++-- 5 files changed, 41 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6645062c/CHANGES.txt --
[5/5] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1
Merge branch 'cassandra-3.0' into cassandra-3.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/798a7a8f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/798a7a8f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/798a7a8f Branch: refs/heads/cassandra-3.1 Commit: 798a7a8fe8e7f28605341ebabc538908c3ff8dba Parents: 36e7677 eb8afb7 Author: Marcus Eriksson Authored: Tue Nov 17 10:13:33 2015 +0100 Committer: Marcus Eriksson Committed: Tue Nov 17 10:13:33 2015 +0100 -- CHANGES.txt | 2 ++ .../cassandra/repair/messages/RepairOption.java | 18 +++--- .../cassandra/service/ActiveRepairService.java| 2 ++ .../apache/cassandra/service/StorageService.java | 9 +++-- .../repair/messages/RepairOptionTest.java | 17 +++-- 5 files changed, 41 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/798a7a8f/CHANGES.txt --
[3/4] cassandra git commit: Don't do anticompaction after subrange repair
Don't do anticompaction after subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99b82dbb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99b82dbb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99b82dbb
Branch: refs/heads/cassandra-3.0
Commit: 99b82dbb43277035562e7b82bb9bdebd84510e96
Parents: d434a33
Author: Ariel Weisberg
Authored: Tue Nov 10 13:08:05 2015 -0500
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:59 2015 +0100
--
CHANGES.txt | 1 +
.../cassandra/repair/messages/RepairOption.java | 19 ++-
.../cassandra/service/ActiveRepairService.java | 2 ++
.../apache/cassandra/service/StorageService.java | 9 +
.../repair/messages/RepairOptionTest.java| 10 +++---
5 files changed, 29 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 489a76d..f5d3416 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.4
+ * Don't do anticompaction after subrange repair (CASSANDRA-10422)
* Fix SimpleDateType type compatibility (CASSANDRA-10027)
* (Hadoop) fix splits calculation (CASSANDRA-10640)
* (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --git a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
index 1780b6b..d50a2ed 100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@ -145,8 +145,9 @@ public class RepairOption
if (rangesStr != null)
{
if (incremental)
-throw new IllegalArgumentException("Incremental repair can't
be requested with subrange repair " +
- "because each subrange
repair would generate an anti-compacted table");
+logger.warn("Incremental repair can't be requested with
subrange repair " +
+"because each subrange repair would generate an
anti-compacted table. " +
+"The repair will occur but without
anti-compaction.");
StringTokenizer tokenizer = new StringTokenizer(rangesStr, ",");
while (tokenizer.hasMoreTokens())
{
@@ -161,7 +162,7 @@ public class RepairOption
}
}
-RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges);
+RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges, !ranges.isEmpty());
// data centers
String dataCentersStr = options.get(DATACENTERS_KEY);
@@ -220,13 +221,14 @@ public class RepairOption
private final boolean incremental;
private final boolean trace;
private final int jobThreads;
+private final boolean isSubrangeRepair;
private final Collection columnFamilies = new HashSet<>();
private final Collection dataCenters = new HashSet<>();
private final Collection hosts = new HashSet<>();
private final Collection> ranges = new HashSet<>();
-public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges)
+public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges, boolean isSubrangeRepair)
{
if (FBUtilities.isWindows() &&
(DatabaseDescriptor.getDiskAccessMode() !=
Config.DiskAccessMode.standard || DatabaseDescriptor.getIndexAccessMode() !=
Config.DiskAccessMode.standard) &&
@@ -243,6 +245,7 @@ public class RepairOption
this.trace = trace;
this.jobThreads = jobThreads;
this.ranges.addAll(ranges);
+this.isSubrangeRepair = isSubrangeRepair;
}
public RepairParallelism getParallelism()
@@ -292,8 +295,14 @@ public class RepairOption
public boolean isGlobal()
{
-return dataCenters.isEmpty() && hosts.isEmpty();
+return dataCenters.isEmpty() && hosts.isEmpty() && !isSubrangeRepair();
}
+
+public boolean isSubrangeRepair()
+{
+return isSubrangeRepair;
+}
+
@Override
public String toS
[2/6] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d434a33a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d434a33a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d434a33a
Branch: refs/heads/trunk
Commit: d434a33ace2dfe6715f4857f9537ee884f4ef410
Parents: 73a730f a8e8a67
Author: Marcus Eriksson
Authored: Tue Nov 17 10:07:04 2015 +0100
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:04 2015 +0100
--
CHANGES.txt| 1 +
.../apache/cassandra/repair/messages/RepairOption.java | 3 +++
.../org/apache/cassandra/service/StorageService.java | 4
.../cassandra/repair/messages/RepairOptionTest.java| 13 +++--
4 files changed, 19 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/CHANGES.txt
--
diff --cc CHANGES.txt
index 5705453,b6b394a..489a76d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
-2.1.12
+2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
+ * (Hadoop) fix splits calculation (CASSANDRA-10640)
+ * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
+ * (cqlsh) show partial trace if incomplete after max_trace_wait
(CASSANDRA-7645)
+ * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
+ * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
+ * Expose phi values from failure detector via JMX and tweak debug
+ and trace logging (CASSANDRA-9526)
+ * Fix RangeNamesQueryPager (CASSANDRA-10509)
+ * Deprecate Pig support (CASSANDRA-10542)
+ * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
+Merged from 2.1:
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --cc src/java/org/apache/cassandra/repair/messages/RepairOption.java
index f3e452c,000..1780b6b
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@@ -1,308 -1,0 +1,311 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.repair.messages;
+
+import java.util.*;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.repair.RepairParallelism;
+import org.apache.cassandra.tools.nodetool.Repair;
+import org.apache.cassandra.utils.FBUtilities;
+
+/**
+ * Repair options.
+ */
+public class RepairOption
+{
+public static final String PARALLELISM_KEY = "parallelism";
+public static final String PRIMARY_RANGE_KEY = "primaryRange";
+public static final String INCREMENTAL_KEY = "incremental";
+public static final String JOB_THREADS_KEY = "jobThreads";
+public static final String RANGES_KEY = "ranges";
+public static final String COLUMNFAMILIES_KEY = "columnFamilies";
+public static final String DATACENTERS_KEY = "dataCenters";
+public static final String HOSTS_KEY = "hosts";
+public static final String TRACE_KEY = "trace";
+
+// we don't want to push nodes too much for repair
+public static final int MAX_JOB_THREADS = 4;
+
+private static final Logger logger =
LoggerFactory.ge
[jira] [Updated] (CASSANDRA-10693) Bootstrapping needs improved error handling for disk failure on existing node
[
https://issues.apache.org/jira/browse/CASSANDRA-10693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heiko Sommer updated CASSANDRA-10693:
-
Description:
(This is the original problem description. Later the title of this ticket was
changed and new findings were added as a comment.)
I'm playing with a test cluster, trying to add a 3rd server while writing to
the cluster at max rate (executeAsync with pushing back). Single table for time
series data, with RF=2, STCS.
The bootstrap failure that I repeatedly get may be the same as in
CASSANDRA-9440 (or also CASSANDRA-6565).
Even if I could join the 3rd node using temporarily modified configuation
({{streaming_socket_timeout_in_ms}}, {{phi_convict_threshold}} etc), I think
that at least improving the logs for this failure case could be worthwhile.
When starting the new node (134.171.18.228), the underlying problem seems to
happen very early in the bootstrapping, on one of the existing cluster nodes
(134.171.18.231):
{noformat}
INFO [STREAM-INIT-/134.171.18.228:45331] 2015-11-11 16:21:30,344
StreamResultFuture.java:109 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc
ID#0] Creating new streaming plan for Bootstrap
INFO [STREAM-INIT-/134.171.18.228:45331] 2015-11-11 16:21:30,344
StreamResultFuture.java:116 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc,
ID#0] Received streaming plan for Bootstrap
INFO [STREAM-INIT-/134.171.18.228:45333] 2015-11-11 16:21:30,350
StreamResultFuture.java:116 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc,
ID#0] Received streaming plan for Bootstrap
ERROR [STREAM-IN-/134.171.18.228] 2015-11-11 16:21:41,505
StreamSession.java:505 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc]
Streaming error occurred
java.lang.AssertionError: Illegal bounds [2270144..2270152); size: 2270144
at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:328)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:98)
~[apache-cassandra-2.1.11.jar:2.1.11]
at org.apache.cassandra.io.util.Memory.getLong(Memory.java:249)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.io.compress.CompressionMetadata.getChunksForSections(CompressionMetadata.java:253)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.streaming.messages.OutgoingFileMessage.(OutgoingFileMessage.java:70)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.streaming.StreamTransferTask.addTransferFile(StreamTransferTask.java:58)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.streaming.StreamSession.addTransferFiles(StreamSession.java:378)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.streaming.StreamSession.addTransferRanges(StreamSession.java:272)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:521)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:453)
~[apache-cassandra-2.1.11.jar:2.1.11]
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:256)
~[apache-cassandra-2.1.11.jar:2.1.11]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
INFO [STREAM-IN-/134.171.18.228] 2015-11-11 16:21:41,510
StreamResultFuture.java:180 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc]
Session with /134.171.18.228 is complete
WARN [STREAM-IN-/134.171.18.228] 2015-11-11 16:21:41,511
StreamResultFuture.java:207 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc]
Stream failed
{noformat}
On the joining node, this failure is not reported at all:
{noformat}
INFO [main] 2015-11-11 16:21:30,000 StorageService.java:1120 - JOINING:
Starting to bootstrap...
INFO [main] 2015-11-11 16:21:30,336 StreamResultFuture.java:86 - [Stream
#43deef20-8890-11e5-a633-c16ac7efddbc] Executing streaming plan for Bootstrap
INFO [StreamConnectionEstablisher:2] 2015-11-11 16:21:30,337
StreamSession.java:220 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc]
Starting streaming to /134.171.18.229
INFO [StreamConnectionEstablisher:1] 2015-11-11 16:21:30,337
StreamSession.java:220 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc]
Starting streaming to /134.171.18.231
INFO [StreamConnectionEstablisher:1] 2015-11-11 16:21:30,348
StreamCoordinator.java:209 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc,
ID#0] Beginning stream session with /134.171.18.231
INFO [StreamConnectionEstablisher:2] 2015-11-11 16:21:30,348
StreamCoordinator.java:209 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc,
ID#0] Beginning stream session with /134.171.18.229
INFO [STREAM-IN-/134.171.18.231] 2015-11-11 16:21:41,508
StreamResultFuture.java:180 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc]
Session with /134.171.18.231 is complete
INFO
[4/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb8afb71 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb8afb71 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb8afb71 Branch: refs/heads/trunk Commit: eb8afb71f52c65855efa1d7e8d3ef314ab7373e2 Parents: c0480d8 99b82db Author: Marcus Eriksson Authored: Tue Nov 17 10:13:18 2015 +0100 Committer: Marcus Eriksson Committed: Tue Nov 17 10:13:18 2015 +0100 -- CHANGES.txt | 2 ++ .../cassandra/repair/messages/RepairOption.java | 18 +++--- .../cassandra/service/ActiveRepairService.java| 2 ++ .../apache/cassandra/service/StorageService.java | 9 +++-- .../repair/messages/RepairOptionTest.java | 17 +++-- 5 files changed, 41 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb8afb71/CHANGES.txt -- diff --cc CHANGES.txt index 8bb67e1,f5d3416..13647cc --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,56 -1,8 +1,58 @@@ -2.2.4 +3.0.1 + * Correctly preserve deletion info on updated rows when notifying indexers + of single-row deletions (CASSANDRA-10694) + * Notify indexers of partition delete during cleanup (CASSANDRA-10685) + * Keep the file open in trySkipCache (CASSANDRA-10669) + * Updated trigger example (CASSANDRA-10257) +Merged from 2.2: + * Don't do anticompaction after subrange repair (CASSANDRA-10422) * Fix SimpleDateType type compatibility (CASSANDRA-10027) * (Hadoop) fix splits calculation (CASSANDRA-10640) * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058) +Merged from 2.1: ++ * Reject incremental repair with subrange repair (CASSANDRA-10422) + * Add a nodetool command to refresh size_estimates (CASSANDRA-9579) + * Invalidate cache after stream receive task is completed (CASSANDRA-10341) + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258) + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605) + + +3.0 + * Fix AssertionError while flushing memtable due to materialized views + incorrectly inserting empty rows (CASSANDRA-10614) + * Store UDA initcond as CQL literal in the schema table, instead of a blob (CASSANDRA-10650) + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491) + * Fix distinct queries in mixed version cluster (CASSANDRA-10573) + * Skip sstable on clustering in names query (CASSANDRA-10571) + * Remove value skipping as it breaks read-repair (CASSANDRA-10655) + * Fix bootstrapping with MVs (CASSANDRA-10621) + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584) + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634) + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates (CASSANDRA-10600) + * Fix reading of legacy sstables (CASSANDRA-10590) + * Use CQL type names in schema metadata tables (CASSANDRA-10365) + * Guard batchlog replay against integer division by zero (CASSANDRA-9223) + * Fix bug when adding a column to thrift with the same name than a primary key (CASSANDRA-10608) + * Add client address argument to IAuthenticator::newSaslNegotiator (CASSANDRA-8068) + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602) + * Don't use 'names query' read path for counters (CASSANDRA-10572) + * Fix backward compatibility for counters (CASSANDRA-10470) + * Remove memory_allocator paramter from cassandra.yaml (CASSANDRA-10581,10628) + * Execute the metadata reload task of all registered indexes on CFS::reload (CASSANDRA-10604) + * Fix thrift cas operations with defined columns (CASSANDRA-10576) + * Fix PartitionUpdate.operationCount()for updates with static column operations (CASSANDRA-10606) + * Fix thrift get() queries with defined columns (CASSANDRA-10586) + * Fix marking of indexes as built and removed (CASSANDRA-10601) + * Skip initialization of non-registered 2i instances, remove Index::getIndexName (CASSANDRA-10595) + * Fix batches on multiple tables (CASSANDRA-10554) + * Ensure compaction options are validated when updating KeyspaceMetadata (CASSANDRA-10569) + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975) + * Remove token generator (CASSANDRA-5261) + * RolesCache should not be created for any authenticator that does not requireAuthentication (CASSANDRA-10562) + * Fix LogTransaction checking only a single directory for files (CASSANDRA-10421) + * Fix handling of range tombstones when reading old format sstables (CASSANDRA-10360) + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367) +Merged from 2.2: * (cqlsh) show partial tr
[3/6] cassandra git commit: Don't do anticompaction after subrange repair
Don't do anticompaction after subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99b82dbb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99b82dbb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99b82dbb
Branch: refs/heads/trunk
Commit: 99b82dbb43277035562e7b82bb9bdebd84510e96
Parents: d434a33
Author: Ariel Weisberg
Authored: Tue Nov 10 13:08:05 2015 -0500
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:59 2015 +0100
--
CHANGES.txt | 1 +
.../cassandra/repair/messages/RepairOption.java | 19 ++-
.../cassandra/service/ActiveRepairService.java | 2 ++
.../apache/cassandra/service/StorageService.java | 9 +
.../repair/messages/RepairOptionTest.java| 10 +++---
5 files changed, 29 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 489a76d..f5d3416 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.4
+ * Don't do anticompaction after subrange repair (CASSANDRA-10422)
* Fix SimpleDateType type compatibility (CASSANDRA-10027)
* (Hadoop) fix splits calculation (CASSANDRA-10640)
* (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --git a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
index 1780b6b..d50a2ed 100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@ -145,8 +145,9 @@ public class RepairOption
if (rangesStr != null)
{
if (incremental)
-throw new IllegalArgumentException("Incremental repair can't
be requested with subrange repair " +
- "because each subrange
repair would generate an anti-compacted table");
+logger.warn("Incremental repair can't be requested with
subrange repair " +
+"because each subrange repair would generate an
anti-compacted table. " +
+"The repair will occur but without
anti-compaction.");
StringTokenizer tokenizer = new StringTokenizer(rangesStr, ",");
while (tokenizer.hasMoreTokens())
{
@@ -161,7 +162,7 @@ public class RepairOption
}
}
-RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges);
+RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges, !ranges.isEmpty());
// data centers
String dataCentersStr = options.get(DATACENTERS_KEY);
@@ -220,13 +221,14 @@ public class RepairOption
private final boolean incremental;
private final boolean trace;
private final int jobThreads;
+private final boolean isSubrangeRepair;
private final Collection columnFamilies = new HashSet<>();
private final Collection dataCenters = new HashSet<>();
private final Collection hosts = new HashSet<>();
private final Collection> ranges = new HashSet<>();
-public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges)
+public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges, boolean isSubrangeRepair)
{
if (FBUtilities.isWindows() &&
(DatabaseDescriptor.getDiskAccessMode() !=
Config.DiskAccessMode.standard || DatabaseDescriptor.getIndexAccessMode() !=
Config.DiskAccessMode.standard) &&
@@ -243,6 +245,7 @@ public class RepairOption
this.trace = trace;
this.jobThreads = jobThreads;
this.ranges.addAll(ranges);
+this.isSubrangeRepair = isSubrangeRepair;
}
public RepairParallelism getParallelism()
@@ -292,8 +295,14 @@ public class RepairOption
public boolean isGlobal()
{
-return dataCenters.isEmpty() && hosts.isEmpty();
+return dataCenters.isEmpty() && hosts.isEmpty() && !isSubrangeRepair();
}
+
+public boolean isSubrangeRepair()
+{
+return isSubrangeRepair;
+}
+
@Override
public String toString()
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1
Merge branch 'cassandra-3.0' into cassandra-3.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/798a7a8f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/798a7a8f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/798a7a8f Branch: refs/heads/trunk Commit: 798a7a8fe8e7f28605341ebabc538908c3ff8dba Parents: 36e7677 eb8afb7 Author: Marcus Eriksson Authored: Tue Nov 17 10:13:33 2015 +0100 Committer: Marcus Eriksson Committed: Tue Nov 17 10:13:33 2015 +0100 -- CHANGES.txt | 2 ++ .../cassandra/repair/messages/RepairOption.java | 18 +++--- .../cassandra/service/ActiveRepairService.java| 2 ++ .../apache/cassandra/service/StorageService.java | 9 +++-- .../repair/messages/RepairOptionTest.java | 17 +++-- 5 files changed, 41 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/798a7a8f/CHANGES.txt --
[1/6] cassandra git commit: Reject incremental repair requests combined with subrange repair
Repository: cassandra
Updated Branches:
refs/heads/trunk 0010fce6d -> 6645062ca
Reject incremental repair requests combined with subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8e8a673
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8e8a673
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8e8a673
Branch: refs/heads/trunk
Commit: a8e8a67306c0b26b8fe9c74a1fb00bacfa224cf7
Parents: 7e056fa
Author: Ariel Weisberg
Authored: Thu Oct 29 12:36:32 2015 -0400
Committer: Marcus Eriksson
Committed: Tue Nov 17 09:57:15 2015 +0100
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/service/StorageService.java | 3 +++
.../org/apache/cassandra/service/StorageServiceServerTest.java | 6 ++
3 files changed, 10 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2eeda94..b6b394a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 665ce3a..03c1960 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2830,6 +2830,9 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
public int forceRepairRangeAsync(String beginToken, String endToken,
String keyspaceName, boolean isSequential, boolean isLocal, boolean fullRepair,
String... columnFamilies)
{
+if (!fullRepair)
+throw new IllegalArgumentException("Incremental repair can't be
requested with subrange repair because " +
+ "each subrange repair would
generate an anti-compacted table");
Collection> repairingRange =
createRepairRangeFrom(beginToken, endToken);
logger.info("starting user-requested repair of range {} for keyspace
{} and column families {}",
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
index dd25b35..564239b 100644
--- a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
+++ b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
@@ -508,4 +508,10 @@ public class StorageServiceServerTest
repairRangeFrom =
StorageService.instance.createRepairRangeFrom("2000", "2000");
assert repairRangeFrom.size() == 0;
}
+
+@Test(expected=IllegalArgumentException.class)
+public void testIncrementalRepairWithSubrangesThrows() throws Exception
+{
+StorageService.instance.forceRepairRangeAsync("", "", "", true,
true, false, "");
+}
}
[3/5] cassandra git commit: Don't do anticompaction after subrange repair
Don't do anticompaction after subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99b82dbb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99b82dbb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99b82dbb
Branch: refs/heads/cassandra-3.1
Commit: 99b82dbb43277035562e7b82bb9bdebd84510e96
Parents: d434a33
Author: Ariel Weisberg
Authored: Tue Nov 10 13:08:05 2015 -0500
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:59 2015 +0100
--
CHANGES.txt | 1 +
.../cassandra/repair/messages/RepairOption.java | 19 ++-
.../cassandra/service/ActiveRepairService.java | 2 ++
.../apache/cassandra/service/StorageService.java | 9 +
.../repair/messages/RepairOptionTest.java| 10 +++---
5 files changed, 29 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 489a76d..f5d3416 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.4
+ * Don't do anticompaction after subrange repair (CASSANDRA-10422)
* Fix SimpleDateType type compatibility (CASSANDRA-10027)
* (Hadoop) fix splits calculation (CASSANDRA-10640)
* (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --git a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
index 1780b6b..d50a2ed 100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@ -145,8 +145,9 @@ public class RepairOption
if (rangesStr != null)
{
if (incremental)
-throw new IllegalArgumentException("Incremental repair can't
be requested with subrange repair " +
- "because each subrange
repair would generate an anti-compacted table");
+logger.warn("Incremental repair can't be requested with
subrange repair " +
+"because each subrange repair would generate an
anti-compacted table. " +
+"The repair will occur but without
anti-compaction.");
StringTokenizer tokenizer = new StringTokenizer(rangesStr, ",");
while (tokenizer.hasMoreTokens())
{
@@ -161,7 +162,7 @@ public class RepairOption
}
}
-RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges);
+RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges, !ranges.isEmpty());
// data centers
String dataCentersStr = options.get(DATACENTERS_KEY);
@@ -220,13 +221,14 @@ public class RepairOption
private final boolean incremental;
private final boolean trace;
private final int jobThreads;
+private final boolean isSubrangeRepair;
private final Collection columnFamilies = new HashSet<>();
private final Collection dataCenters = new HashSet<>();
private final Collection hosts = new HashSet<>();
private final Collection> ranges = new HashSet<>();
-public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges)
+public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges, boolean isSubrangeRepair)
{
if (FBUtilities.isWindows() &&
(DatabaseDescriptor.getDiskAccessMode() !=
Config.DiskAccessMode.standard || DatabaseDescriptor.getIndexAccessMode() !=
Config.DiskAccessMode.standard) &&
@@ -243,6 +245,7 @@ public class RepairOption
this.trace = trace;
this.jobThreads = jobThreads;
this.ranges.addAll(ranges);
+this.isSubrangeRepair = isSubrangeRepair;
}
public RepairParallelism getParallelism()
@@ -292,8 +295,14 @@ public class RepairOption
public boolean isGlobal()
{
-return dataCenters.isEmpty() && hosts.isEmpty();
+return dataCenters.isEmpty() && hosts.isEmpty() && !isSubrangeRepair();
}
+
+public boolean isSubrangeRepair()
+{
+return isSubrangeRepair;
+}
+
@Override
public String toS
[1/4] cassandra git commit: Reject incremental repair requests combined with subrange repair
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 c0480d8bb -> eb8afb71f
Reject incremental repair requests combined with subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8e8a673
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8e8a673
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8e8a673
Branch: refs/heads/cassandra-3.0
Commit: a8e8a67306c0b26b8fe9c74a1fb00bacfa224cf7
Parents: 7e056fa
Author: Ariel Weisberg
Authored: Thu Oct 29 12:36:32 2015 -0400
Committer: Marcus Eriksson
Committed: Tue Nov 17 09:57:15 2015 +0100
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/service/StorageService.java | 3 +++
.../org/apache/cassandra/service/StorageServiceServerTest.java | 6 ++
3 files changed, 10 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2eeda94..b6b394a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 665ce3a..03c1960 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2830,6 +2830,9 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
public int forceRepairRangeAsync(String beginToken, String endToken,
String keyspaceName, boolean isSequential, boolean isLocal, boolean fullRepair,
String... columnFamilies)
{
+if (!fullRepair)
+throw new IllegalArgumentException("Incremental repair can't be
requested with subrange repair because " +
+ "each subrange repair would
generate an anti-compacted table");
Collection> repairingRange =
createRepairRangeFrom(beginToken, endToken);
logger.info("starting user-requested repair of range {} for keyspace
{} and column families {}",
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
index dd25b35..564239b 100644
--- a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
+++ b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
@@ -508,4 +508,10 @@ public class StorageServiceServerTest
repairRangeFrom =
StorageService.instance.createRepairRangeFrom("2000", "2000");
assert repairRangeFrom.size() == 0;
}
+
+@Test(expected=IllegalArgumentException.class)
+public void testIncrementalRepairWithSubrangesThrows() throws Exception
+{
+StorageService.instance.forceRepairRangeAsync("", "", "", true,
true, false, "");
+}
}
[2/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d434a33a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d434a33a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d434a33a
Branch: refs/heads/cassandra-3.0
Commit: d434a33ace2dfe6715f4857f9537ee884f4ef410
Parents: 73a730f a8e8a67
Author: Marcus Eriksson
Authored: Tue Nov 17 10:07:04 2015 +0100
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:04 2015 +0100
--
CHANGES.txt| 1 +
.../apache/cassandra/repair/messages/RepairOption.java | 3 +++
.../org/apache/cassandra/service/StorageService.java | 4
.../cassandra/repair/messages/RepairOptionTest.java| 13 +++--
4 files changed, 19 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/CHANGES.txt
--
diff --cc CHANGES.txt
index 5705453,b6b394a..489a76d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
-2.1.12
+2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
+ * (Hadoop) fix splits calculation (CASSANDRA-10640)
+ * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
+ * (cqlsh) show partial trace if incomplete after max_trace_wait
(CASSANDRA-7645)
+ * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
+ * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
+ * Expose phi values from failure detector via JMX and tweak debug
+ and trace logging (CASSANDRA-9526)
+ * Fix RangeNamesQueryPager (CASSANDRA-10509)
+ * Deprecate Pig support (CASSANDRA-10542)
+ * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
+Merged from 2.1:
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --cc src/java/org/apache/cassandra/repair/messages/RepairOption.java
index f3e452c,000..1780b6b
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@@ -1,308 -1,0 +1,311 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.repair.messages;
+
+import java.util.*;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.repair.RepairParallelism;
+import org.apache.cassandra.tools.nodetool.Repair;
+import org.apache.cassandra.utils.FBUtilities;
+
+/**
+ * Repair options.
+ */
+public class RepairOption
+{
+public static final String PARALLELISM_KEY = "parallelism";
+public static final String PRIMARY_RANGE_KEY = "primaryRange";
+public static final String INCREMENTAL_KEY = "incremental";
+public static final String JOB_THREADS_KEY = "jobThreads";
+public static final String RANGES_KEY = "ranges";
+public static final String COLUMNFAMILIES_KEY = "columnFamilies";
+public static final String DATACENTERS_KEY = "dataCenters";
+public static final String HOSTS_KEY = "hosts";
+public static final String TRACE_KEY = "trace";
+
+// we don't want to push nodes too much for repair
+public static final int MAX_JOB_THREADS = 4;
+
+private static final Logger logger =
LoggerFa
[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d434a33a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d434a33a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d434a33a
Branch: refs/heads/cassandra-2.2
Commit: d434a33ace2dfe6715f4857f9537ee884f4ef410
Parents: 73a730f a8e8a67
Author: Marcus Eriksson
Authored: Tue Nov 17 10:07:04 2015 +0100
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:04 2015 +0100
--
CHANGES.txt| 1 +
.../apache/cassandra/repair/messages/RepairOption.java | 3 +++
.../org/apache/cassandra/service/StorageService.java | 4
.../cassandra/repair/messages/RepairOptionTest.java| 13 +++--
4 files changed, 19 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/CHANGES.txt
--
diff --cc CHANGES.txt
index 5705453,b6b394a..489a76d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
-2.1.12
+2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
+ * (Hadoop) fix splits calculation (CASSANDRA-10640)
+ * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
+ * (cqlsh) show partial trace if incomplete after max_trace_wait
(CASSANDRA-7645)
+ * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
+ * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
+ * Expose phi values from failure detector via JMX and tweak debug
+ and trace logging (CASSANDRA-9526)
+ * Fix RangeNamesQueryPager (CASSANDRA-10509)
+ * Deprecate Pig support (CASSANDRA-10542)
+ * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
+Merged from 2.1:
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --cc src/java/org/apache/cassandra/repair/messages/RepairOption.java
index f3e452c,000..1780b6b
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@@ -1,308 -1,0 +1,311 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.repair.messages;
+
+import java.util.*;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.repair.RepairParallelism;
+import org.apache.cassandra.tools.nodetool.Repair;
+import org.apache.cassandra.utils.FBUtilities;
+
+/**
+ * Repair options.
+ */
+public class RepairOption
+{
+public static final String PARALLELISM_KEY = "parallelism";
+public static final String PRIMARY_RANGE_KEY = "primaryRange";
+public static final String INCREMENTAL_KEY = "incremental";
+public static final String JOB_THREADS_KEY = "jobThreads";
+public static final String RANGES_KEY = "ranges";
+public static final String COLUMNFAMILIES_KEY = "columnFamilies";
+public static final String DATACENTERS_KEY = "dataCenters";
+public static final String HOSTS_KEY = "hosts";
+public static final String TRACE_KEY = "trace";
+
+// we don't want to push nodes too much for repair
+public static final int MAX_JOB_THREADS = 4;
+
+private static final Logger logger =
LoggerFa
[2/5] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d434a33a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d434a33a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d434a33a
Branch: refs/heads/cassandra-3.1
Commit: d434a33ace2dfe6715f4857f9537ee884f4ef410
Parents: 73a730f a8e8a67
Author: Marcus Eriksson
Authored: Tue Nov 17 10:07:04 2015 +0100
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:04 2015 +0100
--
CHANGES.txt| 1 +
.../apache/cassandra/repair/messages/RepairOption.java | 3 +++
.../org/apache/cassandra/service/StorageService.java | 4
.../cassandra/repair/messages/RepairOptionTest.java| 13 +++--
4 files changed, 19 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/CHANGES.txt
--
diff --cc CHANGES.txt
index 5705453,b6b394a..489a76d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
-2.1.12
+2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
+ * (Hadoop) fix splits calculation (CASSANDRA-10640)
+ * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
+ * (cqlsh) show partial trace if incomplete after max_trace_wait
(CASSANDRA-7645)
+ * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
+ * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
+ * Expose phi values from failure detector via JMX and tweak debug
+ and trace logging (CASSANDRA-9526)
+ * Fix RangeNamesQueryPager (CASSANDRA-10509)
+ * Deprecate Pig support (CASSANDRA-10542)
+ * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
+Merged from 2.1:
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d434a33a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --cc src/java/org/apache/cassandra/repair/messages/RepairOption.java
index f3e452c,000..1780b6b
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@@ -1,308 -1,0 +1,311 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.repair.messages;
+
+import java.util.*;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.repair.RepairParallelism;
+import org.apache.cassandra.tools.nodetool.Repair;
+import org.apache.cassandra.utils.FBUtilities;
+
+/**
+ * Repair options.
+ */
+public class RepairOption
+{
+public static final String PARALLELISM_KEY = "parallelism";
+public static final String PRIMARY_RANGE_KEY = "primaryRange";
+public static final String INCREMENTAL_KEY = "incremental";
+public static final String JOB_THREADS_KEY = "jobThreads";
+public static final String RANGES_KEY = "ranges";
+public static final String COLUMNFAMILIES_KEY = "columnFamilies";
+public static final String DATACENTERS_KEY = "dataCenters";
+public static final String HOSTS_KEY = "hosts";
+public static final String TRACE_KEY = "trace";
+
+// we don't want to push nodes too much for repair
+public static final int MAX_JOB_THREADS = 4;
+
+private static final Logger logger =
LoggerFa
[1/5] cassandra git commit: Reject incremental repair requests combined with subrange repair
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.1 36e767714 -> 798a7a8fe
Reject incremental repair requests combined with subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8e8a673
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8e8a673
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8e8a673
Branch: refs/heads/cassandra-3.1
Commit: a8e8a67306c0b26b8fe9c74a1fb00bacfa224cf7
Parents: 7e056fa
Author: Ariel Weisberg
Authored: Thu Oct 29 12:36:32 2015 -0400
Committer: Marcus Eriksson
Committed: Tue Nov 17 09:57:15 2015 +0100
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/service/StorageService.java | 3 +++
.../org/apache/cassandra/service/StorageServiceServerTest.java | 6 ++
3 files changed, 10 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2eeda94..b6b394a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 665ce3a..03c1960 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2830,6 +2830,9 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
public int forceRepairRangeAsync(String beginToken, String endToken,
String keyspaceName, boolean isSequential, boolean isLocal, boolean fullRepair,
String... columnFamilies)
{
+if (!fullRepair)
+throw new IllegalArgumentException("Incremental repair can't be
requested with subrange repair because " +
+ "each subrange repair would
generate an anti-compacted table");
Collection> repairingRange =
createRepairRangeFrom(beginToken, endToken);
logger.info("starting user-requested repair of range {} for keyspace
{} and column families {}",
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
index dd25b35..564239b 100644
--- a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
+++ b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
@@ -508,4 +508,10 @@ public class StorageServiceServerTest
repairRangeFrom =
StorageService.instance.createRepairRangeFrom("2000", "2000");
assert repairRangeFrom.size() == 0;
}
+
+@Test(expected=IllegalArgumentException.class)
+public void testIncrementalRepairWithSubrangesThrows() throws Exception
+{
+StorageService.instance.forceRepairRangeAsync("", "", "", true,
true, false, "");
+}
}
[4/4] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb8afb71 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb8afb71 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb8afb71 Branch: refs/heads/cassandra-3.0 Commit: eb8afb71f52c65855efa1d7e8d3ef314ab7373e2 Parents: c0480d8 99b82db Author: Marcus Eriksson Authored: Tue Nov 17 10:13:18 2015 +0100 Committer: Marcus Eriksson Committed: Tue Nov 17 10:13:18 2015 +0100 -- CHANGES.txt | 2 ++ .../cassandra/repair/messages/RepairOption.java | 18 +++--- .../cassandra/service/ActiveRepairService.java| 2 ++ .../apache/cassandra/service/StorageService.java | 9 +++-- .../repair/messages/RepairOptionTest.java | 17 +++-- 5 files changed, 41 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb8afb71/CHANGES.txt -- diff --cc CHANGES.txt index 8bb67e1,f5d3416..13647cc --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,56 -1,8 +1,58 @@@ -2.2.4 +3.0.1 + * Correctly preserve deletion info on updated rows when notifying indexers + of single-row deletions (CASSANDRA-10694) + * Notify indexers of partition delete during cleanup (CASSANDRA-10685) + * Keep the file open in trySkipCache (CASSANDRA-10669) + * Updated trigger example (CASSANDRA-10257) +Merged from 2.2: + * Don't do anticompaction after subrange repair (CASSANDRA-10422) * Fix SimpleDateType type compatibility (CASSANDRA-10027) * (Hadoop) fix splits calculation (CASSANDRA-10640) * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058) +Merged from 2.1: ++ * Reject incremental repair with subrange repair (CASSANDRA-10422) + * Add a nodetool command to refresh size_estimates (CASSANDRA-9579) + * Invalidate cache after stream receive task is completed (CASSANDRA-10341) + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258) + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605) + + +3.0 + * Fix AssertionError while flushing memtable due to materialized views + incorrectly inserting empty rows (CASSANDRA-10614) + * Store UDA initcond as CQL literal in the schema table, instead of a blob (CASSANDRA-10650) + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491) + * Fix distinct queries in mixed version cluster (CASSANDRA-10573) + * Skip sstable on clustering in names query (CASSANDRA-10571) + * Remove value skipping as it breaks read-repair (CASSANDRA-10655) + * Fix bootstrapping with MVs (CASSANDRA-10621) + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584) + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634) + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates (CASSANDRA-10600) + * Fix reading of legacy sstables (CASSANDRA-10590) + * Use CQL type names in schema metadata tables (CASSANDRA-10365) + * Guard batchlog replay against integer division by zero (CASSANDRA-9223) + * Fix bug when adding a column to thrift with the same name than a primary key (CASSANDRA-10608) + * Add client address argument to IAuthenticator::newSaslNegotiator (CASSANDRA-8068) + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602) + * Don't use 'names query' read path for counters (CASSANDRA-10572) + * Fix backward compatibility for counters (CASSANDRA-10470) + * Remove memory_allocator paramter from cassandra.yaml (CASSANDRA-10581,10628) + * Execute the metadata reload task of all registered indexes on CFS::reload (CASSANDRA-10604) + * Fix thrift cas operations with defined columns (CASSANDRA-10576) + * Fix PartitionUpdate.operationCount()for updates with static column operations (CASSANDRA-10606) + * Fix thrift get() queries with defined columns (CASSANDRA-10586) + * Fix marking of indexes as built and removed (CASSANDRA-10601) + * Skip initialization of non-registered 2i instances, remove Index::getIndexName (CASSANDRA-10595) + * Fix batches on multiple tables (CASSANDRA-10554) + * Ensure compaction options are validated when updating KeyspaceMetadata (CASSANDRA-10569) + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975) + * Remove token generator (CASSANDRA-5261) + * RolesCache should not be created for any authenticator that does not requireAuthentication (CASSANDRA-10562) + * Fix LogTransaction checking only a single directory for files (CASSANDRA-10421) + * Fix handling of range tombstones when reading old format sstables (CASSANDRA-10360) + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367) +Merged from 2.2: * (cqlsh) show pa
[4/5] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb8afb71 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb8afb71 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb8afb71 Branch: refs/heads/cassandra-3.1 Commit: eb8afb71f52c65855efa1d7e8d3ef314ab7373e2 Parents: c0480d8 99b82db Author: Marcus Eriksson Authored: Tue Nov 17 10:13:18 2015 +0100 Committer: Marcus Eriksson Committed: Tue Nov 17 10:13:18 2015 +0100 -- CHANGES.txt | 2 ++ .../cassandra/repair/messages/RepairOption.java | 18 +++--- .../cassandra/service/ActiveRepairService.java| 2 ++ .../apache/cassandra/service/StorageService.java | 9 +++-- .../repair/messages/RepairOptionTest.java | 17 +++-- 5 files changed, 41 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb8afb71/CHANGES.txt -- diff --cc CHANGES.txt index 8bb67e1,f5d3416..13647cc --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,56 -1,8 +1,58 @@@ -2.2.4 +3.0.1 + * Correctly preserve deletion info on updated rows when notifying indexers + of single-row deletions (CASSANDRA-10694) + * Notify indexers of partition delete during cleanup (CASSANDRA-10685) + * Keep the file open in trySkipCache (CASSANDRA-10669) + * Updated trigger example (CASSANDRA-10257) +Merged from 2.2: + * Don't do anticompaction after subrange repair (CASSANDRA-10422) * Fix SimpleDateType type compatibility (CASSANDRA-10027) * (Hadoop) fix splits calculation (CASSANDRA-10640) * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058) +Merged from 2.1: ++ * Reject incremental repair with subrange repair (CASSANDRA-10422) + * Add a nodetool command to refresh size_estimates (CASSANDRA-9579) + * Invalidate cache after stream receive task is completed (CASSANDRA-10341) + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258) + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605) + + +3.0 + * Fix AssertionError while flushing memtable due to materialized views + incorrectly inserting empty rows (CASSANDRA-10614) + * Store UDA initcond as CQL literal in the schema table, instead of a blob (CASSANDRA-10650) + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491) + * Fix distinct queries in mixed version cluster (CASSANDRA-10573) + * Skip sstable on clustering in names query (CASSANDRA-10571) + * Remove value skipping as it breaks read-repair (CASSANDRA-10655) + * Fix bootstrapping with MVs (CASSANDRA-10621) + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584) + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634) + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates (CASSANDRA-10600) + * Fix reading of legacy sstables (CASSANDRA-10590) + * Use CQL type names in schema metadata tables (CASSANDRA-10365) + * Guard batchlog replay against integer division by zero (CASSANDRA-9223) + * Fix bug when adding a column to thrift with the same name than a primary key (CASSANDRA-10608) + * Add client address argument to IAuthenticator::newSaslNegotiator (CASSANDRA-8068) + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602) + * Don't use 'names query' read path for counters (CASSANDRA-10572) + * Fix backward compatibility for counters (CASSANDRA-10470) + * Remove memory_allocator paramter from cassandra.yaml (CASSANDRA-10581,10628) + * Execute the metadata reload task of all registered indexes on CFS::reload (CASSANDRA-10604) + * Fix thrift cas operations with defined columns (CASSANDRA-10576) + * Fix PartitionUpdate.operationCount()for updates with static column operations (CASSANDRA-10606) + * Fix thrift get() queries with defined columns (CASSANDRA-10586) + * Fix marking of indexes as built and removed (CASSANDRA-10601) + * Skip initialization of non-registered 2i instances, remove Index::getIndexName (CASSANDRA-10595) + * Fix batches on multiple tables (CASSANDRA-10554) + * Ensure compaction options are validated when updating KeyspaceMetadata (CASSANDRA-10569) + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975) + * Remove token generator (CASSANDRA-5261) + * RolesCache should not be created for any authenticator that does not requireAuthentication (CASSANDRA-10562) + * Fix LogTransaction checking only a single directory for files (CASSANDRA-10421) + * Fix handling of range tombstones when reading old format sstables (CASSANDRA-10360) + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367) +Merged from 2.2: * (cqlsh) show pa
[3/3] cassandra git commit: Don't do anticompaction after subrange repair
Don't do anticompaction after subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99b82dbb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99b82dbb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99b82dbb
Branch: refs/heads/cassandra-2.2
Commit: 99b82dbb43277035562e7b82bb9bdebd84510e96
Parents: d434a33
Author: Ariel Weisberg
Authored: Tue Nov 10 13:08:05 2015 -0500
Committer: Marcus Eriksson
Committed: Tue Nov 17 10:07:59 2015 +0100
--
CHANGES.txt | 1 +
.../cassandra/repair/messages/RepairOption.java | 19 ++-
.../cassandra/service/ActiveRepairService.java | 2 ++
.../apache/cassandra/service/StorageService.java | 9 +
.../repair/messages/RepairOptionTest.java| 10 +++---
5 files changed, 29 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 489a76d..f5d3416 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.4
+ * Don't do anticompaction after subrange repair (CASSANDRA-10422)
* Fix SimpleDateType type compatibility (CASSANDRA-10027)
* (Hadoop) fix splits calculation (CASSANDRA-10640)
* (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99b82dbb/src/java/org/apache/cassandra/repair/messages/RepairOption.java
--
diff --git a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
index 1780b6b..d50a2ed 100644
--- a/src/java/org/apache/cassandra/repair/messages/RepairOption.java
+++ b/src/java/org/apache/cassandra/repair/messages/RepairOption.java
@@ -145,8 +145,9 @@ public class RepairOption
if (rangesStr != null)
{
if (incremental)
-throw new IllegalArgumentException("Incremental repair can't
be requested with subrange repair " +
- "because each subrange
repair would generate an anti-compacted table");
+logger.warn("Incremental repair can't be requested with
subrange repair " +
+"because each subrange repair would generate an
anti-compacted table. " +
+"The repair will occur but without
anti-compaction.");
StringTokenizer tokenizer = new StringTokenizer(rangesStr, ",");
while (tokenizer.hasMoreTokens())
{
@@ -161,7 +162,7 @@ public class RepairOption
}
}
-RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges);
+RepairOption option = new RepairOption(parallelism, primaryRange,
incremental, trace, jobThreads, ranges, !ranges.isEmpty());
// data centers
String dataCentersStr = options.get(DATACENTERS_KEY);
@@ -220,13 +221,14 @@ public class RepairOption
private final boolean incremental;
private final boolean trace;
private final int jobThreads;
+private final boolean isSubrangeRepair;
private final Collection columnFamilies = new HashSet<>();
private final Collection dataCenters = new HashSet<>();
private final Collection hosts = new HashSet<>();
private final Collection> ranges = new HashSet<>();
-public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges)
+public RepairOption(RepairParallelism parallelism, boolean primaryRange,
boolean incremental, boolean trace, int jobThreads, Collection>
ranges, boolean isSubrangeRepair)
{
if (FBUtilities.isWindows() &&
(DatabaseDescriptor.getDiskAccessMode() !=
Config.DiskAccessMode.standard || DatabaseDescriptor.getIndexAccessMode() !=
Config.DiskAccessMode.standard) &&
@@ -243,6 +245,7 @@ public class RepairOption
this.trace = trace;
this.jobThreads = jobThreads;
this.ranges.addAll(ranges);
+this.isSubrangeRepair = isSubrangeRepair;
}
public RepairParallelism getParallelism()
@@ -292,8 +295,14 @@ public class RepairOption
public boolean isGlobal()
{
-return dataCenters.isEmpty() && hosts.isEmpty();
+return dataCenters.isEmpty() && hosts.isEmpty() && !isSubrangeRepair();
}
+
+public boolean isSubrangeRepair()
+{
+return isSubrangeRepair;
+}
+
@Override
public String toS
[1/3] cassandra git commit: Reject incremental repair requests combined with subrange repair
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 73a730f92 -> 99b82dbb4
Reject incremental repair requests combined with subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8e8a673
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8e8a673
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8e8a673
Branch: refs/heads/cassandra-2.2
Commit: a8e8a67306c0b26b8fe9c74a1fb00bacfa224cf7
Parents: 7e056fa
Author: Ariel Weisberg
Authored: Thu Oct 29 12:36:32 2015 -0400
Committer: Marcus Eriksson
Committed: Tue Nov 17 09:57:15 2015 +0100
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/service/StorageService.java | 3 +++
.../org/apache/cassandra/service/StorageServiceServerTest.java | 6 ++
3 files changed, 10 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2eeda94..b6b394a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 665ce3a..03c1960 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2830,6 +2830,9 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
public int forceRepairRangeAsync(String beginToken, String endToken,
String keyspaceName, boolean isSequential, boolean isLocal, boolean fullRepair,
String... columnFamilies)
{
+if (!fullRepair)
+throw new IllegalArgumentException("Incremental repair can't be
requested with subrange repair because " +
+ "each subrange repair would
generate an anti-compacted table");
Collection> repairingRange =
createRepairRangeFrom(beginToken, endToken);
logger.info("starting user-requested repair of range {} for keyspace
{} and column families {}",
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
index dd25b35..564239b 100644
--- a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
+++ b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
@@ -508,4 +508,10 @@ public class StorageServiceServerTest
repairRangeFrom =
StorageService.instance.createRepairRangeFrom("2000", "2000");
assert repairRangeFrom.size() == 0;
}
+
+@Test(expected=IllegalArgumentException.class)
+public void testIncrementalRepairWithSubrangesThrows() throws Exception
+{
+StorageService.instance.forceRepairRangeAsync("", "", "", true,
true, false, "");
+}
}
cassandra git commit: Reject incremental repair requests combined with subrange repair
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 7e056fa27 -> a8e8a6730
Reject incremental repair requests combined with subrange repair
Patch by Ariel Weisberg; reviewed by marcuse for CASSANDRA-10422
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8e8a673
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8e8a673
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8e8a673
Branch: refs/heads/cassandra-2.1
Commit: a8e8a67306c0b26b8fe9c74a1fb00bacfa224cf7
Parents: 7e056fa
Author: Ariel Weisberg
Authored: Thu Oct 29 12:36:32 2015 -0400
Committer: Marcus Eriksson
Committed: Tue Nov 17 09:57:15 2015 +0100
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/service/StorageService.java | 3 +++
.../org/apache/cassandra/service/StorageServiceServerTest.java | 6 ++
3 files changed, 10 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2eeda94..b6b394a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.12
+ * Reject incremental repair with subrange repair (CASSANDRA-10422)
* Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
* Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
* Invalidate cache after stream receive task is completed (CASSANDRA-10341)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 665ce3a..03c1960 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2830,6 +2830,9 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
public int forceRepairRangeAsync(String beginToken, String endToken,
String keyspaceName, boolean isSequential, boolean isLocal, boolean fullRepair,
String... columnFamilies)
{
+if (!fullRepair)
+throw new IllegalArgumentException("Incremental repair can't be
requested with subrange repair because " +
+ "each subrange repair would
generate an anti-compacted table");
Collection> repairingRange =
createRepairRangeFrom(beginToken, endToken);
logger.info("starting user-requested repair of range {} for keyspace
{} and column families {}",
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8e8a673/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
index dd25b35..564239b 100644
--- a/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
+++ b/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
@@ -508,4 +508,10 @@ public class StorageServiceServerTest
repairRangeFrom =
StorageService.instance.createRepairRangeFrom("2000", "2000");
assert repairRangeFrom.size() == 0;
}
+
+@Test(expected=IllegalArgumentException.class)
+public void testIncrementalRepairWithSubrangesThrows() throws Exception
+{
+StorageService.instance.forceRepairRangeAsync("", "", "", true,
true, false, "");
+}
}
[jira] [Commented] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
[
https://issues.apache.org/jira/browse/CASSANDRA-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008332#comment-15008332
]
Mike Adamson commented on CASSANDRA-10717:
--
[~jlewandowski] Wouldn't it make more sense to have the check for the
authProvider first and then the username & password? Something like this:
{noformat}
if (authProvider.isPresent())
builder.withAuthProvider(authProvider.get());
else if (username.isPresent() && password.isPresent())
builder.withCredentials(username.get(), password.get());
{noformat}
This would allow for a user setting the username and password along with a
non-standard auth provider.
> Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
> --
>
> Key: CASSANDRA-10717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
> Attachments: CqlConfigHelper_authentication_patch.patch
>
>
> Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
> anything but providing username and password in order to work with password
> protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code
> from Thrift to Java Driver, the user needs to explicitly select
> {{AuthProvider}} to make it work.
> I propose a simple change in {{CqlConfigHelper}} which make
> {{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}}
> if they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10677) Improve performance of folderSize function
[ https://issues.apache.org/jira/browse/CASSANDRA-10677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008328#comment-15008328 ] Stefania commented on CASSANDRA-10677: -- There are CI failures but none are related to this change since all unit tests exercising this code path run OK, so +1. > Improve performance of folderSize function > -- > > Key: CASSANDRA-10677 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10677 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths > Environment: Ubuntu 14. JDK 7 >Reporter: Briareus >Priority: Minor > Labels: patch, performance > Fix For: 3.x > > Attachments: > Optimized_folderSize_function_to_use_Java_7_nio_walkFileTree_method_.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > FileUtils.folderSize function recursively traverses the directory tree using > listFiles method. This is no longer efficient as Java 7 offers much better > Files.walkFileTree method. It makes the method work twice faster according to > my tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
[
https://issues.apache.org/jira/browse/CASSANDRA-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008292#comment-15008292
]
Jacek Lewandowski commented on CASSANDRA-10717:
---
ping [~jjordan]
> Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
> --
>
> Key: CASSANDRA-10717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
> Attachments: CqlConfigHelper_authentication_patch.patch
>
>
> Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
> anything but providing username and password in order to work with password
> protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code
> from Thrift to Java Driver, the user needs to explicitly select
> {{AuthProvider}} to make it work.
> I propose a simple change in {{CqlConfigHelper}} which make
> {{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}}
> if they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8879) Alter table on compact storage broken
[
https://issues.apache.org/jira/browse/CASSANDRA-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008256#comment-15008256
]
Sylvain Lebresne commented on CASSANDRA-8879:
-
+1 on the patch, but I'll note the dtests don't seems to have been run except
for the 2.1 branch so I can't vouch for them.
> Alter table on compact storage broken
> -
>
> Key: CASSANDRA-8879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8879
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nick Bailey
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: 8879-2.0.txt
>
>
> In 2.0 HEAD, alter table on compact storage tables seems to be broken. With
> the following table definition, altering the column breaks cqlsh and
> generates a stack trace in the log.
> {noformat}
> CREATE TABLE settings (
> key blob,
> column1 blob,
> value blob,
> PRIMARY KEY ((key), column1)
> ) WITH COMPACT STORAGE
> {noformat}
> {noformat}
> cqlsh:OpsCenter> alter table settings ALTER column1 TYPE ascii ;
> TSocket read 0 bytes
> cqlsh:OpsCenter> DESC TABLE settings;
> {noformat}
> {noformat}
> ERROR [Thrift:7] 2015-02-26 17:20:24,640 CassandraDaemon.java (line 199)
> Exception in thread Thread[Thrift:7,5,main]
> java.lang.AssertionError
> >...at
> >org.apache.cassandra.cql3.statements.AlterTableStatement.announceMigration(AlterTableStatement.java:198)
> >...at
> >org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:79)
> >...at
> >org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:158)
> >...at
> >org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:175)
> >...at
> >org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1958)
> >...at
> >org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4486)
> >...at
> >org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4470)
> >...at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> >...at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> >...at
> >org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
> >...at
> >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >...at
> >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >...at java.lang.Thread.run(Thread.java:724)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10717) Changes from CASSANDRA-9353 makes Hadoop integration backward incompatible
Jacek Lewandowski created CASSANDRA-10717:
-
Summary: Changes from CASSANDRA-9353 makes Hadoop integration
backward incompatible
Key: CASSANDRA-10717
URL: https://issues.apache.org/jira/browse/CASSANDRA-10717
Project: Cassandra
Issue Type: Bug
Reporter: Jacek Lewandowski
Assignee: Jacek Lewandowski
Attachments: CqlConfigHelper_authentication_patch.patch
Previously {{CqlInputFormat.getSplits(JobContext)}} method did not require
anything but providing username and password in order to work with password
protected Cassandra. Now, since CASSANDRA-9353 moved this part of the code from
Thrift to Java Driver, the user needs to explicitly select {{AuthProvider}} to
make it work.
I propose a simple change in {{CqlConfigHelper}} which make
{{CqlConfigHelper.getCluster}} method set credentials on {{Cluster.Builder}} if
they are present in the configuration, prior to setting {{AuthProvider}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8142) prevent the command "cassandra start" from starting a cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-8142: Fix Version/s: 3.x > prevent the command "cassandra start" from starting a cluster > - > > Key: CASSANDRA-8142 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8142 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Steven Lowenthal >Assignee: Robert Stupp > Fix For: 3.x > > > Students often type > "sudo service cassandra start" wrong, and type "sudo cassandra start". > Running a package installation as root messes up their environments. > since "start" is not a valid option on the "cassandra" command, we should > block cassandra from starting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
