[jira] [Commented] (CASSANDRA-12920) Possible messaging latency caused by not flushing in time
[
https://issues.apache.org/jira/browse/CASSANDRA-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678199#comment-15678199
]
Simon Zhou commented on CASSANDRA-12920:
Yes they're time UUIDs. I need to enable GC logging and do another tracing.
> Possible messaging latency caused by not flushing in time
> -
>
> Key: CASSANDRA-12920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12920
> Project: Cassandra
> Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Fix For: 2.2.5
>
>
> Not sure if this can be improved. We're using Cassandra 2.2.5. Cassandra
> considers messages whose payload size <= 64KB as small messages. For response
> for such as digest request, it's very likely a small message. Even quite a
> few such responses cannot fill in the output buffer and thus trigger a flush.
> One possible issue is that, we use conditions count == 1 && backlog.isEmpty()
> to decide whether to flush the output stream. That means if the backlog is
> empty, we won't flush the output stream even we drained, say, 10 messages
> whose sizes sum < 64K. This may cause delay for small messages. Shouldn't we
> flush after writing the last drained message if backlog is empty? Of course
> if backlog is not empty, we can continue draining more messages and very
> likely trigger a flush very soon.
> Here are some tracing events that show the latency. Be noted that the
> involved nodes are in the same data center whose round trip network latency
> is well below 1ms. Alao we have disabled message coalescing due to
> CASSANDRA-12676.
> {noformat}
> 126571b0-aaa6-11e6-ab2b-77eb6f529c59 | 127c7c28-aaa6-11e6-941f-f3a2e2759bd2
> | Enqueuing response to /***32.28 |
> ***29.3 | 1199 | SharedPool-Worker-4
> 126571b0-aaa6-11e6-ab2b-77eb6f529c59 | 127cca45-aaa6-11e6-941f-f3a2e2759bd2
> | Sending REQUEST_RESPONSE message to /***32.28 |
> ***29.3 | 2667 | MessagingService-Outgoing-/***32.28
> 126571b0-aaa6-11e6-ab2b-77eb6f529c59 | 127cca40-aaa6-11e6-ab2b-77eb6f529c59
> | REQUEST_RESPONSE message received from /***29.3 |
> ***32.28 | 152635 | MessagingService-Incoming-/***29.3
> 126571b0-aaa6-11e6-ab2b-77eb6f529c59 | 127cca41-aaa6-11e6-ab2b-77eb6f529c59
> | Processing response from /***29.3 |
> ***32.28 | 152665 |SharedPool-Worker-45
> 1af31490-aaa6-11e6-ab2b-77eb6f529c59 | 1b08e68a-aaa6-11e6-8dce-951d39203586
> | Enqueuing response to /***32.28 |
> ***29.5 |532 | SharedPool-Worker-178
> 1af31490-aaa6-11e6-ab2b-77eb6f529c59 | 1b090d90-aaa6-11e6-8dce-951d39203586
> | Sending REQUEST_RESPONSE message to /***32.28 |
> ***29.5 |579 | MessagingService-Outgoing-/***32.28
> 1af31490-aaa6-11e6-ab2b-77eb6f529c59 | 1b090d90-aaa6-11e6-ab2b-77eb6f529c59
> | REQUEST_RESPONSE message received from /***29.5 |
> ***32.28 | 143865 | MessagingService-Incoming-/***29.5
> 1af31490-aaa6-11e6-ab2b-77eb6f529c59 | 1b090d91-aaa6-11e6-ab2b-77eb6f529c59
> | Processing response from /***29.5 |
> ***32.28 | 143908 |
>SharedPool-Worker-41
> 1af58590-aaa6-11e6-ab2b-77eb6f529c59 | 1b0934a8-aaa6-11e6-8dce-951d39203586
> | Enqueuing response to /***32.28 |
> ***29.5 | 1632 |
> SharedPool-Worker-159 1af58590-aaa6-11e6-ab2b-77eb6f529c59 |
> 1b0934a9-aaa6-11e6-8dce-951d39203586 | Sending
> REQUEST_RESPONSE message to /***32.28 | ***29.5 | 1660 |
> MessagingService
> -Outgoing-/***32.28 1af58590-aaa6-11e6-ab2b-77eb6f529c59 |
> 1b095bb0-aaa6-11e6-ab2b-77eb6f529c59 | REQUEST_RESPONSE
> message received from /***29.5 | ***32.28 | 129813 |
> MessagingServic
> e-Incoming-/***29.5 1af58590-aaa6-11e6-ab2b-77eb6f529c59 |
> 1b095bb1-aaa6-11e6-ab2b-77eb6f529c59 |
> Processing response from /***29.5 | ***32.28 | 129835 |
>
> SharedPool-Worker-3
> 3010cfc0-aaa6-11e6-ab2b-77eb6f529c59 | 3029fd17-aaa6-11e6-a5cc-8b24438aa927
> | Enqueuing response to /***32.28 |
> ***30.8 | 1686 | SharedPool-Worker-271
> 3010cfc0-aaa6-11e6-ab2b-77eb6f529c59 | 302ae770-aaa6-11e6-a5cc-8b24438aa927
> |
cassandra-builds git commit: Enable generated jobs
Repository: cassandra-builds
Updated Branches:
refs/heads/master 5be39659d -> 92c37f78c
Enable generated jobs
Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/92c37f78
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/92c37f78
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/92c37f78
Branch: refs/heads/master
Commit: 92c37f78c36ce5f578cdf26e265d35bcfcf26da1
Parents: 5be3965
Author: Michael Shuler
Authored: Fri Nov 18 17:45:34 2016 -0600
Committer: Michael Shuler
Committed: Fri Nov 18 17:45:34 2016 -0600
--
jenkins-dsl/cassandra_job_dsl_seed.groovy | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/92c37f78/jenkins-dsl/cassandra_job_dsl_seed.groovy
--
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index ae90b3b..146b44d 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -182,7 +182,7 @@ cassandraBranches.each {
* Main branch artifacts and eclipse-warnings job
*/
job("${jobNamePrefix}-artifacts") {
-//disabled(false)
+disabled(false)
using('Cassandra-template-artifacts')
configure { node ->
node / scm / branches / 'hudson.plugins.git.BranchSpec' /
name(branchName)
@@ -199,7 +199,7 @@ cassandraBranches.each {
def targetName = it
job("${jobNamePrefix}-${targetName}") {
-//disabled(false)
+disabled(false)
using('Cassandra-template-test')
configure { node ->
node / scm / branches / 'hudson.plugins.git.BranchSpec' /
name(branchName)
cassandra-builds git commit: Add category throttle and drop empty archives
Repository: cassandra-builds
Updated Branches:
refs/heads/master 1182b97ab -> 5be39659d
Add category throttle and drop empty archives
Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/5be39659
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/5be39659
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/5be39659
Branch: refs/heads/master
Commit: 5be39659d03f01a2a0249241d17511e5b0c5bd1e
Parents: 1182b97
Author: Michael Shuler
Authored: Fri Nov 18 17:31:33 2016 -0600
Committer: Michael Shuler
Committed: Fri Nov 18 17:31:33 2016 -0600
--
jenkins-dsl/cassandra_job_dsl_seed.groovy | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/5be39659/jenkins-dsl/cassandra_job_dsl_seed.groovy
--
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index a4c8a37..ae90b3b 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -41,6 +41,9 @@ job('Cassandra-template-artifacts') {
noActivity(300)
}
}
+throttleConcurrentBuilds {
+categories(['Cassandra'])
+}
scm {
git {
remote {
@@ -84,6 +87,9 @@ job('Cassandra-template-test') {
noActivity(1200)
}
}
+throttleConcurrentBuilds {
+categories(['Cassandra'])
+}
scm {
git {
remote {
@@ -109,7 +115,6 @@ job('Cassandra-template-test') {
stabilityTestDataPublisher()
}
}
-archiveArtifacts('')
}
}
@@ -129,6 +134,9 @@ job('Cassandra-template-dtest') {
noActivity(1200)
}
}
+throttleConcurrentBuilds {
+categories(['Cassandra'])
+}
scm {
git {
remote {
@@ -154,7 +162,6 @@ job('Cassandra-template-dtest') {
stabilityTestDataPublisher()
}
}
-archiveArtifacts('')
}
}
cassandra-builds git commit: Remove console timestamps option (not installed)
Repository: cassandra-builds
Updated Branches:
refs/heads/master 0f40fac31 -> 1182b97ab
Remove console timestamps option (not installed)
Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/1182b97a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/1182b97a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/1182b97a
Branch: refs/heads/master
Commit: 1182b97ab55d8f31017cf63fb4cb8f051fc1a9b9
Parents: 0f40fac
Author: Michael Shuler
Authored: Fri Nov 18 17:16:31 2016 -0600
Committer: Michael Shuler
Committed: Fri Nov 18 17:16:31 2016 -0600
--
jenkins-dsl/cassandra_job_dsl_seed.groovy | 3 ---
1 file changed, 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/1182b97a/jenkins-dsl/cassandra_job_dsl_seed.groovy
--
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 8b9b544..a4c8a37 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -37,7 +37,6 @@ job('Cassandra-template-artifacts') {
artifactNumToKeep(1)
}
wrappers {
-timestamps()
timeout {
noActivity(300)
}
@@ -81,7 +80,6 @@ job('Cassandra-template-test') {
numToKeep(10)
}
wrappers {
-timestamps()
timeout {
noActivity(1200)
}
@@ -127,7 +125,6 @@ job('Cassandra-template-dtest') {
numToKeep(10)
}
wrappers {
-timestamps()
timeout {
noActivity(1200)
}
cassandra-builds git commit: Whatever, DSL.. renamed with underscores..
Repository: cassandra-builds
Updated Branches:
refs/heads/master aebe167d4 -> 0f40fac31
Whatever, DSL.. renamed with underscores..
Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/0f40fac3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/0f40fac3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/0f40fac3
Branch: refs/heads/master
Commit: 0f40fac31968201f6a19608e6afdef647002232b
Parents: aebe167
Author: Michael Shuler
Authored: Fri Nov 18 17:10:06 2016 -0600
Committer: Michael Shuler
Committed: Fri Nov 18 17:10:06 2016 -0600
--
jenkins-dsl/cassandra-job-dsl-seed.groovy | 230 -
jenkins-dsl/cassandra_job_dsl_seed.groovy | 230 +
2 files changed, 230 insertions(+), 230 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/0f40fac3/jenkins-dsl/cassandra-job-dsl-seed.groovy
--
diff --git a/jenkins-dsl/cassandra-job-dsl-seed.groovy
b/jenkins-dsl/cassandra-job-dsl-seed.groovy
deleted file mode 100644
index 8b9b544..000
--- a/jenkins-dsl/cassandra-job-dsl-seed.groovy
+++ /dev/null
@@ -1,230 +0,0 @@
-
-//
-// Common Vars and Branch List
-//
-
-
-def jobDescription = 'Apache Cassandra DSL-generated job - DSL git repo: https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git";>cassandra-builds'
-def jdkLabel = 'jdk1.8.0_66-unlimited-security'
-def slaveLabel = 'cassandra'
-def mainRepo = 'https://git-wip-us.apache.org/repos/asf/cassandra.git'
-def buildsRepo = 'https://git.apache.org/cassandra-builds.git'
-def buildDescStr = 'REF = ${GIT_BRANCH} COMMIT = ${GIT_COMMIT}'
-// Cassandra active branches
-def cassandraBranches = ['cassandra-2.2', 'cassandra-3.0', 'cassandra-3.X',
'trunk']
-// Ant test targets
-def testTargets = ['test', 'test-all', 'test-burn', 'test-cdc',
'test-compression']
-
-
-//
-// Job Templates
-// - disabled by default
-// - running jobs use templates for most configurations
-// and set details like branch
-//
-
-
-/**
- * Artifacts and eclipse-warnings template
- */
-job('Cassandra-template-artifacts') {
-disabled(true)
-description(jobDescription)
-jdk(jdkLabel)
-label(slaveLabel)
-logRotator {
-numToKeep(10)
-artifactNumToKeep(1)
-}
-wrappers {
-timestamps()
-timeout {
-noActivity(300)
-}
-}
-scm {
-git {
-remote {
-url(mainRepo)
-}
-branch('*/null')
-extensions {
-cleanAfterCheckout()
-}
-}
-}
-triggers {
-scm('H/30 * * * *')
-}
-steps {
-buildDescription('', buildDescStr)
-shell("git clean -xdff ; git clone ${buildsRepo}")
-}
-publishers {
-archiveArtifacts('build/*.tar.gz,
build/**/eclipse_compiler_checks.txt')
-archiveJavadoc {
-javadocDir 'build/javadoc'
-keepAll false
-}
-}
-}
-
-/**
- * Ant test template
- */
-job('Cassandra-template-test') {
-disabled(true)
-description(jobDescription)
-jdk(jdkLabel)
-label(slaveLabel)
-logRotator {
-numToKeep(10)
-}
-wrappers {
-timestamps()
-timeout {
-noActivity(1200)
-}
-}
-scm {
-git {
-remote {
-url(mainRepo)
-}
-branch('*/null')
-extensions {
-cleanAfterCheckout()
-}
-}
-}
-triggers {
-scm('@daily')
-}
-steps {
-buildDescription('', buildDescStr)
-shell("git clean -xdff ; git clone ${buildsRepo}")
-}
-publishers {
-junit {
-testResults('**/TEST-*.xml')
-testDataPublishers {
-stabilityTestDataPublisher()
-}
-}
-archiveArtifacts('')
-}
-}
-
-/**
- * Dtest template
- */
-job('Cassandra-template-dtest') {
-disabled(true)
-description(jobDescription)
-jdk(jdkLabel)
-label(slaveLabel)
-logRotator {
-numToKeep(10)
-}
-wrappers {
-timestamps()
-timeout {
-noActivity(1200)
-}
-}
-scm {
-git {
-remote {
-url(mainRepo)
-}
-branch('*/null')
-extensions {
-cleanAfterCheckout()
-}
-}
-}
-triggers {
-sc
cassandra-builds git commit: Disable generated jobs
Repository: cassandra-builds
Updated Branches:
refs/heads/master 8d79f534b -> aebe167d4
Disable generated jobs
Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/aebe167d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/aebe167d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/aebe167d
Branch: refs/heads/master
Commit: aebe167d46caf0f7798873a53e27ba41884eadef
Parents: 8d79f53
Author: Michael Shuler
Authored: Fri Nov 18 16:58:14 2016 -0600
Committer: Michael Shuler
Committed: Fri Nov 18 16:58:14 2016 -0600
--
jenkins-dsl/cassandra-job-dsl-seed.groovy | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/aebe167d/jenkins-dsl/cassandra-job-dsl-seed.groovy
--
diff --git a/jenkins-dsl/cassandra-job-dsl-seed.groovy
b/jenkins-dsl/cassandra-job-dsl-seed.groovy
index ddf5dfe..8b9b544 100644
--- a/jenkins-dsl/cassandra-job-dsl-seed.groovy
+++ b/jenkins-dsl/cassandra-job-dsl-seed.groovy
@@ -178,7 +178,7 @@ cassandraBranches.each {
* Main branch artifacts and eclipse-warnings job
*/
job("${jobNamePrefix}-artifacts") {
-disabled(false)
+//disabled(false)
using('Cassandra-template-artifacts')
configure { node ->
node / scm / branches / 'hudson.plugins.git.BranchSpec' /
name(branchName)
@@ -195,7 +195,7 @@ cassandraBranches.each {
def targetName = it
job("${jobNamePrefix}-${targetName}") {
-disabled(false)
+//disabled(false)
using('Cassandra-template-test')
configure { node ->
node / scm / branches / 'hudson.plugins.git.BranchSpec' /
name(branchName)
cassandra-builds git commit: Add initial DSL builder
Repository: cassandra-builds
Updated Branches:
refs/heads/master cd2f37cc7 -> 8d79f534b
Add initial DSL builder
Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/8d79f534
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/8d79f534
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/8d79f534
Branch: refs/heads/master
Commit: 8d79f534b74c0070dcee719c1c120e28af412df3
Parents: cd2f37c
Author: Michael Shuler
Authored: Fri Nov 18 16:55:58 2016 -0600
Committer: Michael Shuler
Committed: Fri Nov 18 16:55:58 2016 -0600
--
jenkins-dsl/cassandra-job-dsl-seed.groovy | 230 +
1 file changed, 230 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/8d79f534/jenkins-dsl/cassandra-job-dsl-seed.groovy
--
diff --git a/jenkins-dsl/cassandra-job-dsl-seed.groovy
b/jenkins-dsl/cassandra-job-dsl-seed.groovy
new file mode 100644
index 000..ddf5dfe
--- /dev/null
+++ b/jenkins-dsl/cassandra-job-dsl-seed.groovy
@@ -0,0 +1,230 @@
+
+//
+// Common Vars and Branch List
+//
+
+
+def jobDescription = 'Apache Cassandra DSL-generated job - DSL git repo: https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git";>cassandra-builds'
+def jdkLabel = 'jdk1.8.0_66-unlimited-security'
+def slaveLabel = 'cassandra'
+def mainRepo = 'https://git-wip-us.apache.org/repos/asf/cassandra.git'
+def buildsRepo = 'https://git.apache.org/cassandra-builds.git'
+def buildDescStr = 'REF = ${GIT_BRANCH} COMMIT = ${GIT_COMMIT}'
+// Cassandra active branches
+def cassandraBranches = ['cassandra-2.2', 'cassandra-3.0', 'cassandra-3.X',
'trunk']
+// Ant test targets
+def testTargets = ['test', 'test-all', 'test-burn', 'test-cdc',
'test-compression']
+
+
+//
+// Job Templates
+// - disabled by default
+// - running jobs use templates for most configurations
+// and set details like branch
+//
+
+
+/**
+ * Artifacts and eclipse-warnings template
+ */
+job('Cassandra-template-artifacts') {
+disabled(true)
+description(jobDescription)
+jdk(jdkLabel)
+label(slaveLabel)
+logRotator {
+numToKeep(10)
+artifactNumToKeep(1)
+}
+wrappers {
+timestamps()
+timeout {
+noActivity(300)
+}
+}
+scm {
+git {
+remote {
+url(mainRepo)
+}
+branch('*/null')
+extensions {
+cleanAfterCheckout()
+}
+}
+}
+triggers {
+scm('H/30 * * * *')
+}
+steps {
+buildDescription('', buildDescStr)
+shell("git clean -xdff ; git clone ${buildsRepo}")
+}
+publishers {
+archiveArtifacts('build/*.tar.gz,
build/**/eclipse_compiler_checks.txt')
+archiveJavadoc {
+javadocDir 'build/javadoc'
+keepAll false
+}
+}
+}
+
+/**
+ * Ant test template
+ */
+job('Cassandra-template-test') {
+disabled(true)
+description(jobDescription)
+jdk(jdkLabel)
+label(slaveLabel)
+logRotator {
+numToKeep(10)
+}
+wrappers {
+timestamps()
+timeout {
+noActivity(1200)
+}
+}
+scm {
+git {
+remote {
+url(mainRepo)
+}
+branch('*/null')
+extensions {
+cleanAfterCheckout()
+}
+}
+}
+triggers {
+scm('@daily')
+}
+steps {
+buildDescription('', buildDescStr)
+shell("git clean -xdff ; git clone ${buildsRepo}")
+}
+publishers {
+junit {
+testResults('**/TEST-*.xml')
+testDataPublishers {
+stabilityTestDataPublisher()
+}
+}
+archiveArtifacts('')
+}
+}
+
+/**
+ * Dtest template
+ */
+job('Cassandra-template-dtest') {
+disabled(true)
+description(jobDescription)
+jdk(jdkLabel)
+label(slaveLabel)
+logRotator {
+numToKeep(10)
+}
+wrappers {
+timestamps()
+timeout {
+noActivity(1200)
+}
+}
+scm {
+git {
+remote {
+url(mainRepo)
+}
+branch('*/null')
+extensions {
+cleanAfterCheckout()
+}
+}
+}
+triggers {
+scm('@daily')
+}
+steps {
+buildDescription('', buildDescStr)
+shell("git clean -xdff ; git clo
[jira] [Commented] (CASSANDRA-12916) Broken UDT muitations loading from CommitLog
[ https://issues.apache.org/jira/browse/CASSANDRA-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677829#comment-15677829 ] Tyler Hobbs commented on CASSANDRA-12916: - The patch (with the Thrift followup fix) looks correct to me. > Broken UDT muitations loading from CommitLog > > > Key: CASSANDRA-12916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12916 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths > Environment: Cassandra 3.9 >Reporter: Sergey Dobrodey >Assignee: Sam Tunnicliffe >Priority: Critical > Labels: patch > Fix For: 3.x > > Attachments: patch.diff, udt.cql > > > UDT mutatitions seems to be broken. Simple example is attached. After steps > from it, restart cassandra and during commit log reading it will fail with > error: > ERROR 09:34:46 Exiting due to error while processing commit log during > initialization. > org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException: > Unexpected error deserializing mutation; saved to > /tmp/mutation6087238241614604390dat. This may be caused by replaying a > mutation against a table with the same name but incompatible schema. > Exception follows: org.apache.cassandra.serializers.MarshalException: Not > enough bytes to read 0th field data > I resolved this problem, so my patch is in attachment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra-builds git commit: Add eclipse-warnings to artifacts build
Repository: cassandra-builds
Updated Branches:
refs/heads/master 1c49e7003 -> cd2f37cc7
Add eclipse-warnings to artifacts build
Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/cd2f37cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/cd2f37cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/cd2f37cc
Branch: refs/heads/master
Commit: cd2f37cc7b539aa05b9ba9489190149a4863fae3
Parents: 1c49e70
Author: Michael Shuler
Authored: Fri Nov 18 14:33:47 2016 -0600
Committer: Michael Shuler
Committed: Fri Nov 18 14:33:47 2016 -0600
--
build-scripts/cassandra-artifacts.sh | 3 +++
1 file changed, 3 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/cd2f37cc/build-scripts/cassandra-artifacts.sh
--
diff --git a/build-scripts/cassandra-artifacts.sh
b/build-scripts/cassandra-artifacts.sh
index dd480e7..35b632e 100755
--- a/build-scripts/cassandra-artifacts.sh
+++ b/build-scripts/cassandra-artifacts.sh
@@ -23,6 +23,9 @@ for x in $(seq 1 3); do
ant clean artifacts
RETURN="$?"
if [ "${RETURN}" -eq "0" ]; then
+# Run eclipse-warnings if build was successful
+ant eclipse-warnings
+RETURN="$?"
break
fi
done
[jira] [Updated] (CASSANDRA-12281) Gossip blocks on startup when there are pending range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Knighton updated CASSANDRA-12281:
--
Summary: Gossip blocks on startup when there are pending range movements
(was: Gossip blocks on startup when another node is bootstrapping)
> Gossip blocks on startup when there are pending range movements
> ---
>
> Key: CASSANDRA-12281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12281
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Eric Evans
>Assignee: Stefan Podkowinski
> Fix For: 2.2.9, 3.0.11, 3.10
>
> Attachments: 12281-2.2.patch, 12281-3.0.patch, 12281-3.X.patch,
> 12281-trunk.patch, restbase1015-a_jstack.txt
>
>
> In our cluster, normal node startup times (after a drain on shutdown) are
> less than 1 minute. However, when another node in the cluster is
> bootstrapping, the same node startup takes nearly 30 minutes to complete, the
> apparent result of gossip blocking on pending range calculations.
> {noformat}
> $ nodetool-a tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 1840 0
> 0
> ReadStage 0 0 2350 0
> 0
> RequestResponseStage 0 0 53 0
> 0
> ReadRepairStage 0 0 1 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> HintedHandoff 0 0 44 0
> 0
> MiscStage 0 0 0 0
> 0
> CompactionExecutor3 3395 0
> 0
> MemtableReclaimMemory 0 0 30 0
> 0
> PendingRangeCalculator1 2 29 0
> 0
> GossipStage 1 5602164 0
> 0
> MigrationStage0 0 0 0
> 0
> MemtablePostFlush 0 0111 0
> 0
> ValidationExecutor0 0 0 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0 30 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> Message type Dropped
> READ 0
> RANGE_SLICE 0
> _TRACE 0
> MUTATION 0
> COUNTER_MUTATION 0
> REQUEST_RESPONSE 0
> PAGED_RANGE 0
> READ_REPAIR 0
> {noformat}
> A full thread dump is attached, but the relevant bit seems to be here:
> {noformat}
> [ ... ]
> "GossipStage:1" #1801 daemon prio=5 os_prio=0 tid=0x7fe4cd54b000
> nid=0xea9 waiting on condition [0x7fddcf883000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0004c1e922c0> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at
> org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:174)
> at
> org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:160)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2023)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1682)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1182)
> at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossi
[jira] [Resolved] (CASSANDRA-12877) SASI index throwing AssertionError on creation/flush
[
https://issues.apache.org/jira/browse/CASSANDRA-12877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-12877.
Resolution: Fixed
Fix Version/s: (was: 3.10)
Resolved by reverting 11990, per [~ifesdjeen]. Droping fixver 3.10 and closing.
> SASI index throwing AssertionError on creation/flush
>
>
> Key: CASSANDRA-12877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12877
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: 3.9 and 3.10 tested on both linux and osx
>Reporter: Voytek Jarnot
>Assignee: Alex Petrov
>Priority: Blocker
>
> Possibly a 3.10 regression? The exact test shown below does not error in 3.9.
> I built and installed a 3.10 snapshot (built 04-Nov-2016) to get around
> CASSANDRA-11670, CASSANDRA-12689, and CASSANDRA-12223 which are holding me
> back when using 3.9.
> Now I'm able to make nodetool flush (or a scheduled flush) produce an
> unhandled error easily with a SASI:
> {code}
> CREATE KEYSPACE vjtest WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> use vjtest ;
> create table tester(id1 text, id2 text, id3 text, val1 text, primary
> key((id1, id2), id3));
> create custom index tester_idx_val1 on tester(val1) using
> 'org.apache.cassandra.index.sasi.SASIIndex';
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','1-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','2-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','3-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','4-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','5-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','6-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','7-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','8-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','9-3','asdf');
> {code}
> Not enough going on here to trigger a flush, so following a manual {{nodetool
> flush vjtest}} I get the following in {{system.log}}:
> {code}
> INFO [MemtableFlushWriter:3] 2016-11-04 22:19:35,412
> PerSSTableIndexWriter.java:284 - Scheduling index flush to
> /mydir/apache-cassandra-3.10-SNAPSHOT/data/data/vjtest/tester-6f1fdff0a30611e692c087673c5ef8d4/mc-1-big-SI_tester_idx_val1.db
> INFO [SASI-Memtable:1] 2016-11-04 22:19:35,447
> PerSSTableIndexWriter.java:335 - Index flush to
> /mydir/apache-cassandra-3.10-SNAPSHOT/data/data/vjtest/tester-6f1fdff0a30611e692c087673c5ef8d4/mc-1-big-SI_tester_idx_val1.db
> took 16 ms.
> ERROR [SASI-Memtable:1] 2016-11-04 22:19:35,449 CassandraDaemon.java:229 -
> Exception in thread Thread[SASI-Memtable:1,5,RMI Runtime]
> java.lang.AssertionError: cannot have more than 8 overflow collisions per
> leaf, but had: 9
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.createOverflowEntry(AbstractTokenTreeBuilder.java:357)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.createEntry(AbstractTokenTreeBuilder.java:346)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.DynamicTokenTreeBuilder$DynamicLeaf.serializeData(DynamicTokenTreeBuilder.java:180)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.serialize(AbstractTokenTreeBuilder.java:306)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder.write(AbstractTokenTreeBuilder.java:90)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableDataBlock.flushAndClear(OnDiskIndexBuilder.java:629)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableLevel.flush(OnDiskIndexBuilder.java:446)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableLevel.finalFlush(OnDiskIndexBuilder.java:451)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:296)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:258)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:241)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.
[jira] [Updated] (CASSANDRA-11990) Address rows rather than partitions in SASI
[ https://issues.apache.org/jira/browse/CASSANDRA-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-11990: --- Fix Version/s: (was: 3.10) 3.x > Address rows rather than partitions in SASI > --- > > Key: CASSANDRA-11990 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11990 > Project: Cassandra > Issue Type: Improvement > Components: CQL, sasi >Reporter: Alex Petrov >Assignee: Alex Petrov > Fix For: 3.x > > Attachments: perf.pdf, size_comparison.png > > > Currently, the lookup in SASI index would return the key position of the > partition. After the partition lookup, the rows are iterated and the > operators are applied in order to filter out ones that do not match. > bq. TokenTree which accepts variable size keys (such would enable different > partitioners, collections support, primary key indexing etc.), -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12281) Gossip blocks on startup when another node is bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669481#comment-15669481
]
Joel Knighton edited comment on CASSANDRA-12281 at 11/18/16 5:58 PM:
-
Thanks - your changes and CI look good. I also ran CI on your
CASSANDRA-12281-trunk branch.
Note to committer: there are very slight differences in the 2.2/3.0/3.x
branches (not in substantial content, but in comments and other minor fixes).
The 3.x branch should merge cleanly into trunk, I believe.
+1
was (Author: jkni):
Thanks - your changes and CI look good. I also ran CI on your
CASSANDRA-12281-trunk branch.
Note to committer: there are very slight differences in the 2.2/3.0/3.x
branches (not in substantial comment, but in comments and other minor fixes).
The 3.x branch should merge cleanly into trunk, I believe.
+1
> Gossip blocks on startup when another node is bootstrapping
> ---
>
> Key: CASSANDRA-12281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12281
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Eric Evans
>Assignee: Stefan Podkowinski
> Fix For: 2.2.9, 3.0.11, 3.10
>
> Attachments: 12281-2.2.patch, 12281-3.0.patch, 12281-3.X.patch,
> 12281-trunk.patch, restbase1015-a_jstack.txt
>
>
> In our cluster, normal node startup times (after a drain on shutdown) are
> less than 1 minute. However, when another node in the cluster is
> bootstrapping, the same node startup takes nearly 30 minutes to complete, the
> apparent result of gossip blocking on pending range calculations.
> {noformat}
> $ nodetool-a tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 1840 0
> 0
> ReadStage 0 0 2350 0
> 0
> RequestResponseStage 0 0 53 0
> 0
> ReadRepairStage 0 0 1 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> HintedHandoff 0 0 44 0
> 0
> MiscStage 0 0 0 0
> 0
> CompactionExecutor3 3395 0
> 0
> MemtableReclaimMemory 0 0 30 0
> 0
> PendingRangeCalculator1 2 29 0
> 0
> GossipStage 1 5602164 0
> 0
> MigrationStage0 0 0 0
> 0
> MemtablePostFlush 0 0111 0
> 0
> ValidationExecutor0 0 0 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0 30 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> Message type Dropped
> READ 0
> RANGE_SLICE 0
> _TRACE 0
> MUTATION 0
> COUNTER_MUTATION 0
> REQUEST_RESPONSE 0
> PAGED_RANGE 0
> READ_REPAIR 0
> {noformat}
> A full thread dump is attached, but the relevant bit seems to be here:
> {noformat}
> [ ... ]
> "GossipStage:1" #1801 daemon prio=5 os_prio=0 tid=0x7fe4cd54b000
> nid=0xea9 waiting on condition [0x7fddcf883000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0004c1e922c0> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Write
[cassandra] Git Push Summary
Repository: cassandra Updated Tags: refs/tags/3.10-tentative [created] 96d67b109
[jira] [Updated] (CASSANDRA-12281) Gossip blocks on startup when another node is bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-12281:
--
Resolution: Fixed
Fix Version/s: 3.10
3.0.11
2.2.9
Status: Resolved (was: Ready to Commit)
> Gossip blocks on startup when another node is bootstrapping
> ---
>
> Key: CASSANDRA-12281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12281
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Eric Evans
>Assignee: Stefan Podkowinski
> Fix For: 2.2.9, 3.0.11, 3.10
>
> Attachments: 12281-2.2.patch, 12281-3.0.patch, 12281-3.X.patch,
> 12281-trunk.patch, restbase1015-a_jstack.txt
>
>
> In our cluster, normal node startup times (after a drain on shutdown) are
> less than 1 minute. However, when another node in the cluster is
> bootstrapping, the same node startup takes nearly 30 minutes to complete, the
> apparent result of gossip blocking on pending range calculations.
> {noformat}
> $ nodetool-a tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 1840 0
> 0
> ReadStage 0 0 2350 0
> 0
> RequestResponseStage 0 0 53 0
> 0
> ReadRepairStage 0 0 1 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> HintedHandoff 0 0 44 0
> 0
> MiscStage 0 0 0 0
> 0
> CompactionExecutor3 3395 0
> 0
> MemtableReclaimMemory 0 0 30 0
> 0
> PendingRangeCalculator1 2 29 0
> 0
> GossipStage 1 5602164 0
> 0
> MigrationStage0 0 0 0
> 0
> MemtablePostFlush 0 0111 0
> 0
> ValidationExecutor0 0 0 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0 30 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> Message type Dropped
> READ 0
> RANGE_SLICE 0
> _TRACE 0
> MUTATION 0
> COUNTER_MUTATION 0
> REQUEST_RESPONSE 0
> PAGED_RANGE 0
> READ_REPAIR 0
> {noformat}
> A full thread dump is attached, but the relevant bit seems to be here:
> {noformat}
> [ ... ]
> "GossipStage:1" #1801 daemon prio=5 os_prio=0 tid=0x7fe4cd54b000
> nid=0xea9 waiting on condition [0x7fddcf883000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0004c1e922c0> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at
> org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:174)
> at
> org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:160)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2023)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1682)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1182)
> at org.apache.cassandra.gms.Gossiper.applyNewStates
[jira] [Commented] (CASSANDRA-12281) Gossip blocks on startup when another node is bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677239#comment-15677239
]
Aleksey Yeschenko commented on CASSANDRA-12281:
---
Committed to 2.2 as
[9cd7d540de2ea525982d139a4c8a11233c4e98c9|https://github.com/apache/cassandra/commit/9cd7d540de2ea525982d139a4c8a11233c4e98c9]
and merged upwards in accordance with Joel's instructions. Thanks.
> Gossip blocks on startup when another node is bootstrapping
> ---
>
> Key: CASSANDRA-12281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12281
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Eric Evans
>Assignee: Stefan Podkowinski
> Attachments: 12281-2.2.patch, 12281-3.0.patch, 12281-3.X.patch,
> 12281-trunk.patch, restbase1015-a_jstack.txt
>
>
> In our cluster, normal node startup times (after a drain on shutdown) are
> less than 1 minute. However, when another node in the cluster is
> bootstrapping, the same node startup takes nearly 30 minutes to complete, the
> apparent result of gossip blocking on pending range calculations.
> {noformat}
> $ nodetool-a tpstats
> Pool NameActive Pending Completed Blocked All
> time blocked
> MutationStage 0 0 1840 0
> 0
> ReadStage 0 0 2350 0
> 0
> RequestResponseStage 0 0 53 0
> 0
> ReadRepairStage 0 0 1 0
> 0
> CounterMutationStage 0 0 0 0
> 0
> HintedHandoff 0 0 44 0
> 0
> MiscStage 0 0 0 0
> 0
> CompactionExecutor3 3395 0
> 0
> MemtableReclaimMemory 0 0 30 0
> 0
> PendingRangeCalculator1 2 29 0
> 0
> GossipStage 1 5602164 0
> 0
> MigrationStage0 0 0 0
> 0
> MemtablePostFlush 0 0111 0
> 0
> ValidationExecutor0 0 0 0
> 0
> Sampler 0 0 0 0
> 0
> MemtableFlushWriter 0 0 30 0
> 0
> InternalResponseStage 0 0 0 0
> 0
> AntiEntropyStage 0 0 0 0
> 0
> CacheCleanupExecutor 0 0 0 0
> 0
> Message type Dropped
> READ 0
> RANGE_SLICE 0
> _TRACE 0
> MUTATION 0
> COUNTER_MUTATION 0
> REQUEST_RESPONSE 0
> PAGED_RANGE 0
> READ_REPAIR 0
> {noformat}
> A full thread dump is attached, but the relevant bit seems to be here:
> {noformat}
> [ ... ]
> "GossipStage:1" #1801 daemon prio=5 os_prio=0 tid=0x7fe4cd54b000
> nid=0xea9 waiting on condition [0x7fddcf883000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0004c1e922c0> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at
> org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:174)
> at
> org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:160)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2023)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1682)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Go
[01/10] cassandra git commit: Avoid blocking gossip during pending range calculation
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 7d2fdfeb4 -> 9cd7d540d
refs/heads/cassandra-3.0 eb41380cc -> 59b40b317
refs/heads/cassandra-3.X f33cd55a5 -> 96d67b109
refs/heads/trunk 29cb59106 -> f1c3aac76
Avoid blocking gossip during pending range calculation
patch by Stefan Podkowinski; reviewed by Joel Knighton for CASSANDRA-12281
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9cd7d540
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9cd7d540
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9cd7d540
Branch: refs/heads/cassandra-2.2
Commit: 9cd7d540de2ea525982d139a4c8a11233c4e98c9
Parents: 7d2fdfe
Author: Stefan Podkowinski
Authored: Fri Oct 21 11:34:53 2016 +0200
Committer: Aleksey Yeschenko
Committed: Fri Nov 18 17:29:14 2016 +
--
CHANGES.txt | 1 +
.../apache/cassandra/locator/TokenMetadata.java | 211 +++
.../service/PendingRangeCalculatorService.java | 10 +-
.../gms/PendingRangeCalculatorServiceTest.java | 133
4 files changed, 267 insertions(+), 88 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 54dc4b5..5a2e0ab 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
* Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/src/java/org/apache/cassandra/locator/TokenMetadata.java
--
diff --git a/src/java/org/apache/cassandra/locator/TokenMetadata.java
b/src/java/org/apache/cassandra/locator/TokenMetadata.java
index b06c9c8..aafd7f9 100644
--- a/src/java/org/apache/cassandra/locator/TokenMetadata.java
+++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java
@@ -649,12 +649,7 @@ public class TokenMetadata
lock.readLock().lock();
try
{
-TokenMetadata allLeftMetadata = cloneOnlyTokenMap();
-
-for (InetAddress endpoint : leavingEndpoints)
-allLeftMetadata.removeEndpoint(endpoint);
-
-return allLeftMetadata;
+return removeEndpoints(cloneOnlyTokenMap(), leavingEndpoints);
}
finally
{
@@ -662,6 +657,14 @@ public class TokenMetadata
}
}
+private static TokenMetadata removeEndpoints(TokenMetadata
allLeftMetadata, Set leavingEndpoints)
+{
+for (InetAddress endpoint : leavingEndpoints)
+allLeftMetadata.removeEndpoint(endpoint);
+
+return allLeftMetadata;
+}
+
/**
* Create a copy of TokenMetadata with tokenToEndpointMap reflecting
situation after all
* current leave, and move operations have finished.
@@ -787,118 +790,154 @@ public class TokenMetadata
*/
public void calculatePendingRanges(AbstractReplicationStrategy strategy,
String keyspaceName)
{
-lock.readLock().lock();
-try
+// avoid race between both branches - do not use a lock here as this
will block any other unrelated operations!
+synchronized (pendingRanges)
{
-PendingRangeMaps newPendingRanges = new PendingRangeMaps();
-
if (bootstrapTokens.isEmpty() && leavingEndpoints.isEmpty() &&
movingEndpoints.isEmpty())
{
if (logger.isTraceEnabled())
logger.trace("No bootstrapping, leaving or moving nodes ->
empty pending ranges for {}", keyspaceName);
-pendingRanges.put(keyspaceName, newPendingRanges);
-return;
+pendingRanges.put(keyspaceName, new PendingRangeMaps());
}
+else
+{
+if (logger.isDebugEnabled())
+logger.debug("Starting pending range calculation for {}",
keyspaceName);
-Multimap> addressRanges =
strategy.getAddressRanges();
-
-// Copy of metadata reflecting the situation after all leave
operations are finished.
-TokenMetadata allLeftMetadata = cloneAfterAllLeft();
+long startedAt = System.currentTimeMillis();
-// get all ranges that will be affected by leaving nodes
-Set> affectedRanges = new HashSet>();
-for (InetAddress endpoint : leavingEndpoints)
-affectedRanges
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59b40b31 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59b40b31 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59b40b31 Branch: refs/heads/trunk Commit: 59b40b3173933620bc5f30e26366cd09b3a4ca10 Parents: eb41380 9cd7d54 Author: Aleksey Yeschenko Authored: Fri Nov 18 17:29:41 2016 + Committer: Aleksey Yeschenko Committed: Fri Nov 18 17:30:46 2016 + -- CHANGES.txt | 3 + .../apache/cassandra/locator/TokenMetadata.java | 213 +++ .../service/PendingRangeCalculatorService.java | 6 +- .../gms/PendingRangeCalculatorServiceTest.java | 133 4 files changed, 267 insertions(+), 88 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b40b31/CHANGES.txt -- diff --cc CHANGES.txt index 8a3ac65,5a2e0ab..bcd0b5c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,5 +1,45 @@@ -2.2.9 +3.0.11 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) ++Merged from 2.2: + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281) ++ + +3.0.10 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Pass root cause to CorruptBlockException when uncompression failed (CASSANDRA-12889) + * Fix partition count log during compaction (CASSANDRA-12184) + * Batch with multiple conditional updates for the same partition causes AssertionError (CASSANDRA-12867) + * Make AbstractReplicationStrategy extendable from outside its package (CASSANDRA-12788) + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854) + * Don't tell users to turn off consistent rangemovements during rebuild. (CASSANDRA-12296) + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689) + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801) + * Include SSTable filename in compacting large row message (CASSANDRA-12384) + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330) + * Fix ViewTest.testCompaction (CASSANDRA-12789) + * Improve avg aggregate functions (CASSANDRA-12417) + * Preserve quoted reserved keyword column names in MV creation (CASSANDRA-11803) + * nodetool stopdaemon errors out (CASSANDRA-12646) + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268) + * mx4j does not work in 3.0.8 (CASSANDRA-12274) + * Abort cqlsh copy-from in case of no answer after prolonged period of time (CASSANDRA-12740) + * Avoid sstable corrupt exception due to dropped static column (CASSANDRA-12582) + * Make stress use client mode to avoid checking commit log size on startup (CASSANDRA-12478) + * Fix exceptions with new vnode allocation (CASSANDRA-12715) + * Unify drain and shutdown processes (CASSANDRA-12509) + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706) + * Fix failure in LogTransactionTest (CASSANDRA-12632) + * Fix potentially incomplete non-frozen UDT values when querying with the + full primary key specified (CASSANDRA-12605) + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() (CASSANDRA-11670) + * Establish consistent distinction between non-existing partition and NULL value for LWTs on static columns (CASSANDRA-12060) + * Extend ColumnIdentifier.internedInstances key to include the type that generated the byte buffer (CASSANDRA-12516) + * Backport CASSANDRA-10756 (race condition in NativeTransportService shutdown) (CASSANDRA-12472) + * If CF has no clustering columns, any row cache is full partition cache (CASSANDRA-12499) + * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) + * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) +Merged from 2.2: * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b40b31/src/java/org/apache/cassandra/locator/TokenMetadata.java -- diff --cc src/java/org/apache/cassandra/locator/TokenMetadata.java index b50db00,aafd7f9..b44a1a1 --- a/src/java/org/apache/cassandra/locator/TokenMetadata.java +++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java @@@ -128,15 -123,6 +128,15 @@@ public class TokenMetadat sortedTokens = sortTokens(); } +/** - * To be
[04/10] cassandra git commit: Avoid blocking gossip during pending range calculation
Avoid blocking gossip during pending range calculation
patch by Stefan Podkowinski; reviewed by Joel Knighton for CASSANDRA-12281
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9cd7d540
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9cd7d540
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9cd7d540
Branch: refs/heads/trunk
Commit: 9cd7d540de2ea525982d139a4c8a11233c4e98c9
Parents: 7d2fdfe
Author: Stefan Podkowinski
Authored: Fri Oct 21 11:34:53 2016 +0200
Committer: Aleksey Yeschenko
Committed: Fri Nov 18 17:29:14 2016 +
--
CHANGES.txt | 1 +
.../apache/cassandra/locator/TokenMetadata.java | 211 +++
.../service/PendingRangeCalculatorService.java | 10 +-
.../gms/PendingRangeCalculatorServiceTest.java | 133
4 files changed, 267 insertions(+), 88 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 54dc4b5..5a2e0ab 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
* Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/src/java/org/apache/cassandra/locator/TokenMetadata.java
--
diff --git a/src/java/org/apache/cassandra/locator/TokenMetadata.java
b/src/java/org/apache/cassandra/locator/TokenMetadata.java
index b06c9c8..aafd7f9 100644
--- a/src/java/org/apache/cassandra/locator/TokenMetadata.java
+++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java
@@ -649,12 +649,7 @@ public class TokenMetadata
lock.readLock().lock();
try
{
-TokenMetadata allLeftMetadata = cloneOnlyTokenMap();
-
-for (InetAddress endpoint : leavingEndpoints)
-allLeftMetadata.removeEndpoint(endpoint);
-
-return allLeftMetadata;
+return removeEndpoints(cloneOnlyTokenMap(), leavingEndpoints);
}
finally
{
@@ -662,6 +657,14 @@ public class TokenMetadata
}
}
+private static TokenMetadata removeEndpoints(TokenMetadata
allLeftMetadata, Set leavingEndpoints)
+{
+for (InetAddress endpoint : leavingEndpoints)
+allLeftMetadata.removeEndpoint(endpoint);
+
+return allLeftMetadata;
+}
+
/**
* Create a copy of TokenMetadata with tokenToEndpointMap reflecting
situation after all
* current leave, and move operations have finished.
@@ -787,118 +790,154 @@ public class TokenMetadata
*/
public void calculatePendingRanges(AbstractReplicationStrategy strategy,
String keyspaceName)
{
-lock.readLock().lock();
-try
+// avoid race between both branches - do not use a lock here as this
will block any other unrelated operations!
+synchronized (pendingRanges)
{
-PendingRangeMaps newPendingRanges = new PendingRangeMaps();
-
if (bootstrapTokens.isEmpty() && leavingEndpoints.isEmpty() &&
movingEndpoints.isEmpty())
{
if (logger.isTraceEnabled())
logger.trace("No bootstrapping, leaving or moving nodes ->
empty pending ranges for {}", keyspaceName);
-pendingRanges.put(keyspaceName, newPendingRanges);
-return;
+pendingRanges.put(keyspaceName, new PendingRangeMaps());
}
+else
+{
+if (logger.isDebugEnabled())
+logger.debug("Starting pending range calculation for {}",
keyspaceName);
-Multimap> addressRanges =
strategy.getAddressRanges();
-
-// Copy of metadata reflecting the situation after all leave
operations are finished.
-TokenMetadata allLeftMetadata = cloneAfterAllLeft();
+long startedAt = System.currentTimeMillis();
-// get all ranges that will be affected by leaving nodes
-Set> affectedRanges = new HashSet>();
-for (InetAddress endpoint : leavingEndpoints)
-affectedRanges.addAll(addressRanges.get(endpoint));
+// create clone of current state
+BiMultiValMap bootstrapTokens = new
BiMultiValMap<>();
+Set leavingEndpoints = new HashSet<>();
+Set> mo
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59b40b31 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59b40b31 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59b40b31 Branch: refs/heads/cassandra-3.0 Commit: 59b40b3173933620bc5f30e26366cd09b3a4ca10 Parents: eb41380 9cd7d54 Author: Aleksey Yeschenko Authored: Fri Nov 18 17:29:41 2016 + Committer: Aleksey Yeschenko Committed: Fri Nov 18 17:30:46 2016 + -- CHANGES.txt | 3 + .../apache/cassandra/locator/TokenMetadata.java | 213 +++ .../service/PendingRangeCalculatorService.java | 6 +- .../gms/PendingRangeCalculatorServiceTest.java | 133 4 files changed, 267 insertions(+), 88 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b40b31/CHANGES.txt -- diff --cc CHANGES.txt index 8a3ac65,5a2e0ab..bcd0b5c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,5 +1,45 @@@ -2.2.9 +3.0.11 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) ++Merged from 2.2: + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281) ++ + +3.0.10 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Pass root cause to CorruptBlockException when uncompression failed (CASSANDRA-12889) + * Fix partition count log during compaction (CASSANDRA-12184) + * Batch with multiple conditional updates for the same partition causes AssertionError (CASSANDRA-12867) + * Make AbstractReplicationStrategy extendable from outside its package (CASSANDRA-12788) + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854) + * Don't tell users to turn off consistent rangemovements during rebuild. (CASSANDRA-12296) + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689) + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801) + * Include SSTable filename in compacting large row message (CASSANDRA-12384) + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330) + * Fix ViewTest.testCompaction (CASSANDRA-12789) + * Improve avg aggregate functions (CASSANDRA-12417) + * Preserve quoted reserved keyword column names in MV creation (CASSANDRA-11803) + * nodetool stopdaemon errors out (CASSANDRA-12646) + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268) + * mx4j does not work in 3.0.8 (CASSANDRA-12274) + * Abort cqlsh copy-from in case of no answer after prolonged period of time (CASSANDRA-12740) + * Avoid sstable corrupt exception due to dropped static column (CASSANDRA-12582) + * Make stress use client mode to avoid checking commit log size on startup (CASSANDRA-12478) + * Fix exceptions with new vnode allocation (CASSANDRA-12715) + * Unify drain and shutdown processes (CASSANDRA-12509) + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706) + * Fix failure in LogTransactionTest (CASSANDRA-12632) + * Fix potentially incomplete non-frozen UDT values when querying with the + full primary key specified (CASSANDRA-12605) + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() (CASSANDRA-11670) + * Establish consistent distinction between non-existing partition and NULL value for LWTs on static columns (CASSANDRA-12060) + * Extend ColumnIdentifier.internedInstances key to include the type that generated the byte buffer (CASSANDRA-12516) + * Backport CASSANDRA-10756 (race condition in NativeTransportService shutdown) (CASSANDRA-12472) + * If CF has no clustering columns, any row cache is full partition cache (CASSANDRA-12499) + * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) + * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) +Merged from 2.2: * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b40b31/src/java/org/apache/cassandra/locator/TokenMetadata.java -- diff --cc src/java/org/apache/cassandra/locator/TokenMetadata.java index b50db00,aafd7f9..b44a1a1 --- a/src/java/org/apache/cassandra/locator/TokenMetadata.java +++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java @@@ -128,15 -123,6 +128,15 @@@ public class TokenMetadat sortedTokens = sortTokens(); } +/** -
[03/10] cassandra git commit: Avoid blocking gossip during pending range calculation
Avoid blocking gossip during pending range calculation
patch by Stefan Podkowinski; reviewed by Joel Knighton for CASSANDRA-12281
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9cd7d540
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9cd7d540
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9cd7d540
Branch: refs/heads/cassandra-3.X
Commit: 9cd7d540de2ea525982d139a4c8a11233c4e98c9
Parents: 7d2fdfe
Author: Stefan Podkowinski
Authored: Fri Oct 21 11:34:53 2016 +0200
Committer: Aleksey Yeschenko
Committed: Fri Nov 18 17:29:14 2016 +
--
CHANGES.txt | 1 +
.../apache/cassandra/locator/TokenMetadata.java | 211 +++
.../service/PendingRangeCalculatorService.java | 10 +-
.../gms/PendingRangeCalculatorServiceTest.java | 133
4 files changed, 267 insertions(+), 88 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 54dc4b5..5a2e0ab 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
* Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/src/java/org/apache/cassandra/locator/TokenMetadata.java
--
diff --git a/src/java/org/apache/cassandra/locator/TokenMetadata.java
b/src/java/org/apache/cassandra/locator/TokenMetadata.java
index b06c9c8..aafd7f9 100644
--- a/src/java/org/apache/cassandra/locator/TokenMetadata.java
+++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java
@@ -649,12 +649,7 @@ public class TokenMetadata
lock.readLock().lock();
try
{
-TokenMetadata allLeftMetadata = cloneOnlyTokenMap();
-
-for (InetAddress endpoint : leavingEndpoints)
-allLeftMetadata.removeEndpoint(endpoint);
-
-return allLeftMetadata;
+return removeEndpoints(cloneOnlyTokenMap(), leavingEndpoints);
}
finally
{
@@ -662,6 +657,14 @@ public class TokenMetadata
}
}
+private static TokenMetadata removeEndpoints(TokenMetadata
allLeftMetadata, Set leavingEndpoints)
+{
+for (InetAddress endpoint : leavingEndpoints)
+allLeftMetadata.removeEndpoint(endpoint);
+
+return allLeftMetadata;
+}
+
/**
* Create a copy of TokenMetadata with tokenToEndpointMap reflecting
situation after all
* current leave, and move operations have finished.
@@ -787,118 +790,154 @@ public class TokenMetadata
*/
public void calculatePendingRanges(AbstractReplicationStrategy strategy,
String keyspaceName)
{
-lock.readLock().lock();
-try
+// avoid race between both branches - do not use a lock here as this
will block any other unrelated operations!
+synchronized (pendingRanges)
{
-PendingRangeMaps newPendingRanges = new PendingRangeMaps();
-
if (bootstrapTokens.isEmpty() && leavingEndpoints.isEmpty() &&
movingEndpoints.isEmpty())
{
if (logger.isTraceEnabled())
logger.trace("No bootstrapping, leaving or moving nodes ->
empty pending ranges for {}", keyspaceName);
-pendingRanges.put(keyspaceName, newPendingRanges);
-return;
+pendingRanges.put(keyspaceName, new PendingRangeMaps());
}
+else
+{
+if (logger.isDebugEnabled())
+logger.debug("Starting pending range calculation for {}",
keyspaceName);
-Multimap> addressRanges =
strategy.getAddressRanges();
-
-// Copy of metadata reflecting the situation after all leave
operations are finished.
-TokenMetadata allLeftMetadata = cloneAfterAllLeft();
+long startedAt = System.currentTimeMillis();
-// get all ranges that will be affected by leaving nodes
-Set> affectedRanges = new HashSet>();
-for (InetAddress endpoint : leavingEndpoints)
-affectedRanges.addAll(addressRanges.get(endpoint));
+// create clone of current state
+BiMultiValMap bootstrapTokens = new
BiMultiValMap<>();
+Set leavingEndpoints = new HashSet<>();
+
[02/10] cassandra git commit: Avoid blocking gossip during pending range calculation
Avoid blocking gossip during pending range calculation
patch by Stefan Podkowinski; reviewed by Joel Knighton for CASSANDRA-12281
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9cd7d540
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9cd7d540
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9cd7d540
Branch: refs/heads/cassandra-3.0
Commit: 9cd7d540de2ea525982d139a4c8a11233c4e98c9
Parents: 7d2fdfe
Author: Stefan Podkowinski
Authored: Fri Oct 21 11:34:53 2016 +0200
Committer: Aleksey Yeschenko
Committed: Fri Nov 18 17:29:14 2016 +
--
CHANGES.txt | 1 +
.../apache/cassandra/locator/TokenMetadata.java | 211 +++
.../service/PendingRangeCalculatorService.java | 10 +-
.../gms/PendingRangeCalculatorServiceTest.java | 133
4 files changed, 267 insertions(+), 88 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 54dc4b5..5a2e0ab 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
* Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9cd7d540/src/java/org/apache/cassandra/locator/TokenMetadata.java
--
diff --git a/src/java/org/apache/cassandra/locator/TokenMetadata.java
b/src/java/org/apache/cassandra/locator/TokenMetadata.java
index b06c9c8..aafd7f9 100644
--- a/src/java/org/apache/cassandra/locator/TokenMetadata.java
+++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java
@@ -649,12 +649,7 @@ public class TokenMetadata
lock.readLock().lock();
try
{
-TokenMetadata allLeftMetadata = cloneOnlyTokenMap();
-
-for (InetAddress endpoint : leavingEndpoints)
-allLeftMetadata.removeEndpoint(endpoint);
-
-return allLeftMetadata;
+return removeEndpoints(cloneOnlyTokenMap(), leavingEndpoints);
}
finally
{
@@ -662,6 +657,14 @@ public class TokenMetadata
}
}
+private static TokenMetadata removeEndpoints(TokenMetadata
allLeftMetadata, Set leavingEndpoints)
+{
+for (InetAddress endpoint : leavingEndpoints)
+allLeftMetadata.removeEndpoint(endpoint);
+
+return allLeftMetadata;
+}
+
/**
* Create a copy of TokenMetadata with tokenToEndpointMap reflecting
situation after all
* current leave, and move operations have finished.
@@ -787,118 +790,154 @@ public class TokenMetadata
*/
public void calculatePendingRanges(AbstractReplicationStrategy strategy,
String keyspaceName)
{
-lock.readLock().lock();
-try
+// avoid race between both branches - do not use a lock here as this
will block any other unrelated operations!
+synchronized (pendingRanges)
{
-PendingRangeMaps newPendingRanges = new PendingRangeMaps();
-
if (bootstrapTokens.isEmpty() && leavingEndpoints.isEmpty() &&
movingEndpoints.isEmpty())
{
if (logger.isTraceEnabled())
logger.trace("No bootstrapping, leaving or moving nodes ->
empty pending ranges for {}", keyspaceName);
-pendingRanges.put(keyspaceName, newPendingRanges);
-return;
+pendingRanges.put(keyspaceName, new PendingRangeMaps());
}
+else
+{
+if (logger.isDebugEnabled())
+logger.debug("Starting pending range calculation for {}",
keyspaceName);
-Multimap> addressRanges =
strategy.getAddressRanges();
-
-// Copy of metadata reflecting the situation after all leave
operations are finished.
-TokenMetadata allLeftMetadata = cloneAfterAllLeft();
+long startedAt = System.currentTimeMillis();
-// get all ranges that will be affected by leaving nodes
-Set> affectedRanges = new HashSet>();
-for (InetAddress endpoint : leavingEndpoints)
-affectedRanges.addAll(addressRanges.get(endpoint));
+// create clone of current state
+BiMultiValMap bootstrapTokens = new
BiMultiValMap<>();
+Set leavingEndpoints = new HashSet<>();
+
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X
Merge branch 'cassandra-3.0' into cassandra-3.X Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/96d67b10 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/96d67b10 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/96d67b10 Branch: refs/heads/trunk Commit: 96d67b109a2ef858c2753bbb9853d01460cb8f8e Parents: f33cd55 59b40b3 Author: Aleksey Yeschenko Authored: Fri Nov 18 17:31:11 2016 + Committer: Aleksey Yeschenko Committed: Fri Nov 18 17:38:31 2016 + -- CHANGES.txt | 2 +- .../apache/cassandra/locator/TokenMetadata.java | 211 +++ .../service/PendingRangeCalculatorService.java | 6 +- .../gms/PendingRangeCalculatorServiceTest.java | 133 4 files changed, 264 insertions(+), 88 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/96d67b10/CHANGES.txt -- diff --cc CHANGES.txt index ee73b81,bcd0b5c..2826011 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -150,7 -40,6 +150,7 @@@ Merged from 3.0 * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) Merged from 2.2: - === ++ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281) * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) http://git-wip-us.apache.org/repos/asf/cassandra/blob/96d67b10/src/java/org/apache/cassandra/locator/TokenMetadata.java --
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X
Merge branch 'cassandra-3.0' into cassandra-3.X Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/96d67b10 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/96d67b10 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/96d67b10 Branch: refs/heads/cassandra-3.X Commit: 96d67b109a2ef858c2753bbb9853d01460cb8f8e Parents: f33cd55 59b40b3 Author: Aleksey Yeschenko Authored: Fri Nov 18 17:31:11 2016 + Committer: Aleksey Yeschenko Committed: Fri Nov 18 17:38:31 2016 + -- CHANGES.txt | 2 +- .../apache/cassandra/locator/TokenMetadata.java | 211 +++ .../service/PendingRangeCalculatorService.java | 6 +- .../gms/PendingRangeCalculatorServiceTest.java | 133 4 files changed, 264 insertions(+), 88 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/96d67b10/CHANGES.txt -- diff --cc CHANGES.txt index ee73b81,bcd0b5c..2826011 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -150,7 -40,6 +150,7 @@@ Merged from 3.0 * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) Merged from 2.2: - === ++ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281) * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) http://git-wip-us.apache.org/repos/asf/cassandra/blob/96d67b10/src/java/org/apache/cassandra/locator/TokenMetadata.java --
[10/10] cassandra git commit: Merge branch 'cassandra-3.X' into trunk
Merge branch 'cassandra-3.X' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f1c3aac7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f1c3aac7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f1c3aac7 Branch: refs/heads/trunk Commit: f1c3aac7685bd7d847fbb846d91d68a4fa8be1fc Parents: 29cb591 96d67b1 Author: Aleksey Yeschenko Authored: Fri Nov 18 17:38:38 2016 + Committer: Aleksey Yeschenko Committed: Fri Nov 18 17:38:38 2016 + -- CHANGES.txt | 2 +- .../apache/cassandra/locator/TokenMetadata.java | 211 +++ .../service/PendingRangeCalculatorService.java | 6 +- .../gms/PendingRangeCalculatorServiceTest.java | 133 4 files changed, 264 insertions(+), 88 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f1c3aac7/CHANGES.txt --
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59b40b31 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59b40b31 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59b40b31 Branch: refs/heads/cassandra-3.X Commit: 59b40b3173933620bc5f30e26366cd09b3a4ca10 Parents: eb41380 9cd7d54 Author: Aleksey Yeschenko Authored: Fri Nov 18 17:29:41 2016 + Committer: Aleksey Yeschenko Committed: Fri Nov 18 17:30:46 2016 + -- CHANGES.txt | 3 + .../apache/cassandra/locator/TokenMetadata.java | 213 +++ .../service/PendingRangeCalculatorService.java | 6 +- .../gms/PendingRangeCalculatorServiceTest.java | 133 4 files changed, 267 insertions(+), 88 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b40b31/CHANGES.txt -- diff --cc CHANGES.txt index 8a3ac65,5a2e0ab..bcd0b5c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,5 +1,45 @@@ -2.2.9 +3.0.11 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) ++Merged from 2.2: + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281) ++ + +3.0.10 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Pass root cause to CorruptBlockException when uncompression failed (CASSANDRA-12889) + * Fix partition count log during compaction (CASSANDRA-12184) + * Batch with multiple conditional updates for the same partition causes AssertionError (CASSANDRA-12867) + * Make AbstractReplicationStrategy extendable from outside its package (CASSANDRA-12788) + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854) + * Don't tell users to turn off consistent rangemovements during rebuild. (CASSANDRA-12296) + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689) + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801) + * Include SSTable filename in compacting large row message (CASSANDRA-12384) + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330) + * Fix ViewTest.testCompaction (CASSANDRA-12789) + * Improve avg aggregate functions (CASSANDRA-12417) + * Preserve quoted reserved keyword column names in MV creation (CASSANDRA-11803) + * nodetool stopdaemon errors out (CASSANDRA-12646) + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268) + * mx4j does not work in 3.0.8 (CASSANDRA-12274) + * Abort cqlsh copy-from in case of no answer after prolonged period of time (CASSANDRA-12740) + * Avoid sstable corrupt exception due to dropped static column (CASSANDRA-12582) + * Make stress use client mode to avoid checking commit log size on startup (CASSANDRA-12478) + * Fix exceptions with new vnode allocation (CASSANDRA-12715) + * Unify drain and shutdown processes (CASSANDRA-12509) + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706) + * Fix failure in LogTransactionTest (CASSANDRA-12632) + * Fix potentially incomplete non-frozen UDT values when querying with the + full primary key specified (CASSANDRA-12605) + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() (CASSANDRA-11670) + * Establish consistent distinction between non-existing partition and NULL value for LWTs on static columns (CASSANDRA-12060) + * Extend ColumnIdentifier.internedInstances key to include the type that generated the byte buffer (CASSANDRA-12516) + * Backport CASSANDRA-10756 (race condition in NativeTransportService shutdown) (CASSANDRA-12472) + * If CF has no clustering columns, any row cache is full partition cache (CASSANDRA-12499) + * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) + * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) +Merged from 2.2: * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) http://git-wip-us.apache.org/repos/asf/cassandra/blob/59b40b31/src/java/org/apache/cassandra/locator/TokenMetadata.java -- diff --cc src/java/org/apache/cassandra/locator/TokenMetadata.java index b50db00,aafd7f9..b44a1a1 --- a/src/java/org/apache/cassandra/locator/TokenMetadata.java +++ b/src/java/org/apache/cassandra/locator/TokenMetadata.java @@@ -128,15 -123,6 +128,15 @@@ public class TokenMetadat sortedTokens = sortTokens(); } +/** -
[jira] [Updated] (CASSANDRA-12916) Broken UDT muitations loading from CommitLog
[ https://issues.apache.org/jira/browse/CASSANDRA-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-12916: Fix Version/s: 3.x Reproduced In: 3.9, 3.8, 3.7, 3.6 (was: 3.6, 3.7, 3.8, 3.9) Status: Patch Available (was: Open) Looks like this problem has been around since non-frozen UDTs were introduced. I've re-written the patch slightly and added some unit tests, [~thobbs] as the author of CASSANDRA-7423, would you mind casting your eye over it please? The 3.X patch merges cleanly to trunk. ||branch||testall||dtest|| |[12916-3.X|https://github.com/beobal/cassandra/tree/12916-3.X]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12916-3.X-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12916-3.X-dtest]| |[12916-trunk|https://github.com/beobal/cassandra/tree/12916-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12916-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12916-trunk-dtest]| > Broken UDT muitations loading from CommitLog > > > Key: CASSANDRA-12916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12916 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths > Environment: Cassandra 3.9 >Reporter: Sergey Dobrodey >Assignee: Sam Tunnicliffe >Priority: Critical > Labels: patch > Fix For: 3.x > > Attachments: patch.diff, udt.cql > > > UDT mutatitions seems to be broken. Simple example is attached. After steps > from it, restart cassandra and during commit log reading it will fail with > error: > ERROR 09:34:46 Exiting due to error while processing commit log during > initialization. > org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException: > Unexpected error deserializing mutation; saved to > /tmp/mutation6087238241614604390dat. This may be caused by replaying a > mutation against a table with the same name but incompatible schema. > Exception follows: org.apache.cassandra.serializers.MarshalException: Not > enough bytes to read 0th field data > I resolved this problem, so my patch is in attachment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-12916) Broken UDT muitations loading from CommitLog
[ https://issues.apache.org/jira/browse/CASSANDRA-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe reassigned CASSANDRA-12916: --- Assignee: Sam Tunnicliffe > Broken UDT muitations loading from CommitLog > > > Key: CASSANDRA-12916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12916 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths > Environment: Cassandra 3.9 >Reporter: Sergey Dobrodey >Assignee: Sam Tunnicliffe >Priority: Critical > Labels: patch > Attachments: patch.diff, udt.cql > > > UDT mutatitions seems to be broken. Simple example is attached. After steps > from it, restart cassandra and during commit log reading it will fail with > error: > ERROR 09:34:46 Exiting due to error while processing commit log during > initialization. > org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException: > Unexpected error deserializing mutation; saved to > /tmp/mutation6087238241614604390dat. This may be caused by replaying a > mutation against a table with the same name but incompatible schema. > Exception follows: org.apache.cassandra.serializers.MarshalException: Not > enough bytes to read 0th field data > I resolved this problem, so my patch is in attachment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12809) dtest failure in upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_2_x_To_indev_3_0_x.boolean_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673987#comment-15673987
]
Philip Thompson edited comment on CASSANDRA-12809 at 11/18/16 4:29 PM:
---
No repro, retrying:
http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/366/
was (Author: philipthompson):
No repro, retrying:
http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/365/
> dtest failure in
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_2_x_To_indev_3_0_x.boolean_test
> ---
>
> Key: CASSANDRA-12809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12809
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
> Labels: dtest, test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_upgrade/64/testReport/upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_2_2_x_To_indev_3_0_x/boolean_test
> {code}
> Error Message
> Problem starting node node1 due to [Errno 2] No such file or directory:
> '/tmp/dtest-QXmxBV/test/node1/cassandra.pid'
> {code}
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/upgrade_tests/cql_tests.py", line
> 2206, in boolean_test
> for is_upgraded, cursor in self.do_upgrade(cursor):
> File "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_base.py", line
> 153, in do_upgrade
> node1.start(wait_for_binary_proto=True, wait_other_notice=True)
> File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 648, in
> start
> self._update_pid(process)
> File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 1780, in
> _update_pid
> raise NodeError('Problem starting node %s due to %s' % (self.name, e),
> process)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12614) Cassandra stops compacting
[
https://issues.apache.org/jira/browse/CASSANDRA-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Trukhanov resolved CASSANDRA-12614.
-
Resolution: Invalid
It was due to somehow corrupted .data file
As we found out that Maxtimestamp in that sstable was like year 14800+ or
something.
So we fixed that timestamp (by simply overwriting data — thanks to
sstable2json) and it fixed everything
> Cassandra stops compacting
> --
>
> Key: CASSANDRA-12614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12614
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: linux cas-2.1.15
>Reporter: Pavel Trukhanov
>
> After some issues 3 out of 9 our cassandra nodes stopped to compact one
> particular cf. So we see constantly increasing number of sstables for that cf.
> While other cfs on these nodes and other nodes keep compacting.
> This cf has DTCS.
> Here's desc:
> {quote}
> CREATE TABLE okmeter.bunches (
> bunch_key text,
> bunch_ts bigint,
> version uuid,
> value blob,
> PRIMARY KEY (bunch_key, bunch_ts, version)
> ) WITH COMPACT STORAGE
> AND CLUSTERING ORDER BY (bunch_ts DESC, version ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '\{"keys":"ALL", "rows_per_partition":"NONE"\}'
> AND comment = ''
> AND compaction = \{'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy',
> 'max_window_size_seconds': '3686400', 'base_time_seconds': '14400',
> 'max_sstable_age_days': '90'\}
> AND compression = {}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {quote}
> I couldn't find anything interesting in logs.
> I've tried drain & restart - no luck.
> I've tried to disable/enable autocompation - nothing's changed.
> I've tried force major - it works but sstable count starts increasing after
> major is done.
> !https://dl.dropboxusercontent.com/u/8273787/Selection_630_.jpg!
> It seems that it just completely ignores autocompaction enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11307) Inappropriate WARN log message for commit log sync delay when using commitlog_sync: batch
[ https://issues.apache.org/jira/browse/CASSANDRA-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-11307: --- Assignee: (was: Benjamin Lerer) > Inappropriate WARN log message for commit log sync delay when using > commitlog_sync: batch > - > > Key: CASSANDRA-11307 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11307 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Linux >Reporter: Gary Ogasawara >Priority: Minor > > In v2.1.11 when using configuration > commitlog_sync: batch > commitlog_sync_batch_window_in_ms: 2 > will frequently get this WARN log though there is not a real problem. > WARN [COMMIT-LOG-WRITER] 2016-03-03 00:40:11,646 > AbstractCommitLogService.java:105 - Out of 142412 commit log syncs over the > past 300s with average duration of 0.00ms, 35 have exceeded the configured > commit interval by an average of 3.66ms > WARN [COMMIT-LOG-WRITER] 2016-03-03 00:45:11,755 > AbstractCommitLogService.java:105 - Out of 104605 commit log syncs over the > past 300s with average duration of 1.74ms, 18314 have exceeded the configured > commit interval by an average of 6.33ms > Looks like BatchCommitLogService sets AbstractCommitLogService() > pollIntervalMillis to the configured commitlog_sync_batch_window_in_ms. > Since CASSANDRA-9504, this is by default set to 2 which causes this WARN > message to be triggered unnecessarily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11307) Inappropriate WARN log message for commit log sync delay when using commitlog_sync: batch
[ https://issues.apache.org/jira/browse/CASSANDRA-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer reassigned CASSANDRA-11307: -- Assignee: Benjamin Lerer > Inappropriate WARN log message for commit log sync delay when using > commitlog_sync: batch > - > > Key: CASSANDRA-11307 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11307 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Linux >Reporter: Gary Ogasawara >Assignee: Benjamin Lerer >Priority: Minor > > In v2.1.11 when using configuration > commitlog_sync: batch > commitlog_sync_batch_window_in_ms: 2 > will frequently get this WARN log though there is not a real problem. > WARN [COMMIT-LOG-WRITER] 2016-03-03 00:40:11,646 > AbstractCommitLogService.java:105 - Out of 142412 commit log syncs over the > past 300s with average duration of 0.00ms, 35 have exceeded the configured > commit interval by an average of 3.66ms > WARN [COMMIT-LOG-WRITER] 2016-03-03 00:45:11,755 > AbstractCommitLogService.java:105 - Out of 104605 commit log syncs over the > past 300s with average duration of 1.74ms, 18314 have exceeded the configured > commit interval by an average of 6.33ms > Looks like BatchCommitLogService sets AbstractCommitLogService() > pollIntervalMillis to the configured commitlog_sync_batch_window_in_ms. > Since CASSANDRA-9504, this is by default set to 2 which causes this WARN > message to be triggered unnecessarily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676926#comment-15676926
]
Benjamin Lerer edited comment on CASSANDRA-11935 at 11/18/16 3:15 PM:
--
bq. So 100+50 is now going to be varint math?
No, :-)
If you have {{SELECT 100 + 50 ...}} the result will be an {{int}}. The patch
narrow down the type up to the {{int}} level. If a literal can fit in an
{{int}} C* will try to use the {{int}} type for the litteral. If it cannot fit
in an {{int}}, C* will use {{bigint}} or {{varint}} for the literal value.
Depending in which type it can fit in. The output type will then be the one of
the type with the higher precision. If the left operand is fitting in a
{{bigint}} and the right one in an {{int}} the result type will be a {{bigint}}.
The discussion was about the case where you have {{... WHERE pk = 1 + 1}} and
where pk is a {{smallint}} for example.
bq. It's more of a general remark that having intersecting and pretty special
rules doesn't give me the feel of a clean and well understood type system,
which give me slight concerns that some of those rules may prove annoying in
future developments.
This rule is at the end of the chain and is only used for operators. By
consequence, I hope, that if at some point, we have a better alternative, we
will be able to replace it without too much difficulties.
was (Author: blerer):
bq. So 100+50 is now going to be varint math?
No, :-)
If you have {{SELECT 100 + 50 ...}} the result will be an {{int}}. The patch
narrow down the type up to the {{int}} level. If a literal can fit in an
{{int}} C* will try to use the {{int}} type for the litteral. If it cannot fit
in an int C* will use {{bigint}} or {{varint}} for the literal value depending
in which type it can fit in. The output type will then be the one of the type
with the higher precision. So if the left operand is fitting in a {{bigint}}
and the right one in an {{int}} the result type will be a {{bigint}}.
The all discussion is about the case where you have {{... WHERE pk = 1 + 1}}
and where pk is a {{smallint}} for example.
bq. It's more of a general remark that having intersecting and pretty special
rules doesn't give me the feel of a clean and well understood type system,
which give me slight concerns that some of those rules may prove annoying in
future developments.
This rule is at the end of the chain and is only used for operators. By
consequence, I hope that if, at some point, we have a better alternative, we
will be able to replace it without too much difficulties.
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676926#comment-15676926
]
Benjamin Lerer commented on CASSANDRA-11935:
bq. So 100+50 is now going to be varint math?
No, :-)
If you have {{SELECT 100 + 50 ...}} the result will be an {{int}}. The patch
narrow down the type up to the {{int}} level. If a literal can fit in an
{{int}} C* will try to use the {{int}} type for the litteral. If it cannot fit
in an int C* will use {{bigint}} or {{varint}} for the literal value depending
in which type it can fit in. The output type will then be the one of the type
with the higher precision. So if the left operand is fitting in a {{bigint}}
and the right one in an {{int}} the result type will be a {{bigint}}.
The all discussion is about the case where you have {{... WHERE pk = 1 + 1}}
and where pk is a {{smallint}} for example.
bq. It's more of a general remark that having intersecting and pretty special
rules doesn't give me the feel of a clean and well understood type system,
which give me slight concerns that some of those rules may prove annoying in
future developments.
This rule is at the end of the chain and is only used for operators. By
consequence, I hope that if, at some point, we have a better alternative, we
will be able to replace it without too much difficulties.
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12919) Fix inconsistencies in cassandra-stress load balancing policy
[
https://issues.apache.org/jira/browse/CASSANDRA-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676860#comment-15676860
]
T Jake Luciani commented on CASSANDRA-12919:
+1
> Fix inconsistencies in cassandra-stress load balancing policy
> -
>
> Key: CASSANDRA-12919
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12919
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.x
>
>
> The default load balancing policy in cassandra-stress is null, which means
> that the driver will currently wrap {{DCAwareRoundRobinPolicy}} with
> {{TokenAwarePolicy}}.
> However, when a white list of nodes or a data center are specified in the
> command line options, then either {{WhiteListPolicy}} or
> {{DCAwareRoundRobinPolicy}} are used respectively, without wrapping them in a
> {{TokenAwarePolicy}}.
> This means that in the default case token aware routing is applied, but in
> some cases it is not. Token aware routing could give a performance boost of
> up to 2.5 times on bare metal, making comparisons of workloads problematic,
> if it is applied inconsistently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11115) Thrift removal
[ https://issues.apache.org/jira/browse/CASSANDRA-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne reassigned CASSANDRA-5: Assignee: Sylvain Lebresne > Thrift removal > -- > > Key: CASSANDRA-5 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 4.x > > > Thrift removal [has been announced for > 4.0|http://mail-archives.apache.org/mod_mbox/cassandra-user/201601.mbox/%3ccaldd-zgagnldu3pqbd6wp0jb0x73qjdr9phpxmmo+gq+2e5...@mail.gmail.com%3E]. > This ticket is meant to serve as a general task for that removal, but also > to track issue related to that, either things that we should do in 3.x to > make that removal as smooth as possible, or sub-tasks that it makes sense to > separate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676762#comment-15676762
]
Sylvain Lebresne commented on CASSANDRA-11935:
--
bq. I do not think that there is a conflict with the prefered type here.
There is a conflict in principle in the sense that if you have {{d = 3 + 4}}
where {{d}} is a {{bigint}}, then the new rule would lead you to pick {{bigint
add(bigint, bigint)}} while the prefered type suggests {{bigint add(int,
int)}}. Don't get me wrong, both choice happens to be fine and the code pick a
choice. It's more of a general remark that having intersecting and pretty
special rules doesn't give me the feel of a clean and well understood type
system, which give me slight concerns that some of those rules may prove
annoying in future developments.
But I suppose it's not a terribly precise comment, it doesn't seem we're going
anywhere, and as I don't a concrete issue with the current patch, +1.
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676712#comment-15676712
]
Jeremiah Jordan commented on CASSANDRA-11935:
-
bq. The type with the highest precision will always be returned
So 100+50 is now going to be varint math?
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676532#comment-15676532
]
Benjamin Lerer commented on CASSANDRA-11935:
bq. Honestly, if that's the only issue.
Unfortunatly it is not. If you have {{SELECT a + b}} where {{a}} and {{b}} are
columns of {{int}} type you will get the same error message because: {{tinyint
add(int, int)}}, {{smallint add(int, int)}}, {{int add(int, int)}}, etc... will
all match.
bq. It's not a terribly general (it might vaguely make sense for operators, but
I don't think it particuarly does for function in general) and that somewhat
conflict with the "prefered type" choice in some cases on principle.
In my patch this rule is only used for operators.
I do not think that there is a conflict with the prefered type here. The
prefered type is just a way to narrow down the set of functions. Unfortunatly,
it is not enough in this case. The only way to narrow down the set to one here
is to add an extra rule.
Taking into account the fact that the type with the highest precision will
always the one returned I do not think that it is a bad choice.
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12447) Invalid Query : cannot restrict clustering columns when selecting only static columns.
[ https://issues.apache.org/jira/browse/CASSANDRA-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer resolved CASSANDRA-12447. Resolution: Resolved > Invalid Query : cannot restrict clustering columns when selecting only static > columns. > -- > > Key: CASSANDRA-12447 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12447 > Project: Cassandra > Issue Type: Bug > Components: CQL > Environment: cassandra 2.1.13 >Reporter: ZhaoYang >Assignee: Benjamin Lerer >Priority: Minor > > Invalid Query Error: Cannot restrict clustering columns when selecting only > static columns. > Define a table with PartitionKey, ClusteringKey, 1 static Column, 1 normal > column. > "Select PartitionKey from ks.cf where PartitionKey=? and ClusteringKey=?" > will throw the above error.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12447) Invalid Query : cannot restrict clustering columns when selecting only static columns.
[
https://issues.apache.org/jira/browse/CASSANDRA-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676486#comment-15676486
]
Benjamin Lerer commented on CASSANDRA-12447:
I checked {{2.2}} and the query will be accepted there. As {{2.1}} is critical
bug fix only, I think we can close the issue.
> Invalid Query : cannot restrict clustering columns when selecting only static
> columns.
> --
>
> Key: CASSANDRA-12447
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12447
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: cassandra 2.1.13
>Reporter: ZhaoYang
>Assignee: Benjamin Lerer
>Priority: Minor
>
> Invalid Query Error: Cannot restrict clustering columns when selecting only
> static columns.
> Define a table with PartitionKey, ClusteringKey, 1 static Column, 1 normal
> column.
> "Select PartitionKey from ks.cf where PartitionKey=? and ClusteringKey=?"
> will throw the above error..
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12792) delete with timestamp long.MAX_VALUE for the whole key creates tombstone that cannot be removed.
[ https://issues.apache.org/jira/browse/CASSANDRA-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-12792: Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as 7d2fdfeb41eca9badaf10b906b6afe077d166348 to 2.2, eb41380cc27277e34edf2c74f535588fd1382a9a to 3.0, f33cd55a5bbf9a8ba0073c606b971d3b3fc85471 to 3.X and 29cb5910612b603e312d1e99555113d827f6cd9b to trunk. > delete with timestamp long.MAX_VALUE for the whole key creates tombstone that > cannot be removed. > - > > Key: CASSANDRA-12792 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12792 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Ian Ilsley >Assignee: Joel Knighton > > In db/compaction/LazilyCompactedRow.java > we only check for < MaxPurgeableTimeStamp > eg: > (this.maxRowTombstone.markedForDeleteAt < getMaxPurgeableTimestamp()) > this should probably be <= -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12792) delete with timestamp long.MAX_VALUE for the whole key creates tombstone that cannot be removed.
[ https://issues.apache.org/jira/browse/CASSANDRA-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-12792: Status: Ready to Commit (was: Patch Available) > delete with timestamp long.MAX_VALUE for the whole key creates tombstone that > cannot be removed. > - > > Key: CASSANDRA-12792 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12792 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Ian Ilsley >Assignee: Joel Knighton > > In db/compaction/LazilyCompactedRow.java > we only check for < MaxPurgeableTimeStamp > eg: > (this.maxRowTombstone.markedForDeleteAt < getMaxPurgeableTimestamp()) > this should probably be <= -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[05/13] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
--
diff --cc test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
index 26d53ed,4a1f2ca..436b916
--- a/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
@@@ -86,17 -98,16 +86,17 @@@ public class CompactionsPurgeTes
String cfName = "Standard1";
ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(cfName);
-DecoratedKey key = Util.dk("key1");
-Mutation rm;
+String key = "key1";
// inserts
-rm = new Mutation(KEYSPACE1, key.getKey());
for (int i = 0; i < 10; i++)
{
-rm.add(cfName, cellname(String.valueOf(i)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 0);
+RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 0,
key);
+builder.clustering(String.valueOf(i))
- .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
- .build().applyUnsafe();
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
}
-rm.applyUnsafe();
+
cfs.forceBlockingFlush();
// deletes
@@@ -107,19 -120,148 +107,149 @@@
cfs.forceBlockingFlush();
// resurrect one column
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.add(cfName, cellname(String.valueOf(5)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 2);
-rm.applyUnsafe();
+RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 2, key);
+builder.clustering(String.valueOf(5))
- .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
- .build().applyUnsafe();
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
+
cfs.forceBlockingFlush();
// major compact and test that all columns but the resurrected one is
completely gone
FBUtilities.waitOnFutures(CompactionManager.instance.submitMaximal(cfs,
Integer.MAX_VALUE, false));
-cfs.invalidateCachedRow(key);
-ColumnFamily cf =
cfs.getColumnFamily(QueryFilter.getIdentityFilter(key, cfName,
System.currentTimeMillis()));
-assertColumns(cf, "5");
-assertNotNull(cf.getColumn(cellname(String.valueOf(5;
+cfs.invalidateCachedPartition(dk(key));
++
++ImmutableBTreePartition partition =
Util.getOnlyPartitionUnfiltered(Util.cmd(cfs, key).build());
++assertEquals(1, partition.rowCount());
+ }
+
+ @Test
+ public void testMajorCompactionPurgeTombstonesWithMaxTimestamp()
+ {
+ CompactionManager.instance.disableAutoCompaction();
+
+ Keyspace keyspace = Keyspace.open(KEYSPACE1);
+ String cfName = "Standard1";
+ ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(cfName);
+
-DecoratedKey key = Util.dk("key1");
-Mutation rm;
++String key = "key1";
+
+ // inserts
-rm = new Mutation(KEYSPACE1, key.getKey());
+ for (int i = 0; i < 10; i++)
+ {
-rm.add(cfName, cellname(String.valueOf(i)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 0);
++RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 0,
key);
++builder.clustering(String.valueOf(i))
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
+ }
-rm.apply();
+ cfs.forceBlockingFlush();
+
+ // deletes
+ for (int i = 0; i < 10; i++)
+ {
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.delete(cfName, cellname(String.valueOf(i)), Long.MAX_VALUE);
-rm.apply();
++RowUpdateBuilder.deleteRow(cfs.metadata, Long.MAX_VALUE, key,
String.valueOf(i)).applyUnsafe();
+ }
+ cfs.forceBlockingFlush();
+
+ // major compact - tombstones should be purged
+
FBUtilities.waitOnFutures(CompactionManager.instance.submitMaximal(cfs,
Integer.MAX_VALUE, false));
+
+ // resurrect one column
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.add(cfName, cellname(String.valueOf(5)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 2);
-rm.apply();
++RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 2, key);
++builder.clustering(String.valueOf(5))
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
++
+ cfs.forceBlockingFlush();
+
-cfs.invalidateCachedRow(key);
-ColumnFamily cf =
cfs.getColumnFamily(QueryFilter.getIdentityFilter(key, cfName,
System.currentTimeMillis()));
-asser
[13/13] cassandra git commit: Merge branch 'cassandra-3.X' into trunk
Merge branch 'cassandra-3.X' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29cb5910 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29cb5910 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29cb5910 Branch: refs/heads/trunk Commit: 29cb5910612b603e312d1e99555113d827f6cd9b Parents: f55d1b9 f33cd55 Author: Branimir Lambov Authored: Fri Nov 18 12:45:24 2016 +0200 Committer: Branimir Lambov Committed: Fri Nov 18 12:45:24 2016 +0200 -- CHANGES.txt | 2 + .../org/apache/cassandra/db/ReadCommand.java| 5 +- .../db/compaction/CompactionController.java | 46 +-- .../db/compaction/CompactionIterator.java | 22 +-- .../db/compaction/CompactionManager.java| 5 +- .../db/compaction/SSTableSplitter.java | 5 +- .../cassandra/db/compaction/Upgrader.java | 5 +- .../cassandra/db/compaction/Verifier.java | 5 +- .../cassandra/db/partitions/PurgeFunction.java | 6 +- .../db/compaction/CompactionControllerTest.java | 21 ++- .../db/compaction/CompactionsPurgeTest.java | 138 ++- 11 files changed, 213 insertions(+), 47 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/29cb5910/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/29cb5910/src/java/org/apache/cassandra/db/compaction/CompactionManager.java --
[04/13] cassandra git commit: Rework tombstone purgeability checks to not use sentinel timestamp to indicate unconditional purgeability
Rework tombstone purgeability checks to not use sentinel timestamp to indicate
unconditional purgeability
patch by Joel Knighton; reviewed by Branimir Lambov for CASSANDRA-12792
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d2fdfeb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d2fdfeb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d2fdfeb
Branch: refs/heads/trunk
Commit: 7d2fdfeb41eca9badaf10b906b6afe077d166348
Parents: 84b9e72
Author: Joel Knighton
Authored: Thu Oct 20 22:51:51 2016 -0500
Committer: Branimir Lambov
Committed: Fri Nov 18 12:25:29 2016 +0200
--
CHANGES.txt | 1 +
.../db/compaction/CompactionController.java | 58 +---
.../db/compaction/CompactionManager.java| 6 +-
.../db/compaction/LazilyCompactedRow.java | 24 ++--
.../db/compaction/SSTableSplitter.java | 7 +-
.../cassandra/db/compaction/Scrubber.java | 6 +-
.../cassandra/db/compaction/Upgrader.java | 6 +-
.../cassandra/db/compaction/Verifier.java | 6 +-
.../db/compaction/CompactionControllerTest.java | 22 ++--
.../db/compaction/CompactionsPurgeTest.java | 131 +++
10 files changed, 219 insertions(+), 48 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3482052..54dc4b5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
* Fix Util.spinAssertEquals (CASSANDRA-12283)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index 699bc55..e895573 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -19,6 +19,9 @@ package org.apache.cassandra.db.compaction;
import java.util.*;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+
import org.apache.cassandra.io.sstable.format.SSTableReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -186,36 +189,59 @@ public class CompactionController implements AutoCloseable
}
/**
- * @return the largest timestamp before which it's okay to drop tombstones
for the given partition;
- * i.e., after the maxPurgeableTimestamp there may exist newer data that
still needs to be suppressed
- * in other sstables. This returns the minimum timestamp for any SSTable
that contains this partition and is not
- * participating in this compaction, or memtable that contains this
partition,
- * or LONG.MAX_VALUE if no SSTable or memtable exist.
+ * @param key
+ * @return a predicate for whether tombstones marked for deletion at the
given time for the given partition are
+ * purgeable; we calculate this by checking whether the deletion time is
less than the min timestamp of all SSTables
+ * containing his partition and not participating in the compaction. This
means there isn't any data in those
+ * sstables that might still need to be suppressed by a tombstone at this
timestamp.
*/
-public long maxPurgeableTimestamp(DecoratedKey key)
+public Predicate getPurgeEvaluator(DecoratedKey key)
{
if (NEVER_PURGE_TOMBSTONES)
-return Long.MIN_VALUE;
+return Predicates.alwaysFalse();
-long min = Long.MAX_VALUE;
overlapIterator.update(key);
-for (SSTableReader sstable : overlapIterator.overlaps())
+Set filteredSSTables = overlapIterator.overlaps();
+Iterable memtables =
cfs.getTracker().getView().getAllMemtables();
+long minTimestampSeen = Long.MAX_VALUE;
+boolean hasTimestamp = false;
+
+for (SSTableReader sstable: filteredSSTables)
{
// if we don't have bloom filter(bf_fp_chance=1.0 or filter file
is missing),
// we check index file instead.
-if (sstable.getBloomFilter() instanceof AlwaysPresentFilter &&
sstable.getPosition(key, SSTableReader.Operator.EQ, false) != null)
-min = Math.min(min, sstable.getMinTimestamp());
-
[02/13] cassandra git commit: Rework tombstone purgeability checks to not use sentinel timestamp to indicate unconditional purgeability
Rework tombstone purgeability checks to not use sentinel timestamp to indicate
unconditional purgeability
patch by Joel Knighton; reviewed by Branimir Lambov for CASSANDRA-12792
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d2fdfeb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d2fdfeb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d2fdfeb
Branch: refs/heads/cassandra-3.0
Commit: 7d2fdfeb41eca9badaf10b906b6afe077d166348
Parents: 84b9e72
Author: Joel Knighton
Authored: Thu Oct 20 22:51:51 2016 -0500
Committer: Branimir Lambov
Committed: Fri Nov 18 12:25:29 2016 +0200
--
CHANGES.txt | 1 +
.../db/compaction/CompactionController.java | 58 +---
.../db/compaction/CompactionManager.java| 6 +-
.../db/compaction/LazilyCompactedRow.java | 24 ++--
.../db/compaction/SSTableSplitter.java | 7 +-
.../cassandra/db/compaction/Scrubber.java | 6 +-
.../cassandra/db/compaction/Upgrader.java | 6 +-
.../cassandra/db/compaction/Verifier.java | 6 +-
.../db/compaction/CompactionControllerTest.java | 22 ++--
.../db/compaction/CompactionsPurgeTest.java | 131 +++
10 files changed, 219 insertions(+), 48 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3482052..54dc4b5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
* Fix Util.spinAssertEquals (CASSANDRA-12283)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index 699bc55..e895573 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -19,6 +19,9 @@ package org.apache.cassandra.db.compaction;
import java.util.*;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+
import org.apache.cassandra.io.sstable.format.SSTableReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -186,36 +189,59 @@ public class CompactionController implements AutoCloseable
}
/**
- * @return the largest timestamp before which it's okay to drop tombstones
for the given partition;
- * i.e., after the maxPurgeableTimestamp there may exist newer data that
still needs to be suppressed
- * in other sstables. This returns the minimum timestamp for any SSTable
that contains this partition and is not
- * participating in this compaction, or memtable that contains this
partition,
- * or LONG.MAX_VALUE if no SSTable or memtable exist.
+ * @param key
+ * @return a predicate for whether tombstones marked for deletion at the
given time for the given partition are
+ * purgeable; we calculate this by checking whether the deletion time is
less than the min timestamp of all SSTables
+ * containing his partition and not participating in the compaction. This
means there isn't any data in those
+ * sstables that might still need to be suppressed by a tombstone at this
timestamp.
*/
-public long maxPurgeableTimestamp(DecoratedKey key)
+public Predicate getPurgeEvaluator(DecoratedKey key)
{
if (NEVER_PURGE_TOMBSTONES)
-return Long.MIN_VALUE;
+return Predicates.alwaysFalse();
-long min = Long.MAX_VALUE;
overlapIterator.update(key);
-for (SSTableReader sstable : overlapIterator.overlaps())
+Set filteredSSTables = overlapIterator.overlaps();
+Iterable memtables =
cfs.getTracker().getView().getAllMemtables();
+long minTimestampSeen = Long.MAX_VALUE;
+boolean hasTimestamp = false;
+
+for (SSTableReader sstable: filteredSSTables)
{
// if we don't have bloom filter(bf_fp_chance=1.0 or filter file
is missing),
// we check index file instead.
-if (sstable.getBloomFilter() instanceof AlwaysPresentFilter &&
sstable.getPosition(key, SSTableReader.Operator.EQ, false) != null)
-min = Math.min(min, sstable.getMinTimestamp()
[09/13] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
--
diff --cc test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
index 26d53ed,4a1f2ca..436b916
--- a/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
@@@ -86,17 -98,16 +86,17 @@@ public class CompactionsPurgeTes
String cfName = "Standard1";
ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(cfName);
-DecoratedKey key = Util.dk("key1");
-Mutation rm;
+String key = "key1";
// inserts
-rm = new Mutation(KEYSPACE1, key.getKey());
for (int i = 0; i < 10; i++)
{
-rm.add(cfName, cellname(String.valueOf(i)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 0);
+RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 0,
key);
+builder.clustering(String.valueOf(i))
- .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
- .build().applyUnsafe();
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
}
-rm.applyUnsafe();
+
cfs.forceBlockingFlush();
// deletes
@@@ -107,19 -120,148 +107,149 @@@
cfs.forceBlockingFlush();
// resurrect one column
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.add(cfName, cellname(String.valueOf(5)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 2);
-rm.applyUnsafe();
+RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 2, key);
+builder.clustering(String.valueOf(5))
- .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
- .build().applyUnsafe();
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
+
cfs.forceBlockingFlush();
// major compact and test that all columns but the resurrected one is
completely gone
FBUtilities.waitOnFutures(CompactionManager.instance.submitMaximal(cfs,
Integer.MAX_VALUE, false));
-cfs.invalidateCachedRow(key);
-ColumnFamily cf =
cfs.getColumnFamily(QueryFilter.getIdentityFilter(key, cfName,
System.currentTimeMillis()));
-assertColumns(cf, "5");
-assertNotNull(cf.getColumn(cellname(String.valueOf(5;
+cfs.invalidateCachedPartition(dk(key));
++
++ImmutableBTreePartition partition =
Util.getOnlyPartitionUnfiltered(Util.cmd(cfs, key).build());
++assertEquals(1, partition.rowCount());
+ }
+
+ @Test
+ public void testMajorCompactionPurgeTombstonesWithMaxTimestamp()
+ {
+ CompactionManager.instance.disableAutoCompaction();
+
+ Keyspace keyspace = Keyspace.open(KEYSPACE1);
+ String cfName = "Standard1";
+ ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(cfName);
+
-DecoratedKey key = Util.dk("key1");
-Mutation rm;
++String key = "key1";
+
+ // inserts
-rm = new Mutation(KEYSPACE1, key.getKey());
+ for (int i = 0; i < 10; i++)
+ {
-rm.add(cfName, cellname(String.valueOf(i)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 0);
++RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 0,
key);
++builder.clustering(String.valueOf(i))
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
+ }
-rm.apply();
+ cfs.forceBlockingFlush();
+
+ // deletes
+ for (int i = 0; i < 10; i++)
+ {
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.delete(cfName, cellname(String.valueOf(i)), Long.MAX_VALUE);
-rm.apply();
++RowUpdateBuilder.deleteRow(cfs.metadata, Long.MAX_VALUE, key,
String.valueOf(i)).applyUnsafe();
+ }
+ cfs.forceBlockingFlush();
+
+ // major compact - tombstones should be purged
+
FBUtilities.waitOnFutures(CompactionManager.instance.submitMaximal(cfs,
Integer.MAX_VALUE, false));
+
+ // resurrect one column
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.add(cfName, cellname(String.valueOf(5)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 2);
-rm.apply();
++RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 2, key);
++builder.clustering(String.valueOf(5))
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
++
+ cfs.forceBlockingFlush();
+
-cfs.invalidateCachedRow(key);
-ColumnFamily cf =
cfs.getColumnFamily(QueryFilter.getIdentityFilter(key, cfName,
System.currentTimeMillis()));
-asser
[12/13] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X
Merge branch 'cassandra-3.0' into cassandra-3.X
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f33cd55a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f33cd55a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f33cd55a
Branch: refs/heads/trunk
Commit: f33cd55a5bbf9a8ba0073c606b971d3b3fc85471
Parents: 490c1c2 eb41380
Author: Branimir Lambov
Authored: Fri Nov 18 12:43:04 2016 +0200
Committer: Branimir Lambov
Committed: Fri Nov 18 12:44:09 2016 +0200
--
CHANGES.txt | 2 +
.../org/apache/cassandra/db/ReadCommand.java| 5 +-
.../db/compaction/CompactionController.java | 46 +--
.../db/compaction/CompactionIterator.java | 22 +--
.../db/compaction/CompactionManager.java| 5 +-
.../db/compaction/SSTableSplitter.java | 5 +-
.../cassandra/db/compaction/Upgrader.java | 5 +-
.../cassandra/db/compaction/Verifier.java | 5 +-
.../cassandra/db/partitions/PurgeFunction.java | 6 +-
.../db/compaction/CompactionControllerTest.java | 21 ++-
.../db/compaction/CompactionsPurgeTest.java | 138 ++-
11 files changed, 213 insertions(+), 47 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f33cd55a/CHANGES.txt
--
diff --cc CHANGES.txt
index 6ca26f9,8a3ac65..ee73b81
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -150,6 -37,7 +150,8 @@@ Merged from 3.0
* Correct log message for statistics of offheap memtable flush
(CASSANDRA-12776)
* Explicitly set locale for string validation
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
Merged from 2.2:
++===
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
* Fix Util.spinAssertEquals (CASSANDRA-12283)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f33cd55a/src/java/org/apache/cassandra/db/ReadCommand.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f33cd55a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionController.java
index b34eee6,34d093e..64c35d9
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@@ -18,13 -18,10 +18,14 @@@
package org.apache.cassandra.db.compaction;
import java.util.*;
+ import java.util.function.Predicate;
import org.apache.cassandra.db.Memtable;
+import org.apache.cassandra.db.rows.UnfilteredRowIterator;
+
+import com.google.common.base.Predicates;
import com.google.common.collect.Iterables;
+import com.google.common.util.concurrent.RateLimiter;
import org.apache.cassandra.db.partitions.Partition;
import org.apache.cassandra.io.sstable.format.SSTableReader;
@@@ -213,20 -194,24 +214,24 @@@ public class CompactionController imple
}
/**
- * @return the largest timestamp before which it's okay to drop
tombstones for the given partition;
- * i.e., after the maxPurgeableTimestamp there may exist newer data that
still needs to be suppressed
- * in other sstables. This returns the minimum timestamp for any SSTable
that contains this partition and is not
- * participating in this compaction, or memtable that contains this
partition,
- * or LONG.MAX_VALUE if no SSTable or memtable exist.
+ * @param key
+ * @return a predicate for whether tombstones marked for deletion at the
given time for the given partition are
+ * purgeable; we calculate this by checking whether the deletion time is
less than the min timestamp of all SSTables
+ * containing his partition and not participating in the compaction. This
means there isn't any data in those
+ * sstables that might still need to be suppressed by a tombstone at this
timestamp.
*/
- public long maxPurgeableTimestamp(DecoratedKey key)
+ public Predicate getPurgeEvaluator(DecoratedKey key)
{
-if (!compactingRepaired() || NEVER_PURGE_TOMBSTONES)
+if (NEVER_PURGE_TOMBSTONES || !compactingRepaired())
- return Long.MIN_VALUE;
+ return time -> false;
- long min = Long.MAX_VALUE;
overlapIterator.update(key);
- for (SSTableReader sstable : overlapIterator.overlaps())
+ Set filteredSSTables = overlapIterator.ov
[06/13] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb41380c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb41380c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb41380c Branch: refs/heads/cassandra-3.X Commit: eb41380cc27277e34edf2c74f535588fd1382a9a Parents: 14f36fc 7d2fdfe Author: Branimir Lambov Authored: Fri Nov 18 12:35:32 2016 +0200 Committer: Branimir Lambov Committed: Fri Nov 18 12:36:26 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/ReadCommand.java| 5 +- .../db/compaction/CompactionController.java | 50 --- .../db/compaction/CompactionIterator.java | 22 +-- .../db/compaction/CompactionManager.java| 5 +- .../db/compaction/SSTableSplitter.java | 5 +- .../cassandra/db/compaction/Upgrader.java | 5 +- .../cassandra/db/compaction/Verifier.java | 5 +- .../cassandra/db/partitions/PurgeFunction.java | 6 +- .../db/compaction/CompactionControllerTest.java | 21 ++- .../db/compaction/CompactionsPurgeTest.java | 138 ++- 11 files changed, 213 insertions(+), 50 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/CHANGES.txt -- diff --cc CHANGES.txt index efc681d,54dc4b5..8a3ac65 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,5 +1,43 @@@ -2.2.9 +3.0.11 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + +3.0.10 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Pass root cause to CorruptBlockException when uncompression failed (CASSANDRA-12889) + * Fix partition count log during compaction (CASSANDRA-12184) + * Batch with multiple conditional updates for the same partition causes AssertionError (CASSANDRA-12867) + * Make AbstractReplicationStrategy extendable from outside its package (CASSANDRA-12788) + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854) + * Don't tell users to turn off consistent rangemovements during rebuild. (CASSANDRA-12296) + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689) + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801) + * Include SSTable filename in compacting large row message (CASSANDRA-12384) + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330) + * Fix ViewTest.testCompaction (CASSANDRA-12789) + * Improve avg aggregate functions (CASSANDRA-12417) + * Preserve quoted reserved keyword column names in MV creation (CASSANDRA-11803) + * nodetool stopdaemon errors out (CASSANDRA-12646) + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268) + * mx4j does not work in 3.0.8 (CASSANDRA-12274) + * Abort cqlsh copy-from in case of no answer after prolonged period of time (CASSANDRA-12740) + * Avoid sstable corrupt exception due to dropped static column (CASSANDRA-12582) + * Make stress use client mode to avoid checking commit log size on startup (CASSANDRA-12478) + * Fix exceptions with new vnode allocation (CASSANDRA-12715) + * Unify drain and shutdown processes (CASSANDRA-12509) + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706) + * Fix failure in LogTransactionTest (CASSANDRA-12632) + * Fix potentially incomplete non-frozen UDT values when querying with the + full primary key specified (CASSANDRA-12605) + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() (CASSANDRA-11670) + * Establish consistent distinction between non-existing partition and NULL value for LWTs on static columns (CASSANDRA-12060) + * Extend ColumnIdentifier.internedInstances key to include the type that generated the byte buffer (CASSANDRA-12516) + * Backport CASSANDRA-10756 (race condition in NativeTransportService shutdown) (CASSANDRA-12472) + * If CF has no clustering columns, any row cache is full partition cache (CASSANDRA-12499) + * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) + * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) +Merged from 2.2: + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) * Fix Util.spinAssertEquals (CASSANDRA-12283) http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/src/java/org/apache/cassandra/db/ReadCommand.java -- diff
[10/13] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb41380c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb41380c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb41380c Branch: refs/heads/trunk Commit: eb41380cc27277e34edf2c74f535588fd1382a9a Parents: 14f36fc 7d2fdfe Author: Branimir Lambov Authored: Fri Nov 18 12:35:32 2016 +0200 Committer: Branimir Lambov Committed: Fri Nov 18 12:36:26 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/ReadCommand.java| 5 +- .../db/compaction/CompactionController.java | 50 --- .../db/compaction/CompactionIterator.java | 22 +-- .../db/compaction/CompactionManager.java| 5 +- .../db/compaction/SSTableSplitter.java | 5 +- .../cassandra/db/compaction/Upgrader.java | 5 +- .../cassandra/db/compaction/Verifier.java | 5 +- .../cassandra/db/partitions/PurgeFunction.java | 6 +- .../db/compaction/CompactionControllerTest.java | 21 ++- .../db/compaction/CompactionsPurgeTest.java | 138 ++- 11 files changed, 213 insertions(+), 50 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/CHANGES.txt -- diff --cc CHANGES.txt index efc681d,54dc4b5..8a3ac65 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,5 +1,43 @@@ -2.2.9 +3.0.11 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + +3.0.10 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Pass root cause to CorruptBlockException when uncompression failed (CASSANDRA-12889) + * Fix partition count log during compaction (CASSANDRA-12184) + * Batch with multiple conditional updates for the same partition causes AssertionError (CASSANDRA-12867) + * Make AbstractReplicationStrategy extendable from outside its package (CASSANDRA-12788) + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854) + * Don't tell users to turn off consistent rangemovements during rebuild. (CASSANDRA-12296) + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689) + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801) + * Include SSTable filename in compacting large row message (CASSANDRA-12384) + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330) + * Fix ViewTest.testCompaction (CASSANDRA-12789) + * Improve avg aggregate functions (CASSANDRA-12417) + * Preserve quoted reserved keyword column names in MV creation (CASSANDRA-11803) + * nodetool stopdaemon errors out (CASSANDRA-12646) + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268) + * mx4j does not work in 3.0.8 (CASSANDRA-12274) + * Abort cqlsh copy-from in case of no answer after prolonged period of time (CASSANDRA-12740) + * Avoid sstable corrupt exception due to dropped static column (CASSANDRA-12582) + * Make stress use client mode to avoid checking commit log size on startup (CASSANDRA-12478) + * Fix exceptions with new vnode allocation (CASSANDRA-12715) + * Unify drain and shutdown processes (CASSANDRA-12509) + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706) + * Fix failure in LogTransactionTest (CASSANDRA-12632) + * Fix potentially incomplete non-frozen UDT values when querying with the + full primary key specified (CASSANDRA-12605) + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() (CASSANDRA-11670) + * Establish consistent distinction between non-existing partition and NULL value for LWTs on static columns (CASSANDRA-12060) + * Extend ColumnIdentifier.internedInstances key to include the type that generated the byte buffer (CASSANDRA-12516) + * Backport CASSANDRA-10756 (race condition in NativeTransportService shutdown) (CASSANDRA-12472) + * If CF has no clustering columns, any row cache is full partition cache (CASSANDRA-12499) + * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) + * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) +Merged from 2.2: + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) * Fix Util.spinAssertEquals (CASSANDRA-12283) http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/src/java/org/apache/cassandra/db/ReadCommand.java -- diff --cc src
[08/13] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb41380c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb41380c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb41380c Branch: refs/heads/cassandra-3.0 Commit: eb41380cc27277e34edf2c74f535588fd1382a9a Parents: 14f36fc 7d2fdfe Author: Branimir Lambov Authored: Fri Nov 18 12:35:32 2016 +0200 Committer: Branimir Lambov Committed: Fri Nov 18 12:36:26 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/ReadCommand.java| 5 +- .../db/compaction/CompactionController.java | 50 --- .../db/compaction/CompactionIterator.java | 22 +-- .../db/compaction/CompactionManager.java| 5 +- .../db/compaction/SSTableSplitter.java | 5 +- .../cassandra/db/compaction/Upgrader.java | 5 +- .../cassandra/db/compaction/Verifier.java | 5 +- .../cassandra/db/partitions/PurgeFunction.java | 6 +- .../db/compaction/CompactionControllerTest.java | 21 ++- .../db/compaction/CompactionsPurgeTest.java | 138 ++- 11 files changed, 213 insertions(+), 50 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/CHANGES.txt -- diff --cc CHANGES.txt index efc681d,54dc4b5..8a3ac65 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,5 +1,43 @@@ -2.2.9 +3.0.11 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + +3.0.10 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Pass root cause to CorruptBlockException when uncompression failed (CASSANDRA-12889) + * Fix partition count log during compaction (CASSANDRA-12184) + * Batch with multiple conditional updates for the same partition causes AssertionError (CASSANDRA-12867) + * Make AbstractReplicationStrategy extendable from outside its package (CASSANDRA-12788) + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854) + * Don't tell users to turn off consistent rangemovements during rebuild. (CASSANDRA-12296) + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689) + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801) + * Include SSTable filename in compacting large row message (CASSANDRA-12384) + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330) + * Fix ViewTest.testCompaction (CASSANDRA-12789) + * Improve avg aggregate functions (CASSANDRA-12417) + * Preserve quoted reserved keyword column names in MV creation (CASSANDRA-11803) + * nodetool stopdaemon errors out (CASSANDRA-12646) + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268) + * mx4j does not work in 3.0.8 (CASSANDRA-12274) + * Abort cqlsh copy-from in case of no answer after prolonged period of time (CASSANDRA-12740) + * Avoid sstable corrupt exception due to dropped static column (CASSANDRA-12582) + * Make stress use client mode to avoid checking commit log size on startup (CASSANDRA-12478) + * Fix exceptions with new vnode allocation (CASSANDRA-12715) + * Unify drain and shutdown processes (CASSANDRA-12509) + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706) + * Fix failure in LogTransactionTest (CASSANDRA-12632) + * Fix potentially incomplete non-frozen UDT values when querying with the + full primary key specified (CASSANDRA-12605) + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() (CASSANDRA-11670) + * Establish consistent distinction between non-existing partition and NULL value for LWTs on static columns (CASSANDRA-12060) + * Extend ColumnIdentifier.internedInstances key to include the type that generated the byte buffer (CASSANDRA-12516) + * Backport CASSANDRA-10756 (race condition in NativeTransportService shutdown) (CASSANDRA-12472) + * If CF has no clustering columns, any row cache is full partition cache (CASSANDRA-12499) + * Correct log message for statistics of offheap memtable flush (CASSANDRA-12776) + * Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545) +Merged from 2.2: + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792) * Fail repair if participant dies during sync or anticompaction (CASSANDRA-12901) * cqlsh COPY: unprotected pk values before converting them if not using prepared statements (CASSANDRA-12863) * Fix Util.spinAssertEquals (CASSANDRA-12283) http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/src/java/org/apache/cassandra/db/ReadCommand.java -- diff
[07/13] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb41380c/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
--
diff --cc test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
index 26d53ed,4a1f2ca..436b916
--- a/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java
@@@ -86,17 -98,16 +86,17 @@@ public class CompactionsPurgeTes
String cfName = "Standard1";
ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(cfName);
-DecoratedKey key = Util.dk("key1");
-Mutation rm;
+String key = "key1";
// inserts
-rm = new Mutation(KEYSPACE1, key.getKey());
for (int i = 0; i < 10; i++)
{
-rm.add(cfName, cellname(String.valueOf(i)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 0);
+RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 0,
key);
+builder.clustering(String.valueOf(i))
- .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
- .build().applyUnsafe();
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
}
-rm.applyUnsafe();
+
cfs.forceBlockingFlush();
// deletes
@@@ -107,19 -120,148 +107,149 @@@
cfs.forceBlockingFlush();
// resurrect one column
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.add(cfName, cellname(String.valueOf(5)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 2);
-rm.applyUnsafe();
+RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 2, key);
+builder.clustering(String.valueOf(5))
- .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
- .build().applyUnsafe();
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
+
cfs.forceBlockingFlush();
// major compact and test that all columns but the resurrected one is
completely gone
FBUtilities.waitOnFutures(CompactionManager.instance.submitMaximal(cfs,
Integer.MAX_VALUE, false));
-cfs.invalidateCachedRow(key);
-ColumnFamily cf =
cfs.getColumnFamily(QueryFilter.getIdentityFilter(key, cfName,
System.currentTimeMillis()));
-assertColumns(cf, "5");
-assertNotNull(cf.getColumn(cellname(String.valueOf(5;
+cfs.invalidateCachedPartition(dk(key));
++
++ImmutableBTreePartition partition =
Util.getOnlyPartitionUnfiltered(Util.cmd(cfs, key).build());
++assertEquals(1, partition.rowCount());
+ }
+
+ @Test
+ public void testMajorCompactionPurgeTombstonesWithMaxTimestamp()
+ {
+ CompactionManager.instance.disableAutoCompaction();
+
+ Keyspace keyspace = Keyspace.open(KEYSPACE1);
+ String cfName = "Standard1";
+ ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(cfName);
+
-DecoratedKey key = Util.dk("key1");
-Mutation rm;
++String key = "key1";
+
+ // inserts
-rm = new Mutation(KEYSPACE1, key.getKey());
+ for (int i = 0; i < 10; i++)
+ {
-rm.add(cfName, cellname(String.valueOf(i)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 0);
++RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 0,
key);
++builder.clustering(String.valueOf(i))
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
+ }
-rm.apply();
+ cfs.forceBlockingFlush();
+
+ // deletes
+ for (int i = 0; i < 10; i++)
+ {
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.delete(cfName, cellname(String.valueOf(i)), Long.MAX_VALUE);
-rm.apply();
++RowUpdateBuilder.deleteRow(cfs.metadata, Long.MAX_VALUE, key,
String.valueOf(i)).applyUnsafe();
+ }
+ cfs.forceBlockingFlush();
+
+ // major compact - tombstones should be purged
+
FBUtilities.waitOnFutures(CompactionManager.instance.submitMaximal(cfs,
Integer.MAX_VALUE, false));
+
+ // resurrect one column
-rm = new Mutation(KEYSPACE1, key.getKey());
-rm.add(cfName, cellname(String.valueOf(5)),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 2);
-rm.apply();
++RowUpdateBuilder builder = new RowUpdateBuilder(cfs.metadata, 2, key);
++builder.clustering(String.valueOf(5))
++ .add("val", ByteBufferUtil.EMPTY_BYTE_BUFFER)
++ .build().applyUnsafe();
++
+ cfs.forceBlockingFlush();
+
-cfs.invalidateCachedRow(key);
-ColumnFamily cf =
cfs.getColumnFamily(QueryFilter.getIdentityFilter(key, cfName,
System.currentTimeMillis()));
-asser
[11/13] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X
Merge branch 'cassandra-3.0' into cassandra-3.X
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f33cd55a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f33cd55a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f33cd55a
Branch: refs/heads/cassandra-3.X
Commit: f33cd55a5bbf9a8ba0073c606b971d3b3fc85471
Parents: 490c1c2 eb41380
Author: Branimir Lambov
Authored: Fri Nov 18 12:43:04 2016 +0200
Committer: Branimir Lambov
Committed: Fri Nov 18 12:44:09 2016 +0200
--
CHANGES.txt | 2 +
.../org/apache/cassandra/db/ReadCommand.java| 5 +-
.../db/compaction/CompactionController.java | 46 +--
.../db/compaction/CompactionIterator.java | 22 +--
.../db/compaction/CompactionManager.java| 5 +-
.../db/compaction/SSTableSplitter.java | 5 +-
.../cassandra/db/compaction/Upgrader.java | 5 +-
.../cassandra/db/compaction/Verifier.java | 5 +-
.../cassandra/db/partitions/PurgeFunction.java | 6 +-
.../db/compaction/CompactionControllerTest.java | 21 ++-
.../db/compaction/CompactionsPurgeTest.java | 138 ++-
11 files changed, 213 insertions(+), 47 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f33cd55a/CHANGES.txt
--
diff --cc CHANGES.txt
index 6ca26f9,8a3ac65..ee73b81
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -150,6 -37,7 +150,8 @@@ Merged from 3.0
* Correct log message for statistics of offheap memtable flush
(CASSANDRA-12776)
* Explicitly set locale for string validation
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
Merged from 2.2:
++===
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
* Fix Util.spinAssertEquals (CASSANDRA-12283)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f33cd55a/src/java/org/apache/cassandra/db/ReadCommand.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f33cd55a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionController.java
index b34eee6,34d093e..64c35d9
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@@ -18,13 -18,10 +18,14 @@@
package org.apache.cassandra.db.compaction;
import java.util.*;
+ import java.util.function.Predicate;
import org.apache.cassandra.db.Memtable;
+import org.apache.cassandra.db.rows.UnfilteredRowIterator;
+
+import com.google.common.base.Predicates;
import com.google.common.collect.Iterables;
+import com.google.common.util.concurrent.RateLimiter;
import org.apache.cassandra.db.partitions.Partition;
import org.apache.cassandra.io.sstable.format.SSTableReader;
@@@ -213,20 -194,24 +214,24 @@@ public class CompactionController imple
}
/**
- * @return the largest timestamp before which it's okay to drop
tombstones for the given partition;
- * i.e., after the maxPurgeableTimestamp there may exist newer data that
still needs to be suppressed
- * in other sstables. This returns the minimum timestamp for any SSTable
that contains this partition and is not
- * participating in this compaction, or memtable that contains this
partition,
- * or LONG.MAX_VALUE if no SSTable or memtable exist.
+ * @param key
+ * @return a predicate for whether tombstones marked for deletion at the
given time for the given partition are
+ * purgeable; we calculate this by checking whether the deletion time is
less than the min timestamp of all SSTables
+ * containing his partition and not participating in the compaction. This
means there isn't any data in those
+ * sstables that might still need to be suppressed by a tombstone at this
timestamp.
*/
- public long maxPurgeableTimestamp(DecoratedKey key)
+ public Predicate getPurgeEvaluator(DecoratedKey key)
{
-if (!compactingRepaired() || NEVER_PURGE_TOMBSTONES)
+if (NEVER_PURGE_TOMBSTONES || !compactingRepaired())
- return Long.MIN_VALUE;
+ return time -> false;
- long min = Long.MAX_VALUE;
overlapIterator.update(key);
- for (SSTableReader sstable : overlapIterator.overlaps())
+ Set filteredSSTables = overlapIte
[03/13] cassandra git commit: Rework tombstone purgeability checks to not use sentinel timestamp to indicate unconditional purgeability
Rework tombstone purgeability checks to not use sentinel timestamp to indicate
unconditional purgeability
patch by Joel Knighton; reviewed by Branimir Lambov for CASSANDRA-12792
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d2fdfeb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d2fdfeb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d2fdfeb
Branch: refs/heads/cassandra-3.X
Commit: 7d2fdfeb41eca9badaf10b906b6afe077d166348
Parents: 84b9e72
Author: Joel Knighton
Authored: Thu Oct 20 22:51:51 2016 -0500
Committer: Branimir Lambov
Committed: Fri Nov 18 12:25:29 2016 +0200
--
CHANGES.txt | 1 +
.../db/compaction/CompactionController.java | 58 +---
.../db/compaction/CompactionManager.java| 6 +-
.../db/compaction/LazilyCompactedRow.java | 24 ++--
.../db/compaction/SSTableSplitter.java | 7 +-
.../cassandra/db/compaction/Scrubber.java | 6 +-
.../cassandra/db/compaction/Upgrader.java | 6 +-
.../cassandra/db/compaction/Verifier.java | 6 +-
.../db/compaction/CompactionControllerTest.java | 22 ++--
.../db/compaction/CompactionsPurgeTest.java | 131 +++
10 files changed, 219 insertions(+), 48 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3482052..54dc4b5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
* Fix Util.spinAssertEquals (CASSANDRA-12283)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index 699bc55..e895573 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -19,6 +19,9 @@ package org.apache.cassandra.db.compaction;
import java.util.*;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+
import org.apache.cassandra.io.sstable.format.SSTableReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -186,36 +189,59 @@ public class CompactionController implements AutoCloseable
}
/**
- * @return the largest timestamp before which it's okay to drop tombstones
for the given partition;
- * i.e., after the maxPurgeableTimestamp there may exist newer data that
still needs to be suppressed
- * in other sstables. This returns the minimum timestamp for any SSTable
that contains this partition and is not
- * participating in this compaction, or memtable that contains this
partition,
- * or LONG.MAX_VALUE if no SSTable or memtable exist.
+ * @param key
+ * @return a predicate for whether tombstones marked for deletion at the
given time for the given partition are
+ * purgeable; we calculate this by checking whether the deletion time is
less than the min timestamp of all SSTables
+ * containing his partition and not participating in the compaction. This
means there isn't any data in those
+ * sstables that might still need to be suppressed by a tombstone at this
timestamp.
*/
-public long maxPurgeableTimestamp(DecoratedKey key)
+public Predicate getPurgeEvaluator(DecoratedKey key)
{
if (NEVER_PURGE_TOMBSTONES)
-return Long.MIN_VALUE;
+return Predicates.alwaysFalse();
-long min = Long.MAX_VALUE;
overlapIterator.update(key);
-for (SSTableReader sstable : overlapIterator.overlaps())
+Set filteredSSTables = overlapIterator.overlaps();
+Iterable memtables =
cfs.getTracker().getView().getAllMemtables();
+long minTimestampSeen = Long.MAX_VALUE;
+boolean hasTimestamp = false;
+
+for (SSTableReader sstable: filteredSSTables)
{
// if we don't have bloom filter(bf_fp_chance=1.0 or filter file
is missing),
// we check index file instead.
-if (sstable.getBloomFilter() instanceof AlwaysPresentFilter &&
sstable.getPosition(key, SSTableReader.Operator.EQ, false) != null)
-min = Math.min(min, sstable.getMinTimestamp()
[01/13] cassandra git commit: Rework tombstone purgeability checks to not use sentinel timestamp to indicate unconditional purgeability
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 84b9e7278 -> 7d2fdfeb4
refs/heads/cassandra-3.0 14f36fce3 -> eb41380cc
refs/heads/cassandra-3.X 490c1c27c -> f33cd55a5
refs/heads/trunk f55d1b9f0 -> 29cb59106
Rework tombstone purgeability checks to not use sentinel timestamp to indicate
unconditional purgeability
patch by Joel Knighton; reviewed by Branimir Lambov for CASSANDRA-12792
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d2fdfeb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d2fdfeb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d2fdfeb
Branch: refs/heads/cassandra-2.2
Commit: 7d2fdfeb41eca9badaf10b906b6afe077d166348
Parents: 84b9e72
Author: Joel Knighton
Authored: Thu Oct 20 22:51:51 2016 -0500
Committer: Branimir Lambov
Committed: Fri Nov 18 12:25:29 2016 +0200
--
CHANGES.txt | 1 +
.../db/compaction/CompactionController.java | 58 +---
.../db/compaction/CompactionManager.java| 6 +-
.../db/compaction/LazilyCompactedRow.java | 24 ++--
.../db/compaction/SSTableSplitter.java | 7 +-
.../cassandra/db/compaction/Scrubber.java | 6 +-
.../cassandra/db/compaction/Upgrader.java | 6 +-
.../cassandra/db/compaction/Verifier.java | 6 +-
.../db/compaction/CompactionControllerTest.java | 22 ++--
.../db/compaction/CompactionsPurgeTest.java | 131 +++
10 files changed, 219 insertions(+), 48 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3482052..54dc4b5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.9
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
* Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
* cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
* Fix Util.spinAssertEquals (CASSANDRA-12283)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d2fdfeb/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index 699bc55..e895573 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -19,6 +19,9 @@ package org.apache.cassandra.db.compaction;
import java.util.*;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+
import org.apache.cassandra.io.sstable.format.SSTableReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -186,36 +189,59 @@ public class CompactionController implements AutoCloseable
}
/**
- * @return the largest timestamp before which it's okay to drop tombstones
for the given partition;
- * i.e., after the maxPurgeableTimestamp there may exist newer data that
still needs to be suppressed
- * in other sstables. This returns the minimum timestamp for any SSTable
that contains this partition and is not
- * participating in this compaction, or memtable that contains this
partition,
- * or LONG.MAX_VALUE if no SSTable or memtable exist.
+ * @param key
+ * @return a predicate for whether tombstones marked for deletion at the
given time for the given partition are
+ * purgeable; we calculate this by checking whether the deletion time is
less than the min timestamp of all SSTables
+ * containing his partition and not participating in the compaction. This
means there isn't any data in those
+ * sstables that might still need to be suppressed by a tombstone at this
timestamp.
*/
-public long maxPurgeableTimestamp(DecoratedKey key)
+public Predicate getPurgeEvaluator(DecoratedKey key)
{
if (NEVER_PURGE_TOMBSTONES)
-return Long.MIN_VALUE;
+return Predicates.alwaysFalse();
-long min = Long.MAX_VALUE;
overlapIterator.update(key);
-for (SSTableReader sstable : overlapIterator.overlaps())
+Set filteredSSTables = overlapIterator.overlaps();
+Iterable memtables =
cfs.getTracker().getView().getAllMemtables();
+long minTimestampSeen = Long.MAX_VALUE;
+boolean hasTimestamp = false;
+
+for (SSTableReader sstable: filteredSSTables)
{
// if we don't have bloom filter(bf_fp_chance=1.0 or filter file
is missing),
// we
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676457#comment-15676457
]
Sylvain Lebresne commented on CASSANDRA-11935:
--
bq. It would work for {{i = 100 + 50}} but not for {{SELECT 100 + 50}} where we
will get an error like: {{Ambiguous '+' operation: use type casts to
disambiguate}}.
Honestly, if that's the only issue, I think it'd be fairly acceptable. {{SELECT
100 + 50}} is kind of useless (CQL is not meant to be a calculator) so having
to explicit the return type you want is imo fine (it's not crazy to ask the
user which format it wants that returned since there is multiple valid
options). And forcing an explicit cast in that case doesn't even forbid us to
improve later if we really really want.
bq. In such a case, I believe that we should just pick the operation with the
same argument types than the return type.
It's a somewhat random rule though imo. It's not a terribly general (it might
vaguely make sense for operators, but I don't think it particuarly does for
function in general) and that somewhat conflict with the "prefered type" choice
in some cases on principle.
In general, I think it's not a good thing to have too many special cases for
the function resolution mechanism (simple regular rules are always better in my
experience) and I fear doing so could lead to unexpected issue/difficulties in
the future, so I still would prefer "just" adding more overloads. I know there
is a concern on the memory footprint of those overloads, but have we even
measured that overhead before worrying (it's easy to do and it's fixed)? And
even if it's non negligeable, I suspect we could very well reduce that
overhead, it's largely an implementation detail.
bq. For the {{preferedType}} I still believe that we should stop at {{int}} for
integer to minimize the risk of overflow issues in the selection part.
I can live with that (if I'm being honest, I think I'm a bit umconfortable with
the {{preferedType}} concept in the first place, but I don't a simple good
alternative to suggest either).
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12924) GraphiteReporter does not reconnect if graphite restarts
[
https://issues.apache.org/jira/browse/CASSANDRA-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676413#comment-15676413
]
Stefano Ortolani commented on CASSANDRA-12924:
--
Would it be ok to mix minor versions?
In other words, C* provides metrics-core 3.1.0 while metrics-graphite needs to
be included by the user.
Could I just deploy metrics-graphite 3.1.2 and fix the problem, right?
> GraphiteReporter does not reconnect if graphite restarts
>
>
> Key: CASSANDRA-12924
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12924
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stefano Ortolani
>
> Seems like GraphiteReporter does not reconnect after graphite is restarted.
> The consequence is complete loss of reported metrics until Cassandra
> restarts. Logs show this every minute:
> {noformat}
> WARN [metrics-graphite-reporter-1-thread-1] 2016-11-17 10:06:26,549
> GraphiteReporter.java:179 - Unable to report to Graphite
> java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[na:1.8.0_91]
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.write(BufferedWriter.java:230) ~[na:1.8.0_91]
> at java.io.Writer.write(Writer.java:157) ~[na:1.8.0_91]
> at com.codahale.metrics.graphite.Graphite.send(Graphite.java:130)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:283)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
> [metrics-core-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117)
> [metrics-core-3.1.0.jar:3.1.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_91]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_91]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_91]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_91]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN [metrics-graphite-reporter-1-thread-1] 2016-11-17 10:06:26,549
> GraphiteReporter.java:183 - Error closing Graphite
> java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> ~[na:1.8.0_91]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
> ~[na:1.8.0_91]
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[na:1.8.0_91]
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> ~[na:1.8.0_91]
> at java.io.BufferedWriter.write(BufferedWriter.java:230) ~[na:1.8.0_91]
> at java.io.Writer.write(Writer.java:157) ~[na:1.8.0_91]
> at com.codahale.metrics.graphite.Graphite.send(Graphite.java:130)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:283)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
> [metrics-core-3.1.0.jar:3.1.0]
>
[jira] [Updated] (CASSANDRA-12916) Broken UDT muitations loading from CommitLog
[ https://issues.apache.org/jira/browse/CASSANDRA-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-12916: Reproduced In: 3.9, 3.8, 3.7, 3.6 (was: 3.9) Reviewer: Sam Tunnicliffe > Broken UDT muitations loading from CommitLog > > > Key: CASSANDRA-12916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12916 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths > Environment: Cassandra 3.9 >Reporter: Sergey Dobrodey >Priority: Critical > Labels: patch > Attachments: patch.diff, udt.cql > > > UDT mutatitions seems to be broken. Simple example is attached. After steps > from it, restart cassandra and during commit log reading it will fail with > error: > ERROR 09:34:46 Exiting due to error while processing commit log during > initialization. > org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException: > Unexpected error deserializing mutation; saved to > /tmp/mutation6087238241614604390dat. This may be caused by replaying a > mutation against a table with the same name but incompatible schema. > Exception follows: org.apache.cassandra.serializers.MarshalException: Not > enough bytes to read 0th field data > I resolved this problem, so my patch is in attachment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12792) delete with timestamp long.MAX_VALUE for the whole key creates tombstone that cannot be removed.
[ https://issues.apache.org/jira/browse/CASSANDRA-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676339#comment-15676339 ] Branimir Lambov commented on CASSANDRA-12792: - +1 > delete with timestamp long.MAX_VALUE for the whole key creates tombstone that > cannot be removed. > - > > Key: CASSANDRA-12792 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12792 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Ian Ilsley >Assignee: Joel Knighton > > In db/compaction/LazilyCompactedRow.java > we only check for < MaxPurgeableTimeStamp > eg: > (this.maxRowTombstone.markedForDeleteAt < getMaxPurgeableTimestamp()) > this should probably be <= -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12447) Invalid Query : cannot restrict clustering columns when selecting only static columns.
[ https://issues.apache.org/jira/browse/CASSANDRA-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-12447: --- Component/s: CQL > Invalid Query : cannot restrict clustering columns when selecting only static > columns. > -- > > Key: CASSANDRA-12447 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12447 > Project: Cassandra > Issue Type: Bug > Components: CQL > Environment: cassandra 2.1.13 >Reporter: ZhaoYang >Assignee: Benjamin Lerer >Priority: Minor > > Invalid Query Error: Cannot restrict clustering columns when selecting only > static columns. > Define a table with PartitionKey, ClusteringKey, 1 static Column, 1 normal > column. > "Select PartitionKey from ks.cf where PartitionKey=? and ClusteringKey=?" > will throw the above error.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12447) Invalid Query : cannot restrict clustering columns when selecting only static columns.
[ https://issues.apache.org/jira/browse/CASSANDRA-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-12447: --- Summary: Invalid Query : cannot restrict clustering columns when selecting only static columns. (was: Invalid Query : cannot restrict static Column when selecting only static columns.) > Invalid Query : cannot restrict clustering columns when selecting only static > columns. > -- > > Key: CASSANDRA-12447 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12447 > Project: Cassandra > Issue Type: Bug > Environment: cassandra 2.1.13 >Reporter: ZhaoYang >Assignee: Benjamin Lerer >Priority: Minor > > Invalid Query Error: Cannot restrict clustering columns when selecting only > static columns. > Define a table with PartitionKey, ClusteringKey, 1 static Column, 1 normal > column. > "Select PartitionKey from ks.cf where PartitionKey=? and ClusteringKey=?" > will throw the above error.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12447) Invalid Query : cannot restrict static Column when selecting only static columns.
[ https://issues.apache.org/jira/browse/CASSANDRA-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-12447: - Summary: Invalid Query : cannot restrict static Column when selecting only static columns. (was: Invalid Query : cannot restrict static Column) > Invalid Query : cannot restrict static Column when selecting only static > columns. > - > > Key: CASSANDRA-12447 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12447 > Project: Cassandra > Issue Type: Bug > Environment: cassandra 2.1.13 >Reporter: ZhaoYang >Assignee: Benjamin Lerer >Priority: Minor > > Invalid Query Error: Cannot restrict clustering columns when selecting only > static columns. > Define a table with PartitionKey, ClusteringKey, 1 static Column, 1 normal > column. > "Select PartitionKey from ks.cf where PartitionKey=? and ClusteringKey=?" > will throw the above error.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676172#comment-15676172
]
Benjamin Lerer commented on CASSANDRA-11935:
I pushed a new patch which:
* revert the {{preferedType}} change
* use the return type to improve the function selection (CASSANDRA-11946)
* select the operation with the same argument types than the return type when
constant literals are used
||branch||utests||dtests||
|[3.X|https://github.com/apache/cassandra/compare/trunk...blerer:11935-3.X]|[3.X|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-11935-3.X-testall/]|[3.X|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-11935-3.X-dtest/]|
|[trunk|https://github.com/apache/cassandra/compare/trunk...blerer:11935-trunk]|[trunk|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-11935-trunk-testall/]|[trunk|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-11935-trunk-dtest/]|
> Add support for arithmetic operators
>
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep
> the scope as small as possible. Dates and string operations will be adressed
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are
> performed on different types (e.g. double + int).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-12447) Invalid Query : cannot restrict static Column
[ https://issues.apache.org/jira/browse/CASSANDRA-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer reassigned CASSANDRA-12447: -- Assignee: Benjamin Lerer > Invalid Query : cannot restrict static Column > - > > Key: CASSANDRA-12447 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12447 > Project: Cassandra > Issue Type: Bug > Environment: cassandra 2.1.13 >Reporter: ZhaoYang >Assignee: Benjamin Lerer >Priority: Minor > > Invalid Query Error: Cannot restrict clustering columns when selecting only > static columns. > Define a table with PartitionKey, ClusteringKey, 1 static Column, 1 normal > column. > "Select PartitionKey from ks.cf where PartitionKey=? and ClusteringKey=?" > will throw the above error.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12447) Invalid Query : cannot restrict static Column
[ https://issues.apache.org/jira/browse/CASSANDRA-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676133#comment-15676133 ] ZhaoYang commented on CASSANDRA-12447: -- Thanks Alex. I just wonder: should we make the same query behaves the same way no matter whether static column exists? > Invalid Query : cannot restrict static Column > - > > Key: CASSANDRA-12447 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12447 > Project: Cassandra > Issue Type: Bug > Environment: cassandra 2.1.13 >Reporter: ZhaoYang >Priority: Minor > > Invalid Query Error: Cannot restrict clustering columns when selecting only > static columns. > Define a table with PartitionKey, ClusteringKey, 1 static Column, 1 normal > column. > "Select PartitionKey from ks.cf where PartitionKey=? and ClusteringKey=?" > will throw the above error.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12880) setInt/setShort/setFlaot throwing exception from CassandraPreparedStatement
[ https://issues.apache.org/jira/browse/CASSANDRA-12880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer resolved CASSANDRA-12880. Resolution: Invalid Fix Version/s: (was: 2.1.x) > setInt/setShort/setFlaot throwing exception from CassandraPreparedStatement > --- > > Key: CASSANDRA-12880 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12880 > Project: Cassandra > Issue Type: Test > Components: Core > Environment: Cassandra JDBC >Reporter: Raghavendra Pinninti >Priority: Trivial > > Getting Exception as for ps.setInt/ps.setFloat methods > Exception in thread "main" java.lang.NoSuchMethodError: > org.apache.cassandra.cql.jdbc.JdbcInteger.decompose(Ljava/math/BigInteger;)Ljava/nio/ByteBuffer; > at > org.apache.cassandra.cql.jdbc.CassandraPreparedStatement.setShort(CassandraPreparedStatement.java:381) > at com.jdbc.cassandra.CassandraJdbc.Insert(CassandraJdbc.java:63) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12880) setInt/setShort/setFlaot throwing exception from CassandraPreparedStatement
[
https://issues.apache.org/jira/browse/CASSANDRA-12880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676130#comment-15676130
]
Benjamin Lerer commented on CASSANDRA-12880:
{{org.apache.cassandra.cql.jdbc.JdbcInteger}} and the other classes mentioned
are not Cassandra classes.
They seems to belong to the Cassandra JDBC driver and I do not know why they
use the {{org.apache.cassandra}} domain name.
> setInt/setShort/setFlaot throwing exception from CassandraPreparedStatement
> ---
>
> Key: CASSANDRA-12880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12880
> Project: Cassandra
> Issue Type: Test
> Components: Core
> Environment: Cassandra JDBC
>Reporter: Raghavendra Pinninti
>Priority: Trivial
> Fix For: 2.1.x
>
>
> Getting Exception as for ps.setInt/ps.setFloat methods
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.cassandra.cql.jdbc.JdbcInteger.decompose(Ljava/math/BigInteger;)Ljava/nio/ByteBuffer;
> at
> org.apache.cassandra.cql.jdbc.CassandraPreparedStatement.setShort(CassandraPreparedStatement.java:381)
> at com.jdbc.cassandra.CassandraJdbc.Insert(CassandraJdbc.java:63)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
