[jira] [Created] (CASSANDRA-13817) cassandra allow filtering bug
wang huatao created CASSANDRA-13817: --- Summary: cassandra allow filtering bug Key: CASSANDRA-13817 URL: https://issues.apache.org/jira/browse/CASSANDRA-13817 Project: Cassandra Issue Type: Bug Components: CQL, Libraries Reporter: wang huatao Fix For: 3.10 I have one bug about cassandra cql, when I use select * from table where name = 'myName' alllow filtering, sometimes can be found, but sometimes can not found. I am very sure the row data existed. my data was very small, just 2000 rows.and only one node. i use cassandra 3.10, ubuntu 14. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13817) cassandra allow filtering bug
[ https://issues.apache.org/jira/browse/CASSANDRA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wang huatao updated CASSANDRA-13817: Labels: allow-filtering (was: ) > cassandra allow filtering bug > - > > Key: CASSANDRA-13817 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13817 > Project: Cassandra > Issue Type: Bug > Components: CQL, Libraries >Reporter: wang huatao > Labels: allow-filtering > Fix For: 3.10 > > > I have one bug about cassandra cql, when I use select * from table where > name = 'myName' alllow filtering, sometimes can be found, but sometimes can > not found. I am very sure the row data existed. my data was very small, just > 2000 rows.and only one node. i use cassandra 3.10, ubuntu 14. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13770) AssertionError: Lower bound INCL_START_BOUND during select by index
[
https://issues.apache.org/jira/browse/CASSANDRA-13770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rok Doltar updated CASSANDRA-13770:
---
Description:
We are getting the following error:
{code}
DEBUG [Native-Transport-Requests-1] 2017-08-17 07:47:01,815
ReadCallback.java:132 - Failed; received 0 of 1 responses
WARN [ReadStage-2] 2017-08-17 07:47:01,816
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-2,5,main]: {}
java.lang.AssertionError: Lower bound
[INCL_START_BOUND(00283543383338354144333637373731373445443036424134424442463445393233453846394634283836453642373436354546423435334544363636443236344644313935333032363338314542363200,
ab570080-831f-11e7-a81f-417b646547c3, , 1x) ]is bigger than first returned
value [Row:
partition_key=00283543383338354144333637373731373445443036424134424442463445393233453846394634283836453642373436354546423435334544363636443236344644313935333032363338314542363200,
version=null, file_path=null, file_name=null | ] for sstable
/var/lib/cassandra/data/catalog/file-aa90a340831f11e7aca2ed895c1dab3f/.idx_file_path_hash/mc-51-big-Data.db
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:124)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:47)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:500)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:360)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:67)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:695)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:639)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:514)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.index.internal.CassandraIndexSearcher.queryIndex(CassandraIndexSearcher.java:81)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.index.internal.CassandraIndexSearcher.search(CassandraIndexSearcher.java:63)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:408)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1882)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2587)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_141]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
[apache-cassandra-3.11.0.jar:3.11.0]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
[apache-cassandra-3.11.0.jar:3.11.0]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141]
{code}
The related table is:
{code}
CREATE TABLE catalog.file (
path_hash text,
file_hash text,
version timeuuid,
file_path text,
file_name text,
allocations_size bigint,
change_time timestamp,
creation_time timestamp,
dacl frozen,
ea_size bigint,
end_of_file bigi
[jira] [Commented] (CASSANDRA-13798) Disallow filtering on non-primary-key base column for MV
[ https://issues.apache.org/jira/browse/CASSANDRA-13798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144898#comment-16144898 ] Paulo Motta commented on CASSANDRA-13798: - bq. Right OK. If we can do it in 4.1 that's fine. It seems we don't actually have a difference between major and minor then? My (incorrect) assumption was that massive storage changes constituted a major version change, and 4.x would be intended to be minor. Hmm it depends whether the tick tock model is happening or not after 4.0, I'm not sure what the consensus was with that. But I meant that sstable version changes are normally during majors, and assumed 4.1 would be a major in the non-tick model while 4.0.1 would be a minor (following the previous model). In any case, this is just to leave things at a consistent state in case the community decides to release 4.0 without waiting for this feature to be fully enabled, but doesn't prevent us to removing this limitation if the feature is correctly implemented by 4.0 release timeframe. We will create another ticket to properly enable this feature following this. > Disallow filtering on non-primary-key base column for MV > > > Key: CASSANDRA-13798 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13798 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: ZhaoYang >Assignee: ZhaoYang > > We should probably consider disallow filtering conditions on non-primary-key > base column for Materialized View which is introduced in CASSANDRA-10368. > The main problem is that the liveness of view row is now depending on > multiple base columns (multiple filtered non-pk base column + base column > used in view pk) and this semantic could not be properly support without > drastic storage format changes. (SEE CASSANDRA-11500, > [background|https://issues.apache.org/jira/browse/CASSANDRA-11500?focusedCommentId=16119823&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16119823]) > We should step back and re-consider the non-primary-key filtering feature > together with supporting multiple non-PK cols in MV clustering key in > CASSANDRA-10226. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD commented on CASSANDRA-13418:
---
2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this what you
want");
+}
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it gets
> deleted as soon as it can. And in this situation I believe it would be really
> beneficial to allow users to simply ignore overlapping SSTables when looking
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset,
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be
> enough to greatly reduce entropy of the most used data (and if you have
> timeseries, you're likely to have a dashboard doing the same important
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:09 AM:
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this what you
want");
+}
}
{noformat}
was (Author: rgerard):
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this what you
want");
+}
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it gets
> d
[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:09 AM:
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this what you
want");
+}
}
{noformat}
was (Author: rgerard):
2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this what you
want");
+}
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it gets
> deleted as so
[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:10 AM:
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+}
}
{noformat}
was (Author: rgerard):
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this what you
want");
+}
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it g
[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:11 AM:
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
locally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+}
}
{noformat}
was (Author: rgerard):
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+}
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it ge
[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:11 AM:
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+}
}
{noformat}
was (Author: rgerard):
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore *the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
globally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+}
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it g
[jira] [Comment Edited] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901
]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:15 AM:
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
In this case, you will most likely not trigger any compaction to purge
tombstone if you run into an overlaps.
bq. 1. enabling both
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
locally if the current sstable is eligible.
In this case, you will always trigger compaction to purge tombstone even if you
run into an overlaps.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+}
}
{noformat}
was (Author: rgerard):
bq. 2. only enabling unsafe_aggressive_sstable_expiration
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *not ignore* the overlaps and
look globally if the current sstable is eligible.
bq. 1. enabling both
When looking for sstables expired, you will *ignore* the overlaps and only look
locally if the current sstable is eligible.
When looking for sstables to compact, you will *ignore* the overlaps and look
locally if the current sstable is eligible.
-
I made a new version of the patch with uncheckedTombstoneCompaction disabled
and a warning message.
https://github.com/criteo-forks/cassandra/commit/1800b23ddfbb308645c44022e15c1760a0124025
the diff
{noformat}
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index 43c90c7042..d21222c484 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
else
logger.debug("Enabling tombstone compactions for TWCS");
-if (this.options.ignoreOverlaps)
-this.uncheckedTombstoneCompaction = true;
-
+if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) {
+logger.warn("You are running with sstables overlapping checks
disabled but without unchecked tombstone compaction, check that this is what
you want");
+}
}
{noformat}
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
> Attachments: twcs-cleanup.png
>
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing i
[jira] [Commented] (CASSANDRA-13817) cassandra allow filtering bug
[ https://issues.apache.org/jira/browse/CASSANDRA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144918#comment-16144918 ] Benjamin Lerer commented on CASSANDRA-13817: Could you provide the table schema? > cassandra allow filtering bug > - > > Key: CASSANDRA-13817 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13817 > Project: Cassandra > Issue Type: Bug > Components: CQL, Libraries >Reporter: wang huatao > Labels: allow-filtering > Fix For: 3.10 > > > I have one bug about cassandra cql, when I use select * from table where > name = 'myName' alllow filtering, sometimes can be found, but sometimes can > not found. I am very sure the row data existed. my data was very small, just > 2000 rows.and only one node. i use cassandra 3.10, ubuntu 14. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13798) Disallow filtering on non-primary-key base column for MV
[ https://issues.apache.org/jira/browse/CASSANDRA-13798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Bossa updated CASSANDRA-13798: - Reviewer: Paulo Motta > Disallow filtering on non-primary-key base column for MV > > > Key: CASSANDRA-13798 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13798 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: ZhaoYang >Assignee: ZhaoYang > > We should probably consider disallow filtering conditions on non-primary-key > base column for Materialized View which is introduced in CASSANDRA-10368. > The main problem is that the liveness of view row is now depending on > multiple base columns (multiple filtered non-pk base column + base column > used in view pk) and this semantic could not be properly support without > drastic storage format changes. (SEE CASSANDRA-11500, > [background|https://issues.apache.org/jira/browse/CASSANDRA-11500?focusedCommentId=16119823&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16119823]) > We should step back and re-consider the non-primary-key filtering feature > together with supporting multiple non-PK cols in MV clustering key in > CASSANDRA-10226. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145021#comment-16145021
]
Sylvain Lebresne commented on CASSANDRA-13711:
--
The unit tests committed with that ticket are pretty fragile because they
insert with a TTL of 100 and expect a following read to get the same TTL of
100, but a read give you the remaining TTL (and this is returning with a 1
second precision), so on a slow run or through some timing issue you can easily
get a failure with:
{noformat}
junit.framework.AssertionFailedError: Invalid value for row 0 column 3 (ttl(i)
of type int), expected <100> but got <99>
at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1255)
at
org.apache.cassandra.cql3.validation.operations.SelectTest.testMixedTTLOnColumnsWide(SelectTest.java:4806)
{noformat}
[~jjirsa]: Would you mind having a look at making those tests less flaky?
(personally don't mind if you'd rather ninja-fix).
> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b| writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t |x | 1500565131354883
> b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "z" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "e" ],
> "position" : 21
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "r" ],
> "position" : 45
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "w" ],
> "position" : 69
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "t" ],
> "position" : 93
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
> { "name" : "b", "value" : "x" }
> ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "b" ],
> "position" : 121
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
> }
> ]
> }
> ]MacBook-Pro:~ jjirsa$
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.
[jira] [Commented] (CASSANDRA-13812) Missing system keyspace tables are not created
[
https://issues.apache.org/jira/browse/CASSANDRA-13812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145097#comment-16145097
]
Aleksey Yeschenko commented on CASSANDRA-13812:
---
[~slebresne] The purpose of hardcoding a minimal timestamp value there is to
avoid mismatches from automated creation, but still allow custom updates (to
keyspace RF for example). When we do update them in Core, we'd bump the
timestamp too so that the new default would override the old default, but still
not affect the overrides by the user - if any. FWIW we've been doing it like
this since forever, not just CASSANDRA-13441, just not consistently everywhere.
If I recall correctly, so does DSE. Now, this may or may not work as intended,
but that was the intention.
As for dropping those tables - no, it should absolutely not be allowed.
bq. But as said above, even outside that particular case, CASSANDRA-13441 means
(unless I'm missing something) that we cannot ever do any update to a
system_distributed table (we can add stuffs, but we can't update) and that
doesn't feel ideal to me.
It's already been the case for tables themselves before 13441, I think. Just
not for KS metadata. So far we've been lucky to not require any incompatible
changes.
Overall, I think we should not allow user modifications to our distributed
system tables (auth, tracing, system_distributed). So long as that is true, we
can hardcode a timestamp as a generation, and bump it every time we make a
change. But we should allow altering the keyspace itself by the user - at the
very least to allow changing RF. I think it's still fine to hard code a
timestamp there - user's changes will always win, and for keyspaces this is
what we want - given that the only two params we have on keyspaces are RF and
durable_writes.
> Missing system keyspace tables are not created
> --
>
> Key: CASSANDRA-13812
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13812
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: ZhaoYang
>
> Auth/Trace/Distributed Keyspaces or Tables dropped are not created on startup
> although a log message {{MigrationManager.java:220 - Create new table:
> TableMetadata...}} appears.
> Steps to reproduce:
> # Start node
> # {{DROP TABLE system_distributed.view_build_status;}}
> # {{DROP TABLE system_distributed.repair_history;}}
> # Stop node
> # Start node
> # Tables are *not* created, but log messages appear
> Cause:
> System's keyspaces or tables are created with timestamp 0 in CASSANDRA-13441
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13813) Don't let user drop (or generally break) tables in system_distributed
[
https://issues.apache.org/jira/browse/CASSANDRA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145105#comment-16145105
]
Aleksey Yeschenko commented on CASSANDRA-13813:
---
The reason it isn't listed is because we want to allow changing the replication
strategy/factor on those keyspaces themselves.
It's just that it's been implemented sub-diligently. So we want reject any
changes to the tables (including dropping them), reject dropping of those
keyspaces, but allow altering keyspace params. Also reject: creating of new
tables in those keyspaces.
> Don't let user drop (or generally break) tables in system_distributed
> -
>
> Key: CASSANDRA-13813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> There is not currently no particular restrictions on schema modifications to
> tables of the {{system_distributed}} keyspace. This does mean you can drop
> those tables, or even alter them in wrong ways like dropping or renaming
> columns. All of which is guaranteed to break stuffs (that is, repair if you
> mess up with on of it's table, or MVs if you mess up with
> {{view_build_status}}).
> I'm pretty sure this was never intended and is an oversight of the condition
> on {{ALTERABLE_SYSTEM_KEYSPACES}} in
> [ClientState|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ClientState.java#L397].
> That condition is such that any keyspace not listed in
> {{ALTERABLE_SYSTEM_KEYSPACES}} (which happens to be the case for
> {{system_distributed}}) has no specific restrictions whatsoever, while given
> the naming it's fair to assume the intention that exactly the opposite.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13814) unable to perform DDL and DML operation after install apache-cassandra-2.1.18
[ https://issues.apache.org/jira/browse/CASSANDRA-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145109#comment-16145109 ] Aleksey Yeschenko commented on CASSANDRA-13814: --- You have to create and then specify a keyspace first, before you create any tables. > unable to perform DDL and DML operation after install apache-cassandra-2.1.18 > -- > > Key: CASSANDRA-13814 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13814 > Project: Cassandra > Issue Type: Bug > Components: Configuration > Environment: linux(RHEL 6.8) >Reporter: rajasekhar >Priority: Critical > Fix For: 2.1.18 > > > unable to perform DDL and DML operation after install > apache-cassandra-2.1.18. please suggest what steps need to performs and > provide the solution step by step > *Error:* > InvalidRequest: code=2200 [Invalid query] message="No keyspace has been > specified. USE a keyspace, or explicitly specify keyspace.tablename" > assandra home path : /u01/Cassandra_home > cassandra bin path: -/u01/Cassandra_home/apache-cassandra-2.1.18/bin > [cassdb@alsc_dev_db bin]$ pwd > /u01/Cassandra_home/apache-cassandra-2.1.18/bin > cassdb@alsc_dev_db bin]$ ./cqlsh > Connected to Test Cluster at 127.0.0.1:9042. > [cqlsh 5.0.1 | Cassandra 2.1.18 | CQL spec 3.2.1 | Native protocol v3] > Use HELP for help. > cqlsh> > cqlsh> CREATE TABLE test1(sno int); > InvalidRequest: code=2200 [Invalid query] message="No keyspace has been > specified. USE a keyspace, or explicitly specify keyspace.tablename" > cqlsh> -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-13814) unable to perform DDL and DML operation after install apache-cassandra-2.1.18
[ https://issues.apache.org/jira/browse/CASSANDRA-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-13814. --- Resolution: Invalid > unable to perform DDL and DML operation after install apache-cassandra-2.1.18 > -- > > Key: CASSANDRA-13814 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13814 > Project: Cassandra > Issue Type: Bug > Components: Configuration > Environment: linux(RHEL 6.8) >Reporter: rajasekhar >Priority: Critical > Fix For: 2.1.18 > > > unable to perform DDL and DML operation after install > apache-cassandra-2.1.18. please suggest what steps need to performs and > provide the solution step by step > *Error:* > InvalidRequest: code=2200 [Invalid query] message="No keyspace has been > specified. USE a keyspace, or explicitly specify keyspace.tablename" > assandra home path : /u01/Cassandra_home > cassandra bin path: -/u01/Cassandra_home/apache-cassandra-2.1.18/bin > [cassdb@alsc_dev_db bin]$ pwd > /u01/Cassandra_home/apache-cassandra-2.1.18/bin > cassdb@alsc_dev_db bin]$ ./cqlsh > Connected to Test Cluster at 127.0.0.1:9042. > [cqlsh 5.0.1 | Cassandra 2.1.18 | CQL spec 3.2.1 | Native protocol v3] > Use HELP for help. > cqlsh> > cqlsh> CREATE TABLE test1(sno int); > InvalidRequest: code=2200 [Invalid query] message="No keyspace has been > specified. USE a keyspace, or explicitly specify keyspace.tablename" > cqlsh> -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-10786:
Status: Patch Available (was: Open)
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 4.x
>
>
> *_Initial description:_*
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
> -
> *_Resolution (2017/02/13):_*
> The following changes were made to native protocol v5:
> - the PREPARED response includes {{result_metadata_id}}, a hash of the result
> set metadata.
> - every EXECUTE message must provide {{result_metadata_id}} in addition to
> the prepared statement id. If it doesn't match the current one on the server,
> it means the client is operating on a stale schema.
> - to notify the client, the server returns a ROWS response with a new
> {{Metadata_changed}} flag, the new {{result_metadata_id}} and the updated
> result metadata (this overrides the {{No_metadata}} flag, even if the client
> had requested it)
> - the client updates its copy of the result metadata before it decodes the
> results.
> So the scenario above would now look like:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, and
> result set (b, c) that hashes to cde456
> # column a gets added to the table, C* does not invalidate its cache entry,
> but only updates the result set to (a, b, c) which hashes to fff789
> # client sends an EXECUTE request for (statementId=abc123, resultId=cde456)
> and skip_metadata flag
> # cde456!=fff789, so C* responds with ROWS(..., no_metadata=false,
> metadata_changed=true, new_metadata_id=fff789,col specs for (a,b,c))
> # client updates its column specifications, and will send the next execute
> queries with (statementId=abc123, resultId=fff789)
> This works the same with multiple clients.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-10786:
Status: Ready to Commit (was: Patch Available)
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 4.x
>
>
> *_Initial description:_*
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
> -
> *_Resolution (2017/02/13):_*
> The following changes were made to native protocol v5:
> - the PREPARED response includes {{result_metadata_id}}, a hash of the result
> set metadata.
> - every EXECUTE message must provide {{result_metadata_id}} in addition to
> the prepared statement id. If it doesn't match the current one on the server,
> it means the client is operating on a stale schema.
> - to notify the client, the server returns a ROWS response with a new
> {{Metadata_changed}} flag, the new {{result_metadata_id}} and the updated
> result metadata (this overrides the {{No_metadata}} flag, even if the client
> had requested it)
> - the client updates its copy of the result metadata before it decodes the
> results.
> So the scenario above would now look like:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, and
> result set (b, c) that hashes to cde456
> # column a gets added to the table, C* does not invalidate its cache entry,
> but only updates the result set to (a, b, c) which hashes to fff789
> # client sends an EXECUTE request for (statementId=abc123, resultId=cde456)
> and skip_metadata flag
> # cde456!=fff789, so C* responds with ROWS(..., no_metadata=false,
> metadata_changed=true, new_metadata_id=fff789,col specs for (a,b,c))
> # client updates its column specifications, and will send the next execute
> queries with (statementId=abc123, resultId=fff789)
> This works the same with multiple clients.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[2/6] cassandra git commit: Fix AssertionError in short read protection
Fix AssertionError in short read protection
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-13747
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7cb009f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7cb009f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7cb009f
Branch: refs/heads/cassandra-3.11
Commit: a7cb009f8a3f4d0e0293111bfcfff3d404a37a89
Parents: dfbe3fa
Author: Aleksey Yeschenko
Authored: Sun Aug 6 19:42:47 2017 +0100
Committer: Aleksey Yeschenko
Committed: Tue Aug 29 12:22:39 2017 +0100
--
CHANGES.txt | 1 +
.../UnfilteredPartitionIterators.java | 6 ---
.../db/transform/EmptyPartitionsDiscarder.java | 35 +++
.../apache/cassandra/db/transform/Filter.java | 28 +++-
.../db/transform/FilteredPartitions.java| 18 +---
.../cassandra/db/transform/FilteredRows.java| 2 +-
.../apache/cassandra/metrics/TableMetrics.java | 4 ++
.../apache/cassandra/service/DataResolver.java | 47 ++--
8 files changed, 94 insertions(+), 47 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5ccd5cd..6609b05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.15
+ * Fix AssertionError in short read protection (CASSANDRA-13747)
* Don't skip corrupted sstables on startup (CASSANDRA-13620)
* Fix the merging of cells with different user type versions (CASSANDRA-13776)
* Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
--
diff --git
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
index 1abbb19..4e0ac1b 100644
---
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
+++
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
@@ -77,12 +77,6 @@ public abstract class UnfilteredPartitionIterators
return Transformation.apply(toReturn, new Close());
}
-public static PartitionIterator
mergeAndFilter(List iterators, int nowInSec,
MergeListener listener)
-{
-// TODO: we could have a somewhat faster version if we were to merge
the UnfilteredRowIterators directly as RowIterators
-return filter(merge(iterators, nowInSec, listener), nowInSec);
-}
-
public static PartitionIterator filter(final UnfilteredPartitionIterator
iterator, final int nowInSec)
{
return FilteredPartitions.filter(iterator, nowInSec);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
--
diff --git
a/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
b/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
new file mode 100644
index 000..5e41cec
--- /dev/null
+++ b/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.transform;
+
+import org.apache.cassandra.db.rows.BaseRowIterator;
+
+public final class EmptyPartitionsDiscarder extends
Transformation>
+{
+@Override
+protected BaseRowIterator applyToPartition(BaseRowIterator iterator)
+{
+if (iterator.isEmpty())
+{
+iterator.close();
+return null;
+}
+
+return iterator;
+}
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/trans
[3/6] cassandra git commit: Fix AssertionError in short read protection
Fix AssertionError in short read protection
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-13747
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7cb009f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7cb009f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7cb009f
Branch: refs/heads/trunk
Commit: a7cb009f8a3f4d0e0293111bfcfff3d404a37a89
Parents: dfbe3fa
Author: Aleksey Yeschenko
Authored: Sun Aug 6 19:42:47 2017 +0100
Committer: Aleksey Yeschenko
Committed: Tue Aug 29 12:22:39 2017 +0100
--
CHANGES.txt | 1 +
.../UnfilteredPartitionIterators.java | 6 ---
.../db/transform/EmptyPartitionsDiscarder.java | 35 +++
.../apache/cassandra/db/transform/Filter.java | 28 +++-
.../db/transform/FilteredPartitions.java| 18 +---
.../cassandra/db/transform/FilteredRows.java| 2 +-
.../apache/cassandra/metrics/TableMetrics.java | 4 ++
.../apache/cassandra/service/DataResolver.java | 47 ++--
8 files changed, 94 insertions(+), 47 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5ccd5cd..6609b05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.15
+ * Fix AssertionError in short read protection (CASSANDRA-13747)
* Don't skip corrupted sstables on startup (CASSANDRA-13620)
* Fix the merging of cells with different user type versions (CASSANDRA-13776)
* Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
--
diff --git
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
index 1abbb19..4e0ac1b 100644
---
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
+++
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
@@ -77,12 +77,6 @@ public abstract class UnfilteredPartitionIterators
return Transformation.apply(toReturn, new Close());
}
-public static PartitionIterator
mergeAndFilter(List iterators, int nowInSec,
MergeListener listener)
-{
-// TODO: we could have a somewhat faster version if we were to merge
the UnfilteredRowIterators directly as RowIterators
-return filter(merge(iterators, nowInSec, listener), nowInSec);
-}
-
public static PartitionIterator filter(final UnfilteredPartitionIterator
iterator, final int nowInSec)
{
return FilteredPartitions.filter(iterator, nowInSec);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
--
diff --git
a/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
b/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
new file mode 100644
index 000..5e41cec
--- /dev/null
+++ b/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.transform;
+
+import org.apache.cassandra.db.rows.BaseRowIterator;
+
+public final class EmptyPartitionsDiscarder extends
Transformation>
+{
+@Override
+protected BaseRowIterator applyToPartition(BaseRowIterator iterator)
+{
+if (iterator.isEmpty())
+{
+iterator.close();
+return null;
+}
+
+return iterator;
+}
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/transform/Filt
[1/6] cassandra git commit: Fix AssertionError in short read protection
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 dfbe3fabd -> a7cb009f8
refs/heads/cassandra-3.11 809f3b30e -> 826ae9c91
refs/heads/trunk 326f3a7c7 -> 278906c6c
Fix AssertionError in short read protection
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-13747
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7cb009f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7cb009f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7cb009f
Branch: refs/heads/cassandra-3.0
Commit: a7cb009f8a3f4d0e0293111bfcfff3d404a37a89
Parents: dfbe3fa
Author: Aleksey Yeschenko
Authored: Sun Aug 6 19:42:47 2017 +0100
Committer: Aleksey Yeschenko
Committed: Tue Aug 29 12:22:39 2017 +0100
--
CHANGES.txt | 1 +
.../UnfilteredPartitionIterators.java | 6 ---
.../db/transform/EmptyPartitionsDiscarder.java | 35 +++
.../apache/cassandra/db/transform/Filter.java | 28 +++-
.../db/transform/FilteredPartitions.java| 18 +---
.../cassandra/db/transform/FilteredRows.java| 2 +-
.../apache/cassandra/metrics/TableMetrics.java | 4 ++
.../apache/cassandra/service/DataResolver.java | 47 ++--
8 files changed, 94 insertions(+), 47 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5ccd5cd..6609b05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.15
+ * Fix AssertionError in short read protection (CASSANDRA-13747)
* Don't skip corrupted sstables on startup (CASSANDRA-13620)
* Fix the merging of cells with different user type versions (CASSANDRA-13776)
* Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
--
diff --git
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
index 1abbb19..4e0ac1b 100644
---
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
+++
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
@@ -77,12 +77,6 @@ public abstract class UnfilteredPartitionIterators
return Transformation.apply(toReturn, new Close());
}
-public static PartitionIterator
mergeAndFilter(List iterators, int nowInSec,
MergeListener listener)
-{
-// TODO: we could have a somewhat faster version if we were to merge
the UnfilteredRowIterators directly as RowIterators
-return filter(merge(iterators, nowInSec, listener), nowInSec);
-}
-
public static PartitionIterator filter(final UnfilteredPartitionIterator
iterator, final int nowInSec)
{
return FilteredPartitions.filter(iterator, nowInSec);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7cb009f/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
--
diff --git
a/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
b/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
new file mode 100644
index 000..5e41cec
--- /dev/null
+++ b/src/java/org/apache/cassandra/db/transform/EmptyPartitionsDiscarder.java
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.transform;
+
+import org.apache.cassandra.db.rows.BaseRowIterator;
+
+public final class EmptyPartitionsDiscarder extends
Transformation>
+{
+@Override
+protected BaseRowIterator applyToPartition(BaseRowIterator iterator)
+{
+if (iterator.isEmpty())
+{
+iterator.cl
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/826ae9c9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/826ae9c9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/826ae9c9
Branch: refs/heads/trunk
Commit: 826ae9c91e11ebb889b3f1788b9357c2c717f9a0
Parents: 809f3b3 a7cb009
Author: Aleksey Yeschenko
Authored: Tue Aug 29 12:30:40 2017 +0100
Committer: Aleksey Yeschenko
Committed: Tue Aug 29 12:31:27 2017 +0100
--
CHANGES.txt | 1 +
.../UnfilteredPartitionIterators.java | 7 ---
.../db/transform/EmptyPartitionsDiscarder.java | 35 ++
.../apache/cassandra/db/transform/Filter.java | 28 +++
.../db/transform/FilteredPartitions.java| 18 +--
.../cassandra/db/transform/FilteredRows.java| 2 +-
.../apache/cassandra/metrics/TableMetrics.java | 4 ++
.../apache/cassandra/service/DataResolver.java | 51 ++--
.../apache/cassandra/db/ReadCommandTest.java| 23 -
9 files changed, 107 insertions(+), 62 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/826ae9c9/CHANGES.txt
--
diff --cc CHANGES.txt
index b0dbd60,6609b05..c4aee3a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
-3.0.15
+3.11.1
+ * Fix cassandra-stress hang issues when an error during cluster connection
happens (CASSANDRA-12938)
+ * Better bootstrap failure message when blocked by (potential) range
movement (CASSANDRA-13744)
+ * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
+ * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
+ * Duplicate the buffer before passing it to analyser in SASI operation
(CASSANDRA-13512)
+ * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
+Merged from 3.0:
+ * Fix AssertionError in short read protection (CASSANDRA-13747)
* Don't skip corrupted sstables on startup (CASSANDRA-13620)
* Fix the merging of cells with different user type versions
(CASSANDRA-13776)
* Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/826ae9c9/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
--
diff --cc
src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
index fc225e8,4e0ac1b..778c71d
---
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
+++
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
@@@ -78,31 -77,6 +78,24 @@@ public abstract class UnfilteredPartiti
return Transformation.apply(toReturn, new Close());
}
+public static UnfilteredPartitionIterator concat(final
List iterators)
+{
+if (iterators.size() == 1)
+return iterators.get(0);
+
+class Extend implements MorePartitions
+{
+int i = 1;
+public UnfilteredPartitionIterator moreContents()
+{
+if (i >= iterators.size())
+return null;
+return iterators.get(i++);
+}
+}
+return MorePartitions.extend(iterators.get(0), new Extend());
+}
+
-
- public static PartitionIterator
mergeAndFilter(List iterators, int nowInSec,
MergeListener listener)
- {
- // TODO: we could have a somewhat faster version if we were to merge
the UnfilteredRowIterators directly as RowIterators
- return filter(merge(iterators, nowInSec, listener), nowInSec);
- }
-
public static PartitionIterator filter(final UnfilteredPartitionIterator
iterator, final int nowInSec)
{
return FilteredPartitions.filter(iterator, nowInSec);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/826ae9c9/src/java/org/apache/cassandra/metrics/TableMetrics.java
--
diff --cc src/java/org/apache/cassandra/metrics/TableMetrics.java
index 7a84eca,fe88a63..b0f667c
--- a/src/java/org/apache/cassandra/metrics/TableMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/TableMetrics.java
@@@ -167,40 -151,8 +167,42 @@@ public class TableMetric
public final static LatencyMetrics globalWriteLatency = new
LatencyMetrics(globalFactory, globalAliasFactory, "Write");
public final static LatencyMetrics globalRangeLatency = new
LatencyMetrics(globalFactory, globalAliasFactory, "Range");
+public final static Gauge globalPercentRepaired =
Metrics.register(globalFactory.createMetricName("PercentRepaired"),
+
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/826ae9c9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/826ae9c9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/826ae9c9
Branch: refs/heads/cassandra-3.11
Commit: 826ae9c91e11ebb889b3f1788b9357c2c717f9a0
Parents: 809f3b3 a7cb009
Author: Aleksey Yeschenko
Authored: Tue Aug 29 12:30:40 2017 +0100
Committer: Aleksey Yeschenko
Committed: Tue Aug 29 12:31:27 2017 +0100
--
CHANGES.txt | 1 +
.../UnfilteredPartitionIterators.java | 7 ---
.../db/transform/EmptyPartitionsDiscarder.java | 35 ++
.../apache/cassandra/db/transform/Filter.java | 28 +++
.../db/transform/FilteredPartitions.java| 18 +--
.../cassandra/db/transform/FilteredRows.java| 2 +-
.../apache/cassandra/metrics/TableMetrics.java | 4 ++
.../apache/cassandra/service/DataResolver.java | 51 ++--
.../apache/cassandra/db/ReadCommandTest.java| 23 -
9 files changed, 107 insertions(+), 62 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/826ae9c9/CHANGES.txt
--
diff --cc CHANGES.txt
index b0dbd60,6609b05..c4aee3a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
-3.0.15
+3.11.1
+ * Fix cassandra-stress hang issues when an error during cluster connection
happens (CASSANDRA-12938)
+ * Better bootstrap failure message when blocked by (potential) range
movement (CASSANDRA-13744)
+ * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
+ * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
+ * Duplicate the buffer before passing it to analyser in SASI operation
(CASSANDRA-13512)
+ * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
+Merged from 3.0:
+ * Fix AssertionError in short read protection (CASSANDRA-13747)
* Don't skip corrupted sstables on startup (CASSANDRA-13620)
* Fix the merging of cells with different user type versions
(CASSANDRA-13776)
* Copy session properties on cqlsh.py do_login (CASSANDRA-13640)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/826ae9c9/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
--
diff --cc
src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
index fc225e8,4e0ac1b..778c71d
---
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
+++
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
@@@ -78,31 -77,6 +78,24 @@@ public abstract class UnfilteredPartiti
return Transformation.apply(toReturn, new Close());
}
+public static UnfilteredPartitionIterator concat(final
List iterators)
+{
+if (iterators.size() == 1)
+return iterators.get(0);
+
+class Extend implements MorePartitions
+{
+int i = 1;
+public UnfilteredPartitionIterator moreContents()
+{
+if (i >= iterators.size())
+return null;
+return iterators.get(i++);
+}
+}
+return MorePartitions.extend(iterators.get(0), new Extend());
+}
+
-
- public static PartitionIterator
mergeAndFilter(List iterators, int nowInSec,
MergeListener listener)
- {
- // TODO: we could have a somewhat faster version if we were to merge
the UnfilteredRowIterators directly as RowIterators
- return filter(merge(iterators, nowInSec, listener), nowInSec);
- }
-
public static PartitionIterator filter(final UnfilteredPartitionIterator
iterator, final int nowInSec)
{
return FilteredPartitions.filter(iterator, nowInSec);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/826ae9c9/src/java/org/apache/cassandra/metrics/TableMetrics.java
--
diff --cc src/java/org/apache/cassandra/metrics/TableMetrics.java
index 7a84eca,fe88a63..b0f667c
--- a/src/java/org/apache/cassandra/metrics/TableMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/TableMetrics.java
@@@ -167,40 -151,8 +167,42 @@@ public class TableMetric
public final static LatencyMetrics globalWriteLatency = new
LatencyMetrics(globalFactory, globalAliasFactory, "Write");
public final static LatencyMetrics globalRangeLatency = new
LatencyMetrics(globalFactory, globalAliasFactory, "Range");
+public final static Gauge globalPercentRepaired =
Metrics.register(globalFactory.createMetricName("PercentRepaired"
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/278906c6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/278906c6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/278906c6
Branch: refs/heads/trunk
Commit: 278906c6c0424c1ce0d922c24747c97978b0aa14
Parents: 326f3a7 826ae9c
Author: Aleksey Yeschenko
Authored: Tue Aug 29 12:33:33 2017 +0100
Committer: Aleksey Yeschenko
Committed: Tue Aug 29 12:33:50 2017 +0100
--
CHANGES.txt | 1 +
.../UnfilteredPartitionIterators.java | 7 ---
.../db/transform/EmptyPartitionsDiscarder.java | 35 +++
.../apache/cassandra/db/transform/Filter.java | 28 +++-
.../db/transform/FilteredPartitions.java| 15 ---
.../cassandra/db/transform/FilteredRows.java| 2 +-
.../apache/cassandra/metrics/TableMetrics.java | 4 ++
.../apache/cassandra/service/DataResolver.java | 45 ++--
.../apache/cassandra/db/ReadCommandTest.java| 23 +-
9 files changed, 101 insertions(+), 59 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/278906c6/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/278906c6/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/278906c6/src/java/org/apache/cassandra/db/transform/FilteredPartitions.java
--
diff --cc src/java/org/apache/cassandra/db/transform/FilteredPartitions.java
index ed643bb,ad9446d..fa12c9c
--- a/src/java/org/apache/cassandra/db/transform/FilteredPartitions.java
+++ b/src/java/org/apache/cassandra/db/transform/FilteredPartitions.java
@@@ -50,11 -50,19 +50,16 @@@ public final class FilteredPartitions e
/**
* Filter any RangeTombstoneMarker from the iterator's iterators,
transforming it into a PartitionIterator.
*/
- public static PartitionIterator filter(UnfilteredPartitionIterator
iterator, int nowInSecs)
+ public static FilteredPartitions filter(UnfilteredPartitionIterator
iterator, int nowInSecs)
{
- Filter filter = new Filter(true, nowInSecs);
- if (iterator instanceof UnfilteredPartitions)
- return new FilteredPartitions(filter, (UnfilteredPartitions)
iterator);
- return new FilteredPartitions(iterator, filter);
+ FilteredPartitions filtered = filter(iterator, new Filter(nowInSecs));
-
-return iterator.isForThrift()
- ? filtered
- : (FilteredPartitions) Transformation.apply(filtered, new
EmptyPartitionsDiscarder());
++return (FilteredPartitions) Transformation.apply(filtered, new
EmptyPartitionsDiscarder());
+ }
+
+ public static FilteredPartitions filter(UnfilteredPartitionIterator
iterator, Filter filter)
+ {
+ return iterator instanceof UnfilteredPartitions
+ ? new FilteredPartitions(filter, (UnfilteredPartitions) iterator)
+ : new FilteredPartitions(iterator, filter);
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/278906c6/src/java/org/apache/cassandra/metrics/TableMetrics.java
--
diff --cc src/java/org/apache/cassandra/metrics/TableMetrics.java
index 58b017e,b0f667c..7e6ca25
--- a/src/java/org/apache/cassandra/metrics/TableMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/TableMetrics.java
@@@ -240,33 -201,8 +240,35 @@@ public class TableMetric
}
});
+public static final Gauge globalBytesRepaired =
Metrics.register(globalFactory.createMetricName("BytesRepaired"),
+
new Gauge()
+{
+public Long getValue()
+{
+return totalNonSystemTablesSize(SSTableReader::isRepaired).left;
+}
+});
+
+public static final Gauge globalBytesUnrepaired =
Metrics.register(globalFactory.createMetricName("BytesUnrepaired"),
+
new Gauge()
+{
+public Long getValue()
+{
+return totalNonSystemTablesSize(s -> !s.isRepaired() &&
!s.isPendingRepair()).left;
+}
+});
+
+public static final Gauge globalBytesPendingRepair =
Metrics.register(globalFactory.createMetricName("BytesPendingRepair"),
+
new Gauge()
+{
+public Lon
[jira] [Commented] (CASSANDRA-13747) Fix AssertionError in short read protection
[
https://issues.apache.org/jira/browse/CASSANDRA-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145148#comment-16145148
]
Aleksey Yeschenko commented on CASSANDRA-13747:
---
Didn’t get a clean CI run, but these are the issues flagged:
On 3.0 and 3.11, {{RemoveTest}} failed due to a port being used by another
process. Passed locally, with and without compression.
On 3.0, {{ClientWarningsTest}} timed out. Passed locally.
5 dtest failures in 3.0 with unrelated failure reasons. Unfortunately currently
unable to get a clean run on ASF Jenkins.
Committed to 3.0 as
[a7cb009f8a3f4d0e0293111bfcfff3d404a37a89|https://github.com/apache/cassandra/commit/a7cb009f8a3f4d0e0293111bfcfff3d404a37a89]
and merged with 3.11 and trunk.
> Fix AssertionError in short read protection
> ---
>
> Key: CASSANDRA-13747
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13747
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.15, 3.11.1
>
>
> {{ShortReadRowProtection.moreContents()}} expects that by the time we get to
> that method, the global post-reconciliation counter was already applied to
> the current partition. However, sometimes it won’t get applied, and the
> global counter continues counting with {{rowInCurrentPartition}} value not
> reset from previous partition, which in the most obvious case would trigger
> the assertion we are observing - {{assert
> !postReconciliationCounter.isDoneForPartition();}}. In other cases it’s
> possible because of this lack of reset to query a node for too few extra
> rows, causing unnecessary SRP data requests.
> Why is the counter not always applied to the current partition?
> The merged {{PartitionIterator}} returned from {{DataResolver.resolve()}} has
> two transformations applied to it, in the following order:
> {{Filter}} - to purge non-live data from partitions, and to discard empty
> partitions altogether (except for Thrift)
> {{Counter}}, to count and stop iteration
> Problem is, {{Filter}} ’s {{applyToPartition()}} code that discards empty
> partitions ({{closeIfEmpty()}} method) would sometimes consume the iterator,
> triggering short read protection *before* {{Counter}} ’s
> {{applyToPartition()}} gets called and resets its {{rowInCurrentPartition}}
> sub-counter.
> We should not be consuming iterators until all transformations are applied to
> them. For transformations it means that they cannot consume iterators unless
> they are the last transformation on the stack.
> The linked branch fixes the problem by splitting {{Filter}} into two
> transformations. The original - {{Filter}} - that does filtering within
> partitions - and a separate {{EmptyPartitionsDiscarder}}, that discards empty
> partitions from {{PartitionIterators}}. Thus {{DataResolve.resolve()}}, when
> constructing its {{PartitionIterator}}, now does merge first, then applies
> {{Filter}}, then {{Counter}}, and only then, as its last (third)
> transformation - the {{EmptyPartitionsDiscarder}}. Being the last one
> applied, it’s legal for it to consume the iterator, and triggering
> {{moreContents()}} is now no longer a problem.
> Fixes: [3.0|https://github.com/iamaleksey/cassandra/commits/13747-3.0],
> [3.11|https://github.com/iamaleksey/cassandra/commits/13747-3.11],
> [4.0|https://github.com/iamaleksey/cassandra/commits/13747-4.0]. dtest
> [here|https://github.com/iamaleksey/cassandra-dtest/commits/13747].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13747) Fix AssertionError in short read protection
[
https://issues.apache.org/jira/browse/CASSANDRA-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13747:
--
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 3.0.x)
3.11.1
3.0.15
Status: Resolved (was: Patch Available)
> Fix AssertionError in short read protection
> ---
>
> Key: CASSANDRA-13747
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13747
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.15, 3.11.1
>
>
> {{ShortReadRowProtection.moreContents()}} expects that by the time we get to
> that method, the global post-reconciliation counter was already applied to
> the current partition. However, sometimes it won’t get applied, and the
> global counter continues counting with {{rowInCurrentPartition}} value not
> reset from previous partition, which in the most obvious case would trigger
> the assertion we are observing - {{assert
> !postReconciliationCounter.isDoneForPartition();}}. In other cases it’s
> possible because of this lack of reset to query a node for too few extra
> rows, causing unnecessary SRP data requests.
> Why is the counter not always applied to the current partition?
> The merged {{PartitionIterator}} returned from {{DataResolver.resolve()}} has
> two transformations applied to it, in the following order:
> {{Filter}} - to purge non-live data from partitions, and to discard empty
> partitions altogether (except for Thrift)
> {{Counter}}, to count and stop iteration
> Problem is, {{Filter}} ’s {{applyToPartition()}} code that discards empty
> partitions ({{closeIfEmpty()}} method) would sometimes consume the iterator,
> triggering short read protection *before* {{Counter}} ’s
> {{applyToPartition()}} gets called and resets its {{rowInCurrentPartition}}
> sub-counter.
> We should not be consuming iterators until all transformations are applied to
> them. For transformations it means that they cannot consume iterators unless
> they are the last transformation on the stack.
> The linked branch fixes the problem by splitting {{Filter}} into two
> transformations. The original - {{Filter}} - that does filtering within
> partitions - and a separate {{EmptyPartitionsDiscarder}}, that discards empty
> partitions from {{PartitionIterators}}. Thus {{DataResolve.resolve()}}, when
> constructing its {{PartitionIterator}}, now does merge first, then applies
> {{Filter}}, then {{Counter}}, and only then, as its last (third)
> transformation - the {{EmptyPartitionsDiscarder}}. Being the last one
> applied, it’s legal for it to consume the iterator, and triggering
> {{moreContents()}} is now no longer a problem.
> Fixes: [3.0|https://github.com/iamaleksey/cassandra/commits/13747-3.0],
> [3.11|https://github.com/iamaleksey/cassandra/commits/13747-3.11],
> [4.0|https://github.com/iamaleksey/cassandra/commits/13747-4.0]. dtest
> [here|https://github.com/iamaleksey/cassandra-dtest/commits/13747].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13363) Fix racy read command serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13363:
--
Priority: Major (was: Critical)
Summary: Fix racy read command serialization (was:
java.lang.ArrayIndexOutOfBoundsException: null)
> Fix racy read command serialization
> ---
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra-dtest git commit: Fix short read protection
Repository: cassandra-dtest
Updated Branches:
refs/heads/master ac9c95607 -> 2ad557dff
Fix short read protection
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-13747
Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/2ad557df
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/2ad557df
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/2ad557df
Branch: refs/heads/master
Commit: 2ad557dff9f9d4a3c09f0781b3eeeb5fe75b57d0
Parents: ac9c956
Author: Aleksey Yeschenko
Authored: Mon Aug 7 14:06:05 2017 +0100
Committer: Aleksey Yeschenko
Committed: Tue Aug 29 12:47:10 2017 +0100
--
consistency_test.py | 53
1 file changed, 53 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/2ad557df/consistency_test.py
--
diff --git a/consistency_test.py b/consistency_test.py
index 9424b4c..b50d81b 100644
--- a/consistency_test.py
+++ b/consistency_test.py
@@ -772,6 +772,59 @@ class TestAccuracy(TestHelper):
class TestConsistency(Tester):
+@since('3.0')
+def test_13747(self):
+"""
+@jira_ticket CASSANDRA-13747
+"""
+cluster = self.cluster
+
+# disable hinted handoff and set batch commit log so this doesn't
interfere with the test
+cluster.set_configuration_options(values={'hinted_handoff_enabled':
False})
+cluster.set_batch_commitlog(enabled=True)
+
+cluster.populate(2).start(wait_other_notice=True)
+node1, node2 = cluster.nodelist()
+
+session = self.patient_cql_connection(node1)
+
+query = "CREATE KEYSPACE IF NOT EXISTS test WITH replication =
{'class': 'NetworkTopologyStrategy', 'datacenter1': 2};"
+session.execute(query)
+
+query = "CREATE TABLE IF NOT EXISTS test.test (id int PRIMARY KEY);"
+session.execute(query)
+
+#
+# populate the table with 10 rows:
+#
+
+# -7509452495886106294 | 5
+# -4069959284402364209 | 1 x
+# -3799847372828181882 | 8
+# -3485513579396041028 | 0 x
+# -3248873570005575792 | 2
+# -2729420104000364805 | 4 x
+# 1634052884888577606 | 7
+# 2705480034054113608 | 6 x
+# 3728482343045213994 | 9
+# 9010454139840013625 | 3 x
+
+stmt = session.prepare("INSERT INTO test.test (id) VALUES (?);")
+for id in range(0, 10):
+session.execute(stmt, [id], ConsistencyLevel.ALL)
+
+# with node2 down and hints disabled, delete every other row on node1
+node2.stop(wait_other_notice=True)
+session.execute("DELETE FROM test.test WHERE id IN (1, 0, 4, 6, 3);")
+
+# with both nodes up, do a DISTINCT range query with CL.ALL;
+# prior to CASSANDRA-13747 this would cause an assertion in short read
protection code
+node2.start(wait_other_notice=True)
+stmt = SimpleStatement("SELECT DISTINCT token(id), id FROM test.test;",
+ consistency_level = ConsistencyLevel.ALL)
+result = list(session.execute(stmt))
+assert_length_equal(result, 5)
+
def short_read_test(self):
"""
@jira_ticket CASSANDRA-9460
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145183#comment-16145183
]
Marcus Eriksson commented on CASSANDRA-13752:
-
So this is because we reuse the same StreamingHistogram when we [open sstables
early|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/io/sstable/metadata/MetadataCollector.java#L292]
- the early opened sstable will be using the streaming histogram that
compaction is still building. Since CASSANDRA-13038 we can modify the contents
of the StreamingHistogram when we call {{sum()}} ({{spool}} might be compacted
into {{bin}}). So, if someone calls the
{{ColumnFamilyStoreMBean#getDroppableTombstoneRatio}} at the wrong time we
could get either the CME from CASSANDRA-13756 or this corruption.
Making StreamingHistogram thread safe is one way of fixing this, but I would
argue that we should be using a "builder" for StreamingHistogram - we should
never access the SH while building it and for early opened sstables we should
call {{.build()}} on the StreamingHistogramBuilder and get a copy of the
internal state.
Also, we should not query LIVE sstables
[here|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L2589]
- it should be using {{SSTableSet.CANONICAL}} (this is probably enough to fix
this for now - this is the only way I can see that we access the
sstablemetadata in early opened sstables).
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandr
[jira] [Commented] (CASSANDRA-13738) Load is over calculated after each IndexSummaryRedistribution
[
https://issues.apache.org/jira/browse/CASSANDRA-13738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145189#comment-16145189
]
Aleksey Yeschenko commented on CASSANDRA-13738:
---
[~jjirsa] Not my strongest area of the codebase. Maybe [~krummas] has some
spare cycles?
> Load is over calculated after each IndexSummaryRedistribution
> -
>
> Key: CASSANDRA-13738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13738
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: sizeIssue.png
>
>
> For example, here is one of our cluster with about 500GB per node, but
> {{nodetool status}} shows far more load than it actually is and keeps
> increasing, restarting the process will reset the load, but keeps increasing
> afterwards:
> {noformat}
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens Owns (effective) Host ID
> Rack
> UN IP1* 13.52 TB 256 100.0%
> c4c31e0a-3f01-49f7-8a22-33043737975d rac1
> UN IP2* 14.25 TB 256 100.0%
> efec4980-ec9e-4424-8a21-ce7ddaf80aa0 rac1
> UN IP3* 13.52 TB 256 100.0%
> 7dbcfdfc-9c07-4b1a-a4b9-970b715ebed8 rac1
> UN IP4* 22.13 TB 256 100.0%
> 8879e6c4-93e3-4cc5-b957-f999c6b9b563 rac1
> UN IP5* 18.02 TB 256 100.0%
> 4a1eaf22-4a83-4736-9e1c-12f898d685fa rac1
> UN IP6* 11.68 TB 256 100.0%
> d633c591-28af-42cc-bc5e-47d1c8bcf50f rac1
> {noformat}
> !sizeIssue.png|test!
> The root cause is if the SSTable index summary is redistributed (typically
> executes hourly), the updated SSTable size is added again.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13738) Load is over calculated after each IndexSummaryRedistribution
[
https://issues.apache.org/jira/browse/CASSANDRA-13738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-13738:
Reviewer: Marcus Eriksson
sure
> Load is over calculated after each IndexSummaryRedistribution
> -
>
> Key: CASSANDRA-13738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13738
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: sizeIssue.png
>
>
> For example, here is one of our cluster with about 500GB per node, but
> {{nodetool status}} shows far more load than it actually is and keeps
> increasing, restarting the process will reset the load, but keeps increasing
> afterwards:
> {noformat}
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens Owns (effective) Host ID
> Rack
> UN IP1* 13.52 TB 256 100.0%
> c4c31e0a-3f01-49f7-8a22-33043737975d rac1
> UN IP2* 14.25 TB 256 100.0%
> efec4980-ec9e-4424-8a21-ce7ddaf80aa0 rac1
> UN IP3* 13.52 TB 256 100.0%
> 7dbcfdfc-9c07-4b1a-a4b9-970b715ebed8 rac1
> UN IP4* 22.13 TB 256 100.0%
> 8879e6c4-93e3-4cc5-b957-f999c6b9b563 rac1
> UN IP5* 18.02 TB 256 100.0%
> 4a1eaf22-4a83-4736-9e1c-12f898d685fa rac1
> UN IP6* 11.68 TB 256 100.0%
> d633c591-28af-42cc-bc5e-47d1c8bcf50f rac1
> {noformat}
> !sizeIssue.png|test!
> The root cause is if the SSTable index summary is redistributed (typically
> executes hourly), the updated SSTable size is added again.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13812) Missing system keyspace tables are not created
[
https://issues.apache.org/jira/browse/CASSANDRA-13812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145196#comment-16145196
]
Sylvain Lebresne commented on CASSANDRA-13812:
--
bq. When we do update them in Core, we'd bump the timestamp too so that the new
default would override the old default, but still not affect the overrides by
the user - if any.
Fair enough. I can see this working if we do do that. That said, I have never
been privy to this intention and while I may have missed it, I haven't seen any
comment in the code documenting such intention/expected process, so maybe it's
at least worth documenting this.
> Missing system keyspace tables are not created
> --
>
> Key: CASSANDRA-13812
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13812
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: ZhaoYang
>
> Auth/Trace/Distributed Keyspaces or Tables dropped are not created on startup
> although a log message {{MigrationManager.java:220 - Create new table:
> TableMetadata...}} appears.
> Steps to reproduce:
> # Start node
> # {{DROP TABLE system_distributed.view_build_status;}}
> # {{DROP TABLE system_distributed.repair_history;}}
> # Stop node
> # Start node
> # Tables are *not* created, but log messages appear
> Cause:
> System's keyspaces or tables are created with timestamp 0 in CASSANDRA-13441
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13812) Missing system keyspace tables are not created
[
https://issues.apache.org/jira/browse/CASSANDRA-13812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145202#comment-16145202
]
Aleksey Yeschenko commented on CASSANDRA-13812:
---
bq. That said, I have never been privy to this intention and while I may have
missed it, I haven't seen any comment in the code documenting such
intention/expected process, so maybe it's at least worth documenting this.
Also fair enough. Can do that as part of CASSANDRA-13813 maybe?
> Missing system keyspace tables are not created
> --
>
> Key: CASSANDRA-13812
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13812
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: ZhaoYang
>
> Auth/Trace/Distributed Keyspaces or Tables dropped are not created on startup
> although a log message {{MigrationManager.java:220 - Create new table:
> TableMetadata...}} appears.
> Steps to reproduce:
> # Start node
> # {{DROP TABLE system_distributed.view_build_status;}}
> # {{DROP TABLE system_distributed.repair_history;}}
> # Stop node
> # Start node
> # Tables are *not* created, but log messages appear
> Cause:
> System's keyspaces or tables are created with timestamp 0 in CASSANDRA-13441
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13813) Don't let user drop (or generally break) tables in system_distributed
[
https://issues.apache.org/jira/browse/CASSANDRA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145207#comment-16145207
]
Sylvain Lebresne commented on CASSANDRA-13813:
--
bq. The reason it isn't listed is because we want to allow changing the
replication strategy/factor on those keyspaces themselves.
The name of the set makes it clear what the _intention_ was. What I'm saying is
that the code doesn't implement that intention. Currently, you can alter the
replication strategy/factor of _all_ system distributed keyspace, even
{{system_distributed}}, and I'm reasonably convinced that wasn't the intention.
This certainly isn't hard to fix though.
Now with that said, I happen to not particularly see a terribly good reason to
limit which of the system distributed keyspace can have their strategy/factor
changed (and again, despite the original code intention, we currently _don't_
have such limit), so instead of simply making the check on
{{ALTERABLE_SYSTEM_KEYSPACES}} work as originally intended (and thus starting
to forbid changing the RF on {{system_distributed}}), I'd advocate for just
removing {{ALTERABLE_SYSTEM_KEYSPACES}} and deal with all those keyspace
consistently. Just my 2 cents though.
> Don't let user drop (or generally break) tables in system_distributed
> -
>
> Key: CASSANDRA-13813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> There is not currently no particular restrictions on schema modifications to
> tables of the {{system_distributed}} keyspace. This does mean you can drop
> those tables, or even alter them in wrong ways like dropping or renaming
> columns. All of which is guaranteed to break stuffs (that is, repair if you
> mess up with on of it's table, or MVs if you mess up with
> {{view_build_status}}).
> I'm pretty sure this was never intended and is an oversight of the condition
> on {{ALTERABLE_SYSTEM_KEYSPACES}} in
> [ClientState|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ClientState.java#L397].
> That condition is such that any keyspace not listed in
> {{ALTERABLE_SYSTEM_KEYSPACES}} (which happens to be the case for
> {{system_distributed}}) has no specific restrictions whatsoever, while given
> the naming it's fair to assume the intention that exactly the opposite.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13813) Don't let user drop (or generally break) tables in system_distributed
[
https://issues.apache.org/jira/browse/CASSANDRA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145227#comment-16145227
]
Aleksey Yeschenko commented on CASSANDRA-13813:
---
Not having {{system_distributed}} in {{ALTERABLE_SYSTEM_KEYSPACES}} is an
oversight from the patch that added {{system_distributed}}.
Had it been added correctly, CASSANDRA-13812 would not be a problem -
{{ClientState.preventSystemKSSchemaModification()}} would've forbidden those
drops.
The constant is there to enumerate the keyspace whose tables cannot be
modified, and that's the purpose it serves. It might not have the best possible
name, though, or be ideally documented.
I'd start with adding {{system_distributed}} to {{ALTERABLE_SYSTEM_KEYSPACES}}
- that would immediately fix CASSANDRA-13812 and this ticket. Then rename
{{ALTERABLE_SYSTEM_KEYSPACES}} to something more descriptive (even
{{PARTIALLY_ALTERABLE_SYSTEM_KEYSPACES}} would be a start. And document the
intent.
> Don't let user drop (or generally break) tables in system_distributed
> -
>
> Key: CASSANDRA-13813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> There is not currently no particular restrictions on schema modifications to
> tables of the {{system_distributed}} keyspace. This does mean you can drop
> those tables, or even alter them in wrong ways like dropping or renaming
> columns. All of which is guaranteed to break stuffs (that is, repair if you
> mess up with on of it's table, or MVs if you mess up with
> {{view_build_status}}).
> I'm pretty sure this was never intended and is an oversight of the condition
> on {{ALTERABLE_SYSTEM_KEYSPACES}} in
> [ClientState|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ClientState.java#L397].
> That condition is such that any keyspace not listed in
> {{ALTERABLE_SYSTEM_KEYSPACES}} (which happens to be the case for
> {{system_distributed}}) has no specific restrictions whatsoever, while given
> the naming it's fair to assume the intention that exactly the opposite.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13813) Don't let user drop (or generally break) tables in system_distributed
[
https://issues.apache.org/jira/browse/CASSANDRA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko reassigned CASSANDRA-13813:
-
Assignee: Aleksey Yeschenko
> Don't let user drop (or generally break) tables in system_distributed
> -
>
> Key: CASSANDRA-13813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x
>
>
> There is not currently no particular restrictions on schema modifications to
> tables of the {{system_distributed}} keyspace. This does mean you can drop
> those tables, or even alter them in wrong ways like dropping or renaming
> columns. All of which is guaranteed to break stuffs (that is, repair if you
> mess up with on of it's table, or MVs if you mess up with
> {{view_build_status}}).
> I'm pretty sure this was never intended and is an oversight of the condition
> on {{ALTERABLE_SYSTEM_KEYSPACES}} in
> [ClientState|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ClientState.java#L397].
> That condition is such that any keyspace not listed in
> {{ALTERABLE_SYSTEM_KEYSPACES}} (which happens to be the case for
> {{system_distributed}}) has no specific restrictions whatsoever, while given
> the naming it's fair to assume the intention that exactly the opposite.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13649) Uncaught exceptions in Netty pipeline
[
https://issues.apache.org/jira/browse/CASSANDRA-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145265#comment-16145265
]
Ajith Shetty commented on CASSANDRA-13649:
--
Jason/Norman, we are using cassandra ReleaseVersion: 3.0.11.
We see the same issue.
INFO [SharedPool-Worker-1] 2017-08-27 05:35:17,241 Message.java:615 -
Unexpected exception during request; channel = [id: 0xcdc01633, L:/XX:9042
! R:/:58310]
io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
Connection reset by peer
at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
And from the application side we see the below error:
2017-08-27 05:34:05,121 ERROR [mainLog] (nioEventLoopGroup-22-47)
[/:9042] Timed out waiting for server response; nested exception is
com.datastax.driver.core.exceptions.OperationTimedOutException: [/XXX:9042]
Timed out waiting for server response:
org.springframework.cassandra.support.exception.CassandraUncategorizedException:
[/XXX:9042] Timed out waiting for server response; nested exception is
com.datastax.driver.core.exceptions.OperationTimedOutException: [/XXX:9042]
Timed out waiting for server response
at
org.springframework.cassandra.support.CassandraExceptionTranslator.translateExceptionIfPossible(CassandraExceptionTranslator.java:129)
[spring-cql-1.5.0.RELEASE.jar:]
at
org.springframework.cassandra.core.CqlTemplate.potentiallyConvertRuntimeException(CqlTemplate.java:946)
[spring-cql-1.5.0.RELEASE.jar:]
Has it been fixed int the 3.0.11 version??
Thanks,
Ajith Shetty
> Uncaught exceptions in Netty pipeline
> -
>
> Key: CASSANDRA-13649
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13649
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging, Testing
>Reporter: Stefan Podkowinski
>Assignee: Norman Maurer
> Labels: patch
> Attachments:
> 0001-CASSANDRA-13649-Ensure-all-exceptions-are-correctly-.patch,
> test_stdout.txt
>
>
> I've noticed some netty related errors in trunk in [some of the dtest
> results|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/106/#showFailuresLink].
> Just want to make sure that we don't have to change anything related to the
> exception handling in our pipeline and that this isn't a netty issue.
> Actually if this causes flakiness but is otherwise harmless, we should do
> something about it, even if it's just on the dtest side.
> {noformat}
> WARN [epollEventLoopGroup-2-9] 2017-06-28 17:23:49,699 Slf4JLogger.java:151
> - An exceptionCaught() event was fired, and it reached at the tail of the
> pipeline. It usually means the last handler in the pipeline did not handle
> the exception.
> io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
> Connection reset by peer
> at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
> Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> {noformat}
> And again in another test:
> {noformat}
> WARN [epollEventLoopGroup-2-8] 2017-06-29 02:27:31,300 Slf4JLogger.java:151
> - An exceptionCaught() event was fired, and it reached at the tail of the
> pipeline. It usually means the last handler in the pipeline did not handle
> the exception.
> io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
> Connection reset by peer
> at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
> Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> {noformat}
> Edit:
> The {{io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)()
> failed}} error also causes tests to fail for 3.0 and 3.11.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13813) Don't let user drop (or generally break) tables in system_distributed
[
https://issues.apache.org/jira/browse/CASSANDRA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13813:
--
Reviewer: Sylvain Lebresne
> Don't let user drop (or generally break) tables in system_distributed
> -
>
> Key: CASSANDRA-13813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x
>
>
> There is not currently no particular restrictions on schema modifications to
> tables of the {{system_distributed}} keyspace. This does mean you can drop
> those tables, or even alter them in wrong ways like dropping or renaming
> columns. All of which is guaranteed to break stuffs (that is, repair if you
> mess up with on of it's table, or MVs if you mess up with
> {{view_build_status}}).
> I'm pretty sure this was never intended and is an oversight of the condition
> on {{ALTERABLE_SYSTEM_KEYSPACES}} in
> [ClientState|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ClientState.java#L397].
> That condition is such that any keyspace not listed in
> {{ALTERABLE_SYSTEM_KEYSPACES}} (which happens to be the case for
> {{system_distributed}}) has no specific restrictions whatsoever, while given
> the naming it's fair to assume the intention that exactly the opposite.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13813) Don't let user drop (or generally break) tables in system_distributed
[
https://issues.apache.org/jira/browse/CASSANDRA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145227#comment-16145227
]
Aleksey Yeschenko edited comment on CASSANDRA-13813 at 8/29/17 1:23 PM:
Not having {{system_distributed}} in {{ALTERABLE_SYSTEM_KEYSPACES}} is an
oversight from the patch that added {{system_distributed}}.
Had it been added correctly, CASSANDRA-13812 would not be a problem -
{{ClientState.preventSystemKSSchemaModification()}} would've forbidden those
drops.
The constant is there to enumerate the keyspace whose tables cannot be
modified, and that's the purpose it serves. It might not have the best possible
name, though, or be ideally documented.
I'd start with adding {{system_distributed}} to {{ALTERABLE_SYSTEM_KEYSPACES}}
- that would immediately fix CASSANDRA-13812 and this ticket. Then rename
{{ALTERABLE_SYSTEM_KEYSPACES}} to something more descriptive (even
{{PARTIALLY_ALTERABLE_SYSTEM_KEYSPACES}} would be a start. And document the
intent.
EDIT: As Sylvain reminded me offline, we don't really need
{{ALTERABLE_SYSTEM_KEYSPACES}} at all. The intended set of keyspaces in there
is duplicated in {{Schema.REPLICATED_SYSTEM_KEYSPACE_NAMES}}, which is what we
should be using instead - directly or via
{{SchemaConstants.isReplicatedSystemKeyspace()}}.
was (Author: iamaleksey):
Not having {{system_distributed}} in {{ALTERABLE_SYSTEM_KEYSPACES}} is an
oversight from the patch that added {{system_distributed}}.
Had it been added correctly, CASSANDRA-13812 would not be a problem -
{{ClientState.preventSystemKSSchemaModification()}} would've forbidden those
drops.
The constant is there to enumerate the keyspace whose tables cannot be
modified, and that's the purpose it serves. It might not have the best possible
name, though, or be ideally documented.
I'd start with adding {{system_distributed}} to {{ALTERABLE_SYSTEM_KEYSPACES}}
- that would immediately fix CASSANDRA-13812 and this ticket. Then rename
{{ALTERABLE_SYSTEM_KEYSPACES}} to something more descriptive (even
{{PARTIALLY_ALTERABLE_SYSTEM_KEYSPACES}} would be a start. And document the
intent.
> Don't let user drop (or generally break) tables in system_distributed
> -
>
> Key: CASSANDRA-13813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x
>
>
> There is not currently no particular restrictions on schema modifications to
> tables of the {{system_distributed}} keyspace. This does mean you can drop
> those tables, or even alter them in wrong ways like dropping or renaming
> columns. All of which is guaranteed to break stuffs (that is, repair if you
> mess up with on of it's table, or MVs if you mess up with
> {{view_build_status}}).
> I'm pretty sure this was never intended and is an oversight of the condition
> on {{ALTERABLE_SYSTEM_KEYSPACES}} in
> [ClientState|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ClientState.java#L397].
> That condition is such that any keyspace not listed in
> {{ALTERABLE_SYSTEM_KEYSPACES}} (which happens to be the case for
> {{system_distributed}}) has no specific restrictions whatsoever, while given
> the naming it's fair to assume the intention that exactly the opposite.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12907) Different data directories for SSDs and HDDs at configuration level
[ https://issues.apache.org/jira/browse/CASSANDRA-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145285#comment-16145285 ] Hannu Kröger commented on CASSANDRA-12907: -- One idea would be also to extract the file placing logic to a different extensible class which could be then replaced by configurable plugin like "SSDMetadataFilePlacingStrategy" with parameter "ssd_directory_locations". Then it would be possible to implement both: one strategy to store only metadata and indexes on ssds and another to store certain tables fully in different location". When opening files, all locations probably should be checked for easy migration from one approach to another. > Different data directories for SSDs and HDDs at configuration level > --- > > Key: CASSANDRA-12907 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12907 > Project: Cassandra > Issue Type: Improvement >Reporter: Natale Galioto > Labels: performance > > Currently, users can speed up some CFs by symlinking its data directory to > fast media such as SSDs. In my opinion, instead, configuration file should > allow two different sets of directory: one dedicated to spindles, one > dedicated to SSDs. > This would allow a "once and for all mixed SSD & HDD configuration", instead > of continuously symlinking the "right" directory each time a CF is created > (due to the name mangling of the CF directories). > And this in turn would allow a priori knowledge on disk structures, and would > allow to place indexes of all sort (lookup, partition, etc... everything that > is needed to "just" locate data) on fast SSDs, speeding up ALL the CFs > instead of only one, while the HDDs could be used just for data retrieval and > sequential reads. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-13812) Missing system keyspace tables are not created
[
https://issues.apache.org/jira/browse/CASSANDRA-13812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko resolved CASSANDRA-13812.
---
Resolution: Duplicate
Reproduced In: 3.11.0, 3.0.14, 4.0 (was: 3.0.14, 3.11.0, 4.0)
[~jasonstack] Thanks for catching the bug. Closing this JIRA as duplicate of
CASSANDRA-13813, where we'll deal with the root cause (there are no better
fitting issues resolutions unfortunately).
> Missing system keyspace tables are not created
> --
>
> Key: CASSANDRA-13812
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13812
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: ZhaoYang
>
> Auth/Trace/Distributed Keyspaces or Tables dropped are not created on startup
> although a log message {{MigrationManager.java:220 - Create new table:
> TableMetadata...}} appears.
> Steps to reproduce:
> # Start node
> # {{DROP TABLE system_distributed.view_build_status;}}
> # {{DROP TABLE system_distributed.repair_history;}}
> # Stop node
> # Start node
> # Tables are *not* created, but log messages appear
> Cause:
> System's keyspaces or tables are created with timestamp 0 in CASSANDRA-13441
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13215) Cassandra nodes startup time 20x more after upgarding to 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145453#comment-16145453 ] Corentin Chary commented on CASSANDRA-13215: I can confirm that this is affecting us too (startup and repairs). [~krummas] did you end up doing something for this issue ? Else I might give it a shot. > Cassandra nodes startup time 20x more after upgarding to 3.x > > > Key: CASSANDRA-13215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13215 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Cluster setup: two datacenters (dc-main, dc-backup). > dc-main - 9 servers, no vnodes > dc-backup - 6 servers, vnodes >Reporter: Viktor Kuzmin >Assignee: Marcus Eriksson > Attachments: simple-cache.patch > > > CompactionStrategyManage.getCompactionStrategyIndex is called on each sstable > at startup. And this function calls StorageService.getDiskBoundaries. And > getDiskBoundaries calls AbstractReplicationStrategy.getAddressRanges. > It appears that last function can be really slow. In our environment we have > 1545 tokens and with NetworkTopologyStrategy it can make 1545*1545 > computations in worst case (maybe I'm wrong, but it really takes lot's of > cpu). > Also this function can affect runtime later, cause it is called not only > during startup. > I've tried to implement simple cache for getDiskBoundaries results and now > startup time is about one minute instead of 25m, but I'm not sure if it's a > good solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13363) Fix racy read command serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145451#comment-16145451
]
Sam Tunnicliffe commented on CASSANDRA-13363:
-
LGTM except now that {{findIndex}} is baked into
{{SinglePartitionReadCommand}}, we perform a lookup through
{{SIM::getBestIndexFor}} when creating the commands to actually read from index
tables in {{CompositesSearcher}} & {{KeysSearcher}}. Also, because those
commands are created using the base table's CFM, we actually end up finding an
index (the one we're in the process of searching). In practice, I don't suppose
this is much of a problem as the execution of those commands is done directly
through {{queryMemtableAndDisk}}, so we don't attempt to erroneously use the
index. However, it is confusing and more seriously, it breaks
{{secondary_index_test:TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query}}.
Aside from that, I just have a couple of tiny nits, feel free to ignore
either/both:
{{getBestIndexFor(ReadCommand)}} is now only used by tests which could easily
be tweaked to use the version which takes a {{RowFilter}}. OFC, that trivial
method doesn't really muck up the the {{SIM}} API, so nbd if it stays, but it
isn't really adding anything either so ¯\_(ツ)_/¯
In some methods of {{*ReadCommand}}, the long lists of args are formatted 1 per
line, and in others are in a single line. e.g. {{SPRC::create}} vs
{{SPRC::copy}} etc. All of these are already touched by this patch, so they may
as well be consistently formatted.
> Fix racy read command serialization
> ---
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13215) Cassandra nodes startup time 20x more after upgarding to 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145464#comment-16145464 ] Marcus Eriksson commented on CASSANDRA-13215: - [~iksaif] yeah I'm working on it, should have a patch ready soon > Cassandra nodes startup time 20x more after upgarding to 3.x > > > Key: CASSANDRA-13215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13215 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Cluster setup: two datacenters (dc-main, dc-backup). > dc-main - 9 servers, no vnodes > dc-backup - 6 servers, vnodes >Reporter: Viktor Kuzmin >Assignee: Marcus Eriksson > Attachments: simple-cache.patch > > > CompactionStrategyManage.getCompactionStrategyIndex is called on each sstable > at startup. And this function calls StorageService.getDiskBoundaries. And > getDiskBoundaries calls AbstractReplicationStrategy.getAddressRanges. > It appears that last function can be really slow. In our environment we have > 1545 tokens and with NetworkTopologyStrategy it can make 1545*1545 > computations in worst case (maybe I'm wrong, but it really takes lot's of > cpu). > Also this function can affect runtime later, cause it is called not only > during startup. > I've tried to implement simple cache for getDiskBoundaries results and now > startup time is about one minute instead of 25m, but I'm not sure if it's a > good solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[09/12] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/82d68408 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/82d68408 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/82d68408 Branch: refs/heads/cassandra-3.11 Commit: 82d684080dac7e1aac61b41395dc5dd84acfda60 Parents: cf0b6d1 4b52a68 Author: Jason Brown Authored: Tue Aug 29 08:01:00 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:02:29 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/82d68408/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[03/12] cassandra git commit: followup to CASSANDRA-13700; minor cleanup
followup to CASSANDRA-13700; minor cleanup
Patch by jasobrown; reviewed by Joel Knighton for CASSANDRA-13700
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b927783
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b927783
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b927783
Branch: refs/heads/cassandra-3.11
Commit: 6b927783ba0777d3dd7c2c3311b246a8dbce5b59
Parents: a3498d5
Author: Jason Brown
Authored: Tue Aug 29 07:52:40 2017 -0700
Committer: Jason Brown
Committed: Tue Aug 29 07:52:40 2017 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b927783/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 7c37917..d3db37f 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -860,8 +860,9 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
* than the version passed in. In this case we also send the old
* heart beat and throw it away on the receiver if it is redundant.
*/
-int localHbGeneration =
epState.getHeartBeatState().getGeneration();
-int localHbVersion =
epState.getHeartBeatState().getHeartBeatVersion();
+HeartBeatState heartBeatState = epState.getHeartBeatState();
+int localHbGeneration = heartBeatState.getGeneration();
+int localHbVersion = heartBeatState.getHeartBeatVersion();
if (localHbVersion > version)
{
reqdEndpointState = new EndpointState(new
HeartBeatState(localHbGeneration, localHbVersion));
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[11/12] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35dc3b65 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35dc3b65 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35dc3b65 Branch: refs/heads/trunk Commit: 35dc3b652f0ba94ff9977035efe9074174f9c69c Parents: 826ae9c 82d6840 Author: Jason Brown Authored: Tue Aug 29 08:22:16 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:23:50 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/35dc3b65/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[01/12] cassandra git commit: followup to CASSANDRA-13700; minor cleanup
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 a3498d5eb -> 6b927783b
refs/heads/cassandra-2.2 c6dec2f0e -> 4b52a6860
refs/heads/cassandra-3.11 826ae9c91 -> 35dc3b652
refs/heads/trunk 278906c6c -> 6e395dd3b
followup to CASSANDRA-13700; minor cleanup
Patch by jasobrown; reviewed by Joel Knighton for CASSANDRA-13700
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b927783
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b927783
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b927783
Branch: refs/heads/cassandra-2.1
Commit: 6b927783ba0777d3dd7c2c3311b246a8dbce5b59
Parents: a3498d5
Author: Jason Brown
Authored: Tue Aug 29 07:52:40 2017 -0700
Committer: Jason Brown
Committed: Tue Aug 29 07:52:40 2017 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b927783/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 7c37917..d3db37f 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -860,8 +860,9 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
* than the version passed in. In this case we also send the old
* heart beat and throw it away on the receiver if it is redundant.
*/
-int localHbGeneration =
epState.getHeartBeatState().getGeneration();
-int localHbVersion =
epState.getHeartBeatState().getHeartBeatVersion();
+HeartBeatState heartBeatState = epState.getHeartBeatState();
+int localHbGeneration = heartBeatState.getGeneration();
+int localHbVersion = heartBeatState.getHeartBeatVersion();
if (localHbVersion > version)
{
reqdEndpointState = new EndpointState(new
HeartBeatState(localHbGeneration, localHbVersion));
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[04/12] cassandra git commit: followup to CASSANDRA-13700; minor cleanup
followup to CASSANDRA-13700; minor cleanup
Patch by jasobrown; reviewed by Joel Knighton for CASSANDRA-13700
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b927783
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b927783
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b927783
Branch: refs/heads/trunk
Commit: 6b927783ba0777d3dd7c2c3311b246a8dbce5b59
Parents: a3498d5
Author: Jason Brown
Authored: Tue Aug 29 07:52:40 2017 -0700
Committer: Jason Brown
Committed: Tue Aug 29 07:52:40 2017 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b927783/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 7c37917..d3db37f 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -860,8 +860,9 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
* than the version passed in. In this case we also send the old
* heart beat and throw it away on the receiver if it is redundant.
*/
-int localHbGeneration =
epState.getHeartBeatState().getGeneration();
-int localHbVersion =
epState.getHeartBeatState().getHeartBeatVersion();
+HeartBeatState heartBeatState = epState.getHeartBeatState();
+int localHbGeneration = heartBeatState.getGeneration();
+int localHbVersion = heartBeatState.getHeartBeatVersion();
if (localHbVersion > version)
{
reqdEndpointState = new EndpointState(new
HeartBeatState(localHbGeneration, localHbVersion));
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[07/12] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b52a686 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b52a686 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b52a686 Branch: refs/heads/trunk Commit: 4b52a6860488280073c5b107d9e7ae824a327fd9 Parents: c6dec2f 6b92778 Author: Jason Brown Authored: Tue Aug 29 07:58:38 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:00:34 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b52a686/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[05/12] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b52a686 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b52a686 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b52a686 Branch: refs/heads/cassandra-3.11 Commit: 4b52a6860488280073c5b107d9e7ae824a327fd9 Parents: c6dec2f 6b92778 Author: Jason Brown Authored: Tue Aug 29 07:58:38 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:00:34 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b52a686/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[06/12] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b52a686 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b52a686 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b52a686 Branch: refs/heads/cassandra-2.2 Commit: 4b52a6860488280073c5b107d9e7ae824a327fd9 Parents: c6dec2f 6b92778 Author: Jason Brown Authored: Tue Aug 29 07:58:38 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:00:34 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b52a686/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[08/12] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/82d68408 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/82d68408 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/82d68408 Branch: refs/heads/trunk Commit: 82d684080dac7e1aac61b41395dc5dd84acfda60 Parents: cf0b6d1 4b52a68 Author: Jason Brown Authored: Tue Aug 29 08:01:00 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:02:29 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/82d68408/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[02/12] cassandra git commit: followup to CASSANDRA-13700; minor cleanup
followup to CASSANDRA-13700; minor cleanup
Patch by jasobrown; reviewed by Joel Knighton for CASSANDRA-13700
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b927783
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b927783
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b927783
Branch: refs/heads/cassandra-2.2
Commit: 6b927783ba0777d3dd7c2c3311b246a8dbce5b59
Parents: a3498d5
Author: Jason Brown
Authored: Tue Aug 29 07:52:40 2017 -0700
Committer: Jason Brown
Committed: Tue Aug 29 07:52:40 2017 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b927783/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 7c37917..d3db37f 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -860,8 +860,9 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
* than the version passed in. In this case we also send the old
* heart beat and throw it away on the receiver if it is redundant.
*/
-int localHbGeneration =
epState.getHeartBeatState().getGeneration();
-int localHbVersion =
epState.getHeartBeatState().getHeartBeatVersion();
+HeartBeatState heartBeatState = epState.getHeartBeatState();
+int localHbGeneration = heartBeatState.getGeneration();
+int localHbVersion = heartBeatState.getHeartBeatVersion();
if (localHbVersion > version)
{
reqdEndpointState = new EndpointState(new
HeartBeatState(localHbGeneration, localHbVersion));
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[10/12] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/35dc3b65 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/35dc3b65 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/35dc3b65 Branch: refs/heads/cassandra-3.11 Commit: 35dc3b652f0ba94ff9977035efe9074174f9c69c Parents: 826ae9c 82d6840 Author: Jason Brown Authored: Tue Aug 29 08:22:16 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:23:50 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/35dc3b65/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[12/12] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e395dd3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e395dd3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e395dd3 Branch: refs/heads/trunk Commit: 6e395dd3b844627586b688163ee00a0882c76485 Parents: 278906c 35dc3b6 Author: Jason Brown Authored: Tue Aug 29 08:24:09 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:24:56 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e395dd3/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13363) Fix racy read command serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13363:
--
Status: Open (was: Patch Available)
> Fix racy read command serialization
> ---
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[2/8] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b52a686 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b52a686 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b52a686 Branch: refs/heads/cassandra-3.0 Commit: 4b52a6860488280073c5b107d9e7ae824a327fd9 Parents: c6dec2f 6b92778 Author: Jason Brown Authored: Tue Aug 29 07:58:38 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:00:34 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b52a686/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[4/8] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e817c83b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e817c83b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e817c83b Branch: refs/heads/cassandra-3.11 Commit: e817c83bc06b7b23aa94c8cd1099103b129801a5 Parents: a7cb009 4b52a68 Author: Jason Brown Authored: Tue Aug 29 08:28:09 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:28:28 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e817c83b/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[1/8] cassandra git commit: followup to CASSANDRA-13700; minor cleanup
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 a7cb009f8 -> e817c83bc
refs/heads/cassandra-3.11 35dc3b652 -> 031d76c9b
refs/heads/trunk 6e395dd3b -> 38ae30685
followup to CASSANDRA-13700; minor cleanup
Patch by jasobrown; reviewed by Joel Knighton for CASSANDRA-13700
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b927783
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b927783
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b927783
Branch: refs/heads/cassandra-3.0
Commit: 6b927783ba0777d3dd7c2c3311b246a8dbce5b59
Parents: a3498d5
Author: Jason Brown
Authored: Tue Aug 29 07:52:40 2017 -0700
Committer: Jason Brown
Committed: Tue Aug 29 07:52:40 2017 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b927783/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 7c37917..d3db37f 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -860,8 +860,9 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
* than the version passed in. In this case we also send the old
* heart beat and throw it away on the receiver if it is redundant.
*/
-int localHbGeneration =
epState.getHeartBeatState().getGeneration();
-int localHbVersion =
epState.getHeartBeatState().getHeartBeatVersion();
+HeartBeatState heartBeatState = epState.getHeartBeatState();
+int localHbGeneration = heartBeatState.getGeneration();
+int localHbVersion = heartBeatState.getHeartBeatVersion();
if (localHbVersion > version)
{
reqdEndpointState = new EndpointState(new
HeartBeatState(localHbGeneration, localHbVersion));
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[6/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/031d76c9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/031d76c9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/031d76c9 Branch: refs/heads/trunk Commit: 031d76c9b4bcf7a3da735cc15d32b5a88be7aeda Parents: 35dc3b6 e817c83 Author: Jason Brown Authored: Tue Aug 29 08:36:58 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:36:58 2017 -0700 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[3/8] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e817c83b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e817c83b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e817c83b Branch: refs/heads/cassandra-3.0 Commit: e817c83bc06b7b23aa94c8cd1099103b129801a5 Parents: a7cb009 4b52a68 Author: Jason Brown Authored: Tue Aug 29 08:28:09 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:28:28 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e817c83b/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[8/8] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/38ae3068 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/38ae3068 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/38ae3068 Branch: refs/heads/trunk Commit: 38ae30685106d745c070114ad9916a9dfcc4ebaf Parents: 6e395dd 031d76c Author: Jason Brown Authored: Tue Aug 29 08:37:22 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:37:22 2017 -0700 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[5/8] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e817c83b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e817c83b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e817c83b Branch: refs/heads/trunk Commit: e817c83bc06b7b23aa94c8cd1099103b129801a5 Parents: a7cb009 4b52a68 Author: Jason Brown Authored: Tue Aug 29 08:28:09 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:28:28 2017 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e817c83b/src/java/org/apache/cassandra/gms/Gossiper.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[7/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/031d76c9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/031d76c9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/031d76c9 Branch: refs/heads/cassandra-3.11 Commit: 031d76c9b4bcf7a3da735cc15d32b5a88be7aeda Parents: 35dc3b6 e817c83 Author: Jason Brown Authored: Tue Aug 29 08:36:58 2017 -0700 Committer: Jason Brown Committed: Tue Aug 29 08:36:58 2017 -0700 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra-dtest git commit: ninja-fix misspelled variable name
Repository: cassandra-dtest
Updated Branches:
refs/heads/master 2ad557dff -> 19b6613d7
ninja-fix misspelled variable name
Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/19b6613d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/19b6613d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/19b6613d
Branch: refs/heads/master
Commit: 19b6613d7c1cd220432af1157b07dbba8fd4a0bb
Parents: 2ad557d
Author: Jason Brown
Authored: Tue Aug 29 08:52:25 2017 -0700
Committer: Jason Brown
Committed: Tue Aug 29 08:52:25 2017 -0700
--
bootstrap_test.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/19b6613d/bootstrap_test.py
--
diff --git a/bootstrap_test.py b/bootstrap_test.py
index 54c49c1..d29390c 100644
--- a/bootstrap_test.py
+++ b/bootstrap_test.py
@@ -150,7 +150,7 @@ class TestBootstrap(BaseBootstrapTest):
cluster = self.cluster
yaml_opts = {'streaming_keep_alive_period_in_secs': 2}
if cluster.version() < '4.0':
-yamp_opts['streaming_socket_timeout_in_ms'] = 1000
+yaml_opts['streaming_socket_timeout_in_ms'] = 1000
cluster.set_configuration_options(values=yaml_opts)
# Create a single node cluster
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13700) Heartbeats can cause gossip information to go permanently missing on certain nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-13700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145523#comment-16145523
]
Jason Brown commented on CASSANDRA-13700:
-
pushed the minor fix as sha {{6b927783ba0777d3dd7c2c3311b246a8dbce5b59}} to
2.1+.
> Heartbeats can cause gossip information to go permanently missing on certain
> nodes
> --
>
> Key: CASSANDRA-13700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13700
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>Priority: Critical
> Fix For: 2.2.11, 3.0.15, 3.11.1, 4.0, 2.1.19
>
>
> In {{Gossiper.getStateForVersionBiggerThan}}, we add the {{HeartBeatState}}
> from the corresponding {{EndpointState}} to the {{EndpointState}} to send.
> When we're getting state for ourselves, this means that we add a reference to
> the local {{HeartBeatState}}. Then, once we've built a message (in either the
> Syn or Ack handler), we send it through the {{MessagingService}}. In the case
> that the {{MessagingService}} is sufficiently slow, the {{GossipTask}} may
> run before serialization of the Syn or Ack. This means that when the
> {{GossipTask}} acquires the gossip {{taskLock}}, it may increment the
> {{HeartBeatState}} version of the local node as stored in the endpoint state
> map. Then, when we finally serialize the Syn or Ack, we'll follow the
> reference to the {{HeartBeatState}} and serialize it with a higher version
> than we saw when constructing the Ack or Ack2.
> Consider the case where we see {{HeartBeatState}} with version 4 when
> constructing an Ack and send it through the {{MessagingService}}. Then, we
> add some piece of state with version 5 to our local {{EndpointState}}. If
> {{GossipTask}} runs and increases the {{HeartBeatState}} version to 6 before
> the {{MessageOut}} containing the Ack is serialized, the node receiving the
> Ack will believe it is current to version 6, despite the fact that it has
> never received a message containing the {{ApplicationState}} tagged with
> version 5.
> I've reproduced in this in several versions; so far, I believe this is
> possible in all versions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys
[
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145667#comment-16145667
]
Jay Zhuang commented on CASSANDRA-13785:
[~aweisberg] do you mind reviewing the patch?
> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13785
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Every a few minutes there're "LEAK DTECTED" messages in the log:
> {noformat}
> ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> {noformat}
> Debugged the issue and found it's triggered by failed compactions, if the
> compacted SSTable has more than 51m {{Integer.MAX_VALUE / 40}}) keys, it will
> fail to create the IndexSummary:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84].
> Cassandra compaction tried to compact every a few minutes and keeps failing.
> The root cause is while [creating
> SafeMemoryWriter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
> with {{> Integer.MAX_VALUE}} space, it returns the tailing
> {{Integer.MAX_VALUE}} space
> [SafeMemoryWriter.java:83|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/util/SafeMemoryWriter.java#L83],
> which makes the first
> [entries.length()|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L173]
> not 0. So the assert fails here:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13363) Fix racy read command serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13363:
--
Status: Patch Available (was: In Progress)
> Fix racy read command serialization
> ---
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13363) Fix racy read command serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145701#comment-16145701
]
Aleksey Yeschenko commented on CASSANDRA-13363:
---
Thanks for the review. Pushed a commit addressing the issues to the same branch
(and rebased on top of most recent 3.0 while at it). Tests scheduled/running:
[utest|https://circleci.com/gh/iamaleksey/cassandra/17],
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/223/].
> Fix racy read command serialization
> ---
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys
[
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145740#comment-16145740
]
Ariel Weisberg commented on CASSANDRA-13785:
Sure I'm looking at it now. What about the leak? Even if this fails it
shouldn't leak memory right?
> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13785
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Every a few minutes there're "LEAK DTECTED" messages in the log:
> {noformat}
> ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> {noformat}
> Debugged the issue and found it's triggered by failed compactions, if the
> compacted SSTable has more than 51m {{Integer.MAX_VALUE / 40}}) keys, it will
> fail to create the IndexSummary:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84].
> Cassandra compaction tried to compact every a few minutes and keeps failing.
> The root cause is while [creating
> SafeMemoryWriter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
> with {{> Integer.MAX_VALUE}} space, it returns the tailing
> {{Integer.MAX_VALUE}} space
> [SafeMemoryWriter.java:83|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/util/SafeMemoryWriter.java#L83],
> which makes the first
> [entries.length()|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L173]
> not 0. So the assert fails here:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13818) Add support for --hosts, --force, and subrange repair to incremental repair
Blake Eggleston created CASSANDRA-13818: --- Summary: Add support for --hosts, --force, and subrange repair to incremental repair Key: CASSANDRA-13818 URL: https://issues.apache.org/jira/browse/CASSANDRA-13818 Project: Cassandra Issue Type: Improvement Reporter: Blake Eggleston Assignee: Blake Eggleston It should be possible to run incremental repair with nodes down, we just shouldn't promote the data to repaired afterwards -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys
[
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145767#comment-16145767
]
Ariel Weisberg commented on CASSANDRA-13785:
Isn't this also a bug in SafeMemoryWriter? Length is supposed to be the amount
of memory written not the size of the allocated buffrer?
> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13785
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Every a few minutes there're "LEAK DTECTED" messages in the log:
> {noformat}
> ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> {noformat}
> Debugged the issue and found it's triggered by failed compactions, if the
> compacted SSTable has more than 51m {{Integer.MAX_VALUE / 40}}) keys, it will
> fail to create the IndexSummary:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84].
> Cassandra compaction tried to compact every a few minutes and keeps failing.
> The root cause is while [creating
> SafeMemoryWriter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
> with {{> Integer.MAX_VALUE}} space, it returns the tailing
> {{Integer.MAX_VALUE}} space
> [SafeMemoryWriter.java:83|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/util/SafeMemoryWriter.java#L83],
> which makes the first
> [entries.length()|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L173]
> not 0. So the assert fails here:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys
[
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145767#comment-16145767
]
Ariel Weisberg edited comment on CASSANDRA-13785 at 8/29/17 5:48 PM:
-
Isn't this also a bug in SafeMemoryWriter? Length is supposed to be the amount
of memory written not the size of the allocated buffer?
was (Author: aweisberg):
Isn't this also a bug in SafeMemoryWriter? Length is supposed to be the amount
of memory written not the size of the allocated buffrer?
> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13785
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Every a few minutes there're "LEAK DTECTED" messages in the log:
> {noformat}
> ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> {noformat}
> Debugged the issue and found it's triggered by failed compactions, if the
> compacted SSTable has more than 51m {{Integer.MAX_VALUE / 40}}) keys, it will
> fail to create the IndexSummary:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84].
> Cassandra compaction tried to compact every a few minutes and keeps failing.
> The root cause is while [creating
> SafeMemoryWriter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
> with {{> Integer.MAX_VALUE}} space, it returns the tailing
> {{Integer.MAX_VALUE}} space
> [SafeMemoryWriter.java:83|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/util/SafeMemoryWriter.java#L83],
> which makes the first
> [entries.length()|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L173]
> not 0. So the assert fails here:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13363) Fix racy read command serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-13363:
Status: Ready to Commit (was: Patch Available)
> Fix racy read command serialization
> ---
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13363) Fix racy read command serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145771#comment-16145771
]
Sam Tunnicliffe commented on CASSANDRA-13363:
-
LGTM and the failing dtest is passing for me locally now, so +1 assuming CI is
still happy.
> Fix racy read command serialization
> ---
>
> Key: CASSANDRA-13363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13363
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6, Cassandra 3.10
>Reporter: Artem Rokhin
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Constantly see this error in the log without any additional information or a
> stack trace.
> {code}
> Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main]
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: null
> {code}
> Logger: org.apache.cassandra.service.CassandraDaemon
> Thrdead: MessagingService-Incoming-/10.0.1.12
> Method: uncaughtException
> File: CassandraDaemon.java
> Line: 229
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145789#comment-16145789
]
Marcus Eriksson commented on CASSANDRA-13752:
-
bq. we should be using a "builder" for StreamingHistogram
looks like we already do this in trunk
so lets just change to CANONICAL and do CASSANDRA-13756 - wdyt [~jjirsa]?
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys
[
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145799#comment-16145799
]
Ariel Weisberg commented on CASSANDRA-13785:
New stuff and summarizing.
* So you are the unlucky person to catch SafeMemoryWriter misbehaving. If we
want our test hygiene to be good you would be the person to write a basic set
of unit tests for it. Go through each external method in SafeMemoryWriter and
test it's API (null handling, largest value, smallest value, negative value, 0)
and internal state (0 length, empty buffer, full buffer, etc.).
* This will still hit the same issue if the decorated key size is larger than
expected right? I know almost everyone uses fixed length keys, but it looks
fragile to not at least fail fast if they use variable length keys or if the
size changes in future iterations.
* This is a bug in {{SafeMemoryWriter.length()}}?
* Even if this throws AssertionError it still shouldn't leak memory right? We
should log the error, but not leak. I think (I'll check) that we don't treat
assertion failures as fatal and crash the process.
> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13785
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>
> Every a few minutes there're "LEAK DTECTED" messages in the log:
> {noformat}
> ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4),
> Memory@[0..d8123468)] was not released before the reference was garbage
> collected
> {noformat}
> Debugged the issue and found it's triggered by failed compactions, if the
> compacted SSTable has more than 51m {{Integer.MAX_VALUE / 40}}) keys, it will
> fail to create the IndexSummary:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84].
> Cassandra compaction tried to compact every a few minutes and keeps failing.
> The root cause is while [creating
> SafeMemoryWriter|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L112]
> with {{> Integer.MAX_VALUE}} space, it returns the tailing
> {{Integer.MAX_VALUE}} space
> [SafeMemoryWriter.java:83|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/util/SafeMemoryWriter.java#L83],
> which makes the first
> [entries.length()|https://github.com/apache/cassandra/blob/6a1b1f26b7174e8c9bf86a96514ab626ce2a4117/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L173]
> not 0. So the assert fails here:
> [IndexSummary:84|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummary.java#L84]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13819) Surprising under-documented behavior with DELETE...USING TIMESTAMP
Eric Wolak created CASSANDRA-13819: -- Summary: Surprising under-documented behavior with DELETE...USING TIMESTAMP Key: CASSANDRA-13819 URL: https://issues.apache.org/jira/browse/CASSANDRA-13819 Project: Cassandra Issue Type: Bug Reporter: Eric Wolak Priority: Minor While investigating differences between various Bigtable derivatives, I‘ve run into an odd behavior of Cassandra. I’m guessing this is intended behavior, but it's surprising enough to me that I think it should be explicitly documented. Let‘s say I have a sensor device reporting data with timestamps. It has a great clock, so I use its timestamps in a USING TIMESTAMP clause in my INSERT statements. One day Jeff realizes that we had a hardware bug with the sensor, and data before timestamp T is incorrect. He issues a DELETE...USING TIMESTAMP T to remove the old data. In the meantime, Sam figures out a way to backfill the data, and she writes a job to insert corrected data into the same table. In keeping with the schema, her job issues INSERT...USING TIMESTAMP statememts, with timestamps before T (because that’s the time the data points correspond to). When testing her job, Sam discovers that the backfilled data isn‘t appearing in the database! In fact, there’s no way for her to insert data with a TIMESTAMP <= T, because the tombstone written by Jeff several days ago is masking them. How can Sam backfill the corrected data? This behavior seems to match the HBase “Current Limitation” that Deletes Mask Puts, documented at http://hbase.apache.org/book.html#_deletes_mask_puts. Should the Cassandra docs also explicitly call-out this behavior? Related: http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html https://docs.datastax.com/en/cassandra/3.0/cassandra/dml/dmlAboutDeletes.html -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145896#comment-16145896
]
Jeff Jirsa commented on CASSANDRA-13752:
Let's just call this a dupe of 13756, and fix it all there.
I'm pushing changes to 3.0 and 3.11 that transform this into a builder, but
also changes to CANONICAL for safety. If you have time, can you do a second
review alongside Jason so we can squash this once and be confident it's fixed?
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13752:
---
Comment: was deleted
(was: I'd rather just change it to a builder in 3.0 and 3.11 for safety in case
someone else comes along and tries to misuse it again in the future?
I've got the 3.0 version done, working on 3.11 now. Would you do a second
review with Jason just to be safe?
)
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145900#comment-16145900
]
Jeff Jirsa commented on CASSANDRA-13752:
I'd rather just change it to a builder in 3.0 and 3.11 for safety in case
someone else comes along and tries to misuse it again in the future?
I've got the 3.0 version done, working on 3.11 now. Would you do a second
review with Jason just to be safe?
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13756) StreamingHistogram is not thread safe
[ https://issues.apache.org/jira/browse/CASSANDRA-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145903#comment-16145903 ] Jeff Jirsa commented on CASSANDRA-13756: Pushed new branches up based on what we learned from 13752: || branch || utest || dtest || | [3.0|https://github.com/jeffjirsa/cassandra/tree/cassandra-3.0-13756] | [3.0 circle|https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-3.0-13756] | [3.0 dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/224/] | | [3.11|https://github.com/jeffjirsa/cassandra/tree/cassandra-3.11-13756] | [3.11 circle|https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-3.11-13756] | [3.11 dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/225/] | > StreamingHistogram is not thread safe > - > > Key: CASSANDRA-13756 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13756 > Project: Cassandra > Issue Type: Bug >Reporter: xiangzhou xia >Assignee: Jeff Jirsa > Fix For: 3.0.x, 3.11.x > > > When we test C*3 in shadow cluster, we notice after a period of time, several > data node suddenly run into 100% cpu and stop process query anymore. > After investigation, we found that threads are stuck on the sum() in > streaminghistogram class. Those are jmx threads that working on expose > getTombStoneRatio metrics (since jmx is kicked off every 3 seconds, there is > a chance that multiple jmx thread is access streaminghistogram at the same > time). > After further investigation, we find that the optimization in CASSANDRA-13038 > led to a spool flush every time when we call sum(). Since TreeMap is not > thread safe, threads will be stuck when multiple threads visit sum() at the > same time. > There are two approaches to solve this issue. > The first one is to add a lock to the flush in sum() which will introduce > some extra overhead to streaminghistogram. > The second one is to avoid streaminghistogram to be access by multiple > threads. For our specific case, is to remove the metrics we added. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145906#comment-16145906
]
Jeff Jirsa commented on CASSANDRA-13752:
[~hkroger] thanks so much for the detailed report and your patience while we
fix this right. Do you mind if I close this as a dupe of 13756, the new patches
there should fix it (and ultimately, it's the same issue - it wasn't thread
safe)?
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145910#comment-16145910
]
Hannu Kröger commented on CASSANDRA-13752:
--
I'm happy as long as it's fixed! Thanks for the support here! :)
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13785) Compaction fails for SSTables with large number of keys
[
https://issues.apache.org/jira/browse/CASSANDRA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145915#comment-16145915
]
Jay Zhuang commented on CASSANDRA-13785:
[~aweisberg] Thanks for the quick response.
{quote}
So you are the unlucky person to catch SafeMemoryWriter misbehaving. If we want
our test hygiene to be good you would be the person to write a basic set of
unit tests for it. Go through each external method in SafeMemoryWriter and test
it's API (null handling, largest value, smallest value, negative value, 0) and
internal state (0 length, empty buffer, full buffer, etc.).
{quote}
Make sense, I'll add more unittests.
{quote}
This will still hit the same issue if the decorated key size is larger than
expected right? I know almost everyone uses fixed length keys, but it looks
fragile to not at least fail fast if they use variable length keys or if the
size changes in future iterations.
{quote}
Yes :( It will fail the assert here: {{[assert entries.length() <=
Integer.MAX_VALUE;|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L172]}}
and trigger the same problem.
{quote}
This is a bug in SafeMemoryWriter.length()?
{quote}
I don't think so, maybe it's a bad name.
{{[length()|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/util/SafeMemoryWriter.java#L81]}}
means the current position. But if the memory longer than Int.MAX_VALUE, it
moves the length() forward to only cover the Int.MAX_VALUE length, so we can
only use the last Int.MAX_VALUE size. e.g. if memory size is {{10 +
Int.MAX_VALUE}}, then the first length() is 10, if memory size is {{10 + 2 *
Int.MAX_VALUE}}, the first length() is {{10 + Int.MAX_VALUE}}. I think the
purpose is to support Int.MAX_VALUE memory, but not sure why, as it only used
by {{IndexSummaryBuilder}}.
{quote}
Even if this throws AssertionError it still shouldn't leak memory right? We
should log the error, but not leak. I think (I'll check) that we don't treat
assertion failures as fatal and crash the process.
{quote}
It doesn't log the error, the compaction thread just fails silently. Here is
the call stack (it's 3.0.14 with a few debugging code, so the line number might
not exactly match):
{{noformat}}
at org.apache.cassandra.io.util.Memory.(Memory.java:82)
at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
at
org.apache.cassandra.io.util.SafeMemoryWriter.(SafeMemoryWriter.java:31)
at
org.apache.cassandra.io.sstable.IndexSummaryBuilder.(IndexSummaryBuilder.java:112)
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.(BigTableWriter.java:377)
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:84)
at
org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
at
org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
at
org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.switchCompactionLocation(DefaultCompactionWriter.java:64)
at
org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.maybeSwitchWriter(CompactionAwareWriter.java:128)
at
org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:108)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
at java.lang.Thread.run(Thread.java:745)
{{noformat}}
I think the leak should be because the {{writer.finish()}} is not called:
[CompactionTask.java:206|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L206]
> Compaction fails for SSTables with large number of keys
> ---
>
> Key: CASSANDRA-13785
>
[jira] [Updated] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13752:
---
Resolution: Duplicate
Status: Resolved (was: Patch Available)
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13752) Corrupted SSTables created in 3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145917#comment-16145917
]
Jeff Jirsa commented on CASSANDRA-13752:
Short term (until you can upgrade), you probably want to avoid calling the
{{getDroppableTombstoneRatio()}} mbean.
> Corrupted SSTables created in 3.11
> --
>
> Key: CASSANDRA-13752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13752
> Project: Cassandra
> Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Hannu Kröger
>Priority: Blocker
> Fix For: 3.11.1
>
>
> We have discovered issues with corrupted SSTables.
> {code}
> ERROR [SSTableBatchOpen:22] 2017-08-03 20:19:53,195 SSTableReader.java:577 -
> Cannot read sstable
> /cassandra/data/mykeyspace/mytable-7a4992800d5611e7b782cb90016f2d17/mc-35556-big=[Data.db,
> Statistics.db, Summary.db, Digest.crc32, CompressionInfo.db, TOC.txt,
> Index.db, Filter.db]; other IO error, skipping table
> java.io.EOFException: EOF after 1898 bytes out of 21093
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:377)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:325)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:231)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:122)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:93)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:488)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> {code}
> Files look like this:
> {code}
> -rw-r--r--. 1 cassandra cassandra 3899251 Aug 7 08:37
> mc-6166-big-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 16874421686 Aug 7 08:37 mc-6166-big-Data.db
> -rw-r--r--. 1 cassandra cassandra 10 Aug 7 08:37
> mc-6166-big-Digest.crc32
> -rw-r--r--. 1 cassandra cassandra 2930904 Aug 7 08:37
> mc-6166-big-Filter.db
> -rw-r--r--. 1 cassandra cassandra 75880 Aug 7 08:37
> mc-6166-big-Index.db
> -rw-r--r--. 1 cassandra cassandra 13762 Aug 7 08:37
> mc-6166-big-Statistics.db
> -rw-r--r--. 1 cassandra cassandra 882008 Aug 7 08:37
> mc-6166-big-Summary.db
> -rw-r--r--. 1 cassandra cassandra 92 Aug 7 08:37 mc-6166-big-TOC.txt
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145925#comment-16145925
]
Jeff Jirsa commented on CASSANDRA-13711:
I saw him open it as CASSANDRA-13764 - let me fix it there just so we don't
re-open this
> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b| writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t |x | 1500565131354883
> b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "z" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "e" ],
> "position" : 21
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "r" ],
> "position" : 45
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "w" ],
> "position" : 69
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "t" ],
> "position" : 93
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
> { "name" : "b", "value" : "x" }
> ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "b" ],
> "position" : 121
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
> }
> ]
> }
> ]MacBook-Pro:~ jjirsa$
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13764) SelectTest.testMixedTTLOnColumnsWide is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa reassigned CASSANDRA-13764:
--
Assignee: Jeff Jirsa
> SelectTest.testMixedTTLOnColumnsWide is flaky
> -
>
> Key: CASSANDRA-13764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13764
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Joel Knighton
>Assignee: Jeff Jirsa
>Priority: Trivial
>
> {{org.apache.cassandra.cql3.validation.operations.SelectTest.testMixedTTLOnColumnsWide}}
> is flaky. This is because it inserts rows and then asserts their contents
> using {{ttl()}} in the select, but if the test is sufficiently slow, the
> remaining ttl may change by the time the select is run. Anecdotally,
> {{testSelectWithAlias}} in the same class uses a fudge factor of 1 second
> that would fix all the failures I've seen, but it might make more sense to
> measure the elapsed time in the test and calculate the acceptable variation
> from that time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13756) StreamingHistogram is not thread safe
[ https://issues.apache.org/jira/browse/CASSANDRA-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145944#comment-16145944 ] Jason Brown commented on CASSANDRA-13756: - I'm +1 on the snapshot version of the code. You may want to wait a day to see if [~krummas] has any additional feedback. > StreamingHistogram is not thread safe > - > > Key: CASSANDRA-13756 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13756 > Project: Cassandra > Issue Type: Bug >Reporter: xiangzhou xia >Assignee: Jeff Jirsa > Fix For: 3.0.x, 3.11.x > > > When we test C*3 in shadow cluster, we notice after a period of time, several > data node suddenly run into 100% cpu and stop process query anymore. > After investigation, we found that threads are stuck on the sum() in > streaminghistogram class. Those are jmx threads that working on expose > getTombStoneRatio metrics (since jmx is kicked off every 3 seconds, there is > a chance that multiple jmx thread is access streaminghistogram at the same > time). > After further investigation, we find that the optimization in CASSANDRA-13038 > led to a spool flush every time when we call sum(). Since TreeMap is not > thread safe, threads will be stuck when multiple threads visit sum() at the > same time. > There are two approaches to solve this issue. > The first one is to add a lock to the flush in sum() which will introduce > some extra overhead to streaminghistogram. > The second one is to avoid streaminghistogram to be access by multiple > threads. For our specific case, is to remove the metrics we added. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13433) RPM distribution improvements and known issues
[
https://issues.apache.org/jira/browse/CASSANDRA-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145967#comment-16145967
]
Nathaniel Tabernero commented on CASSANDRA-13433:
-
I've attached a patch file against cassandra-3.9 for creating a rpm which works
on Centos 6. At the time we upgraded Cassandra, version 3.9 was the latest
stable version. This patch file includes a rpm spec file which has a dependency
on python27 from the centos-release-scl repo. The python27 package will install
python into an alternate location which will not conflict the Centos 6's
default installed python 2.6 . Additionally, the spec file will modify the
cqlsh script to use python 2.7.
Here are the steps we use to install Cassandra 3.9 on Centos 6.9 in our
environment:
{panel}
#Install Java 8, then
sudo yum install centos-release-scl # install SCL repo
sudo yum localinstall cassandra-3.9-2.scl.el6.noarch.rpm #install custom build
cassandra 3.9 rpm
sudo service cassandra start
sudo chkconfig cassandra on
{panel}
If this custom rpm is not available and cassandra 3.9 is already installed
through other means, then:
{panel}
sudo yum install centos-release-scl # install SCL repo
sudo yum install python27 # install Python 2.7
{panel}
Then, manually modify the cqlsh script. Add the following lines after the
comment header
{panel}
# Enable python2.7 from centos 6 SCL
source /opt/rh/python27/enable
{panel}
I hope this is helpful!
> RPM distribution improvements and known issues
> --
>
> Key: CASSANDRA-13433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13433
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>
> Starting with CASSANDRA-13252, new releases will be provided as both official
> RPM and Debian packages. While the Debian packages are already well
> established with our user base, the RPMs just have been release for the first
> time and still require some attention.
> Feel free to discuss RPM related issues in this ticket and open a sub-task to
> fill a bug report.
> Please note that native systemd support will be implemented with
> CASSANDRA-13148 and this is not strictly a RPM specific issue. We still
> intent to offer non-systemd support based on the already working init scripts
> that we ship. Therefor the first step is to make use of systemd backward
> compatibility for SysV/LSB scripts, so we can provide RPMs for both systemd
> and non-systemd environments.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13433) RPM distribution improvements and known issues
[
https://issues.apache.org/jira/browse/CASSANDRA-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145967#comment-16145967
]
Nathaniel Tabernero edited comment on CASSANDRA-13433 at 8/29/17 7:37 PM:
--
I've attached a patch file against cassandra-3.9 for creating a rpm which works
on Centos 6. At the time we upgraded Cassandra, version 3.9 was the latest
stable version. This patch file includes a rpm spec file which has a dependency
on python27 from the centos-release-scl repo. The python27 package will install
python into an alternate location which will not conflict the Centos 6's
default installed python 2.6 . Additionally, the spec file will modify the
cqlsh script to use python 2.7.
Here are the steps we use to install Cassandra 3.9 on Centos 6.9 in our
environment:
{panel}
#Install Java 8, then
sudo yum install centos-release-scl # install SCL repo
sudo yum localinstall cassandra-3.9-2.scl.el6.noarch.rpm #install custom build
cassandra 3.9 rpm
sudo service cassandra start
sudo chkconfig cassandra on
{panel}
If this custom rpm is not available and cassandra 3.9 is already installed
through other means, then:
{panel}
sudo yum install centos-release-scl # install SCL repo
sudo yum install python27 # install Python 2.7
{panel}
Then, manually modify the cqlsh script. Add the following lines after the
comment header
{panel}
\# Enable python2.7 from centos 6 SCL
source /opt/rh/python27/enable
{panel}
I hope this is helpful!
was (Author: ntabernero):
I've attached a patch file against cassandra-3.9 for creating a rpm which works
on Centos 6. At the time we upgraded Cassandra, version 3.9 was the latest
stable version. This patch file includes a rpm spec file which has a dependency
on python27 from the centos-release-scl repo. The python27 package will install
python into an alternate location which will not conflict the Centos 6's
default installed python 2.6 . Additionally, the spec file will modify the
cqlsh script to use python 2.7.
Here are the steps we use to install Cassandra 3.9 on Centos 6.9 in our
environment:
{panel}
#Install Java 8, then
sudo yum install centos-release-scl # install SCL repo
sudo yum localinstall cassandra-3.9-2.scl.el6.noarch.rpm #install custom build
cassandra 3.9 rpm
sudo service cassandra start
sudo chkconfig cassandra on
{panel}
If this custom rpm is not available and cassandra 3.9 is already installed
through other means, then:
{panel}
sudo yum install centos-release-scl # install SCL repo
sudo yum install python27 # install Python 2.7
{panel}
Then, manually modify the cqlsh script. Add the following lines after the
comment header
{panel}
# Enable python2.7 from centos 6 SCL
source /opt/rh/python27/enable
{panel}
I hope this is helpful!
> RPM distribution improvements and known issues
> --
>
> Key: CASSANDRA-13433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13433
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Attachments: cassandra-3.9-centos6.patch
>
>
> Starting with CASSANDRA-13252, new releases will be provided as both official
> RPM and Debian packages. While the Debian packages are already well
> established with our user base, the RPMs just have been release for the first
> time and still require some attention.
> Feel free to discuss RPM related issues in this ticket and open a sub-task to
> fill a bug report.
> Please note that native systemd support will be implemented with
> CASSANDRA-13148 and this is not strictly a RPM specific issue. We still
> intent to offer non-systemd support based on the already working init scripts
> that we ship. Therefor the first step is to make use of systemd backward
> compatibility for SysV/LSB scripts, so we can provide RPMs for both systemd
> and non-systemd environments.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13433) RPM distribution improvements and known issues
[ https://issues.apache.org/jira/browse/CASSANDRA-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathaniel Tabernero updated CASSANDRA-13433: Attachment: cassandra-3.9-centos6.patch patch file against cassandra-3.9 for creating a rpm which works on Centos 6 > RPM distribution improvements and known issues > -- > > Key: CASSANDRA-13433 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13433 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski > Attachments: cassandra-3.9-centos6.patch > > > Starting with CASSANDRA-13252, new releases will be provided as both official > RPM and Debian packages. While the Debian packages are already well > established with our user base, the RPMs just have been release for the first > time and still require some attention. > Feel free to discuss RPM related issues in this ticket and open a sub-task to > fill a bug report. > Please note that native systemd support will be implemented with > CASSANDRA-13148 and this is not strictly a RPM specific issue. We still > intent to offer non-systemd support based on the already working init scripts > that we ship. Therefor the first step is to make use of systemd backward > compatibility for SysV/LSB scripts, so we can provide RPMs for both systemd > and non-systemd environments. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13433) RPM distribution improvements and known issues
[
https://issues.apache.org/jira/browse/CASSANDRA-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145967#comment-16145967
]
Nathaniel Tabernero edited comment on CASSANDRA-13433 at 8/29/17 7:40 PM:
--
I've attached a patch file against cassandra-3.9 for creating a rpm which works
on Centos 6 ([^cassandra-3.9-centos6.patch]). At the time we upgraded
Cassandra, version 3.9 was the latest stable version. This patch file includes
a rpm spec file which has a dependency on python27 from the centos-release-scl
repo. The python27 package will install python into an alternate location which
will not conflict the Centos 6's default installed python 2.6 . Additionally,
the spec file will modify the cqlsh script to use python 2.7.
Here are the steps we use to install Cassandra 3.9 on Centos 6.9 in our
environment:
{panel}
#Install Java 8, then
sudo yum install centos-release-scl # install SCL repo
sudo yum localinstall cassandra-3.9-2.scl.el6.noarch.rpm #install custom build
cassandra 3.9 rpm
sudo service cassandra start
sudo chkconfig cassandra on
{panel}
If this custom rpm is not available and cassandra 3.9 is already installed
through other means, then:
{panel}
sudo yum install centos-release-scl # install SCL repo
sudo yum install python27 # install Python 2.7
{panel}
Then, manually modify the cqlsh script. Add the following lines after the
comment header
{panel}
\# Enable python2.7 from centos 6 SCL
source /opt/rh/python27/enable
{panel}
I hope this is helpful!
was (Author: ntabernero):
I've attached a patch file against cassandra-3.9 for creating a rpm which works
on Centos 6. At the time we upgraded Cassandra, version 3.9 was the latest
stable version. This patch file includes a rpm spec file which has a dependency
on python27 from the centos-release-scl repo. The python27 package will install
python into an alternate location which will not conflict the Centos 6's
default installed python 2.6 . Additionally, the spec file will modify the
cqlsh script to use python 2.7.
Here are the steps we use to install Cassandra 3.9 on Centos 6.9 in our
environment:
{panel}
#Install Java 8, then
sudo yum install centos-release-scl # install SCL repo
sudo yum localinstall cassandra-3.9-2.scl.el6.noarch.rpm #install custom build
cassandra 3.9 rpm
sudo service cassandra start
sudo chkconfig cassandra on
{panel}
If this custom rpm is not available and cassandra 3.9 is already installed
through other means, then:
{panel}
sudo yum install centos-release-scl # install SCL repo
sudo yum install python27 # install Python 2.7
{panel}
Then, manually modify the cqlsh script. Add the following lines after the
comment header
{panel}
\# Enable python2.7 from centos 6 SCL
source /opt/rh/python27/enable
{panel}
I hope this is helpful!
> RPM distribution improvements and known issues
> --
>
> Key: CASSANDRA-13433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13433
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Attachments: cassandra-3.9-centos6.patch
>
>
> Starting with CASSANDRA-13252, new releases will be provided as both official
> RPM and Debian packages. While the Debian packages are already well
> established with our user base, the RPMs just have been release for the first
> time and still require some attention.
> Feel free to discuss RPM related issues in this ticket and open a sub-task to
> fill a bug report.
> Please note that native systemd support will be implemented with
> CASSANDRA-13148 and this is not strictly a RPM specific issue. We still
> intent to offer non-systemd support based on the already working init scripts
> that we ship. Therefor the first step is to make use of systemd backward
> compatibility for SysV/LSB scripts, so we can provide RPMs for both systemd
> and non-systemd environments.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13764) SelectTest.testMixedTTLOnColumnsWide is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145997#comment-16145997
]
Jeff Jirsa commented on CASSANDRA-13764:
[~slebresne] says he's ok with ninja'ing this in, but in case you ( [~jkni] )
want to review before I do it (I need to push for Circle anyway just to be
safe, so this'll sit here until Circle gives me a green run):
3.0: https://github.com/jeffjirsa/cassandra/tree/cassandra-3.0-13764 (Circle:
https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-3.0-13764 )
3.11: https://github.com/jeffjirsa/cassandra/tree/cassandra-3.11-13764 (Circle:
https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-3.11-13764 )
trunk: https://github.com/jeffjirsa/cassandra/tree/cassandra-13764 (Circle:
https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-13764 )
> SelectTest.testMixedTTLOnColumnsWide is flaky
> -
>
> Key: CASSANDRA-13764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13764
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Joel Knighton
>Assignee: Jeff Jirsa
>Priority: Trivial
>
> {{org.apache.cassandra.cql3.validation.operations.SelectTest.testMixedTTLOnColumnsWide}}
> is flaky. This is because it inserts rows and then asserts their contents
> using {{ttl()}} in the select, but if the test is sufficiently slow, the
> remaining ttl may change by the time the select is run. Anecdotally,
> {{testSelectWithAlias}} in the same class uses a fudge factor of 1 second
> that would fix all the failures I've seen, but it might make more sense to
> measure the elapsed time in the test and calculate the acceptable variation
> from that time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13764) SelectTest.testMixedTTLOnColumnsWide is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13764:
---
Status: Patch Available (was: In Progress)
> SelectTest.testMixedTTLOnColumnsWide is flaky
> -
>
> Key: CASSANDRA-13764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13764
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Joel Knighton
>Assignee: Jeff Jirsa
>Priority: Trivial
>
> {{org.apache.cassandra.cql3.validation.operations.SelectTest.testMixedTTLOnColumnsWide}}
> is flaky. This is because it inserts rows and then asserts their contents
> using {{ttl()}} in the select, but if the test is sufficiently slow, the
> remaining ttl may change by the time the select is run. Anecdotally,
> {{testSelectWithAlias}} in the same class uses a fudge factor of 1 second
> that would fix all the failures I've seen, but it might make more sense to
> measure the elapsed time in the test and calculate the acceptable variation
> from that time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13692) CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146000#comment-16146000
]
Dimitar Dimitrov commented on CASSANDRA-13692:
--
Sorry for the late promised update.
I'm currently finalizing the test, which is a bit of a mix between
{{OutOfSpaceTest}}, {{CompactionAwareWriterTest}}, and
{{CompactionsBytemanTest}}.
I reckon that (1) the {{OutOfSpaceTest}}s approach translates well enough for
checking whether failure policies are correctly triggered in this case; (2) the
trickiest part (for me) would be to figure out whether the way
{{CompactionAwareWriter#getWriteDirectory(Iterable, long)}} gets
reached and executed, is correct and relevant.
> CompactionAwareWriter_getWriteDirectory throws incompatible exceptions
> --
>
> Key: CASSANDRA-13692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13692
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Hao Zhong
>Assignee: Dimitar Dimitrov
> Labels: lhf
>
> The CompactionAwareWriter_getWriteDirectory throws RuntimeException:
> {code}
> public Directories.DataDirectory getWriteDirectory(Iterable
> sstables, long estimatedWriteSize)
> {
> File directory = null;
> for (SSTableReader sstable : sstables)
> {
> if (directory == null)
> directory = sstable.descriptor.directory;
> if (!directory.equals(sstable.descriptor.directory))
> {
> logger.trace("All sstables not from the same disk - putting
> results in {}", directory);
> break;
> }
> }
> Directories.DataDirectory d =
> getDirectories().getDataDirectoryForFile(directory);
> if (d != null)
> {
> long availableSpace = d.getAvailableSpace();
> if (availableSpace < estimatedWriteSize)
> throw new RuntimeException(String.format("Not enough space to
> write %s to %s (%s available)",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize),
> d.location,
>
> FBUtilities.prettyPrintMemory(availableSpace)));
> logger.trace("putting compaction results in {}", directory);
> return d;
> }
> d = getDirectories().getWriteableLocation(estimatedWriteSize);
> if (d == null)
> throw new RuntimeException(String.format("Not enough disk space
> to store %s",
>
> FBUtilities.prettyPrintMemory(estimatedWriteSize)));
> return d;
> }
> {code}
> However, the thrown exception does not trigger the failure policy.
> CASSANDRA-11448 fixed a similar problem. The buggy code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new RuntimeException("Insufficient disk space to write " +
> writeSize + " bytes");
> return directory;
> }
> {code}
> The fixed code is:
> {code}
> protected Directories.DataDirectory getWriteDirectory(long writeSize)
> {
> Directories.DataDirectory directory =
> getDirectories().getWriteableLocation(writeSize);
> if (directory == null)
> throw new FSWriteError(new IOException("Insufficient disk space
> to write " + writeSize + " bytes"), "");
> return directory;
> }
> {code}
> The fixed code throws FSWE and triggers the failure policy.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
