[jira] [Comment Edited] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted
[
https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101603#comment-16101603
]
ZhaoYang edited comment on CASSANDRA-11500 at 8/1/17 6:27 AM:
--
WIP
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-11500-cell]
[dtest|https://github.com/riptano/cassandra-dtest/commits/CASSANDRA-11500]
Changed:
* Extra "VirtualCell"(kind of special LivenessInfo for MV) in Row to maintain
consistency of view row. It stores: 1. base column used in view PK and base
column used in view filter conditions. if any of such column dead, entire view
row dead, regardless LivenessInfo or DeletionTime status. or 2. unselected base
columns. if any of such column alive, view's pk should be alive.
* blocked dropping filter base column
* fix issue of creating view with token() filter
* remove Row.Deletion, it was used to wrap DeletionTime with "Shadowable" flag
* fix missing partition deletion in view-update
Todo:
more dtest
was (Author: jasonstack):
WIP
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-11500-cell]
Changed:
* Extra "VirtualCell"(kind of special LivenessInfo for MV) in Row to maintain
liveness of view row. It stores: 1. base column used in view PK and base column
used in view filter conditions. if any of such column dead, entire view row
dead, regardless LivenessInfo or DeletionTime status. 2. unselected base
columns. if any of such column alive, view's pk should be alive if it's not
deleted by DeletionTime or those columns in <1>.
* blocked dropping filter base column
* fix issue of creating view with token() filter
* remove Row.Deletion, it was used to wrap DeletionTime with "Shadowable" flag
Todo:
optimize in-memory/storage representation for "virtual-cells" to re-use
AbstractRow/BTree
more dtest
optimize ViewUpdateGenerator process to reduce payload. eg, if view row has
live pk or other live column, "virtualCells' unselected payload" is not
necessary.
> Obsolete MV entry may not be properly deleted
> -
>
> Key: CASSANDRA-11500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11500
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Sylvain Lebresne
>Assignee: ZhaoYang
>
> When a Materialized View uses a non-PK base table column in its PK, if an
> update changes that column value, we add the new view entry and remove the
> old one. When doing that removal, the current code uses the same timestamp
> than for the liveness info of the new entry, which is the max timestamp for
> any columns participating to the view PK. This is not correct for the
> deletion as the old view entry could have other columns with higher timestamp
> which won't be deleted as can easily shown by the failing of the following
> test:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 4 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1;
> SELECT * FROM mv WHERE k = 1; // This currently return 2 entries, the old
> (invalid) and the new one
> {noformat}
> So the correct timestamp to use for the deletion is the biggest timestamp in
> the old view entry (which we know since we read the pre-existing base row),
> and that is what CASSANDRA-11475 does (the test above thus doesn't fail on
> that branch).
> Unfortunately, even then we can still have problems if further updates
> requires us to overide the old entry. Consider the following case:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 10 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1; // This will delete the
> entry for a=1 with timestamp 10
> UPDATE t USING TIMESTAMP 3 SET a = 1 WHERE k = 1; // This needs to re-insert
> an entry for a=1 but shouldn't be deleted by the prior deletion
> UPDATE t USING TIMESTAMP 4 SET a = 2 WHERE k = 1; // ... and we can play this
> game more than once
> UPDATE t USING TIMESTAMP 5 SET a = 1 WHERE k = 1;
> ...
> {noformat}
> In a way, this is saying that the "shadowable" deletion mechanism is not
> general enough: we need to be able to re-insert an entry when a prior one had
> been deleted before, but we can't rely on timestamps being strictly bigger on
> the re-insert. In that sense, this can be though as a similar problem than
> CASSANDRA-10965, though the solution there of a single flag is not enoug
[jira] [Commented] (CASSANDRA-13736) CASSANDRA-9673 cause atomic batch p99 increase 3x
[
https://issues.apache.org/jira/browse/CASSANDRA-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108355#comment-16108355
]
Jeff Jirsa commented on CASSANDRA-13736:
I think blocking for {{TWO}} on batchlog sync is the intended behavior, though
it's not technically in the
[docs|http://cassandra.apache.org/doc/latest/cql/dml.html#batch]
Having a batch only sync'd to one node doesn't seem like much of a guarantee.
> CASSANDRA-9673 cause atomic batch p99 increase 3x
> -
>
> Key: CASSANDRA-13736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13736
> Project: Cassandra
> Issue Type: Bug
>Reporter: xiangzhou xia
>Assignee: xiangzhou xia
>
> When we testing atomic batch in production traffic, we found that p99 latency
> in atomic batch write is 2x-3x worse than 2.2.
> After debuging, we found that the regression is causing by CASSANDRA-9673.
> This patch changed consistency level in batchlog store from ONE to TWO.
> [~iamaleksey] think only block for one batchlog message is a bug in batchlog
> and change it to block for two in CASSANDRA-9673, I think it's actually a
> very good optimization to reduce latency.
> Set the consistency to one will decrease the possibility of slow data node
> (GC, long message queue, etc) affect the latency of atomic batch. In our
> shadow cluster, when we change consistency from two to one, we notice a 2x-3x
> p99 latency drop in atomic batch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13736) CASSANDRA-9673 cause atomic batch p99 increase 3x
[ https://issues.apache.org/jira/browse/CASSANDRA-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiangzhou xia updated CASSANDRA-13736: -- Description: When we testing atomic batch in production traffic, we found that p99 latency in atomic batch write is 2x-3x worse than 2.2. After debuging, we found that the regression is causing by CASSANDRA-9673. This patch changed consistency level in batchlog store from ONE to TWO. [~iamaleksey] think only block for one batchlog message is a bug in batchlog and change it to block for two in CASSANDRA-9673, I think it's actually a very good optimization to reduce latency. Set the consistency to one will decrease the possibility of slow data node (GC, long message queue, etc) affect the latency of atomic batch. In our shadow cluster, when we change consistency from two to one, we notice a 2x-3x p99 latency drop in atomic batch. was:we notice that changing consistency level from ONE to TWO dramatically increased p99 latency in 3.0 atomic batch. [~iamaleksey] think only block for one batchlog message is a bug in batchlog and change it to block for two in CASSANDRA-9673, I think it's actually a very good optimization to reduce latency. Set the consistency to one will decrease the possibility of slow data node (GC, long message queue, etc) affect the latency of atomic batch. In our shadow cluster, when we change consistency from two to one, we notice a 2x-3x p99 latency drop in atomic batch. Summary: CASSANDRA-9673 cause atomic batch p99 increase 3x (was: consistency level change in batchlog send from CASSANDRA-9673 cause atomic batch p99 increase 3x) > CASSANDRA-9673 cause atomic batch p99 increase 3x > - > > Key: CASSANDRA-13736 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13736 > Project: Cassandra > Issue Type: Bug >Reporter: xiangzhou xia >Assignee: xiangzhou xia > > When we testing atomic batch in production traffic, we found that p99 latency > in atomic batch write is 2x-3x worse than 2.2. > After debuging, we found that the regression is causing by CASSANDRA-9673. > This patch changed consistency level in batchlog store from ONE to TWO. > [~iamaleksey] think only block for one batchlog message is a bug in batchlog > and change it to block for two in CASSANDRA-9673, I think it's actually a > very good optimization to reduce latency. > Set the consistency to one will decrease the possibility of slow data node > (GC, long message queue, etc) affect the latency of atomic batch. In our > shadow cluster, when we change consistency from two to one, we notice a 2x-3x > p99 latency drop in atomic batch. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13736) consistency level change in batchlog send from CASSANDRA-9673 cause atomic batch p99 increase 3x
[ https://issues.apache.org/jira/browse/CASSANDRA-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu reassigned CASSANDRA-13736: - Assignee: xiangzhou xia > consistency level change in batchlog send from CASSANDRA-9673 cause atomic > batch p99 increase 3x > > > Key: CASSANDRA-13736 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13736 > Project: Cassandra > Issue Type: Bug >Reporter: xiangzhou xia >Assignee: xiangzhou xia > > we notice that changing consistency level from ONE to TWO dramatically > increased p99 latency in 3.0 atomic batch. [~iamaleksey] think only block for > one batchlog message is a bug in batchlog and change it to block for two in > CASSANDRA-9673, I think it's actually a very good optimization to reduce > latency. Set the consistency to one will decrease the possibility of slow > data node (GC, long message queue, etc) affect the latency of atomic batch. > In our shadow cluster, when we change consistency from two to one, we notice > a 2x-3x p99 latency drop in atomic batch. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13736) consistency level change in batchlog send from CASSANDRA-9673 cause atomic batch p99 increase 3x
[ https://issues.apache.org/jira/browse/CASSANDRA-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiangzhou xia updated CASSANDRA-13736: -- Description: we notice that changing consistency level from ONE to TWO dramatically increased p99 latency in 3.0 atomic batch. [~iamaleksey] think only block for one batchlog message is a bug in batchlog and change it to block for two in CASSANDRA-9673, I think it's actually a very good optimization to reduce latency. Set the consistency to one will decrease the possibility of slow data node (GC, long message queue, etc) affect the latency of atomic batch. In our shadow cluster, when we change consistency from two to one, we notice a 2x-3x p99 latency drop in atomic batch. (was: we notice that changing consistency level from ONE to TWO dramatically increased p99 latency in 3.0 atomic batch. [~iamaleksey] think only block for one batchlog message is a bug in batchlog, I think it's actually a very good optimization to reduce latency. Set the consistency to one will decrease the possibility of slow data node (GC, long message queue, etc) affect the latency of atomic batch. In our shadow cluster, when we change consistency from two to one, we notice a 2x-3x p99 latency drop in atomic batch. ) > consistency level change in batchlog send from CASSANDRA-9673 cause atomic > batch p99 increase 3x > > > Key: CASSANDRA-13736 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13736 > Project: Cassandra > Issue Type: Bug >Reporter: xiangzhou xia > > we notice that changing consistency level from ONE to TWO dramatically > increased p99 latency in 3.0 atomic batch. [~iamaleksey] think only block for > one batchlog message is a bug in batchlog and change it to block for two in > CASSANDRA-9673, I think it's actually a very good optimization to reduce > latency. Set the consistency to one will decrease the possibility of slow > data node (GC, long message queue, etc) affect the latency of atomic batch. > In our shadow cluster, when we change consistency from two to one, we notice > a 2x-3x p99 latency drop in atomic batch. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13736) consistency level change in batchlog send from CASSANDRA-9673 cause atomic batch p99 increase 3x
xiangzhou xia created CASSANDRA-13736: - Summary: consistency level change in batchlog send from CASSANDRA-9673 cause atomic batch p99 increase 3x Key: CASSANDRA-13736 URL: https://issues.apache.org/jira/browse/CASSANDRA-13736 Project: Cassandra Issue Type: Bug Reporter: xiangzhou xia we notice that changing consistency level from ONE to TWO dramatically increased p99 latency in 3.0 atomic batch. [~iamaleksey] think only block for one batchlog message is a bug in batchlog, I think it's actually a very good optimization to reduce latency. Set the consistency to one will decrease the possibility of slow data node (GC, long message queue, etc) affect the latency of atomic batch. In our shadow cluster, when we change consistency from two to one, we notice a 2x-3x p99 latency drop in atomic batch. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-13441) Schema version changes for each upgraded node in a rolling upgrade, causing migration storms
[ https://issues.apache.org/jira/browse/CASSANDRA-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tania S Engel updated CASSANDRA-13441: -- Comment: was deleted (was: We also see this running Cassandra 3.10, not upgrading rather when a new node is joining. The join seems to complete but there are no stream session logs and we end up failing with UnknownColumnFamily. !screenshot-1.png! ) > Schema version changes for each upgraded node in a rolling upgrade, causing > migration storms > > > Key: CASSANDRA-13441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13441 > Project: Cassandra > Issue Type: Bug > Components: Distributed Metadata >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 3.0.14, 3.11.0, 4.0 > > > In versions < 3.0, during a rolling upgrade (say 2.0 -> 2.1), the first node > to upgrade to 2.1 would add the new tables, setting the new 2.1 version ID, > and subsequently upgraded hosts would settle on that version. > When a 3.0 node upgrades and writes its own new-in-3.0 system tables, it'll > write the same tables that exist in the schema with brand new timestamps. As > written, this will cause all nodes in the cluster to change schema (to the > version with the newest timestamp). On a sufficiently large cluster with a > non-trivial schema, this could cause (literally) millions of migration tasks > to needlessly bounce across the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13441) Schema version changes for each upgraded node in a rolling upgrade, causing migration storms
[ https://issues.apache.org/jira/browse/CASSANDRA-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tania S Engel updated CASSANDRA-13441: -- Attachment: (was: screenshot-1.png) > Schema version changes for each upgraded node in a rolling upgrade, causing > migration storms > > > Key: CASSANDRA-13441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13441 > Project: Cassandra > Issue Type: Bug > Components: Distributed Metadata >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 3.0.14, 3.11.0, 4.0 > > > In versions < 3.0, during a rolling upgrade (say 2.0 -> 2.1), the first node > to upgrade to 2.1 would add the new tables, setting the new 2.1 version ID, > and subsequently upgraded hosts would settle on that version. > When a 3.0 node upgrades and writes its own new-in-3.0 system tables, it'll > write the same tables that exist in the schema with brand new timestamps. As > written, this will cause all nodes in the cluster to change schema (to the > version with the newest timestamp). On a sufficiently large cluster with a > non-trivial schema, this could cause (literally) millions of migration tasks > to needlessly bounce across the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-13441) Schema version changes for each upgraded node in a rolling upgrade, causing migration storms
[ https://issues.apache.org/jira/browse/CASSANDRA-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tania S Engel updated CASSANDRA-13441: -- Comment: was deleted (was: It is unclear to me if the migration task timing out caused none of our tables to stream. I am 100% certain we have keyspaces on the seed node but none of it was streamed. From reading the various related/duplicate cases, it is unclear if the suggestion is to increase -Dcassandra.migration_task_wait_in_seconds or upgrade to 3.11 or both. This problem is intermittent and it happens even when we are bootstrapping the first node to form a cluster of 2. Is the idea that if we "wait for schema agreement...", we will eventually get the tables? It seems that is not the case for this scenario. Right now our code will wait for >nodetool netstats MODE=NORMAL, but in this case, the CQL port is already open and we have already seen "boostrap complete" so I am 99% sure the MODE=NORMAL (no longer JOINING). Also, I have seen a bootstrap scenario where just one "ERROR Migration task failed to complete" logged and then the bootstrap succeeded and streamed all our tables. ) > Schema version changes for each upgraded node in a rolling upgrade, causing > migration storms > > > Key: CASSANDRA-13441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13441 > Project: Cassandra > Issue Type: Bug > Components: Distributed Metadata >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 3.0.14, 3.11.0, 4.0 > > > In versions < 3.0, during a rolling upgrade (say 2.0 -> 2.1), the first node > to upgrade to 2.1 would add the new tables, setting the new 2.1 version ID, > and subsequently upgraded hosts would settle on that version. > When a 3.0 node upgrades and writes its own new-in-3.0 system tables, it'll > write the same tables that exist in the schema with brand new timestamps. As > written, this will cause all nodes in the cluster to change schema (to the > version with the newest timestamp). On a sufficiently large cluster with a > non-trivial schema, this could cause (literally) millions of migration tasks > to needlessly bounce across the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11995) Commitlog replaced with all NULs
[
https://issues.apache.org/jira/browse/CASSANDRA-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108122#comment-16108122
]
Jeff Jirsa commented on CASSANDRA-11995:
[~blambov] - I do think there's a few areas for improvement, though this is the
low hanging fruit of it. The one case where I was able to see this myself was
in a test environment where failures were expected (and indeed, injected), but
based on the descriptions above, I believe happens on any unclean shutdown
triggered during startup.
I'd personally like to commit this as an intermediate step, and not try to fix
ALL of the potential problems at once (I think this removes the most obvious
cause, and does it with very little risk). Are you ok with that, and we can
open some follow-up tickets as they arise?
> Commitlog replaced with all NULs
>
>
> Key: CASSANDRA-11995
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11995
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows 10 Enterprise 1511
> DataStax Cassandra Community Server 2.2.3
>Reporter: James Howe
>Assignee: Jeff Jirsa
>
> I noticed this morning that Cassandra was failing to start, after being shut
> down on Friday.
> {code}
> ERROR 09:13:37 Exiting due to error while processing commit log during
> initialization.
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException:
> Could not read commit log descriptor in file C:\Program Files\DataStax
> Community\data\commitlog\CommitLog-5-1465571056722.log
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:622)
> [apache-cassandra-2.2.3.jar:2.2.3]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:302)
> [apache-cassandra-2.2.3.jar:2.2.3]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:147)
> [apache-cassandra-2.2.3.jar:2.2.3]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189)
> [apache-cassandra-2.2.3.jar:2.2.3]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169)
> [apache-cassandra-2.2.3.jar:2.2.3]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:273)
> [apache-cassandra-2.2.3.jar:2.2.3]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:513)
> [apache-cassandra-2.2.3.jar:2.2.3]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:622)
> [apache-cassandra-2.2.3.jar:2.2.3]
> {code}
> Checking the referenced file reveals it comprises 33,554,432 (32 * 1024 *
> 1024) NUL bytes.
> No logs (stdout, stderr, prunsrv) from the shutdown show any other issues and
> appear exactly as normal.
> Is installed as a service via DataStax's distribution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13711:
---
Status: Ready to Commit (was: Patch Available)
> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b| writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t |x | 1500565131354883
> b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "z" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "e" ],
> "position" : 21
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "r" ],
> "position" : 45
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "w" ],
> "position" : 69
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "t" ],
> "position" : 93
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
> { "name" : "b", "value" : "x" }
> ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "b" ],
> "position" : 121
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
> }
> ]
> }
> ]MacBook-Pro:~ jjirsa$
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13711:
---
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 4.x)
(was: 3.0.x)
4.0
3.11.1
3.0.15
Status: Resolved (was: Ready to Commit)
Committed to 3.0 as {{9dc896f4ea51276de4ea76ffca3fd719e0c8b8a1}} and merged
3.11 and 4.0
> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b| writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t |x | 1500565131354883
> b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "z" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "e" ],
> "position" : 21
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "r" ],
> "position" : 45
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "w" ],
> "position" : 69
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "t" ],
> "position" : 93
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
> { "name" : "b", "value" : "x" }
> ]
> }
> ]
> },
> {
> "partition" : {
> "key" : [ "b" ],
> "position" : 121
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
> }
> ]
> }
> ]MacBook-Pro:~ jjirsa$
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/de60cf07
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/de60cf07
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/de60cf07
Branch: refs/heads/trunk
Commit: de60cf0759e9721ddd1aab725c511a8f910cf415
Parents: 2842716 99196cb
Author: Jeff Jirsa
Authored: Mon Jul 31 15:30:10 2017 -0700
Committer: Jeff Jirsa
Committed: Mon Jul 31 15:30:54 2017 -0700
--
CHANGES.txt | 1 +
.../cql3/selection/ResultSetBuilder.java| 6
.../cql3/validation/entities/TimestampTest.java | 30
.../cql3/validation/operations/SelectTest.java | 29 +++
4 files changed, 66 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de60cf07/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de60cf07/src/java/org/apache/cassandra/cql3/selection/ResultSetBuilder.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/ResultSetBuilder.java
index d179f93,000..9b7abe1
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/cql3/selection/ResultSetBuilder.java
+++ b/src/java/org/apache/cassandra/cql3/selection/ResultSetBuilder.java
@@@ -1,165 -1,0 +1,171 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.cql3.selection;
+
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import org.apache.cassandra.cql3.ResultSet;
+import org.apache.cassandra.cql3.ResultSet.ResultMetadata;
+import org.apache.cassandra.cql3.selection.Selection.Selectors;
+import org.apache.cassandra.db.Clustering;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.aggregation.GroupMaker;
+import org.apache.cassandra.db.context.CounterContext;
+import org.apache.cassandra.db.rows.Cell;
+import org.apache.cassandra.utils.ByteBufferUtil;
+
+public final class ResultSetBuilder
+{
+private final ResultSet resultSet;
+
+/**
+ * As multiple thread can access a Selection instance each
ResultSetBuilder will use
+ * its own Selectors instance.
+ */
+private final Selectors selectors;
+
+/**
+ * The GroupMaker used to build the aggregates.
+ */
+private final GroupMaker groupMaker;
+
+/*
+ * We'll build CQL3 row one by one.
+ * The currentRow is the values for the (CQL3) columns we've fetched.
+ * We also collect timestamps and ttls for the case where the writetime
and
+ * ttl functions are used. Note that we might collect timestamp and/or
ttls
+ * we don't care about, but since the array below are allocated just once,
+ * it doesn't matter performance wise.
+ */
+List current;
+final long[] timestamps;
+final int[] ttls;
+
+public ResultSetBuilder(ResultMetadata metadata, Selectors selectors)
+{
+this(metadata, selectors, null);
+}
+
+public ResultSetBuilder(ResultMetadata metadata, Selectors selectors,
GroupMaker groupMaker)
+{
+this.resultSet = new ResultSet(metadata.copy(), new
ArrayList>());
+this.selectors = selectors;
+this.groupMaker = groupMaker;
+this.timestamps = selectors.collectTimestamps() ? new
long[selectors.numberOfFetchedColumns()] : null;
+this.ttls = selectors.collectTTLs() ? new
int[selectors.numberOfFetchedColumns()] : null;
+
+// We use MIN_VALUE to indicate no timestamp and -1 for no ttl
+if (timestamps != null)
+Arrays.fill(timestamps, Long.MIN_VALUE);
+if (ttls != null)
+Arrays.fill(ttls, -1);
+}
+
+public void add(ByteBuffer v)
+{
+current.add(v);
+}
+
+public void add(Cell c
[3/6] cassandra git commit: Invalid writetime for null columns in cqlsh
Invalid writetime for null columns in cqlsh
Patch by Jeff Jirsa; Reviewed by Aleksey Yeschenko for CASSANDRA-13711
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9dc896f4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9dc896f4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9dc896f4
Branch: refs/heads/trunk
Commit: 9dc896f4ea51276de4ea76ffca3fd719e0c8b8a1
Parents: 9ac01ba
Author: Jeff Jirsa
Authored: Thu Jul 27 11:40:52 2017 -0700
Committer: Jeff Jirsa
Committed: Mon Jul 31 15:29:10 2017 -0700
--
CHANGES.txt | 1 +
.../cassandra/cql3/selection/Selection.java | 6
.../cql3/validation/entities/TimestampTest.java | 29
.../cql3/validation/operations/SelectTest.java | 27 ++
4 files changed, 63 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5e95b75..e3f5fe6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.15
+ * Fix invalid writetime for null cells (CASSANDRA-13711)
* Fix ALTER TABLE statement to atomically propagate changes to the table and
its MVs (CASSANDRA-12952)
* Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
* JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize
(Backport CASSANDRA-13329)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java
b/src/java/org/apache/cassandra/cql3/selection/Selection.java
index 0ecf063..425ae52 100644
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@ -346,6 +346,12 @@ public abstract class Selection
}
}
current = new ArrayList<>(columns.size());
+
+// Timestamps and TTLs are arrays per row, we must null them out
between row
+if (timestamps != null)
+Arrays.fill(timestamps, Long.MIN_VALUE);
+if (ttls != null)
+Arrays.fill(ttls, -1);
}
public ResultSet build(int protocolVersion) throws
InvalidRequestException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
b/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
index 3e70cd0..b41163c 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
@@ -152,4 +152,33 @@ public class TimestampTest extends CQLTester
execute("INSERT INTO %s (k, i) VALUES (1, 1) USING TIMESTAMP ?",
unset()); // treat as 'now'
}
+@Test
+public void testTimestampsOnUnsetColumns() throws Throwable
+{
+createTable("CREATE TABLE %s (k int PRIMARY KEY, i int)");
+execute("INSERT INTO %s (k, i) VALUES (1, 1) USING TIMESTAMP 1;");
+execute("INSERT INTO %s (k) VALUES (2) USING TIMESTAMP 2;");
+execute("INSERT INTO %s (k, i) VALUES (3, 3) USING TIMESTAMP 1;");
+assertRows(execute("SELECT k, i, writetime(i) FROM %s "),
+ row(1, 1, 1L),
+ row(2, null, null),
+ row(3, 3, 1L));
+}
+
+@Test
+public void testTimestampsOnUnsetColumnsWide() throws Throwable
+{
+createTable("CREATE TABLE %s (k int , c int, i int, PRIMARY KEY (k,
c))");
+execute("INSERT INTO %s (k, c, i) VALUES (1, 1, 1) USING TIMESTAMP
1;");
+execute("INSERT INTO %s (k, c) VALUES (1, 2) USING TIMESTAMP 1;");
+execute("INSERT INTO %s (k, c, i) VALUES (1, 3, 1) USING TIMESTAMP
1;");
+execute("INSERT INTO %s (k, c) VALUES (2, 2) USING TIMESTAMP 2;");
+execute("INSERT INTO %s (k, c, i) VALUES (3, 3, 3) USING TIMESTAMP
1;");
+assertRows(execute("SELECT k, c, i, writetime(i) FROM %s "),
+ row(1, 1, 1, 1L),
+ row(1, 2, null, null),
+ row(1, 3, 1, 1L),
+ row(2, 2, null, null),
+ row(3, 3, 3, 1L));
+}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
[1/6] cassandra git commit: Invalid writetime for null columns in cqlsh
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 9ac01baef -> 9dc896f4e
refs/heads/cassandra-3.11 d47d87cd5 -> 99196cb72
refs/heads/trunk 284271670 -> de60cf075
Invalid writetime for null columns in cqlsh
Patch by Jeff Jirsa; Reviewed by Aleksey Yeschenko for CASSANDRA-13711
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9dc896f4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9dc896f4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9dc896f4
Branch: refs/heads/cassandra-3.0
Commit: 9dc896f4ea51276de4ea76ffca3fd719e0c8b8a1
Parents: 9ac01ba
Author: Jeff Jirsa
Authored: Thu Jul 27 11:40:52 2017 -0700
Committer: Jeff Jirsa
Committed: Mon Jul 31 15:29:10 2017 -0700
--
CHANGES.txt | 1 +
.../cassandra/cql3/selection/Selection.java | 6
.../cql3/validation/entities/TimestampTest.java | 29
.../cql3/validation/operations/SelectTest.java | 27 ++
4 files changed, 63 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5e95b75..e3f5fe6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.15
+ * Fix invalid writetime for null cells (CASSANDRA-13711)
* Fix ALTER TABLE statement to atomically propagate changes to the table and
its MVs (CASSANDRA-12952)
* Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
* JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize
(Backport CASSANDRA-13329)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java
b/src/java/org/apache/cassandra/cql3/selection/Selection.java
index 0ecf063..425ae52 100644
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@ -346,6 +346,12 @@ public abstract class Selection
}
}
current = new ArrayList<>(columns.size());
+
+// Timestamps and TTLs are arrays per row, we must null them out
between row
+if (timestamps != null)
+Arrays.fill(timestamps, Long.MIN_VALUE);
+if (ttls != null)
+Arrays.fill(ttls, -1);
}
public ResultSet build(int protocolVersion) throws
InvalidRequestException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
b/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
index 3e70cd0..b41163c 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
@@ -152,4 +152,33 @@ public class TimestampTest extends CQLTester
execute("INSERT INTO %s (k, i) VALUES (1, 1) USING TIMESTAMP ?",
unset()); // treat as 'now'
}
+@Test
+public void testTimestampsOnUnsetColumns() throws Throwable
+{
+createTable("CREATE TABLE %s (k int PRIMARY KEY, i int)");
+execute("INSERT INTO %s (k, i) VALUES (1, 1) USING TIMESTAMP 1;");
+execute("INSERT INTO %s (k) VALUES (2) USING TIMESTAMP 2;");
+execute("INSERT INTO %s (k, i) VALUES (3, 3) USING TIMESTAMP 1;");
+assertRows(execute("SELECT k, i, writetime(i) FROM %s "),
+ row(1, 1, 1L),
+ row(2, null, null),
+ row(3, 3, 1L));
+}
+
+@Test
+public void testTimestampsOnUnsetColumnsWide() throws Throwable
+{
+createTable("CREATE TABLE %s (k int , c int, i int, PRIMARY KEY (k,
c))");
+execute("INSERT INTO %s (k, c, i) VALUES (1, 1, 1) USING TIMESTAMP
1;");
+execute("INSERT INTO %s (k, c) VALUES (1, 2) USING TIMESTAMP 1;");
+execute("INSERT INTO %s (k, c, i) VALUES (1, 3, 1) USING TIMESTAMP
1;");
+execute("INSERT INTO %s (k, c) VALUES (2, 2) USING TIMESTAMP 2;");
+execute("INSERT INTO %s (k, c, i) VALUES (3, 3, 3) USING TIMESTAMP
1;");
+assertRows(execute("SELECT k, c, i, writetime(i) FROM %s "),
+ row(1, 1, 1, 1L),
+ row(1, 2, null, null),
+ row(1, 3, 1, 1L),
+ row(2, 2, null, null),
+ row(3, 3, 3,
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99196cb7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99196cb7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99196cb7
Branch: refs/heads/trunk
Commit: 99196cb72c15aa540f4cb78d5fd8aa9faa645dd9
Parents: d47d87c 9dc896f
Author: Jeff Jirsa
Authored: Mon Jul 31 15:29:21 2017 -0700
Committer: Jeff Jirsa
Committed: Mon Jul 31 15:30:02 2017 -0700
--
CHANGES.txt | 2 +-
.../cassandra/cql3/selection/Selection.java | 6
.../cql3/validation/entities/TimestampTest.java | 29
.../cql3/validation/operations/SelectTest.java | 28 +++
4 files changed, 64 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99196cb7/CHANGES.txt
--
diff --cc CHANGES.txt
index 5be7e1f,e3f5fe6..9c595fc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,12 -1,8 +1,12 @@@
-3.0.15
+3.11.1
+ * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
+ * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
+ * Duplicate the buffer before passing it to analyser in SASI operation
(CASSANDRA-13512)
+ * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
+Merged from 3.0:
- 3.0.15
+ * Fix invalid writetime for null cells (CASSANDRA-13711)
* Fix ALTER TABLE statement to atomically propagate changes to the table and
its MVs (CASSANDRA-12952)
* Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
- * JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize
(Backport CASSANDRA-13329)
* Fix Digest mismatch Exception if hints file has UnknownColumnFamily
(CASSANDRA-13696)
* Purge tombstones created by expired cells (CASSANDRA-13643)
* Make concat work with iterators that have different subsets of columns
(CASSANDRA-13482)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99196cb7/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/Selection.java
index 4ea1071,425ae52..428634c
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@@ -404,12 -346,15 +404,18 @@@ public abstract class Selectio
}
}
current = new ArrayList<>(columns.size());
+
-// Timestamps and TTLs are arrays per row, we must null them out
between row
++// Timestamps and TTLs are arrays per row, we must null them out
between rows
+ if (timestamps != null)
+ Arrays.fill(timestamps, Long.MIN_VALUE);
+ if (ttls != null)
+ Arrays.fill(ttls, -1);
}
-public ResultSet build(int protocolVersion) throws
InvalidRequestException
+/**
+ * Builds the ResultSet
+ */
+public ResultSet build()
{
if (current != null)
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99196cb7/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/99196cb7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/99196cb7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/99196cb7
Branch: refs/heads/cassandra-3.11
Commit: 99196cb72c15aa540f4cb78d5fd8aa9faa645dd9
Parents: d47d87c 9dc896f
Author: Jeff Jirsa
Authored: Mon Jul 31 15:29:21 2017 -0700
Committer: Jeff Jirsa
Committed: Mon Jul 31 15:30:02 2017 -0700
--
CHANGES.txt | 2 +-
.../cassandra/cql3/selection/Selection.java | 6
.../cql3/validation/entities/TimestampTest.java | 29
.../cql3/validation/operations/SelectTest.java | 28 +++
4 files changed, 64 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99196cb7/CHANGES.txt
--
diff --cc CHANGES.txt
index 5be7e1f,e3f5fe6..9c595fc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,12 -1,8 +1,12 @@@
-3.0.15
+3.11.1
+ * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
+ * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
+ * Duplicate the buffer before passing it to analyser in SASI operation
(CASSANDRA-13512)
+ * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
+Merged from 3.0:
- 3.0.15
+ * Fix invalid writetime for null cells (CASSANDRA-13711)
* Fix ALTER TABLE statement to atomically propagate changes to the table and
its MVs (CASSANDRA-12952)
* Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
- * JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize
(Backport CASSANDRA-13329)
* Fix Digest mismatch Exception if hints file has UnknownColumnFamily
(CASSANDRA-13696)
* Purge tombstones created by expired cells (CASSANDRA-13643)
* Make concat work with iterators that have different subsets of columns
(CASSANDRA-13482)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99196cb7/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --cc src/java/org/apache/cassandra/cql3/selection/Selection.java
index 4ea1071,425ae52..428634c
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@@ -404,12 -346,15 +404,18 @@@ public abstract class Selectio
}
}
current = new ArrayList<>(columns.size());
+
-// Timestamps and TTLs are arrays per row, we must null them out
between row
++// Timestamps and TTLs are arrays per row, we must null them out
between rows
+ if (timestamps != null)
+ Arrays.fill(timestamps, Long.MIN_VALUE);
+ if (ttls != null)
+ Arrays.fill(ttls, -1);
}
-public ResultSet build(int protocolVersion) throws
InvalidRequestException
+/**
+ * Builds the ResultSet
+ */
+public ResultSet build()
{
if (current != null)
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/99196cb7/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[2/6] cassandra git commit: Invalid writetime for null columns in cqlsh
Invalid writetime for null columns in cqlsh
Patch by Jeff Jirsa; Reviewed by Aleksey Yeschenko for CASSANDRA-13711
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9dc896f4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9dc896f4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9dc896f4
Branch: refs/heads/cassandra-3.11
Commit: 9dc896f4ea51276de4ea76ffca3fd719e0c8b8a1
Parents: 9ac01ba
Author: Jeff Jirsa
Authored: Thu Jul 27 11:40:52 2017 -0700
Committer: Jeff Jirsa
Committed: Mon Jul 31 15:29:10 2017 -0700
--
CHANGES.txt | 1 +
.../cassandra/cql3/selection/Selection.java | 6
.../cql3/validation/entities/TimestampTest.java | 29
.../cql3/validation/operations/SelectTest.java | 27 ++
4 files changed, 63 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5e95b75..e3f5fe6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.15
+ * Fix invalid writetime for null cells (CASSANDRA-13711)
* Fix ALTER TABLE statement to atomically propagate changes to the table and
its MVs (CASSANDRA-12952)
* Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
* JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize
(Backport CASSANDRA-13329)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/src/java/org/apache/cassandra/cql3/selection/Selection.java
--
diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java
b/src/java/org/apache/cassandra/cql3/selection/Selection.java
index 0ecf063..425ae52 100644
--- a/src/java/org/apache/cassandra/cql3/selection/Selection.java
+++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java
@@ -346,6 +346,12 @@ public abstract class Selection
}
}
current = new ArrayList<>(columns.size());
+
+// Timestamps and TTLs are arrays per row, we must null them out
between row
+if (timestamps != null)
+Arrays.fill(timestamps, Long.MIN_VALUE);
+if (ttls != null)
+Arrays.fill(ttls, -1);
}
public ResultSet build(int protocolVersion) throws
InvalidRequestException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
b/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
index 3e70cd0..b41163c 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/TimestampTest.java
@@ -152,4 +152,33 @@ public class TimestampTest extends CQLTester
execute("INSERT INTO %s (k, i) VALUES (1, 1) USING TIMESTAMP ?",
unset()); // treat as 'now'
}
+@Test
+public void testTimestampsOnUnsetColumns() throws Throwable
+{
+createTable("CREATE TABLE %s (k int PRIMARY KEY, i int)");
+execute("INSERT INTO %s (k, i) VALUES (1, 1) USING TIMESTAMP 1;");
+execute("INSERT INTO %s (k) VALUES (2) USING TIMESTAMP 2;");
+execute("INSERT INTO %s (k, i) VALUES (3, 3) USING TIMESTAMP 1;");
+assertRows(execute("SELECT k, i, writetime(i) FROM %s "),
+ row(1, 1, 1L),
+ row(2, null, null),
+ row(3, 3, 1L));
+}
+
+@Test
+public void testTimestampsOnUnsetColumnsWide() throws Throwable
+{
+createTable("CREATE TABLE %s (k int , c int, i int, PRIMARY KEY (k,
c))");
+execute("INSERT INTO %s (k, c, i) VALUES (1, 1, 1) USING TIMESTAMP
1;");
+execute("INSERT INTO %s (k, c) VALUES (1, 2) USING TIMESTAMP 1;");
+execute("INSERT INTO %s (k, c, i) VALUES (1, 3, 1) USING TIMESTAMP
1;");
+execute("INSERT INTO %s (k, c) VALUES (2, 2) USING TIMESTAMP 2;");
+execute("INSERT INTO %s (k, c, i) VALUES (3, 3, 3) USING TIMESTAMP
1;");
+assertRows(execute("SELECT k, c, i, writetime(i) FROM %s "),
+ row(1, 1, 1, 1L),
+ row(1, 2, null, null),
+ row(1, 3, 1, 1L),
+ row(2, 2, null, null),
+ row(3, 3, 3, 1L));
+}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9dc896f4/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
-
[jira] [Updated] (CASSANDRA-13730) Dropping a table doesn't drop its dropped columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-13730:
-
Status: Patch Available (was: Open)
> Dropping a table doesn't drop its dropped columns
> -
>
> Key: CASSANDRA-13730
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13730
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
> Fix For: 3.0.x, 3.11.x
>
>
> I'm not sure if this is intended or not, but currently a table's dropped
> columns are not dropped when the table itself is dropped:
> {noformat}
> cqlsh> create keyspace ks WITH replication={ 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 } ;
> cqlsh> use ks;
> cqlsh:ks> create table test (pk text primary key, c1 int);
> cqlsh:ks> alter table test drop c1;
> cqlsh:ks> drop table test;
> cqlsh:ks> select * from system_schema.dropped_columns where keyspace_name =
> 'ks' and table_name = 'test';
> keyspace_name | table_name | column_name | dropped_time|
> kind| type
> ---++-+-+-+--
> ks | test | c1 | 2017-07-25 17:53:47.651000+ |
> regular | int
> (1 rows)
> {noformat}
> This can have surprising consequences when creating another table with the
> same name.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13730) Dropping a table doesn't drop its dropped columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-13730:
-
Priority: Minor (was: Major)
Component/s: Distributed Metadata
> Dropping a table doesn't drop its dropped columns
> -
>
> Key: CASSANDRA-13730
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13730
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
>
> I'm not sure if this is intended or not, but currently a table's dropped
> columns are not dropped when the table itself is dropped:
> {noformat}
> cqlsh> create keyspace ks WITH replication={ 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 } ;
> cqlsh> use ks;
> cqlsh:ks> create table test (pk text primary key, c1 int);
> cqlsh:ks> alter table test drop c1;
> cqlsh:ks> drop table test;
> cqlsh:ks> select * from system_schema.dropped_columns where keyspace_name =
> 'ks' and table_name = 'test';
> keyspace_name | table_name | column_name | dropped_time|
> kind| type
> ---++-+-+-+--
> ks | test | c1 | 2017-07-25 17:53:47.651000+ |
> regular | int
> (1 rows)
> {noformat}
> This can have surprising consequences when creating another table with the
> same name.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13730) Dropping a table doesn't drop its dropped columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106746#comment-16106746
]
ZhaoYang edited comment on CASSANDRA-13730 at 7/31/17 2:34 PM:
---
| [trunk|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13730-trunk]
| [unit|https://circleci.com/gh/jasonstack/cassandra/295] | irrelevant
compaction_test.TestCompaction_with_LeveledCompactionStrategy.fanout_size_test
materialized_views_test.TestMaterializedViews.view_tombstone_test (known)
bootstrap_test.TestBootstrap.consistent_range_movement_false_with_rf1_should_succeed_test(known)
|
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13730-3.11] |
[unit|https://circleci.com/gh/jasonstack/cassandra/296] | passed dtest |
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13730-3.0] |
[unit|https://circleci.com/gh/jasonstack/cassandra/287] | irrelevant
offline_tools_test.TestOfflineTools.sstableofflinerelevel_test
auth_test.TestAuth.system_auth_ks_is_alterable_test |
| [dtest|https://github.com/riptano/cassandra-dtest/commits/CASSANDRA-13730] |
clear corresponding dropped_columns entries when table is dropped.
was (Author: jasonstack):
| [trunk|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13730-trunk]
| [unit|https://circleci.com/gh/jasonstack/cassandra/295] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13730-3.11] |
[unit|https://circleci.com/gh/jasonstack/cassandra/296] | passed dtest |
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13730-3.0] |
[unit|https://circleci.com/gh/jasonstack/cassandra/287] | |
| [dtest|https://github.com/riptano/cassandra-dtest/commits/CASSANDRA-13730] |
clear corresponding dropped_columns entries when table is dropped.
> Dropping a table doesn't drop its dropped columns
> -
>
> Key: CASSANDRA-13730
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13730
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
> Fix For: 3.0.x, 3.11.x
>
>
> I'm not sure if this is intended or not, but currently a table's dropped
> columns are not dropped when the table itself is dropped:
> {noformat}
> cqlsh> create keyspace ks WITH replication={ 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 } ;
> cqlsh> use ks;
> cqlsh:ks> create table test (pk text primary key, c1 int);
> cqlsh:ks> alter table test drop c1;
> cqlsh:ks> drop table test;
> cqlsh:ks> select * from system_schema.dropped_columns where keyspace_name =
> 'ks' and table_name = 'test';
> keyspace_name | table_name | column_name | dropped_time|
> kind| type
> ---++-+-+-+--
> ks | test | c1 | 2017-07-25 17:53:47.651000+ |
> regular | int
> (1 rows)
> {noformat}
> This can have surprising consequences when creating another table with the
> same name.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13371) Remove legacy auth tables support
[
https://issues.apache.org/jira/browse/CASSANDRA-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107356#comment-16107356
]
Robert Stupp commented on CASSANDRA-13371:
--
One thing in the 3.0 + 3.11 branches: Better don't reference the classes
{{PasswordAuthenticator}}, {{CassandraRoleManager}}, {{CassandraAuthorizer}}
but use the table names as strings for the {{LEGACY_AUTH_TABLES}} constant.
Referencing the classes may unintentionally initialize stuff.
Beside that: +1 - feel free to fix that on commit.
> Remove legacy auth tables support
> -
>
> Key: CASSANDRA-13371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13371
> Project: Cassandra
> Issue Type: Improvement
> Components: Auth
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Labels: security
> Fix For: 4.x
>
>
> Starting with Cassandra 3.0, we include support for converting pre
> CASSANDRA-7653 user tables, until they will be dropped by the operator.
> Converting e.g. permissions happens by simply copying all of them from
> {{permissions}} -> {{role_permissions}}, until the {{permissions}} table has
> been dropped.
> Upgrading to 4.0 will only be possible from 3.0 upwards, so I think it's safe
> to assume that the new permissions table has already been populated, whether
> the old table was dropped or not. Therefor I'd suggest to just get rid of the
> legacy support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13371) Remove legacy auth tables support
[
https://issues.apache.org/jira/browse/CASSANDRA-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107335#comment-16107335
]
Stefan Podkowinski commented on CASSANDRA-13371:
Had to change the API for the startup check a bit and had to update the
corresponding test for backporting...
||trunk||3.11||3.0||
|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13371-trunk]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13371-3.11]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13371-3.0]|
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/158]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/157]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/156]|
|[testall|https://circleci.com/gh/spodkowinski/cassandra/tree/CASSANDRA-13371-trunk]|[testall|https://circleci.com/gh/spodkowinski/cassandra/tree/CASSANDRA-13371-3.11]|[testall|https://circleci.com/gh/spodkowinski/cassandra/tree/CASSANDRA-13371-3.0]|
> Remove legacy auth tables support
> -
>
> Key: CASSANDRA-13371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13371
> Project: Cassandra
> Issue Type: Improvement
> Components: Auth
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Labels: security
> Fix For: 4.x
>
>
> Starting with Cassandra 3.0, we include support for converting pre
> CASSANDRA-7653 user tables, until they will be dropped by the operator.
> Converting e.g. permissions happens by simply copying all of them from
> {{permissions}} -> {{role_permissions}}, until the {{permissions}} table has
> been dropped.
> Upgrading to 4.0 will only be possible from 3.0 upwards, so I think it's safe
> to assume that the new permissions table has already been populated, whether
> the old table was dropped or not. Therefor I'd suggest to just get rid of the
> legacy support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13371) Remove legacy auth tables support
[
https://issues.apache.org/jira/browse/CASSANDRA-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107269#comment-16107269
]
Robert Stupp commented on CASSANDRA-13371:
--
Yea - just log a warning in 3.0 & 3.x.
> Remove legacy auth tables support
> -
>
> Key: CASSANDRA-13371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13371
> Project: Cassandra
> Issue Type: Improvement
> Components: Auth
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Labels: security
> Fix For: 4.x
>
>
> Starting with Cassandra 3.0, we include support for converting pre
> CASSANDRA-7653 user tables, until they will be dropped by the operator.
> Converting e.g. permissions happens by simply copying all of them from
> {{permissions}} -> {{role_permissions}}, until the {{permissions}} table has
> been dropped.
> Upgrading to 4.0 will only be possible from 3.0 upwards, so I think it's safe
> to assume that the new permissions table has already been populated, whether
> the old table was dropped or not. Therefor I'd suggest to just get rid of the
> legacy support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13371) Remove legacy auth tables support
[
https://issues.apache.org/jira/browse/CASSANDRA-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107264#comment-16107264
]
Stefan Podkowinski commented on CASSANDRA-13371:
How about using the startup check for that and make it log to warn in 3.0/3.11
and start throwing a StartupException in 4.0?
> Remove legacy auth tables support
> -
>
> Key: CASSANDRA-13371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13371
> Project: Cassandra
> Issue Type: Improvement
> Components: Auth
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Labels: security
> Fix For: 4.x
>
>
> Starting with Cassandra 3.0, we include support for converting pre
> CASSANDRA-7653 user tables, until they will be dropped by the operator.
> Converting e.g. permissions happens by simply copying all of them from
> {{permissions}} -> {{role_permissions}}, until the {{permissions}} table has
> been dropped.
> Upgrading to 4.0 will only be possible from 3.0 upwards, so I think it's safe
> to assume that the new permissions table has already been populated, whether
> the old table was dropped or not. Therefor I'd suggest to just get rid of the
> legacy support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-7868) Sporadic CL switch from LOCAL_QUORUM to ALL
[
https://issues.apache.org/jira/browse/CASSANDRA-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107084#comment-16107084
]
Christian Spriegel edited comment on CASSANDRA-7868 at 7/31/17 12:31 PM:
-
[~brandon.williams]: Sorry to warm up this old ticket, but we are having the
same issue in C* 3.0.13.
Are sure this is so harmless? The ReadTimeoutException is being thrown on the
client side. I would expect that a failing Read-Repair does not throw an
exception on the client. Is my expectation incorrect?
Edit: It seems that
StorageProxy.SinglePartitionReadLifecycle.awaitResultsAndRetryOnDigestMismatch()
is the culprit. It indeed does a blocking repair. I assume it is necessary
that it is blocking, but does it really have to be a CL.ALL ?
was (Author: christianmovi):
[~brandon.williams]: Sorry to warm up this old ticket, but we are having the
same issue in C* 3.0.13.
Are sure this is so harmless? The ReadTimeoutException is being thrown on the
client side. I would expect that a failing Read-Repair does not throw an
exception on the client. Is my expectation incorrect?
> Sporadic CL switch from LOCAL_QUORUM to ALL
> ---
>
> Key: CASSANDRA-7868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7868
> Project: Cassandra
> Issue Type: Bug
> Environment: Client: cassandra-java-driver 2.0.4
> Server: 2.0.9
>Reporter: Dmitry Schitinin
>
> Hi!
> We have keyspace described as
> {code}
> CREATE KEYSPACE subscriptions WITH replication = {
> 'class': 'NetworkTopologyStrategy',
> 'FOL': '3',
> 'SAS': '3',
> 'AMS': '0',
> 'IVA': '3',
> 'UGR': '0'
> } AND durable_writes = 'false';
> {code}
> There is simple table
> {code}
> CREATE TABLE processed_documents (
> id text,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {code}
> in the keyspace.
> On client we execute next prepared statement:
> {code}
> session.prepare(
> "SELECT id FROM processed_documents WHERE id IN :ids
> ).setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM)
> {code}
> Used Cassandra session has next main properties:
> * Load balancing policy - DCAwareRoundRobinPolicy(localDc,
> usedHostPerRemoteDc = 3, allowRemoteDcForLocalConsistencyLevel = true)
> * Retry policy - DefaultRetryPolicy
> * Query options - QueryOptions with set consistency level to
> ConsistencyLevel.LOCAL_QUORUM
> Our problem is next.
> Since some moment there are next errors in the client application log:
> {code}
> com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout
> during read query at consistency ALL (9 responses were required but only 8
> replica responded)
> at
> com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.Responses$Error.asException(Responses.java:94)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:108)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:235)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.RequestHandler.onSet(RequestHandler.java:379)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.Connection$Dispatcher.messageReceived(Connection.java:571)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPip
[jira] [Updated] (CASSANDRA-12245) initial view build can be parallel
[ https://issues.apache.org/jira/browse/CASSANDRA-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-12245: Reviewer: Paulo Motta > initial view build can be parallel > -- > > Key: CASSANDRA-12245 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12245 > Project: Cassandra > Issue Type: Improvement > Components: Materialized Views >Reporter: Tom van der Woerdt >Assignee: Andrés de la Peña > Fix For: 4.x > > > On a node with lots of data (~3TB) building a materialized view takes several > weeks, which is not ideal. It's doing this in a single thread. > There are several potential ways this can be optimized : > * do vnodes in parallel, instead of going through the entire range in one > thread > * just iterate through sstables, not worrying about duplicates, and include > the timestamp of the original write in the MV mutation. since this doesn't > exclude duplicates it does increase the amount of work and could temporarily > surface ghost rows (yikes) but I guess that's why they call it eventual > consistency. doing it this way can avoid holding references to all tables on > disk, allows parallelization, and removes the need to check other sstables > for existing data. this is essentially the 'do a full repair' path -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13149) AssertionError prepending to a list
[
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrés de la Peña reassigned CASSANDRA-13149:
-
Assignee: Andrés de la Peña
> AssertionError prepending to a list
> ---
>
> Key: CASSANDRA-13149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: 3.0.8
>Reporter: Steven Warren
>Assignee: Andrés de la Peña
>
> Prepending to a list produces the following AssertionError randomly. Changing
> the update to append (and sort in the client) works around the issue.
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
> ~[apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
> [apache-cassandra-3.0.8.jar:3.0.8]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
> [apache-cassandra-3.0.8.jar:3.0.8]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [apache-cassandra-3.0.8.jar:3.0.8]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.0.8.jar:3.0.8]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12245) initial view build can be parallel
[
https://issues.apache.org/jira/browse/CASSANDRA-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107194#comment-16107194
]
Andrés de la Peña commented on CASSANDRA-12245:
---
Good point, [~doanduyhai].
I have
[updated|https://github.com/adelapena/cassandra/commit/35588ca00337465fedfb42e6cc001773bb739d2f]
the patch to use a new separate table, {{system.views_builds_in_progress_v2}}.
The downside is that pending view builds will be restarted during an upgrade to
4.x, which seems reasonable to me.
I have also [updated the
dtests|https://github.com/adelapena/cassandra-dtest/commit/91f6d3d88de97ec7e18b5243f60c359022bf41c3]
to choose the proper system table depending on the version.
> initial view build can be parallel
> --
>
> Key: CASSANDRA-12245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12245
> Project: Cassandra
> Issue Type: Improvement
> Components: Materialized Views
>Reporter: Tom van der Woerdt
>Assignee: Andrés de la Peña
> Fix For: 4.x
>
>
> On a node with lots of data (~3TB) building a materialized view takes several
> weeks, which is not ideal. It's doing this in a single thread.
> There are several potential ways this can be optimized :
> * do vnodes in parallel, instead of going through the entire range in one
> thread
> * just iterate through sstables, not worrying about duplicates, and include
> the timestamp of the original write in the MV mutation. since this doesn't
> exclude duplicates it does increase the amount of work and could temporarily
> surface ghost rows (yikes) but I guess that's why they call it eventual
> consistency. doing it this way can avoid holding references to all tables on
> disk, allows parallelization, and removes the need to check other sstables
> for existing data. this is essentially the 'do a full repair' path
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13730) Dropping a table doesn't drop its dropped columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13730:
--
Reviewer: Aleksey Yeschenko
> Dropping a table doesn't drop its dropped columns
> -
>
> Key: CASSANDRA-13730
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13730
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
> Fix For: 3.0.x, 3.11.x
>
>
> I'm not sure if this is intended or not, but currently a table's dropped
> columns are not dropped when the table itself is dropped:
> {noformat}
> cqlsh> create keyspace ks WITH replication={ 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 } ;
> cqlsh> use ks;
> cqlsh:ks> create table test (pk text primary key, c1 int);
> cqlsh:ks> alter table test drop c1;
> cqlsh:ks> drop table test;
> cqlsh:ks> select * from system_schema.dropped_columns where keyspace_name =
> 'ks' and table_name = 'test';
> keyspace_name | table_name | column_name | dropped_time|
> kind| type
> ---++-+-+-+--
> ks | test | c1 | 2017-07-25 17:53:47.651000+ |
> regular | int
> (1 rows)
> {noformat}
> This can have surprising consequences when creating another table with the
> same name.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7868) Sporadic CL switch from LOCAL_QUORUM to ALL
[
https://issues.apache.org/jira/browse/CASSANDRA-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107084#comment-16107084
]
Christian Spriegel commented on CASSANDRA-7868:
---
[~brandon.williams]: Sorry to warm up this old ticket, but we are having the
same issue in C* 3.0.13.
Are sure this is so harmless? The ReadTimeoutException is being thrown on the
client side. I would expect that a failing Read-Repair does not throw an
exception on the client. Is my expectation incorrect?
> Sporadic CL switch from LOCAL_QUORUM to ALL
> ---
>
> Key: CASSANDRA-7868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7868
> Project: Cassandra
> Issue Type: Bug
> Environment: Client: cassandra-java-driver 2.0.4
> Server: 2.0.9
>Reporter: Dmitry Schitinin
>
> Hi!
> We have keyspace described as
> {code}
> CREATE KEYSPACE subscriptions WITH replication = {
> 'class': 'NetworkTopologyStrategy',
> 'FOL': '3',
> 'SAS': '3',
> 'AMS': '0',
> 'IVA': '3',
> 'UGR': '0'
> } AND durable_writes = 'false';
> {code}
> There is simple table
> {code}
> CREATE TABLE processed_documents (
> id text,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {code}
> in the keyspace.
> On client we execute next prepared statement:
> {code}
> session.prepare(
> "SELECT id FROM processed_documents WHERE id IN :ids
> ).setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM)
> {code}
> Used Cassandra session has next main properties:
> * Load balancing policy - DCAwareRoundRobinPolicy(localDc,
> usedHostPerRemoteDc = 3, allowRemoteDcForLocalConsistencyLevel = true)
> * Retry policy - DefaultRetryPolicy
> * Query options - QueryOptions with set consistency level to
> ConsistencyLevel.LOCAL_QUORUM
> Our problem is next.
> Since some moment there are next errors in the client application log:
> {code}
> com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout
> during read query at consistency ALL (9 responses were required but only 8
> replica responded)
> at
> com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.Responses$Error.asException(Responses.java:94)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:108)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:235)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.RequestHandler.onSet(RequestHandler.java:379)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> com.datastax.driver.core.Connection$Dispatcher.messageReceived(Connection.java:571)
> ~[cassandra-driver-core-2.0.2.jar:na]
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> ~[netty-3.9.0.Final.jar:na]
> at
> org.
[jira] [Resolved] (CASSANDRA-13735) Cassandra V1 to V2 migration broken when attachments
[ https://issues.apache.org/jira/browse/CASSANDRA-13735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tellier Benoit resolved CASSANDRA-13735. Resolution: Fixed Wrong JIRA, sorry. > Cassandra V1 to V2 migration broken when attachments > > > Key: CASSANDRA-13735 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13735 > Project: Cassandra > Issue Type: Improvement >Reporter: Tellier Benoit > > The return value of V1ToV2Migration migration tests was not checked. > Tha attachment stream was processed two times, leading to an error while > migrating, harming performance and returning errors. > We should write unit tests on the returned value also. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13735) Cassandra V1 to V2 migration broken when attachments
Tellier Benoit created CASSANDRA-13735: -- Summary: Cassandra V1 to V2 migration broken when attachments Key: CASSANDRA-13735 URL: https://issues.apache.org/jira/browse/CASSANDRA-13735 Project: Cassandra Issue Type: Improvement Reporter: Tellier Benoit The return value of V1ToV2Migration migration tests was not checked. Tha attachment stream was processed two times, leading to an error while migrating, harming performance and returning errors. We should write unit tests on the returned value also. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13387) Metrics for repair
[
https://issues.apache.org/jira/browse/CASSANDRA-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Podkowinski updated CASSANDRA-13387:
---
Resolution: Fixed
Reviewer: Stefan Podkowinski
Fix Version/s: 4.0
Status: Resolved (was: Ready to Commit)
Merged as 284271670c389f2b9fd11
Thanks Simon!
> Metrics for repair
> --
>
> Key: CASSANDRA-13387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13387
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
> Fix For: 4.0
>
>
> We're missing metrics for repair, especially for errors. From what I observed
> now, the exception will be caught by UncaughtExceptionHandler set in
> CassandraDaemon and is categorized as StorageMetrics.exceptions. This is one
> example:
> {code}
> ERROR [AntiEntropyStage:1] 2017-03-27 18:17:08,385 CassandraDaemon.java:207 -
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id =
> 8c85d260-1319-11e7-82a2-25090a89015f has failed.
> at
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:377)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:392)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:172)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Introduce error metrics for repair
Repository: cassandra
Updated Branches:
refs/heads/trunk 9b8282752 -> 284271670
Introduce error metrics for repair
patch by Simon Zhou; reviewed by Stefan Podkowinski for CASSANDRA-13387
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28427167
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28427167
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28427167
Branch: refs/heads/trunk
Commit: 284271670c389f2b9fd11271c33eca5bb2a228fd
Parents: 9b82827
Author: Simon Zhou
Authored: Thu Mar 30 20:45:03 2017 -0700
Committer: Stefan Podkowinski
Committed: Mon Jul 31 09:54:05 2017 +0200
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/metrics/StorageMetrics.java | 3 ++-
src/java/org/apache/cassandra/repair/RepairRunnable.java | 10 +-
.../org/apache/cassandra/service/CassandraDaemon.java | 2 +-
src/java/org/apache/cassandra/tools/NodeProbe.java| 2 +-
5 files changed, 14 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/28427167/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d747d50..c0abed7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Introduce error metrics for repair (CASSANDRA-13387)
* Refactoring to primitive functional interfaces in AuthCache
(CASSANDRA-13732)
* Update metrics to 3.1.5 (CASSANDRA-13648)
* batch_size_warn_threshold_in_kb can now be set at runtime (CASSANDRA-13699)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/28427167/src/java/org/apache/cassandra/metrics/StorageMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/StorageMetrics.java
b/src/java/org/apache/cassandra/metrics/StorageMetrics.java
index 12196f7..9399ba6 100644
--- a/src/java/org/apache/cassandra/metrics/StorageMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/StorageMetrics.java
@@ -29,7 +29,8 @@ public class StorageMetrics
private static final MetricNameFactory factory = new
DefaultNameFactory("Storage");
public static final Counter load =
Metrics.counter(factory.createMetricName("Load"));
-public static final Counter exceptions =
Metrics.counter(factory.createMetricName("Exceptions"));
+public static final Counter uncaughtExceptions =
Metrics.counter(factory.createMetricName("Exceptions"));
public static final Counter totalHintsInProgress =
Metrics.counter(factory.createMetricName("TotalHintsInProgress"));
public static final Counter totalHints =
Metrics.counter(factory.createMetricName("TotalHints"));
+public static final Counter repairExceptions =
Metrics.counter(factory.createMetricName("RepairExceptions"));
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/28427167/src/java/org/apache/cassandra/repair/RepairRunnable.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairRunnable.java
b/src/java/org/apache/cassandra/repair/RepairRunnable.java
index 3f761ee..b581ebd 100644
--- a/src/java/org/apache/cassandra/repair/RepairRunnable.java
+++ b/src/java/org/apache/cassandra/repair/RepairRunnable.java
@@ -17,6 +17,7 @@
*/
package org.apache.cassandra.repair;
+import java.io.IOException;
import java.net.InetAddress;
import java.nio.ByteBuffer;
import java.util.*;
@@ -50,6 +51,7 @@ import org.apache.cassandra.db.ConsistencyLevel;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.repair.consistent.CoordinatorSession;
+import org.apache.cassandra.metrics.StorageMetrics;
import org.apache.cassandra.repair.messages.RepairOption;
import org.apache.cassandra.service.ActiveRepairService;
import org.apache.cassandra.service.QueryState;
@@ -121,6 +123,7 @@ public class RepairRunnable extends WrappedRunnable
implements ProgressEventNoti
protected void fireErrorAndComplete(int progressCount, int totalProgress,
String message)
{
+StorageMetrics.repairExceptions.inc();
fireProgressEvent(new ProgressEvent(ProgressEventType.ERROR,
progressCount, totalProgress, message));
String completionMessage = String.format("Repair command #%d finished
with error", cmd);
fireProgressEvent(new ProgressEvent(ProgressEventType.COMPLETE,
progressCount, totalProgress, completionMessage));
@@ -144,7 +147,7 @@ public class RepairRunnable extends WrappedRunnable
implements ProgressEventNoti
validColumnFamilies = storageService.getValidColumnFamilies(false,
false, keyspace, columnFamilies);
progress.incrementAndGet();
}
-
