[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365643#comment-15365643 ] Stefania commented on CASSANDRA-11521: -- Adding links to patch and CI results, not yet ready for review but in case someone wants to take an early look: |[patch|https://github.com/stef1927/cassandra/commits/11521]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11521-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11521-dtest/]| > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[
https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365639#comment-15365639
]
Stefania commented on CASSANDRA-11521:
--
Right. In this case it gets emulated over multiple pages, see the distributed
case
[here|https://github.com/stef1927/cassandra/blob/11521/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java#L450]
vs. the local case
[here|https://github.com/stef1927/cassandra/blob/11521/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java#L546].
It's actually {{Selection.RowBuilder}}, implemented by
[{{AsyncPagingService.PageBuilder}}|https://github.com/stef1927/cassandra/blob/11521/src/java/org/apache/cassandra/cql3/async/paging/AsyncPagingService.java#L132]
that monitors pages and sends them to the client when they are available.
> Implement streaming for bulk read requests
> --
>
> Key: CASSANDRA-11521
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11521
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Stefania
> Labels: client-impacting
> Fix For: 3.x
>
>
> Allow clients to stream data from a C* host, bypassing the coordination layer
> and eliminating the need to query individual pages one by one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365637#comment-15365637 ] Stefania commented on CASSANDRA-11521: -- Yes it will be delivered together with this patch. > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365627#comment-15365627 ] Jonathan Ellis commented on CASSANDRA-11521: Excellent writeup, thanks! > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365625#comment-15365625 ] Jonathan Ellis commented on CASSANDRA-11521: bq. The first thought was to have two new commands, a STREAM request and a STREAM response but I chose against this so that moving forward we could phase out the existing request-response mechanism, clients could simply use streaming for everything and we could set the maximum number of pages to one by default But we still need the storageproxy path for CL > ONE, right? So how do we get to a "streaming for everything" world? > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365623#comment-15365623 ] Jonathan Ellis commented on CASSANDRA-11521: bq. Optimize local range reads at CL.ONE by keeping the iterators open across pages and avoiding the storage proxy layer Does this subsume CASSANDRA-11520 then? > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12051) JSON does not take functions
[
https://issues.apache.org/jira/browse/CASSANDRA-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365564#comment-15365564
]
Tianshi Wang commented on CASSANDRA-12051:
--
It actually supports function now in JSON format.
For anyone who want to insert a current timestamp by JSON format, here is the
query:
{code}
The ts column should be timestamp.
insert into test JSON '{"id":2,"ts":"now"}';
{code}
> JSON does not take functions
>
>
> Key: CASSANDRA-12051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12051
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tianshi Wang
>
> toTimestamp(now()) does not work in JSON format.
> {code}
> cqlsh:ops> create table test (
>... id int,
>... ts timestamp,
>... primary key(id)
>... );
> cqlsh:ops> insert into test (id, ts) values (1, toTimestamp(now()));
> cqlsh:ops> select * from test;
> id | ts
> +-
> 1 | 2016-06-21 18:46:28.753000+
> (1 rows)
> cqlsh:ops> insert into test JSON '{"id":2,"ts":toTimestamp(now())}';
> InvalidRequest: code=2200 [Invalid query] message="Could not decode JSON
> string as a map: org.codehaus.jackson.JsonParseException: Unrecognized token
> 'toTimestamp': was expecting
> at [Source: java.io.StringReader@2da0329d; line: 1, column: 25]. (String
> was: {"id":2,"ts":toTimestamp(now())})"
> cqlsh:ops> insert into test JSON '{"id":2,"ts":"toTimestamp(now())"}';
> InvalidRequest: code=2200 [Invalid query] message="Error decoding JSON value
> for ts: Unable to coerce 'toTimestamp(now())' to a formatted date (long)"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12051) JSON does not take functions
[
https://issues.apache.org/jira/browse/CASSANDRA-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365564#comment-15365564
]
Tianshi Wang edited comment on CASSANDRA-12051 at 7/7/16 3:54 AM:
--
It actually supports function "now" in JSON format. Just remove the parenthesis.
For anyone who want to insert a current timestamp by JSON format, here is the
query:
{code}
The ts column should be timestamp.
insert into test JSON '{"id":2,"ts":"now"}';
{code}
was (Author: postgres):
It actually supports function now in JSON format.
For anyone who want to insert a current timestamp by JSON format, here is the
query:
{code}
The ts column should be timestamp.
insert into test JSON '{"id":2,"ts":"now"}';
{code}
> JSON does not take functions
>
>
> Key: CASSANDRA-12051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12051
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tianshi Wang
>
> toTimestamp(now()) does not work in JSON format.
> {code}
> cqlsh:ops> create table test (
>... id int,
>... ts timestamp,
>... primary key(id)
>... );
> cqlsh:ops> insert into test (id, ts) values (1, toTimestamp(now()));
> cqlsh:ops> select * from test;
> id | ts
> +-
> 1 | 2016-06-21 18:46:28.753000+
> (1 rows)
> cqlsh:ops> insert into test JSON '{"id":2,"ts":toTimestamp(now())}';
> InvalidRequest: code=2200 [Invalid query] message="Could not decode JSON
> string as a map: org.codehaus.jackson.JsonParseException: Unrecognized token
> 'toTimestamp': was expecting
> at [Source: java.io.StringReader@2da0329d; line: 1, column: 25]. (String
> was: {"id":2,"ts":toTimestamp(now())})"
> cqlsh:ops> insert into test JSON '{"id":2,"ts":"toTimestamp(now())"}';
> InvalidRequest: code=2200 [Invalid query] message="Error decoding JSON value
> for ts: Unable to coerce 'toTimestamp(now())' to a formatted date (long)"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365479#comment-15365479 ] Stefania commented on CASSANDRA-11521: -- We bypass StorageProxy for local range queries that use no index, at CL 1. The decision is taken by SelectStatement. I've drafted a quick design doc [here|https://docs.google.com/document/d/1YqKGSU1P8EJIfMrO--29VaSoCy5mUu-ePfAiIOLsY7o/edit?usp=sharing]. > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12147) Static thrift tables with non UTF8Type comparators can have column names converted incorrectly
[
https://issues.apache.org/jira/browse/CASSANDRA-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365474#comment-15365474
]
Jeremiah Jordan commented on CASSANDRA-12147:
-
Tested this out, this patch fixes issues I was seeing with DCT based thrift
tables after an upgrade from Cassandra 2.1 to 3.0.
> Static thrift tables with non UTF8Type comparators can have column names
> converted incorrectly
> --
>
> Key: CASSANDRA-12147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12147
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.8, 3.0.x
>
>
> {{CompactTables::columnDefinitionComparator()}} has been broken since
> CASSANDRA-8099 for non-super columnfamilies, if the comparator is not
> {{UTF8Type}}. This results in being unable to read some pre-existing 2.x data
> post upgrade (it's not lost, but becomes inaccessible).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator
[ https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-9318: Tester: Eduard Tudenhoefner > Bound the number of in-flight requests at the coordinator > - > > Key: CASSANDRA-9318 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9318 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths, Streaming and Messaging >Reporter: Ariel Weisberg >Assignee: Sergio Bossa > Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png, > limit.btm, no_backpressure.png > > > It's possible to somewhat bound the amount of load accepted into the cluster > by bounding the number of in-flight requests and request bytes. > An implementation might do something like track the number of outstanding > bytes and requests and if it reaches a high watermark disable read on client > connections until it goes back below some low watermark. > Need to make sure that disabling read on the client connection won't > introduce other issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365458#comment-15365458 ] sankalp kohli edited comment on CASSANDRA-4650 at 7/7/16 1:32 AM: -- It requires https://mvnrepository.com/artifact/org.psjava/psjava/0.1.19 Also this applies cleanly to 3.0 as well. was (Author: kohlisankalp): It requires https://mvnrepository.com/artifact/org.psjava/psjava/0.1.19 > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12143) NPE when trying to remove purgable tombstones from result
[
https://issues.apache.org/jira/browse/CASSANDRA-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-12143:
Attachment: (was: 12143-2.2.txt)
> NPE when trying to remove purgable tombstones from result
> -
>
> Key: CASSANDRA-12143
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12143
> Project: Cassandra
> Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Fix For: 2.2.7
>
> Attachments: 12143-2.2.txt
>
>
> A cluster running 2.2.6 started throwing NPEs.
> (500K exceptions on a node was seen.)
> {noformat}WARN … AbstractLocalAwareExecutorService.java:169 - Uncaught
> exception on thread Thread[SharedPool-Worker-5,5,main]: {}
> java.lang.NullPointerException: null{noformat}
> Bisecting this highlighted commit d3db33c008542c7044f3ed8c19f3a45679fcf52e as
> the culprit, which was a fix for CASSANDRA-11427.
> This commit added a line to "remove purgable tombstones from result" but
> failed to null check the {{data}} variable first. This variable comes from
> {{Row.cf}} which is permitted to be null where the CFS has no data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12143) NPE when trying to remove purgable tombstones from result
[
https://issues.apache.org/jira/browse/CASSANDRA-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-12143:
Attachment: 12143-2.2.txt
> NPE when trying to remove purgable tombstones from result
> -
>
> Key: CASSANDRA-12143
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12143
> Project: Cassandra
> Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Fix For: 2.2.7
>
> Attachments: 12143-2.2.txt
>
>
> A cluster running 2.2.6 started throwing NPEs.
> (500K exceptions on a node was seen.)
> {noformat}WARN … AbstractLocalAwareExecutorService.java:169 - Uncaught
> exception on thread Thread[SharedPool-Worker-5,5,main]: {}
> java.lang.NullPointerException: null{noformat}
> Bisecting this highlighted commit d3db33c008542c7044f3ed8c19f3a45679fcf52e as
> the culprit, which was a fix for CASSANDRA-11427.
> This commit added a line to "remove purgable tombstones from result" but
> failed to null check the {{data}} variable first. This variable comes from
> {{Row.cf}} which is permitted to be null where the CFS has no data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12143) NPE when trying to remove purgable tombstones from result
[
https://issues.apache.org/jira/browse/CASSANDRA-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365459#comment-15365459
]
mck commented on CASSANDRA-12143:
-
Fair call. The patch is updated with a unit test.
> NPE when trying to remove purgable tombstones from result
> -
>
> Key: CASSANDRA-12143
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12143
> Project: Cassandra
> Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Fix For: 2.2.7
>
> Attachments: 12143-2.2.txt
>
>
> A cluster running 2.2.6 started throwing NPEs.
> (500K exceptions on a node was seen.)
> {noformat}WARN … AbstractLocalAwareExecutorService.java:169 - Uncaught
> exception on thread Thread[SharedPool-Worker-5,5,main]: {}
> java.lang.NullPointerException: null{noformat}
> Bisecting this highlighted commit d3db33c008542c7044f3ed8c19f3a45679fcf52e as
> the culprit, which was a fix for CASSANDRA-11427.
> This commit added a line to "remove purgable tombstones from result" but
> failed to null check the {{data}} variable first. This variable comes from
> {{Row.cf}} which is permitted to be null where the CFS has no data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365458#comment-15365458 ] sankalp kohli commented on CASSANDRA-4650: -- It requires https://mvnrepository.com/artifact/org.psjava/psjava/0.1.19 > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-4650: - Attachment: CASSANDRA-4650_trunk.txt > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-4650: - Status: Patch Available (was: Reopened) > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli reopened CASSANDRA-4650: -- > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Attachments: photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12132) two cassandra nodes have common records,we need to calculate it
[ https://issues.apache.org/jira/browse/CASSANDRA-12132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365435#comment-15365435 ] stone commented on CASSANDRA-12132: --- yes,this is what i want > two cassandra nodes have common records,we need to calculate it > --- > > Key: CASSANDRA-12132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12132 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: stone > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12040) If a level compaction fails due to no space it should schedule the next one
[ https://issues.apache.org/jira/browse/CASSANDRA-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-12040: -- Status: Patch Available (was: Open) > If a level compaction fails due to no space it should schedule the next one > - > > Key: CASSANDRA-12040 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12040 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Attachments: CASSANDRA-12040_3.0.diff, CASSANDRA-12040_trunk.txt > > > If a level compaction fails the space check, it aborts but next time the > compactions are scheduled it will attempt the same one. It should skip it and > go to the next so it can find smaller compactions to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12040) If a level compaction fails due to no space it should schedule the next one
[ https://issues.apache.org/jira/browse/CASSANDRA-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-12040: -- Attachment: CASSANDRA-12040_trunk.txt CASSANDRA-12040_3.0.diff > If a level compaction fails due to no space it should schedule the next one > - > > Key: CASSANDRA-12040 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12040 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Attachments: CASSANDRA-12040_3.0.diff, CASSANDRA-12040_trunk.txt > > > If a level compaction fails the space check, it aborts but next time the > compactions are scheduled it will attempt the same one. It should skip it and > go to the next so it can find smaller compactions to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12040) If a level compaction fails due to no space it should schedule the next one
[ https://issues.apache.org/jira/browse/CASSANDRA-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-12040: -- Attachment: (was: CASSANDRA-12040_trunk.txt) > If a level compaction fails due to no space it should schedule the next one > - > > Key: CASSANDRA-12040 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12040 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > > If a level compaction fails the space check, it aborts but next time the > compactions are scheduled it will attempt the same one. It should skip it and > go to the next so it can find smaller compactions to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12127) Queries with empty ByteBuffer values in clustering column restrictions fail for non-composite compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-12127:
-
Attachment: 12127.txt
> Queries with empty ByteBuffer values in clustering column restrictions fail
> for non-composite compact tables
>
>
> Key: CASSANDRA-12127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12127
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 12127.txt
>
>
> For the following table:
> {code}
> CREATE TABLE myTable (pk int,
> c blob,
> value int,
> PRIMARY KEY (pk, c)) WITH COMPACT STORAGE;
> INSERT INTO myTable (pk, c, value) VALUES (1, textAsBlob('1'), 1);
> INSERT INTO myTable (pk, c, value) VALUES (1, textAsBlob('2'), 2);
> {code}
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}}
> Will result in the following Exception:
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast
> to org.apache.cassandra.db.composites.CellName
> at
> org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:188)
> at
> org.apache.cassandra.db.composites.AbstractSimpleCellNameType.makeCellName(AbstractSimpleCellNameType.java:125)
> at
> org.apache.cassandra.db.composites.AbstractCellNameType.makeCellName(AbstractCellNameType.java:254)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.makeExclusiveSliceBound(SelectStatement.java:1206)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.applySliceRestriction(SelectStatement.java:1214)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.processColumnFamily(SelectStatement.java:1292)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:1259)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:299)
> [...]
> {code}
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c < textAsBlob('');}}
> Will return 2 rows instead of 0.
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c = textAsBlob('');}}
> {code}
> java.lang.AssertionError
> at
> org.apache.cassandra.db.composites.SimpleDenseCellNameType.create(SimpleDenseCellNameType.java:60)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.addSelectedColumns(SelectStatement.java:853)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getRequestedColumns(SelectStatement.java:846)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:583)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:383)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getPageableCommand(SelectStatement.java:253)
> [...]
> {code}
> I checked 2.0 and {{SELECT * FROM myTable WHERE pk = 1 AND c >
> textAsBlob('');}} works properly but {{SELECT * FROM myTable WHERE pk = 1 AND
> c < textAsBlob('');}} return the same wrong results than in 2.1.
> The {{SELECT * FROM myTable WHERE pk = 1 AND c = textAsBlob('');}} is
> rejected if a clear error message: {{Invalid empty value for clustering
> column of COMPACT TABLE}}.
> As it is not possible to insert an empty ByteBuffer value within the
> clustering column of a non-composite compact tables those queries do not
> have a lot of meaning. {{SELECT * FROM myTable WHERE pk = 1 AND c <
> textAsBlob('');}} and {{SELECT * FROM myTable WHERE pk = 1 AND c =
> textAsBlob('');}} will return nothing
> and {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}} will
> return the entire partition (pk = 1).
> In my opinion those queries should probably all be rejected as it seems that
> the fact that {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}}
> was accepted in {{2.0}} was due to a bug.
> I am of course open to discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12127) Queries with empty ByteBuffer values in clustering column restrictions fail for non-composite compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-12127:
-
Attachment: (was: 12127.txt)
> Queries with empty ByteBuffer values in clustering column restrictions fail
> for non-composite compact tables
>
>
> Key: CASSANDRA-12127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12127
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.1.x, 2.2.x
>
>
> For the following table:
> {code}
> CREATE TABLE myTable (pk int,
> c blob,
> value int,
> PRIMARY KEY (pk, c)) WITH COMPACT STORAGE;
> INSERT INTO myTable (pk, c, value) VALUES (1, textAsBlob('1'), 1);
> INSERT INTO myTable (pk, c, value) VALUES (1, textAsBlob('2'), 2);
> {code}
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}}
> Will result in the following Exception:
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast
> to org.apache.cassandra.db.composites.CellName
> at
> org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:188)
> at
> org.apache.cassandra.db.composites.AbstractSimpleCellNameType.makeCellName(AbstractSimpleCellNameType.java:125)
> at
> org.apache.cassandra.db.composites.AbstractCellNameType.makeCellName(AbstractCellNameType.java:254)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.makeExclusiveSliceBound(SelectStatement.java:1206)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.applySliceRestriction(SelectStatement.java:1214)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.processColumnFamily(SelectStatement.java:1292)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:1259)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:299)
> [...]
> {code}
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c < textAsBlob('');}}
> Will return 2 rows instead of 0.
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c = textAsBlob('');}}
> {code}
> java.lang.AssertionError
> at
> org.apache.cassandra.db.composites.SimpleDenseCellNameType.create(SimpleDenseCellNameType.java:60)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.addSelectedColumns(SelectStatement.java:853)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getRequestedColumns(SelectStatement.java:846)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:583)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:383)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getPageableCommand(SelectStatement.java:253)
> [...]
> {code}
> I checked 2.0 and {{SELECT * FROM myTable WHERE pk = 1 AND c >
> textAsBlob('');}} works properly but {{SELECT * FROM myTable WHERE pk = 1 AND
> c < textAsBlob('');}} return the same wrong results than in 2.1.
> The {{SELECT * FROM myTable WHERE pk = 1 AND c = textAsBlob('');}} is
> rejected if a clear error message: {{Invalid empty value for clustering
> column of COMPACT TABLE}}.
> As it is not possible to insert an empty ByteBuffer value within the
> clustering column of a non-composite compact tables those queries do not
> have a lot of meaning. {{SELECT * FROM myTable WHERE pk = 1 AND c <
> textAsBlob('');}} and {{SELECT * FROM myTable WHERE pk = 1 AND c =
> textAsBlob('');}} will return nothing
> and {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}} will
> return the entire partition (pk = 1).
> In my opinion those queries should probably all be rejected as it seems that
> the fact that {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}}
> was accepted in {{2.0}} was due to a bug.
> I am of course open to discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12040) If a level compaction fails due to no space it should schedule the next one
[ https://issues.apache.org/jira/browse/CASSANDRA-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-12040: -- Attachment: CASSANDRA-12040_trunk.txt > If a level compaction fails due to no space it should schedule the next one > - > > Key: CASSANDRA-12040 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12040 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Attachments: CASSANDRA-12040_trunk.txt > > > If a level compaction fails the space check, it aborts but next time the > compactions are scheduled it will attempt the same one. It should skip it and > go to the next so it can find smaller compactions to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12127) Queries with empty ByteBuffer values in clustering column restrictions fail for non-composite compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-12127:
-
Attachment: 12127.txt
One possible solution.
> Queries with empty ByteBuffer values in clustering column restrictions fail
> for non-composite compact tables
>
>
> Key: CASSANDRA-12127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12127
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 12127.txt
>
>
> For the following table:
> {code}
> CREATE TABLE myTable (pk int,
> c blob,
> value int,
> PRIMARY KEY (pk, c)) WITH COMPACT STORAGE;
> INSERT INTO myTable (pk, c, value) VALUES (1, textAsBlob('1'), 1);
> INSERT INTO myTable (pk, c, value) VALUES (1, textAsBlob('2'), 2);
> {code}
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}}
> Will result in the following Exception:
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast
> to org.apache.cassandra.db.composites.CellName
> at
> org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:188)
> at
> org.apache.cassandra.db.composites.AbstractSimpleCellNameType.makeCellName(AbstractSimpleCellNameType.java:125)
> at
> org.apache.cassandra.db.composites.AbstractCellNameType.makeCellName(AbstractCellNameType.java:254)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.makeExclusiveSliceBound(SelectStatement.java:1206)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.applySliceRestriction(SelectStatement.java:1214)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.processColumnFamily(SelectStatement.java:1292)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:1259)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:299)
> [...]
> {code}
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c < textAsBlob('');}}
> Will return 2 rows instead of 0.
> The query: {{SELECT * FROM myTable WHERE pk = 1 AND c = textAsBlob('');}}
> {code}
> java.lang.AssertionError
> at
> org.apache.cassandra.db.composites.SimpleDenseCellNameType.create(SimpleDenseCellNameType.java:60)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.addSelectedColumns(SelectStatement.java:853)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getRequestedColumns(SelectStatement.java:846)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:583)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:383)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.getPageableCommand(SelectStatement.java:253)
> [...]
> {code}
> I checked 2.0 and {{SELECT * FROM myTable WHERE pk = 1 AND c >
> textAsBlob('');}} works properly but {{SELECT * FROM myTable WHERE pk = 1 AND
> c < textAsBlob('');}} return the same wrong results than in 2.1.
> The {{SELECT * FROM myTable WHERE pk = 1 AND c = textAsBlob('');}} is
> rejected if a clear error message: {{Invalid empty value for clustering
> column of COMPACT TABLE}}.
> As it is not possible to insert an empty ByteBuffer value within the
> clustering column of a non-composite compact tables those queries do not
> have a lot of meaning. {{SELECT * FROM myTable WHERE pk = 1 AND c <
> textAsBlob('');}} and {{SELECT * FROM myTable WHERE pk = 1 AND c =
> textAsBlob('');}} will return nothing
> and {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}} will
> return the entire partition (pk = 1).
> In my opinion those queries should probably all be rejected as it seems that
> the fact that {{SELECT * FROM myTable WHERE pk = 1 AND c > textAsBlob('');}}
> was accepted in {{2.0}} was due to a bug.
> I am of course open to discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365172#comment-15365172 ] Jonathan Ellis edited comment on CASSANDRA-11521 at 7/6/16 9:50 PM: Is the plan still to "bypass the coordination layer?" From the comments here it doesn't look like that's the case, but I can't make that determination with confidence. Could you post an updated design sketch? was (Author: jbellis): If this bypasses the coordinator does that mean we're limited to unfiltered seq scan, or do we support all of SELECT still? > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365172#comment-15365172 ] Jonathan Ellis commented on CASSANDRA-11521: If this bypasses the coordinator does that mean we're limited to unfiltered seq scan, or do we support all of SELECT still? > Implement streaming for bulk read requests > -- > > Key: CASSANDRA-11521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11521 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Stefania >Assignee: Stefania > Labels: client-impacting > Fix For: 3.x > > > Allow clients to stream data from a C* host, bypassing the coordination layer > and eliminating the need to query individual pages one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365165#comment-15365165
]
Stanislav Vishnevskiy commented on CASSANDRA-12144:
---
I sent you an email.
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365148#comment-15365148
]
Alex Petrov commented on CASSANDRA-12144:
-
Hi Stanislav,
Could you also CC me? I can take a look at it, too.
Thanks!
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11697) Improve Compaction Throughput
[
https://issues.apache.org/jira/browse/CASSANDRA-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365050#comment-15365050
]
T Jake Luciani edited comment on CASSANDRA-11697 at 7/6/16 8:53 PM:
Just a general note about my findings so far:
There seems to be some conflation about the performance of compaction mainly
that low IO is slower than high IO. Our compaction is primarily CPU bound and
since we compress by default it can appear that compaction is going slow but
it's actually doing ~the same rows/sec as uncompressed. Here is an example of
compressed vs uncompressed notice the new log message from CASSANDRA-10805
shows ~ the same rows/sec.
{quote}
#UNCOMPRESSED
INFO 20:22:01 Compacted (3206a030-43b7-11e6-bf87-63e169949442) 2 sstables to
[/home/jake/workspace/cassandra/build/test/cassandra/data/cql_keyspace/table1-041d8b2043b711e6bf8763e169949442/mb-3-big,]
to level=0.
782.013MiB to 361.930MiB (~46% of original) in 43,211ms.
Read Throughput = 18.097MiB/s, Write Throughput = 8.376MiB/s,
Row Throughput = ~454,545/s. 20,000,000 total partitions merged to
10,000,000. Partition merge counts were \{2:1000, \}
#COMPRESSED
INFO 20:27:49 Compacted (00d23b40-43b8-11e6-ad90-8f9393166cb4) 2 sstables to
[/home/jake/workspace/cassandra/build/test/cassandra/data/cql_keyspace/table1-da46ad8043b711e6ad908f9393166cb4/mb-3-big,]
to level=0.
123.201MiB to 57.405MiB (~46% of original) in 44,527ms. Read Throughput =
2.767MiB/s, Write Throughput = 1.289MiB/s,
Row Throughput = ~444,444/s. 20,000,000 total partitions merged to
10,000,000. Partition merge counts were \{2:1000,\}
{quote}
*I'd like to start talking about compaction in terms of row throughput vs
MB/sec.*
There are some things we can do to make our current compaction faster like
CASSANDRA-10309 should give us a ~10% boost but the *bigger* issue and what I
think most users are seeing in the real world is the fact that under load
compaction effectively grinds to a halt. I can easily produce a 10x drop in
throughput by applying load during a regular compaction. This to me is the
problem we should be focused on since we can see this when looking at sample
user logs. I have a POC I'm working on to address this that I'll be presenting
soon.
was (Author: tjake):
Just a general note about my findings so far:
There seems to be some conflation about the performance of compaction mainly
that low IO is slower than high IO. Our compaction is primarily CPU bound and
since we compress by default it can appear that compaction is going slow but
it's actually doing ~the same rows/sec as uncompressed. Here is an example of
compressed vs uncompressed notice the new log message from CASSANDRA-10805
shows ~ the same rows/sec.
{quote}
#UNCOMPRESSED
INFO 20:22:01 Compacted (3206a030-43b7-11e6-bf87-63e169949442) 2 sstables to
[/home/jake/workspace/cassandra/build/test/cassandra/data/cql_keyspace/table1-041d8b2043b711e6bf8763e169949442/mb-3-big,]
to level=0.
782.013MiB to 361.930MiB (~46% of original) in 43,211ms.
Read Throughput = 18.097MiB/s, Write Throughput = 8.376MiB/s,
Row Throughput = ~454,545/s. 20,000,000 total partitions merged to
10,000,000. Partition merge counts were {2:1000, }
#COMPRESSED
INFO 20:27:49 Compacted (00d23b40-43b8-11e6-ad90-8f9393166cb4) 2 sstables to
[/home/jake/workspace/cassandra/build/test/cassandra/data/cql_keyspace/table1-da46ad8043b711e6ad908f9393166cb4/mb-3-big,]
to level=0.
123.201MiB to 57.405MiB (~46% of original) in 44,527ms. Read Throughput =
2.767MiB/s, Write Throughput = 1.289MiB/s,
Row Throughput = ~444,444/s. 20,000,000 total partitions merged to
10,000,000. Partition merge counts were {2:1000, }
{quote}
*I'd like to start talking about compaction in terms of row throughput vs
MB/sec.*
There are some things we can do to make our current compaction faster like
CASSANDRA-10309 should give us a ~10% boost but the *bigger* issue and what I
think most users are seeing in the real world is the fact that under load
compaction effectively grinds to a halt. I can easily produce a 10x drop in
throughput by applying load during a regular compaction. This to me is the
problem we should be focused on since we can see this when looking at sample
user logs. I have a POC I'm working on to address this that I'll be presenting
soon to address this.
> Improve Compaction Throughput
> -
>
> Key: CASSANDRA-11697
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11697
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
>
> The goal of this ticket is to improve/understand the bottlenecks during
> compactions. At a high level this will inv
[jira] [Commented] (CASSANDRA-12018) CDC follow-ups
[
https://issues.apache.org/jira/browse/CASSANDRA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365051#comment-15365051
]
Joshua McKenzie commented on CASSANDRA-12018:
-
Absolutely no rush on this. Looking at 3.10 for it so you have some time.
> CDC follow-ups
> --
>
> Key: CASSANDRA-12018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12018
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
>
> h6. Platform independent implementation of DirectorySizeCalculator
> On linux, simplify to
> {{Arrays.stream(path.listFiles()).mapToLong(File::length).sum();}}
> h6. Refactor DirectorySizeCalculator
> bq. I don't get the DirectorySizeCalculator. Why the alive and visited sets,
> the listFiles step? Either list the files and just loop through them, or do
> the walkFileTree operation – you are now doing the same work twice. Use a
> plain long instead of the atomic as the class is still thread-unsafe.
> h6. TolerateErrorsInSection should not depend on previous SyncSegment status
> in CommitLogReader
> bq. tolerateErrorsInSection &=: I don't think it was intended for the value
> to depend on previous iterations.
> h6. Refactor interface of SImpleCachedBufferPool
> bq. SimpleCachedBufferPool should provide getThreadLocalReusableBuffer(int
> size) which should automatically reallocate if the available size is less,
> and not expose a setter at all.
> h6. Change CDC exception to WriteFailureException instead of
> WriteTimeoutException
> h6. Remove unused CommitLogTest.testRecovery(byte[] logData)
> h6. NoSpamLogger a message when at CDC capacity
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11697) Improve Compaction Throughput
[
https://issues.apache.org/jira/browse/CASSANDRA-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365050#comment-15365050
]
T Jake Luciani commented on CASSANDRA-11697:
Just a general note about my findings so far:
There seems to be some conflation about the performance of compaction mainly
that low IO is slower than high IO. Our compaction is primarily CPU bound and
since we compress by default it can appear that compaction is going slow but
it's actually doing ~the same rows/sec as uncompressed. Here is an example of
compressed vs uncompressed notice the new log message from CASSANDRA-10805
shows ~ the same rows/sec.
{quote}
#UNCOMPRESSED
INFO 20:22:01 Compacted (3206a030-43b7-11e6-bf87-63e169949442) 2 sstables to
[/home/jake/workspace/cassandra/build/test/cassandra/data/cql_keyspace/table1-041d8b2043b711e6bf8763e169949442/mb-3-big,]
to level=0.
782.013MiB to 361.930MiB (~46% of original) in 43,211ms.
Read Throughput = 18.097MiB/s, Write Throughput = 8.376MiB/s,
Row Throughput = ~454,545/s. 20,000,000 total partitions merged to
10,000,000. Partition merge counts were {2:1000, }
#COMPRESSED
INFO 20:27:49 Compacted (00d23b40-43b8-11e6-ad90-8f9393166cb4) 2 sstables to
[/home/jake/workspace/cassandra/build/test/cassandra/data/cql_keyspace/table1-da46ad8043b711e6ad908f9393166cb4/mb-3-big,]
to level=0.
123.201MiB to 57.405MiB (~46% of original) in 44,527ms. Read Throughput =
2.767MiB/s, Write Throughput = 1.289MiB/s,
Row Throughput = ~444,444/s. 20,000,000 total partitions merged to
10,000,000. Partition merge counts were {2:1000, }
{quote}
*I'd like to start talking about compaction in terms of row throughput vs
MB/sec.*
There are some things we can do to make our current compaction faster like
CASSANDRA-10309 should give us a ~10% boost but the *bigger* issue and what I
think most users are seeing in the real world is the fact that under load
compaction effectively grinds to a halt. I can easily produce a 10x drop in
throughput by applying load during a regular compaction. This to me is the
problem we should be focused on since we can see this when looking at sample
user logs. I have a POC I'm working on to address this that I'll be presenting
soon to address this.
> Improve Compaction Throughput
> -
>
> Key: CASSANDRA-11697
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11697
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
>
> The goal of this ticket is to improve/understand the bottlenecks during
> compactions. At a high level this will involve:
> * A test system for measuring compaction time for different workloads and
> compaction strategies.
> * Profiling and analysis
> * Make improvements
> * Add throughput regression tests so we can track
> We have a lot of random tickets that relate to this so I'll link them to this
> ticket
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365044#comment-15365044
]
Stanislav Vishnevskiy commented on CASSANDRA-12144:
---
Email sent.
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12018) CDC follow-ups
[
https://issues.apache.org/jira/browse/CASSANDRA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365041#comment-15365041
]
Branimir Lambov commented on CASSANDRA-12018:
-
No problem, if Friday isn't too late for it.
> CDC follow-ups
> --
>
> Key: CASSANDRA-12018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12018
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
>
> h6. Platform independent implementation of DirectorySizeCalculator
> On linux, simplify to
> {{Arrays.stream(path.listFiles()).mapToLong(File::length).sum();}}
> h6. Refactor DirectorySizeCalculator
> bq. I don't get the DirectorySizeCalculator. Why the alive and visited sets,
> the listFiles step? Either list the files and just loop through them, or do
> the walkFileTree operation – you are now doing the same work twice. Use a
> plain long instead of the atomic as the class is still thread-unsafe.
> h6. TolerateErrorsInSection should not depend on previous SyncSegment status
> in CommitLogReader
> bq. tolerateErrorsInSection &=: I don't think it was intended for the value
> to depend on previous iterations.
> h6. Refactor interface of SImpleCachedBufferPool
> bq. SimpleCachedBufferPool should provide getThreadLocalReusableBuffer(int
> size) which should automatically reallocate if the available size is less,
> and not expose a setter at all.
> h6. Change CDC exception to WriteFailureException instead of
> WriteTimeoutException
> h6. Remove unused CommitLogTest.testRecovery(byte[] logData)
> h6. NoSpamLogger a message when at CDC capacity
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12018) CDC follow-ups
[
https://issues.apache.org/jira/browse/CASSANDRA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365034#comment-15365034
]
Joshua McKenzie commented on CASSANDRA-12018:
-
bq. On linux, simplify to
{{Arrays.stream(path.listFiles()).mapToLong(File::length).sum();}}
Done.
bq. I don't get the DirectorySizeCalculator. Why the alive and visited sets,
the listFiles step? Either list the files and just loop through them, or do the
walkFileTree operation – you are now doing the same work twice. Use a plain
long instead of the atomic as the class is still thread-unsafe.
Cleaned up.
bq. tolerateErrorsInSection &=: I don't think it was intended for the value to
depend on previous iterations.
Removed &
bq. SimpleCachedBufferPool should provide getThreadLocalReusableBuffer(int
size) which should automatically reallocate if the available size is less, and
not expose a setter at all.
Updated interface to get and setPreferredBufferType
Lastly, removed unused CommitLogTest.testRecovery(byte[] logData) and added
NoSpamLogger message once every 10 seconds when at CDC capacity.
||branch||testall||dtest||
|[12018|https://github.com/josh-mckenzie/cassandra/tree/12018]|[testall|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-12018-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-12018-dtest]|
[~blambov] - up for a review?
> CDC follow-ups
> --
>
> Key: CASSANDRA-12018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12018
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
>
> h6. Platform independent implementation of DirectorySizeCalculator
> On linux, simplify to
> {{Arrays.stream(path.listFiles()).mapToLong(File::length).sum();}}
> h6. Refactor DirectorySizeCalculator
> bq. I don't get the DirectorySizeCalculator. Why the alive and visited sets,
> the listFiles step? Either list the files and just loop through them, or do
> the walkFileTree operation – you are now doing the same work twice. Use a
> plain long instead of the atomic as the class is still thread-unsafe.
> h6. TolerateErrorsInSection should not depend on previous SyncSegment status
> in CommitLogReader
> bq. tolerateErrorsInSection &=: I don't think it was intended for the value
> to depend on previous iterations.
> h6. Refactor interface of SImpleCachedBufferPool
> bq. SimpleCachedBufferPool should provide getThreadLocalReusableBuffer(int
> size) which should automatically reallocate if the available size is less,
> and not expose a setter at all.
> h6. Change CDC exception to WriteFailureException instead of
> WriteTimeoutException
> h6. Remove unused CommitLogTest.testRecovery(byte[] logData)
> h6. NoSpamLogger a message when at CDC capacity
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12018) CDC follow-ups
[
https://issues.apache.org/jira/browse/CASSANDRA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12018:
Status: Patch Available (was: In Progress)
> CDC follow-ups
> --
>
> Key: CASSANDRA-12018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12018
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
>
> h6. Platform independent implementation of DirectorySizeCalculator
> On linux, simplify to
> {{Arrays.stream(path.listFiles()).mapToLong(File::length).sum();}}
> h6. Refactor DirectorySizeCalculator
> bq. I don't get the DirectorySizeCalculator. Why the alive and visited sets,
> the listFiles step? Either list the files and just loop through them, or do
> the walkFileTree operation – you are now doing the same work twice. Use a
> plain long instead of the atomic as the class is still thread-unsafe.
> h6. TolerateErrorsInSection should not depend on previous SyncSegment status
> in CommitLogReader
> bq. tolerateErrorsInSection &=: I don't think it was intended for the value
> to depend on previous iterations.
> h6. Refactor interface of SImpleCachedBufferPool
> bq. SimpleCachedBufferPool should provide getThreadLocalReusableBuffer(int
> size) which should automatically reallocate if the available size is less,
> and not expose a setter at all.
> h6. Change CDC exception to WriteFailureException instead of
> WriteTimeoutException
> h6. Remove unused CommitLogTest.testRecovery(byte[] logData)
> h6. NoSpamLogger a message when at CDC capacity
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-12144:
---
Assignee: Alex Petrov
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12137) dtest failure in sstable_generation_loading_test.TestSSTableGenerationAndLoading.sstableloader_compression_deflate_to_snappy_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Witschey updated CASSANDRA-12137:
-
Assignee: Jim Witschey (was: DS Test Eng)
> dtest failure in
> sstable_generation_loading_test.TestSSTableGenerationAndLoading.sstableloader_compression_deflate_to_snappy_test
> -
>
> Key: CASSANDRA-12137
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12137
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Jim Witschey
> Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/764/testReport/sstable_generation_loading_test/TestSSTableGenerationAndLoading/sstableloader_compression_deflate_to_snappy_test
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/sstable_generation_loading_test.py",
> line 75, in sstableloader_compression_deflate_to_snappy_test
> self.load_sstable_with_configuration('Deflate', 'Snappy')
> File "/home/automaton/cassandra-dtest/sstable_generation_loading_test.py",
> line 178, in load_sstable_with_configuration
> "sstableloader exited with a non-zero status: {}".format(exit_status))
> File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
> File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> "sstableloader exited with a non-zero status: 1
> {code}
> Related failures:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/764/testReport/sstable_generation_loading_test/TestSSTableGenerationAndLoading/sstableloader_compression_none_to_snappy_test/
> http://cassci.datastax.com/job/cassandra-3.0_dtest/764/testReport/sstable_generation_loading_test/TestSSTableGenerationAndLoading/sstableloader_with_mv_test/
> Failed on CassCI build cassandra-3.0_dtest #764
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364882#comment-15364882
]
Tyler Hobbs commented on CASSANDRA-12144:
-
Is it possible for you to upload the sstables somewhere for the node that owns
that partition?
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11414) dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-11414:
---
Resolution: Fixed
Fix Version/s: (was: 3.x)
3.9
3.0.9
2.2.8
Status: Resolved (was: Ready to Commit)
Committed as {00e7ecf1394f8704e2f13369f7950e129459ce2c}.
> dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
> --
>
> Key: CASSANDRA-11414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11414
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Philip Thompson
>Assignee: Paulo Motta
> Labels: dtest
> Fix For: 2.2.8, 3.0.9, 3.9
>
>
> Stress is failing to read back all data. We can see this output from the
> stress read
> {code}
> java.io.IOException: Operation x0 on key(s) [314c384f304f4c325030]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> java.io.IOException: Operation x0 on key(s) [33383438363931353131]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> FAILURE
> {code}
> Started happening with build 1075. Does not appear flaky on CI.
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1076/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test
> Failed on CassCI build trunk_dtest #1076
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[03/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 * cassandra-2.1: Range tombstones that are masked by row tombstones should not be written out Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43c741e2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43c741e2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43c741e2 Branch: refs/heads/trunk Commit: 43c741e251102bf5651ff8aa1b5ca078eb0ddc0b Parents: d5a15e4 98f5f77 Author: Sylvain Lebresne Authored: Wed Jul 6 14:39:13 2016 +0200 Committer: Sylvain Lebresne Committed: Wed Jul 6 14:39:13 2016 +0200 -- CHANGES.txt | 1 + .../db/compaction/LazilyCompactedRow.java | 3 +- .../apache/cassandra/db/RangeTombstoneTest.java | 40 3 files changed, 43 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/43c741e2/CHANGES.txt -- diff --cc CHANGES.txt index 65c7c1f,7fa995d..bfd8aa2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,43 -1,11 +1,44 @@@ -2.1.16 +2.2.8 + * MemoryUtil.getShort() should return an unsigned short also for architectures not supporting unaligned memory accesses (CASSANDRA-11973) +Merged from 2.1: + * Don't write shadowed range tombstone (CASSANDRA-12030) - * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907) - * Reduce contention getting instances of CompositeType (CASSANDRA-10433) * Improve digest calculation in the presence of overlapping tombstones (CASSANDRA-11349) - -2.1.15 + * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907) * Account for partition deletions in tombstone histogram (CASSANDRA-12112) + + +2.2.7 + * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755) + * Validate bloom_filter_fp_chance against lowest supported + value when the table is created (CASSANDRA-11920) + * RandomAccessReader: call isEOF() only when rebuffering, not for every read operation (CASSANDRA-12013) + * Don't send erroneous NEW_NODE notifications on restart (CASSANDRA-11038) + * StorageService shutdown hook should use a volatile variable (CASSANDRA-11984) + * Persist local metadata earlier in startup sequence (CASSANDRA-11742) + * Run CommitLog tests with different compression settings (CASSANDRA-9039) + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043) * Remove finished incoming streaming connections from MessagingService (CASSANDRA-11854) * Don't try to get sstables for non-repairing column families (CASSANDRA-12077) http://git-wip-us.apache.org/repos/asf/cassandra/blob/43c741e2/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/43c741e2/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java -- diff --cc test/unit/org/apache/cassandra/db/RangeTombstoneTest.java index 9
[04/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 * cassandra-2.1: Range tombstones that are masked by row tombstones should not be written out Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43c741e2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43c741e2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43c741e2 Branch: refs/heads/cassandra-3.9 Commit: 43c741e251102bf5651ff8aa1b5ca078eb0ddc0b Parents: d5a15e4 98f5f77 Author: Sylvain Lebresne Authored: Wed Jul 6 14:39:13 2016 +0200 Committer: Sylvain Lebresne Committed: Wed Jul 6 14:39:13 2016 +0200 -- CHANGES.txt | 1 + .../db/compaction/LazilyCompactedRow.java | 3 +- .../apache/cassandra/db/RangeTombstoneTest.java | 40 3 files changed, 43 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/43c741e2/CHANGES.txt -- diff --cc CHANGES.txt index 65c7c1f,7fa995d..bfd8aa2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,43 -1,11 +1,44 @@@ -2.1.16 +2.2.8 + * MemoryUtil.getShort() should return an unsigned short also for architectures not supporting unaligned memory accesses (CASSANDRA-11973) +Merged from 2.1: + * Don't write shadowed range tombstone (CASSANDRA-12030) - * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907) - * Reduce contention getting instances of CompositeType (CASSANDRA-10433) * Improve digest calculation in the presence of overlapping tombstones (CASSANDRA-11349) - -2.1.15 + * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907) * Account for partition deletions in tombstone histogram (CASSANDRA-12112) + + +2.2.7 + * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755) + * Validate bloom_filter_fp_chance against lowest supported + value when the table is created (CASSANDRA-11920) + * RandomAccessReader: call isEOF() only when rebuffering, not for every read operation (CASSANDRA-12013) + * Don't send erroneous NEW_NODE notifications on restart (CASSANDRA-11038) + * StorageService shutdown hook should use a volatile variable (CASSANDRA-11984) + * Persist local metadata earlier in startup sequence (CASSANDRA-11742) + * Run CommitLog tests with different compression settings (CASSANDRA-9039) + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: * Avoid stalling paxos when the paxos state expires (CASSANDRA-12043) * Remove finished incoming streaming connections from MessagingService (CASSANDRA-11854) * Don't try to get sstables for non-repairing column families (CASSANDRA-12077) http://git-wip-us.apache.org/repos/asf/cassandra/blob/43c741e2/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/43c741e2/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java -- diff --cc test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
[08/16] cassandra git commit: Improve streaming synchronization and fault tolerance
Improve streaming synchronization and fault tolerance
Patch by Paulo Motta; Reviewed by yukim for CASSANDRA-11414
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00e7ecf1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00e7ecf1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00e7ecf1
Branch: refs/heads/trunk
Commit: 00e7ecf1394f8704e2f13369f7950e129459ce2c
Parents: 43c741e
Author: Paulo Motta
Authored: Wed Jul 6 12:16:16 2016 -0500
Committer: Yuki Morishita
Committed: Wed Jul 6 12:32:39 2016 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/streaming/ConnectionHandler.java| 8 +++-
.../org/apache/cassandra/streaming/StreamReceiveTask.java| 2 --
3 files changed, 4 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bfd8aa2..7d62f97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.8
+ * Improve streaming synchronization and fault tolerance (CASSANDRA-11414)
* MemoryUtil.getShort() should return an unsigned short also for
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
Merged from 2.1:
* Don't write shadowed range tombstone (CASSANDRA-12030)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index c497a39..364435e 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -233,6 +233,9 @@ public class ConnectionHandler
protected void signalCloseDone()
{
+if (closeFuture == null)
+close();
+
closeFuture.get().set(null);
// We can now close the socket
@@ -294,11 +297,6 @@ public class ConnectionHandler
}
}
}
-catch (SocketException e)
-{
-// socket is closed
-close();
-}
catch (Throwable t)
{
JVMStabilityInspector.inspectThrowable(t);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
index 6911ec6..b342edc 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
@@ -18,8 +18,6 @@
package org.apache.cassandra.streaming;
import java.io.File;
-import java.io.IOError;
-import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
[12/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/778f2a46 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/778f2a46 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/778f2a46 Branch: refs/heads/cassandra-3.9 Commit: 778f2a46e2df52aa8451aceaf17046e6b8c86ace Parents: 9ed3b42 00e7ecf Author: Yuki Morishita Authored: Wed Jul 6 12:33:54 2016 -0500 Committer: Yuki Morishita Committed: Wed Jul 6 12:33:54 2016 -0500 -- CHANGES.txt | 1 + .../cassandra/streaming/ConnectionHandler.java | 8 ++-- .../cassandra/streaming/StreamReceiveTask.java | 50 +++- .../cassandra/streaming/StreamSession.java | 17 +-- .../streaming/StreamingTransferTest.java| 30 ++-- 5 files changed, 83 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/778f2a46/CHANGES.txt -- diff --cc CHANGES.txt index 02786c5,7d62f97..8118de1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,14 +1,27 @@@ -2.2.8 +3.0.9 + * Improve streaming synchronization and fault tolerance (CASSANDRA-11414) + * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098) + * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996) + * Always select the live sstables when getting sstables in bounds (CASSANDRA-11944) + * Fix column ordering of results with static columns for Thrift requests in + a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of + those static columns in query results (CASSANDRA-12123) + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090) + * Fix EOF exception when altering column type (CASSANDRA-11820) +Merged from 2.2: * MemoryUtil.getShort() should return an unsigned short also for architectures not supporting unaligned memory accesses (CASSANDRA-11973) Merged from 2.1: - * Don't write shadowed range tombstone (CASSANDRA-12030) - * Improve digest calculation in the presence of overlapping tombstones (CASSANDRA-11349) * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907) - * Account for partition deletions in tombstone histogram (CASSANDRA-12112) -2.2.7 +3.0.8 + * Fix potential race in schema during new table creation (CASSANDRA-12083) + * cqlsh: fix error handling in rare COPY FROM failure scenario (CASSANDRA-12070) + * Disable autocompaction during drain (CASSANDRA-11878) + * Add a metrics timer to MemtablePool and use it to track time spent blocked on memory in MemtableAllocator (CASSANDRA-11327) + * Fix upgrading schema with super columns with non-text subcomparators (CASSANDRA-12023) + * Add TimeWindowCompactionStrategy (CASSANDRA-9666) +Merged from 2.2: * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755) * Validate bloom_filter_fp_chance against lowest supported value when the table is created (CASSANDRA-11920) http://git-wip-us.apache.org/repos/asf/cassandra/blob/778f2a46/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java -- diff --cc src/java/org/apache/cassandra/streaming/StreamReceiveTask.java index 6280f3a,b342edc..040906b --- a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java +++ b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java @@@ -17,9 -17,7 +17,6 @@@ */ package org.apache.cassandra.streaming; --import java.io.File; - import java.io.IOError; - import java.io.IOException; import java.util.ArrayList; import java.util.Collection; import java.util.List; @@@ -36,19 -33,13 +33,20 @@@ import org.apache.cassandra.concurrent. import org.apache.cassandra.config.Schema; import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Keyspace; +import org.apache.cassandra.db.Mutation; +import org.apache.cassandra.db.compaction.OperationType; +import org.apache.cassandra.db.lifecycle.LifecycleTransaction; +import org.apache.cassandra.db.partitions.PartitionUpdate; +import org.apache.cassandra.db.rows.UnfilteredRowIterator; +import org.apache.cassandra.db.view.View; import org.apache.cassandra.dht.Bounds; import org.apache.cassandra.dht.Token; +import org.apache.cassandra.io.sstable.ISSTableScanner; +import org.apache.cassandra.io.sstable.SSTableMultiWriter; import org.apache.cassandra.io.sstable.format.SSTableReader; -import org.apache.cassandra.io.sstable.format.SSTableWriter; import org.apache.cassandra.utils.JVMStabilityInspector; import org.apache.cassandra.utils.Pair; - ++import org.apache.cassandra.utils.Throwables; import org.apache.cassandra.utils.concurre
[07/16] cassandra git commit: Improve streaming synchronization and fault tolerance
Improve streaming synchronization and fault tolerance
Patch by Paulo Motta; Reviewed by yukim for CASSANDRA-11414
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00e7ecf1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00e7ecf1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00e7ecf1
Branch: refs/heads/cassandra-3.0
Commit: 00e7ecf1394f8704e2f13369f7950e129459ce2c
Parents: 43c741e
Author: Paulo Motta
Authored: Wed Jul 6 12:16:16 2016 -0500
Committer: Yuki Morishita
Committed: Wed Jul 6 12:32:39 2016 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/streaming/ConnectionHandler.java| 8 +++-
.../org/apache/cassandra/streaming/StreamReceiveTask.java| 2 --
3 files changed, 4 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bfd8aa2..7d62f97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.8
+ * Improve streaming synchronization and fault tolerance (CASSANDRA-11414)
* MemoryUtil.getShort() should return an unsigned short also for
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
Merged from 2.1:
* Don't write shadowed range tombstone (CASSANDRA-12030)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index c497a39..364435e 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -233,6 +233,9 @@ public class ConnectionHandler
protected void signalCloseDone()
{
+if (closeFuture == null)
+close();
+
closeFuture.get().set(null);
// We can now close the socket
@@ -294,11 +297,6 @@ public class ConnectionHandler
}
}
}
-catch (SocketException e)
-{
-// socket is closed
-close();
-}
catch (Throwable t)
{
JVMStabilityInspector.inspectThrowable(t);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
index 6911ec6..b342edc 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
@@ -18,8 +18,6 @@
package org.apache.cassandra.streaming;
import java.io.File;
-import java.io.IOError;
-import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
[02/16] cassandra git commit: Range tombstones that are masked by row tombstones should not be written out
Range tombstones that are masked by row tombstones should not be written out
patch by Nachiket Patil; reviewed by Sylvain Lebresne for CASSANDRA-12030
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/98f5f77b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/98f5f77b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/98f5f77b
Branch: refs/heads/trunk
Commit: 98f5f77bb3c5d50e52cbb6f577a463ca8a5134ad
Parents: 3c1653f
Author: Nachiket Patil
Authored: Wed Jul 6 11:22:56 2016 +0200
Committer: Sylvain Lebresne
Committed: Wed Jul 6 14:35:10 2016 +0200
--
CHANGES.txt | 1 +
.../db/compaction/LazilyCompactedRow.java | 3 +-
.../apache/cassandra/db/RangeTombstoneTest.java | 40
3 files changed, 43 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/98f5f77b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b1dcbe1..7fa995d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.16
+ * Don't write shadowed range tombstone (CASSANDRA-12030)
* Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
* Reduce contention getting instances of CompositeType (CASSANDRA-10433)
* Improve digest calculation in the presence of overlapping tombstones
(CASSANDRA-11349)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/98f5f77b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
index f912da2..dab5eeb 100644
--- a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
@@ -286,7 +286,8 @@ public class LazilyCompactedRow extends AbstractCompactedRow
RangeTombstone t = tombstone;
tombstone = null;
-if (t.data.isGcAble(controller.gcBefore) && t.timestamp() <
getMaxPurgeableTimestamp())
+if (t.data.isGcAble(controller.gcBefore) && t.timestamp() <
getMaxPurgeableTimestamp() ||
+maxRowTombstone.markedForDeleteAt >= t.timestamp())
{
indexBuilder.tombstoneTracker().update(t, true);
return null;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/98f5f77b/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
b/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
index 3292422..dfd6960 100644
--- a/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
+++ b/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
@@ -39,6 +39,7 @@ import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.config.IndexType;
import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.db.compaction.LeveledCompactionStrategy;
import org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy;
import org.apache.cassandra.db.composites.CellName;
import org.apache.cassandra.db.composites.CellNames;
@@ -543,6 +544,45 @@ public class RangeTombstoneTest extends SchemaLoader
}
@Test
+public void testCompactionOfRangeTombstonesCoveredByRowTombstone() throws
Exception
+{
+long testTimeStamp = 1451606400L; // 01/01/2016 : 00:00:00 GMT
+Keyspace table = Keyspace.open(KSNAME);
+ColumnFamilyStore cfs = table.getColumnFamilyStore(CFNAME);
+ByteBuffer key = ByteBufferUtil.bytes("k4");
+
+// remove any existing sstables before starting
+cfs.truncateBlocking();
+cfs.disableAutoCompaction();
+
cfs.setCompactionStrategyClass(LeveledCompactionStrategy.class.getCanonicalName());
+
+Mutation rm = new Mutation(KSNAME, key);
+for (int i = 1; i < 11; i += 2, testTimeStamp += i * 10)
+add(rm, i, testTimeStamp);
+rm.apply();
+cfs.forceBlockingFlush();
+
+rm = new Mutation(KSNAME, key);
+ColumnFamily cf = rm.addOrGet(CFNAME);
+
+// Write the covering row tombstone
+cf.delete(new DeletionTime(++testTimeStamp, (int) testTimeStamp));
+
+// Create range tombstones covered by row tombstone above.
+for (int i = 1; i < 11; i += 2, testTimeStamp -= i * 5)
+delete(cf, 0, 7, testTimeStamp);
+rm.apply();
+
[16/16] cassandra git commit: Merge branch 'cassandra-3.9' into trunk
Merge branch 'cassandra-3.9' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9fd60777 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9fd60777 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9fd60777 Branch: refs/heads/trunk Commit: 9fd607778091c48910db557d7a95029cac077244 Parents: b4133f3 59ee46e Author: Yuki Morishita Authored: Wed Jul 6 12:34:30 2016 -0500 Committer: Yuki Morishita Committed: Wed Jul 6 12:34:30 2016 -0500 -- CHANGES.txt | 1 + .../cassandra/streaming/ConnectionHandler.java | 8 ++-- .../cassandra/streaming/StreamReceiveTask.java | 50 +++- .../cassandra/streaming/StreamSession.java | 17 +-- .../streaming/StreamingTransferTest.java| 30 ++-- 5 files changed, 83 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fd60777/CHANGES.txt --
[05/16] cassandra git commit: Merge commit '43c741e251102bf5651ff8aa1b5ca078eb0ddc0b' into cassandra-3.0
Merge commit '43c741e251102bf5651ff8aa1b5ca078eb0ddc0b' into cassandra-3.0 * commit '43c741e251102bf5651ff8aa1b5ca078eb0ddc0b': Range tombstones that are masked by row tombstones should not be written out Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ed3b42d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ed3b42d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ed3b42d Branch: refs/heads/trunk Commit: 9ed3b42d3b50237f99485233857a7b34d5238d9a Parents: dd05e46 43c741e Author: Sylvain Lebresne Authored: Wed Jul 6 14:39:52 2016 +0200 Committer: Sylvain Lebresne Committed: Wed Jul 6 14:39:52 2016 +0200 -- --
[09/16] cassandra git commit: Improve streaming synchronization and fault tolerance
Improve streaming synchronization and fault tolerance
Patch by Paulo Motta; Reviewed by yukim for CASSANDRA-11414
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00e7ecf1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00e7ecf1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00e7ecf1
Branch: refs/heads/cassandra-3.9
Commit: 00e7ecf1394f8704e2f13369f7950e129459ce2c
Parents: 43c741e
Author: Paulo Motta
Authored: Wed Jul 6 12:16:16 2016 -0500
Committer: Yuki Morishita
Committed: Wed Jul 6 12:32:39 2016 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/streaming/ConnectionHandler.java| 8 +++-
.../org/apache/cassandra/streaming/StreamReceiveTask.java| 2 --
3 files changed, 4 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bfd8aa2..7d62f97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.8
+ * Improve streaming synchronization and fault tolerance (CASSANDRA-11414)
* MemoryUtil.getShort() should return an unsigned short also for
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
Merged from 2.1:
* Don't write shadowed range tombstone (CASSANDRA-12030)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index c497a39..364435e 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -233,6 +233,9 @@ public class ConnectionHandler
protected void signalCloseDone()
{
+if (closeFuture == null)
+close();
+
closeFuture.get().set(null);
// We can now close the socket
@@ -294,11 +297,6 @@ public class ConnectionHandler
}
}
}
-catch (SocketException e)
-{
-// socket is closed
-close();
-}
catch (Throwable t)
{
JVMStabilityInspector.inspectThrowable(t);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
index 6911ec6..b342edc 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
@@ -18,8 +18,6 @@
package org.apache.cassandra.streaming;
import java.io.File;
-import java.io.IOError;
-import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
[14/16] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9
Merge branch 'cassandra-3.0' into cassandra-3.9 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59ee46e5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59ee46e5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59ee46e5 Branch: refs/heads/trunk Commit: 59ee46e55a15775a49edde86de81b9b79875731d Parents: 5ad1763 778f2a4 Author: Yuki Morishita Authored: Wed Jul 6 12:34:22 2016 -0500 Committer: Yuki Morishita Committed: Wed Jul 6 12:34:22 2016 -0500 -- CHANGES.txt | 1 + .../cassandra/streaming/ConnectionHandler.java | 8 ++-- .../cassandra/streaming/StreamReceiveTask.java | 50 +++- .../cassandra/streaming/StreamSession.java | 17 +-- .../streaming/StreamingTransferTest.java| 30 ++-- 5 files changed, 83 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/CHANGES.txt -- diff --cc CHANGES.txt index 2861cf7,8118de1..d459e34 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,7 -1,5 +1,8 @@@ -3.0.9 +3.9 + * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073) + * Increase size of flushExecutor thread pool (CASSANDRA-12071) +Merged from 3.0: + * Improve streaming synchronization and fault tolerance (CASSANDRA-11414) * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098) * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996) * Always select the live sstables when getting sstables in bounds (CASSANDRA-11944) http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/src/java/org/apache/cassandra/streaming/ConnectionHandler.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/src/java/org/apache/cassandra/streaming/StreamSession.java --
[11/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/778f2a46 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/778f2a46 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/778f2a46 Branch: refs/heads/trunk Commit: 778f2a46e2df52aa8451aceaf17046e6b8c86ace Parents: 9ed3b42 00e7ecf Author: Yuki Morishita Authored: Wed Jul 6 12:33:54 2016 -0500 Committer: Yuki Morishita Committed: Wed Jul 6 12:33:54 2016 -0500 -- CHANGES.txt | 1 + .../cassandra/streaming/ConnectionHandler.java | 8 ++-- .../cassandra/streaming/StreamReceiveTask.java | 50 +++- .../cassandra/streaming/StreamSession.java | 17 +-- .../streaming/StreamingTransferTest.java| 30 ++-- 5 files changed, 83 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/778f2a46/CHANGES.txt -- diff --cc CHANGES.txt index 02786c5,7d62f97..8118de1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,14 +1,27 @@@ -2.2.8 +3.0.9 + * Improve streaming synchronization and fault tolerance (CASSANDRA-11414) + * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098) + * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996) + * Always select the live sstables when getting sstables in bounds (CASSANDRA-11944) + * Fix column ordering of results with static columns for Thrift requests in + a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of + those static columns in query results (CASSANDRA-12123) + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090) + * Fix EOF exception when altering column type (CASSANDRA-11820) +Merged from 2.2: * MemoryUtil.getShort() should return an unsigned short also for architectures not supporting unaligned memory accesses (CASSANDRA-11973) Merged from 2.1: - * Don't write shadowed range tombstone (CASSANDRA-12030) - * Improve digest calculation in the presence of overlapping tombstones (CASSANDRA-11349) * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907) - * Account for partition deletions in tombstone histogram (CASSANDRA-12112) -2.2.7 +3.0.8 + * Fix potential race in schema during new table creation (CASSANDRA-12083) + * cqlsh: fix error handling in rare COPY FROM failure scenario (CASSANDRA-12070) + * Disable autocompaction during drain (CASSANDRA-11878) + * Add a metrics timer to MemtablePool and use it to track time spent blocked on memory in MemtableAllocator (CASSANDRA-11327) + * Fix upgrading schema with super columns with non-text subcomparators (CASSANDRA-12023) + * Add TimeWindowCompactionStrategy (CASSANDRA-9666) +Merged from 2.2: * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755) * Validate bloom_filter_fp_chance against lowest supported value when the table is created (CASSANDRA-11920) http://git-wip-us.apache.org/repos/asf/cassandra/blob/778f2a46/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java -- diff --cc src/java/org/apache/cassandra/streaming/StreamReceiveTask.java index 6280f3a,b342edc..040906b --- a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java +++ b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java @@@ -17,9 -17,7 +17,6 @@@ */ package org.apache.cassandra.streaming; --import java.io.File; - import java.io.IOError; - import java.io.IOException; import java.util.ArrayList; import java.util.Collection; import java.util.List; @@@ -36,19 -33,13 +33,20 @@@ import org.apache.cassandra.concurrent. import org.apache.cassandra.config.Schema; import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Keyspace; +import org.apache.cassandra.db.Mutation; +import org.apache.cassandra.db.compaction.OperationType; +import org.apache.cassandra.db.lifecycle.LifecycleTransaction; +import org.apache.cassandra.db.partitions.PartitionUpdate; +import org.apache.cassandra.db.rows.UnfilteredRowIterator; +import org.apache.cassandra.db.view.View; import org.apache.cassandra.dht.Bounds; import org.apache.cassandra.dht.Token; +import org.apache.cassandra.io.sstable.ISSTableScanner; +import org.apache.cassandra.io.sstable.SSTableMultiWriter; import org.apache.cassandra.io.sstable.format.SSTableReader; -import org.apache.cassandra.io.sstable.format.SSTableWriter; import org.apache.cassandra.utils.JVMStabilityInspector; import org.apache.cassandra.utils.Pair; - ++import org.apache.cassandra.utils.Throwables; import org.apache.cassandra.utils.concurrent.Refs;
[13/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/778f2a46 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/778f2a46 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/778f2a46 Branch: refs/heads/cassandra-3.0 Commit: 778f2a46e2df52aa8451aceaf17046e6b8c86ace Parents: 9ed3b42 00e7ecf Author: Yuki Morishita Authored: Wed Jul 6 12:33:54 2016 -0500 Committer: Yuki Morishita Committed: Wed Jul 6 12:33:54 2016 -0500 -- CHANGES.txt | 1 + .../cassandra/streaming/ConnectionHandler.java | 8 ++-- .../cassandra/streaming/StreamReceiveTask.java | 50 +++- .../cassandra/streaming/StreamSession.java | 17 +-- .../streaming/StreamingTransferTest.java| 30 ++-- 5 files changed, 83 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/778f2a46/CHANGES.txt -- diff --cc CHANGES.txt index 02786c5,7d62f97..8118de1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,14 +1,27 @@@ -2.2.8 +3.0.9 + * Improve streaming synchronization and fault tolerance (CASSANDRA-11414) + * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098) + * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996) + * Always select the live sstables when getting sstables in bounds (CASSANDRA-11944) + * Fix column ordering of results with static columns for Thrift requests in + a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of + those static columns in query results (CASSANDRA-12123) + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090) + * Fix EOF exception when altering column type (CASSANDRA-11820) +Merged from 2.2: * MemoryUtil.getShort() should return an unsigned short also for architectures not supporting unaligned memory accesses (CASSANDRA-11973) Merged from 2.1: - * Don't write shadowed range tombstone (CASSANDRA-12030) - * Improve digest calculation in the presence of overlapping tombstones (CASSANDRA-11349) * Fix filtering on clustering columns when 2i is used (CASSANDRA-11907) - * Account for partition deletions in tombstone histogram (CASSANDRA-12112) -2.2.7 +3.0.8 + * Fix potential race in schema during new table creation (CASSANDRA-12083) + * cqlsh: fix error handling in rare COPY FROM failure scenario (CASSANDRA-12070) + * Disable autocompaction during drain (CASSANDRA-11878) + * Add a metrics timer to MemtablePool and use it to track time spent blocked on memory in MemtableAllocator (CASSANDRA-11327) + * Fix upgrading schema with super columns with non-text subcomparators (CASSANDRA-12023) + * Add TimeWindowCompactionStrategy (CASSANDRA-9666) +Merged from 2.2: * Allow nodetool info to run with readonly JMX access (CASSANDRA-11755) * Validate bloom_filter_fp_chance against lowest supported value when the table is created (CASSANDRA-11920) http://git-wip-us.apache.org/repos/asf/cassandra/blob/778f2a46/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java -- diff --cc src/java/org/apache/cassandra/streaming/StreamReceiveTask.java index 6280f3a,b342edc..040906b --- a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java +++ b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java @@@ -17,9 -17,7 +17,6 @@@ */ package org.apache.cassandra.streaming; --import java.io.File; - import java.io.IOError; - import java.io.IOException; import java.util.ArrayList; import java.util.Collection; import java.util.List; @@@ -36,19 -33,13 +33,20 @@@ import org.apache.cassandra.concurrent. import org.apache.cassandra.config.Schema; import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Keyspace; +import org.apache.cassandra.db.Mutation; +import org.apache.cassandra.db.compaction.OperationType; +import org.apache.cassandra.db.lifecycle.LifecycleTransaction; +import org.apache.cassandra.db.partitions.PartitionUpdate; +import org.apache.cassandra.db.rows.UnfilteredRowIterator; +import org.apache.cassandra.db.view.View; import org.apache.cassandra.dht.Bounds; import org.apache.cassandra.dht.Token; +import org.apache.cassandra.io.sstable.ISSTableScanner; +import org.apache.cassandra.io.sstable.SSTableMultiWriter; import org.apache.cassandra.io.sstable.format.SSTableReader; -import org.apache.cassandra.io.sstable.format.SSTableWriter; import org.apache.cassandra.utils.JVMStabilityInspector; import org.apache.cassandra.utils.Pair; - ++import org.apache.cassandra.utils.Throwables; import org.apache.cassandra.utils.concurre
[06/16] cassandra git commit: Merge commit '43c741e251102bf5651ff8aa1b5ca078eb0ddc0b' into cassandra-3.0
Merge commit '43c741e251102bf5651ff8aa1b5ca078eb0ddc0b' into cassandra-3.0 * commit '43c741e251102bf5651ff8aa1b5ca078eb0ddc0b': Range tombstones that are masked by row tombstones should not be written out Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ed3b42d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ed3b42d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ed3b42d Branch: refs/heads/cassandra-3.9 Commit: 9ed3b42d3b50237f99485233857a7b34d5238d9a Parents: dd05e46 43c741e Author: Sylvain Lebresne Authored: Wed Jul 6 14:39:52 2016 +0200 Committer: Sylvain Lebresne Committed: Wed Jul 6 14:39:52 2016 +0200 -- --
[15/16] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.9
Merge branch 'cassandra-3.0' into cassandra-3.9 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59ee46e5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59ee46e5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59ee46e5 Branch: refs/heads/cassandra-3.9 Commit: 59ee46e55a15775a49edde86de81b9b79875731d Parents: 5ad1763 778f2a4 Author: Yuki Morishita Authored: Wed Jul 6 12:34:22 2016 -0500 Committer: Yuki Morishita Committed: Wed Jul 6 12:34:22 2016 -0500 -- CHANGES.txt | 1 + .../cassandra/streaming/ConnectionHandler.java | 8 ++-- .../cassandra/streaming/StreamReceiveTask.java | 50 +++- .../cassandra/streaming/StreamSession.java | 17 +-- .../streaming/StreamingTransferTest.java| 30 ++-- 5 files changed, 83 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/CHANGES.txt -- diff --cc CHANGES.txt index 2861cf7,8118de1..d459e34 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,7 -1,5 +1,8 @@@ -3.0.9 +3.9 + * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073) + * Increase size of flushExecutor thread pool (CASSANDRA-12071) +Merged from 3.0: + * Improve streaming synchronization and fault tolerance (CASSANDRA-11414) * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098) * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996) * Always select the live sstables when getting sstables in bounds (CASSANDRA-11944) http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/src/java/org/apache/cassandra/streaming/ConnectionHandler.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59ee46e5/src/java/org/apache/cassandra/streaming/StreamSession.java --
[01/16] cassandra git commit: Range tombstones that are masked by row tombstones should not be written out
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 43c741e25 -> 00e7ecf13
refs/heads/cassandra-3.0 9ed3b42d3 -> 778f2a46e
refs/heads/cassandra-3.9 5ad17634a -> 59ee46e55
refs/heads/trunk b4133f38d -> 9fd607778
Range tombstones that are masked by row tombstones should not be written out
patch by Nachiket Patil; reviewed by Sylvain Lebresne for CASSANDRA-12030
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/98f5f77b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/98f5f77b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/98f5f77b
Branch: refs/heads/cassandra-3.9
Commit: 98f5f77bb3c5d50e52cbb6f577a463ca8a5134ad
Parents: 3c1653f
Author: Nachiket Patil
Authored: Wed Jul 6 11:22:56 2016 +0200
Committer: Sylvain Lebresne
Committed: Wed Jul 6 14:35:10 2016 +0200
--
CHANGES.txt | 1 +
.../db/compaction/LazilyCompactedRow.java | 3 +-
.../apache/cassandra/db/RangeTombstoneTest.java | 40
3 files changed, 43 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/98f5f77b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b1dcbe1..7fa995d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.16
+ * Don't write shadowed range tombstone (CASSANDRA-12030)
* Fix filtering on clustering columns when 2i is used (CASSANDRA-11907)
* Reduce contention getting instances of CompositeType (CASSANDRA-10433)
* Improve digest calculation in the presence of overlapping tombstones
(CASSANDRA-11349)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/98f5f77b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
index f912da2..dab5eeb 100644
--- a/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java
@@ -286,7 +286,8 @@ public class LazilyCompactedRow extends AbstractCompactedRow
RangeTombstone t = tombstone;
tombstone = null;
-if (t.data.isGcAble(controller.gcBefore) && t.timestamp() <
getMaxPurgeableTimestamp())
+if (t.data.isGcAble(controller.gcBefore) && t.timestamp() <
getMaxPurgeableTimestamp() ||
+maxRowTombstone.markedForDeleteAt >= t.timestamp())
{
indexBuilder.tombstoneTracker().update(t, true);
return null;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/98f5f77b/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
b/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
index 3292422..dfd6960 100644
--- a/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
+++ b/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java
@@ -39,6 +39,7 @@ import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.config.IndexType;
import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.db.compaction.LeveledCompactionStrategy;
import org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy;
import org.apache.cassandra.db.composites.CellName;
import org.apache.cassandra.db.composites.CellNames;
@@ -543,6 +544,45 @@ public class RangeTombstoneTest extends SchemaLoader
}
@Test
+public void testCompactionOfRangeTombstonesCoveredByRowTombstone() throws
Exception
+{
+long testTimeStamp = 1451606400L; // 01/01/2016 : 00:00:00 GMT
+Keyspace table = Keyspace.open(KSNAME);
+ColumnFamilyStore cfs = table.getColumnFamilyStore(CFNAME);
+ByteBuffer key = ByteBufferUtil.bytes("k4");
+
+// remove any existing sstables before starting
+cfs.truncateBlocking();
+cfs.disableAutoCompaction();
+
cfs.setCompactionStrategyClass(LeveledCompactionStrategy.class.getCanonicalName());
+
+Mutation rm = new Mutation(KSNAME, key);
+for (int i = 1; i < 11; i += 2, testTimeStamp += i * 10)
+add(rm, i, testTimeStamp);
+rm.apply();
+cfs.forceBlockingFlush();
+
+rm = new Mutation(KSNAME, key);
+ColumnFamily cf = rm.addOrGet(CFNAME);
+
+// Write the covering row tombstone
+cf.delete(new DeletionTime(++testT
[10/16] cassandra git commit: Improve streaming synchronization and fault tolerance
Improve streaming synchronization and fault tolerance
Patch by Paulo Motta; Reviewed by yukim for CASSANDRA-11414
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00e7ecf1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00e7ecf1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00e7ecf1
Branch: refs/heads/cassandra-2.2
Commit: 00e7ecf1394f8704e2f13369f7950e129459ce2c
Parents: 43c741e
Author: Paulo Motta
Authored: Wed Jul 6 12:16:16 2016 -0500
Committer: Yuki Morishita
Committed: Wed Jul 6 12:32:39 2016 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/streaming/ConnectionHandler.java| 8 +++-
.../org/apache/cassandra/streaming/StreamReceiveTask.java| 2 --
3 files changed, 4 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bfd8aa2..7d62f97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.8
+ * Improve streaming synchronization and fault tolerance (CASSANDRA-11414)
* MemoryUtil.getShort() should return an unsigned short also for
architectures not supporting unaligned memory accesses (CASSANDRA-11973)
Merged from 2.1:
* Don't write shadowed range tombstone (CASSANDRA-12030)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index c497a39..364435e 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -233,6 +233,9 @@ public class ConnectionHandler
protected void signalCloseDone()
{
+if (closeFuture == null)
+close();
+
closeFuture.get().set(null);
// We can now close the socket
@@ -294,11 +297,6 @@ public class ConnectionHandler
}
}
}
-catch (SocketException e)
-{
-// socket is closed
-close();
-}
catch (Throwable t)
{
JVMStabilityInspector.inspectThrowable(t);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/00e7ecf1/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
--
diff --git a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
index 6911ec6..b342edc 100644
--- a/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
+++ b/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java
@@ -18,8 +18,6 @@
package org.apache.cassandra.streaming;
import java.io.File;
-import java.io.IOError;
-import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364833#comment-15364833
]
Stanislav Vishnevskiy commented on CASSANDRA-12144:
---
Yup for some rows somehow it manages to have multiple rows for the same primary
key. Only old rows it seems that existed prior to the upgrade.
We followed the steps outlines in this document.
https://docs.datastax.com/en/latest-upgrade/upgrade/cassandra/upgrdBestPractCassandra.html
Which included running upgradesstables before and after the upgrade.
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12105) ThriftServer.stop is not thread safe
[ https://issues.apache.org/jira/browse/CASSANDRA-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-12105: -- Assignee: Brian Wawok Status: Patch Available (was: Open) > ThriftServer.stop is not thread safe > > > Key: CASSANDRA-12105 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12105 > Project: Cassandra > Issue Type: Bug >Reporter: Brian Wawok >Assignee: Brian Wawok >Priority: Critical > Fix For: 2.1.x > > Attachments: patch1.txt, patch2.txt > > > There is a small thread safety issue in ThriftServer.stop(). If we have > multiple calls to stop, one thread may NPE or otherwise do bad stuff. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364824#comment-15364824
]
Tyler Hobbs commented on CASSANDRA-12144:
-
Hmm, so there are multiple rows with the same primary key. Have you run
upgradesstables yet?
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11315) Upgrade from 2.2.6 to 3.0.5 Fails with AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-11315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-11315:
--
Reproduced In: 3.0.7, 3.0.5, 3.0.4, 3.0.3 (was: 3.0.3, 3.0.4, 3.0.5, 3.0.7)
Status: Patch Available (was: In Progress)
Working on dtest coverage for this with [~rhatch] - also for CASSANDRA-12147
and CASSANDRA-12023 (all of similar structure). The patch is ready for review
though.
> Upgrade from 2.2.6 to 3.0.5 Fails with AssertionError
> -
>
> Key: CASSANDRA-11315
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11315
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
> Environment: Ubuntu 14.04, Oracle Java 8, Apache Cassandra 2.2.5 ->
> 3.0.3, Apache Cassandra 2.2.6 -> 3.0.5
>Reporter: Dominik Keil
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> Hi,
> when trying to upgrade our development cluster from C* 2.2.5 to 3.0.3
> Cassandra fails during startup.
> Here's the relevant log snippet:
> {noformat}
> [...]
> INFO [main] 2016-03-08 11:42:01,291 ColumnFamilyStore.java:381 -
> Initializing system.schema_triggers
> INFO [main] 2016-03-08 11:42:01,302 ColumnFamilyStore.java:381 -
> Initializing system.schema_usertypes
> INFO [main] 2016-03-08 11:42:01,313 ColumnFamilyStore.java:381 -
> Initializing system.schema_functions
> INFO [main] 2016-03-08 11:42:01,324 ColumnFamilyStore.java:381 -
> Initializing system.schema_aggregates
> INFO [main] 2016-03-08 11:42:01,576 SystemKeyspace.java:1284 - Detected
> version upgrade from 2.2.5 to 3.0.3, snapshotting system keyspace
> WARN [main] 2016-03-08 11:42:01,911 CompressionParams.java:382 - The
> sstable_compression option has been deprecated. You should use class instead
> WARN [main] 2016-03-08 11:42:01,959 CompressionParams.java:333 - The
> chunk_length_kb option has been deprecated. You should use chunk_length_in_kb
> instead
> ERROR [main] 2016-03-08 11:42:02,638 CassandraDaemon.java:692 - Exception
> encountered during startup
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.CompactTables.getCompactValueColumn(CompactTables.java:90)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:315)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:291)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at org.apache.cassandra.config.CFMetaData.create(CFMetaData.java:367)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableMetadata(LegacySchemaMigrator.java:337)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.readTableMetadata(LegacySchemaMigrator.java:273)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.readTable(LegacySchemaMigrator.java:244)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readTables$227(LegacySchemaMigrator.java:237)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_74]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.readTables(LegacySchemaMigrator.java:237)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.readKeyspace(LegacySchemaMigrator.java:186)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readSchema$224(LegacySchemaMigrator.java:177)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_74]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.readSchema(LegacySchemaMigrator.java:177)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.schema.LegacySchemaMigrator.migrate(LegacySchemaMigrator.java:77)
> ~[apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:223)
> [apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
> [apache-cassandra-3.0.3.jar:3.0.3]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:679)
> [apache-cassandra-3.0.3.jar:3.0.3]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364659#comment-15364659
]
Stanislav Vishnevskiy edited comment on CASSANDRA-12144 at 7/6/16 5:16 PM:
---
The writetime on the rows is in the GitHub gist.
{noformat}
cqlsh> SELECT WRITETIME(since), WRITETIME(type) FROM
discord_relationships.relationships WHERE user_id = 116138050710536192 AND id
= 153047019424972800;
writetime(since) | writetime(type)
--+--
1464619988173052 | 1464619988173052
{noformat}
So that was written on May 30th.
This cluster is half a year old and exhibited zero issues until yesterday
pretty much right after the upgrade finished. We also are noticing a weird key
cache hit rate right from when the upgrade finished.
http://i.imgur.com/JDihdGO.png
The schema is as follows.
{noformat}
CREATE KEYSPACE discord_relationships WITH replication = {'class':
'NetworkTopologyStrategy', 'us-east1': '3'} AND durable_writes = true;
CREATE TABLE discord_relationships.relationships (
user_id bigint,
id bigint,
since timestamp,
type tinyint,
PRIMARY KEY (user_id, id)
) WITH CLUSTERING ORDER BY (id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{noformat}
Thanks.
was (Author: stanislav):
The writetime on the rows is in the GitHub
{noformat}
cqlsh> SELECT WRITETIME(since), WRITETIME(type) FROM
discord_relationships.relationships WHERE user_id = 116138050710536192 AND id
= 153047019424972800;
writetime(since) | writetime(type)
--+--
1464619988173052 | 1464619988173052
{noformat}
So that was written on May 30th.
This cluster is half a year old and exhibited zero issues until yesterday
pretty much right after the upgrade finished. We also are noticing a weird key
cache hit rate right from when the upgrade finished.
http://i.imgur.com/JDihdGO.png
The schema is as follows.
{noformat}
CREATE KEYSPACE discord_relationships WITH replication = {'class':
'NetworkTopologyStrategy', 'us-east1': '3'} AND durable_writes = true;
CREATE TABLE discord_relationships.relationships (
user_id bigint,
id bigint,
since timestamp,
type tinyint,
PRIMARY KEY (user_id, id)
) WITH CLUSTERING ORDER BY (id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{noformat}
Thanks.
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364659#comment-15364659
]
Stanislav Vishnevskiy commented on CASSANDRA-12144:
---
The writetime on the rows is in the GitHub
{noformat}
cqlsh> SELECT WRITETIME(since), WRITETIME(type) FROM
discord_relationships.relationships WHERE user_id = 116138050710536192 AND id
= 153047019424972800;
writetime(since) | writetime(type)
--+--
1464619988173052 | 1464619988173052
{noformat}
So that was written on May 30th.
This cluster is half a year old and exhibited zero issues until yesterday
pretty much right after the upgrade finished. We also are noticing a weird key
cache hit rate right from when the upgrade finished.
http://i.imgur.com/JDihdGO.png
The schema is as follows.
{noformat}
CREATE KEYSPACE discord_relationships WITH replication = {'class':
'NetworkTopologyStrategy', 'us-east1': '3'} AND durable_writes = true;
CREATE TABLE discord_relationships.relationships (
user_id bigint,
id bigint,
since timestamp,
type tinyint,
PRIMARY KEY (user_id, id)
) WITH CLUSTERING ORDER BY (id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{noformat}
Thanks.
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12135) dtest failure in repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364640#comment-15364640
]
Michael Shuler commented on CASSANDRA-12135:
Not sure, but I can't rule it out.
> dtest failure in
> repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test
>
>
> Key: CASSANDRA-12135
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12135
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
> Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/240/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test
> Failed on CassCI build cassandra-2.1_dtest_jdk8 #240
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File
> "/home/automaton/cassandra-dtest/repair_tests/incremental_repair_test.py",
> line 42, in sstable_marking_test
> node3.stop(gently=True)
> File "/home/automaton/ccm/ccmlib/node.py", line 701, in stop
> raise NodeError("Problem stopping node %s" % self.name)
> "Problem stopping node node3
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12147) Static thrift tables with non UTF8Type comparators can have column names converted incorrectly
[
https://issues.apache.org/jira/browse/CASSANDRA-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-12147:
--
Status: Patch Available (was: Open)
> Static thrift tables with non UTF8Type comparators can have column names
> converted incorrectly
> --
>
> Key: CASSANDRA-12147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12147
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.8, 3.0.x
>
>
> {{CompactTables::columnDefinitionComparator()}} has been broken since
> CASSANDRA-8099 for non-super columnfamilies, if the comparator is not
> {{UTF8Type}}. This results in being unable to read some pre-existing 2.x data
> post upgrade (it's not lost, but becomes inaccessible).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12147) Static thrift tables with non UTF8Type comparators can have column names converted incorrectly
[
https://issues.apache.org/jira/browse/CASSANDRA-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364630#comment-15364630
]
Aleksey Yeschenko commented on CASSANDRA-12147:
---
||branch||testall||dtest||
|[12147-3.0|https://github.com/iamaleksey/cassandra/tree/12147-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12147-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12147-3.0-dtest]|
|[12147-3.9|https://github.com/iamaleksey/cassandra/tree/12147-3.9]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12147-3.9-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12147-3.9-dtest]|
> Static thrift tables with non UTF8Type comparators can have column names
> converted incorrectly
> --
>
> Key: CASSANDRA-12147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12147
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.8, 3.0.x
>
>
> {{CompactTables::columnDefinitionComparator()}} has been broken since
> CASSANDRA-8099 for non-super columnfamilies, if the comparator is not
> {{UTF8Type}}. This results in being unable to read some pre-existing 2.x data
> post upgrade (it's not lost, but becomes inaccessible).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7900) Snapshot tests are not platform neutral
[ https://issues.apache.org/jira/browse/CASSANDRA-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-7900: --- Assignee: DS Test Eng (was: Philip Thompson) > Snapshot tests are not platform neutral > --- > > Key: CASSANDRA-7900 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7900 > Project: Cassandra > Issue Type: Test > Environment: Windows >Reporter: Philip Thompson >Assignee: DS Test Eng > Labels: Windows > > The various snapshot and commit log archiving tests in snapshot_test.py are > failing on Windows platforms. This appears to be the fault of test behavior > due to extensive operations in the file system that fail to consider the test > may be running on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8036) Add dtest for ipv6 functionality
[ https://issues.apache.org/jira/browse/CASSANDRA-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-8036: --- Assignee: DS Test Eng (was: Philip Thompson) > Add dtest for ipv6 functionality > > > Key: CASSANDRA-8036 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8036 > Project: Cassandra > Issue Type: Test >Reporter: Philip Thompson >Assignee: DS Test Eng > > Cassandra can run with ipv6 addresses, and cqlsh should be able to connect > via ipv6. We need a dtest to verify this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8202) Live Size Tests
[ https://issues.apache.org/jira/browse/CASSANDRA-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-8202: --- Assignee: DS Test Eng (was: Philip Thompson) > Live Size Tests > --- > > Key: CASSANDRA-8202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8202 > Project: Cassandra > Issue Type: Test >Reporter: Philip Thompson >Assignee: DS Test Eng > > 2.1.X has seen more than a handful of tickets related to incorrectly > computing live size. We need to expand our coverage of tests on this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-12147) Static thrift tables with non UTF8Type comparators can have column names converted incorrectly
Aleksey Yeschenko created CASSANDRA-12147:
-
Summary: Static thrift tables with non UTF8Type comparators can
have column names converted incorrectly
Key: CASSANDRA-12147
URL: https://issues.apache.org/jira/browse/CASSANDRA-12147
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 3.8, 3.0.x
{{CompactTables::columnDefinitionComparator()}} has been broken since
CASSANDRA-8099 for non-super columnfamilies, if the comparator is not
{{UTF8Type}}. This results in being unable to read some pre-existing 2.x data
post upgrade (it's not lost, but becomes inaccessible).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12139) dtest failure in snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlog
[
https://issues.apache.org/jira/browse/CASSANDRA-12139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson resolved CASSANDRA-12139.
-
Resolution: Cannot Reproduce
Only failure was the netty bug. Closing
> dtest failure in
> snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlog
> ---
>
> Key: CASSANDRA-12139
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12139
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/409/testReport/snapshot_test/TestArchiveCommitlog/dont_test_archive_commitlog
> Failed on CassCI build trunk_novnode_dtest #409
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/snapshot_test.py", line 168, in
> dont_test_archive_commitlog
> self.run_archive_commitlog(restore_point_in_time=False,
> restore_archived_commitlog=False)
> File "/home/automaton/cassandra-dtest/snapshot_test.py", line 279, in
> run_archive_commitlog
> set())
> File "/usr/lib/python2.7/unittest/case.py", line 522, in assertNotEqual
> raise self.failureException(msg)
> "set([]) == set([])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12068) dtest failure in paging_test.TestPagingData.static_columns_paging_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson resolved CASSANDRA-12068.
-
Resolution: Cannot Reproduce
> dtest failure in paging_test.TestPagingData.static_columns_paging_test
> --
>
> Key: CASSANDRA-12068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12068
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/261/testReport/paging_test/TestPagingData/static_columns_paging_test
> Failed on CassCI build trunk_offheap_dtest #261
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
> f(obj)
> File "/home/automaton/cassandra-dtest/paging_test.py", line 715, in
> static_columns_paging_test
> self.assertEqual(16, len(results))
> File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
> File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> "16 != 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12068) dtest failure in paging_test.TestPagingData.static_columns_paging_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364616#comment-15364616
]
Philip Thompson commented on CASSANDRA-12068:
-
Didn't repro after 500 runs, closing.
> dtest failure in paging_test.TestPagingData.static_columns_paging_test
> --
>
> Key: CASSANDRA-12068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12068
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Philip Thompson
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/261/testReport/paging_test/TestPagingData/static_columns_paging_test
> Failed on CassCI build trunk_offheap_dtest #261
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
> f(obj)
> File "/home/automaton/cassandra-dtest/paging_test.py", line 715, in
> static_columns_paging_test
> self.assertEqual(16, len(results))
> File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
> File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> "16 != 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10992) Hanging streaming sessions
[
https://issues.apache.org/jira/browse/CASSANDRA-10992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364617#comment-15364617
]
Yuki Morishita commented on CASSANDRA-10992:
bq. I wonder if we should disable retry altogether because I think that in its
current state it brings more pain than benefits.
I agree that retry in stream is not considered well enough, so I am ok if we
remove it.
> Hanging streaming sessions
> --
>
> Key: CASSANDRA-10992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10992
> Project: Cassandra
> Issue Type: Bug
> Environment: C* 2.1.12, Debian Wheezy
>Reporter: mlowicki
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
> Attachments: apache-cassandra-2.1.12-SNAPSHOT.jar, db1.ams.jstack,
> db6.analytics.jstack
>
>
> I've started recently running repair using [Cassandra
> Reaper|https://github.com/spotify/cassandra-reaper] (built-in {{nodetool
> repair}} doesn't work for me - CASSANDRA-9935). It behaves fine but I've
> noticed hanging streaming sessions:
> {code}
> root@db1:~# date
> Sat Jan 9 16:43:00 UTC 2016
> root@db1:~# nt netstats -H | grep total
> Receiving 5 files, 46.59 MB total. Already received 1 files, 11.32 MB
> total
> Sending 7 files, 46.28 MB total. Already sent 7 files, 46.28 MB total
> Receiving 6 files, 64.15 MB total. Already received 1 files, 12.14 MB
> total
> Sending 5 files, 61.15 MB total. Already sent 5 files, 61.15 MB total
> Receiving 4 files, 7.75 MB total. Already received 3 files, 7.58 MB
> total
> Sending 4 files, 4.29 MB total. Already sent 4 files, 4.29 MB total
> Receiving 12 files, 13.79 MB total. Already received 11 files, 7.66
> MB total
> Sending 5 files, 15.32 MB total. Already sent 5 files, 15.32 MB total
> Receiving 8 files, 20.35 MB total. Already received 1 files, 13.63 MB
> total
> Sending 38 files, 125.34 MB total. Already sent 38 files, 125.34 MB
> total
> root@db1:~# date
> Sat Jan 9 17:45:42 UTC 2016
> root@db1:~# nt netstats -H | grep total
> Receiving 5 files, 46.59 MB total. Already received 1 files, 11.32 MB
> total
> Sending 7 files, 46.28 MB total. Already sent 7 files, 46.28 MB total
> Receiving 6 files, 64.15 MB total. Already received 1 files, 12.14 MB
> total
> Sending 5 files, 61.15 MB total. Already sent 5 files, 61.15 MB total
> Receiving 4 files, 7.75 MB total. Already received 3 files, 7.58 MB
> total
> Sending 4 files, 4.29 MB total. Already sent 4 files, 4.29 MB total
> Receiving 12 files, 13.79 MB total. Already received 11 files, 7.66
> MB total
> Sending 5 files, 15.32 MB total. Already sent 5 files, 15.32 MB total
> Receiving 8 files, 20.35 MB total. Already received 1 files, 13.63 MB
> total
> Sending 38 files, 125.34 MB total. Already sent 38 files, 125.34 MB
> total
> {code}
> Such sessions are left even when repair job is long time done (confirmed by
> checking Reaper's and Cassandra's logs). {{streaming_socket_timeout_in_ms}}
> in cassandra.yaml is set to default value (360).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12132) two cassandra nodes have common records,we need to calculate it
[ https://issues.apache.org/jira/browse/CASSANDRA-12132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364600#comment-15364600 ] Tyler Hobbs commented on CASSANDRA-12132: - Basically, there should be a metric that exposes how many token ranges have one replica down, how many have two replicas down, etc? > two cassandra nodes have common records,we need to calculate it > --- > > Key: CASSANDRA-12132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12132 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: stone > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12011) Logback shutdown hook races with StorageServiceShutdownHook
[
https://issues.apache.org/jira/browse/CASSANDRA-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-12011:
---
Assignee: (was: Benjamin Lerer)
> Logback shutdown hook races with StorageServiceShutdownHook
> ---
>
> Key: CASSANDRA-12011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12011
> Project: Cassandra
> Issue Type: Bug
>Reporter: Joel Knighton
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> In {{StorageService}}, we add a JVM shutdown hook called
> {{StorageServiceShutdownHook}} that shuts down gossip and flushes writes for
> non durable keyspaces, among other things.
> In {{logback.xml}}, we add a JVM shutdown hook that tears down logback
> resources.
> Since JVM shutdown hooks are started concurrently, the logback shutdown hook
> almost always completes during the {{StorageServiceShutdownHook}}, which
> means that any log statements in this hook will not work since the logback
> resources have been torn down.
> This makes debugging the {{StorageServiceShutdownHook}} challenging. We
> should do our own logback teardown that does not run until after the
> completion of any Cassandra-oriented JVM shutdown hooks such as the
> {{StorageServiceShutdownHook}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9377) Untested commit log code found via code coverage
[
https://issues.apache.org/jira/browse/CASSANDRA-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-9377:
--
Assignee: (was: Benjamin Lerer)
> Untested commit log code found via code coverage
>
>
> Key: CASSANDRA-9377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9377
> Project: Cassandra
> Issue Type: Test
>Reporter: Ariel Weisberg
> Attachments: jacoco.tgz
>
>
> It took some doing but I was finally able to extract coverage for just the
> unit tests that test the commit log. Attached is the jacoco output as well as
> the build.xml I used to get test-compression and test to run just the commit
> log tests.
> This includes
> {noformat}
> CommitLogTest
> CommitLogFailurePolicyTest
> RecoveryManagerTest
> RecoveryManager2Test
> RecoveryManager3Test
> CommitLogStressTest
> {noformat}
> All tests were run with and without test-compression.
> Coverage is pretty good for some things with the missing coverage being
> exceptional paths for things like files that aren't doing anything
> exceptional in the tests.
> ReplayPosition implements equals and hashCode but has no coverage.
> CommitLogSegment.waitForFinalSync has no coverage.
> CommitLogDescriptor.fromFileName and fromHeader. CommitLogDescriptor
> implements several equals methods that are not fully tested and also doesn't
> implement hashCode to match the equality changes.
> CommitLog does not cover handleCommitError, nor forceRecyle*
> CommitLogReplayer is not well off. Not worth enumerating the issues just a
> lot of error handling that is untested.
> CommitLogArchiver is in poor shape with no coverage for maybeRestoreArchive().
> CommitLogSegmentManager has a few important looking functions with 0 coverage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10715) Allow filtering on NULL
[
https://issues.apache.org/jira/browse/CASSANDRA-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-10715:
---
Assignee: Alex Petrov (was: Benjamin Lerer)
> Allow filtering on NULL
> ---
>
> Key: CASSANDRA-10715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10715
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
> Environment: C* 3.0.0 | cqlsh | C# driver 3.0.0beta2 | Windows 2012 R2
>Reporter: Kishan Karunaratne
>Assignee: Alex Petrov
>Priority: Minor
> Labels: protocolv5
> Fix For: 4.x
>
> Attachments:
> 0001-Allow-null-values-in-filtered-searches-reuse-Operato.patch
>
>
> This is an issue I first noticed through the C# driver, but I was able to
> repro on cqlsh, leading me to believe this is a Cassandra bug.
> Given the following schema:
> {noformat}
> CREATE TABLE "TestKeySpace_4928dc892922"."coolMovies" (
> unique_movie_title text,
> movie_maker text,
> director text,
> list list,
> "mainGuy" text,
> "yearMade" int,
> PRIMARY KEY ((unique_movie_title, movie_maker), director)
> ) WITH CLUSTERING ORDER BY (director ASC)
> {noformat}
> Executing a SELECT with FILTERING on a non-PK column, using a NULL as the
> argument:
> {noformat}
> SELECT "mainGuy", "movie_maker", "unique_movie_title", "list", "director",
> "yearMade" FROM "coolMovies" WHERE "mainGuy" = null ALLOW FILTERING
> {noformat}
> returns a ReadFailure exception:
> {noformat}
> cqlsh:TestKeySpace_4c8f2cf8d5cc> SELECT "mainGuy", "movie_maker",
> "unique_movie_title", "list", "director", "yearMade" FROM "coolMovies" WHERE
> "mainGuy" = null ALLOW FILTERING;
> ←[0;1;31mTraceback (most recent call last):
> File "C:\Users\Kishan\.ccm\repository\3.0.0\bin\\cqlsh.py", line 1216, in
> perform_simple_statement
> result = future.result()
> File
> "C:\Users\Kishan\.ccm\repository\3.0.0\bin\..\lib\cassandra-driver-internal-only-3.0.0a3.post0-3f15725.zip\cassandra-driver-3.0.0a3.post0-3f15725\cassandra\cluster.py",
> line 3118, in result
> raise self._final_exception
> ReadFailure: code=1300 [Replica(s) failed to execute read] message="Operation
> failed - received 0 responses and 1 failures" info={'failures': 1,
> 'received_responses': 0, 'required_responses': 1, 'cons
> istency': 'ONE'}
> ←[0m
> {noformat}
> Cassandra log shows:
> {noformat}
> WARN [SharedPool-Worker-2] 2015-11-16 13:51:00,259
> AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-2,10,main]: {}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:581)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:243)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.transform.BaseRows.applyOne(BaseRows.java:95)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at org.apache.cassandra.db.transform.BaseRows.add(BaseRows.java:86)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.transform.UnfilteredRows.add(UnfilteredRows.java:21)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.transform.Transformation.add(Transformation.java:136)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.transform.Transformation.apply(Transformation.java:102)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:233)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:227)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:293)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65)
> ~[apache-cassandra-3.0.0.jar:3.0.0]
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:288)
> ~[apache-cassandra-3.0.0.jar:3.
[jira] [Commented] (CASSANDRA-12051) JSON does not take functions
[
https://issues.apache.org/jira/browse/CASSANDRA-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364595#comment-15364595
]
Tyler Hobbs commented on CASSANDRA-12051:
-
I'll also note that you can use a normal insert with the {{fromJson()}}
function on JSON fields and use functions for other columns. For example:
{noformat}
INSERT INTO mytable (id, ts) VALUES (fromJson(?), toTimestamp(now()));
{noformat}
> JSON does not take functions
>
>
> Key: CASSANDRA-12051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12051
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tianshi Wang
>
> toTimestamp(now()) does not work in JSON format.
> {code}
> cqlsh:ops> create table test (
>... id int,
>... ts timestamp,
>... primary key(id)
>... );
> cqlsh:ops> insert into test (id, ts) values (1, toTimestamp(now()));
> cqlsh:ops> select * from test;
> id | ts
> +-
> 1 | 2016-06-21 18:46:28.753000+
> (1 rows)
> cqlsh:ops> insert into test JSON '{"id":2,"ts":toTimestamp(now())}';
> InvalidRequest: code=2200 [Invalid query] message="Could not decode JSON
> string as a map: org.codehaus.jackson.JsonParseException: Unrecognized token
> 'toTimestamp': was expecting
> at [Source: java.io.StringReader@2da0329d; line: 1, column: 25]. (String
> was: {"id":2,"ts":toTimestamp(now())})"
> cqlsh:ops> insert into test JSON '{"id":2,"ts":"toTimestamp(now())"}';
> InvalidRequest: code=2200 [Invalid query] message="Error decoding JSON value
> for ts: Unable to coerce 'toTimestamp(now())' to a formatted date (long)"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10890) I am using centos 6. I upgraded from cassandra 2.0.8 to 3.0.1. when i run cqlsh, it shows 'cql shell requires python 2.7' . i installed python 2.7 but still my cql
[
https://issues.apache.org/jira/browse/CASSANDRA-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364584#comment-15364584
]
Alex Petrov commented on CASSANDRA-10890:
-
There might be a problem with packaging on centOS, although as far as I know
Cassandra team is not in control of centOS package.
If the operating system is still using the default python (which might be lower
or higher) by default, there's little we can do in that case as default
interpreter is picked up.
If there's an equivalent of debian {{update-alternatives}} on centOS, I'd
suggest trying that first.
> I am using centos 6. I upgraded from cassandra 2.0.8 to 3.0.1. when i run
> cqlsh, it shows 'cql shell requires python 2.7' . i installed python 2.7 but
> still my cql shell is not working.
> --
>
> Key: CASSANDRA-10890
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10890
> Project: Cassandra
> Issue Type: Test
> Components: CQL
> Environment: centOS 6
>Reporter: Naveen Achyuta
> Fix For: 3.0.x
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11414) dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-11414:
---
Status: Ready to Commit (was: Patch Available)
> dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
> --
>
> Key: CASSANDRA-11414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11414
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Philip Thompson
>Assignee: Paulo Motta
> Labels: dtest
> Fix For: 3.x
>
>
> Stress is failing to read back all data. We can see this output from the
> stress read
> {code}
> java.io.IOException: Operation x0 on key(s) [314c384f304f4c325030]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> java.io.IOException: Operation x0 on key(s) [33383438363931353131]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> FAILURE
> {code}
> Started happening with build 1075. Does not appear flaky on CI.
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1076/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test
> Failed on CassCI build trunk_dtest #1076
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11414) dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364571#comment-15364571
]
Yuki Morishita commented on CASSANDRA-11414:
Patch LGTM, +1.
> dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
> --
>
> Key: CASSANDRA-11414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11414
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Philip Thompson
>Assignee: Paulo Motta
> Labels: dtest
> Fix For: 3.x
>
>
> Stress is failing to read back all data. We can see this output from the
> stress read
> {code}
> java.io.IOException: Operation x0 on key(s) [314c384f304f4c325030]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> java.io.IOException: Operation x0 on key(s) [33383438363931353131]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> FAILURE
> {code}
> Started happening with build 1075. Does not appear flaky on CI.
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1076/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test
> Failed on CassCI build trunk_dtest #1076
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364569#comment-15364569
]
Tyler Hobbs commented on CASSANDRA-12144:
-
Can you also paste the schema for that table?
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-10786:
Issue Type: Sub-task (was: Bug)
Parent: CASSANDRA-9362
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12144) Undeletable rows after upgrading from 2.2.4 to 3.0.7
[
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-12144:
Description:
We upgraded our cluster today and now have a some rows that refuse to delete.
Here are some example traces.
https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
Even weirder.
Updating the row and querying it back results in 2 rows even though the id is
the clustering key.
{noformat}
user_id| id | since| type
---++--+--
116138050710536192 | 153047019424972800 | null |0
116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
{noformat}
And then deleting it again only removes the new one.
{noformat}
cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
116138050710536192 AND id = 153047019424972800;
cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
116138050710536192 AND id = 153047019424972800;
user_id| id | since| type
++--+--
116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
{noformat}
We tried repairing, compacting, scrubbing. No Luck.
Not sure what to do. Is anyone aware of this?
was:
We upgraded our cluster today and now have a some rows that refuse to delete.
Here are some example traces.
https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
Even weirder.
Updating the row and querying it back results in 2 rows even though the id is
the clustering key.
user_id| id | since| type
++--+--
116138050710536192 | 153047019424972800 | null |0
116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
And then deleting it again only removes the new one.
cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
116138050710536192 AND id = 153047019424972800;
cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
116138050710536192 AND id = 153047019424972800;
user_id| id | since| type
++--+--
116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
We tried repairing, compacting, scrubbing. No Luck.
Not sure what to do. Is anyone aware of this?
> Undeletable rows after upgrading from 2.2.4 to 3.0.7
>
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id =
> 116138050710536192 AND id = 153047019424972800;
> user_id| id | since| type
> ++--+--
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12142) Add "beta" version native protocol flag
[
https://issues.apache.org/jira/browse/CASSANDRA-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-12142:
Issue Type: Sub-task (was: Improvement)
Parent: CASSANDRA-9362
> Add "beta" version native protocol flag
> ---
>
> Key: CASSANDRA-12142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12142
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Tyler Hobbs
> Labels: protocolv5
>
> As discussed in CASSANDRA-10786, we'd like to add a new flag to the native
> protocol to allow drivers to connect using a "beta" native protocol version.
> This would be used for native protocol versions that are still in development
> and may not have all of the final features. Without the "beta" flag, drivers
> will be prevented from using the protocol version.
> This is primarily useful for driver authors to start work against a new
> protocol version when the work on that spans multiple releases. Users would
> not generally be expected to utilize this flag, although it could potentially
> be used to offer early feedback on new protocol features.
> It seems like the {{STARTUP}} message body is the best place for the new beta
> flag. We may also considering adding protocol information to the
> {{SUPPORTED}} message as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10449) OOM on bootstrap after long GC pause
[
https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364543#comment-15364543
]
Jeff Jirsa commented on CASSANDRA-10449:
If anything, it's a combination of CASSANDRA-10474 and CASSANDRA-9681 (as
[~mishail] pointed out), and not CASSANDRA-10680.
> OOM on bootstrap after long GC pause
>
>
> Key: CASSANDRA-10449
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
> Project: Cassandra
> Issue Type: Bug
> Environment: Ubuntu 14.04, AWS
>Reporter: Robbie Strickland
> Labels: gc
> Fix For: 2.1.x
>
> Attachments: GCpath.txt, heap_dump.png, system.log.10-05,
> thread_dump.log, threads.txt
>
>
> I have a 20-node cluster (i2.4xlarge) with vnodes (default of 256) and
> 500-700GB per node. SSTable counts are <10 per table. I am attempting to
> provision additional nodes, but bootstrapping OOMs every time after about 10
> hours with a sudden long GC pause:
> {noformat}
> INFO [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old
> Generation GC in 1586126ms. G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380
> CassandraDaemon.java:223 - Exception in thread
> Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to
> no avail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11950) dtest failure in consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
[
https://issues.apache.org/jira/browse/CASSANDRA-11950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-11950:
Assignee: Stefania
> dtest failure in
> consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
> -
>
> Key: CASSANDRA-11950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11950
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Stefania
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log,
> node3.log, node3_debug.log, node4.log, node4_debug.log, node5.log,
> node5_debug.log, node6.log, node6_debug.log, node7.log, node7_debug.log,
> node8.log, node8_debug.log, node9.log, node9_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/10/testReport/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum
> Failed on CassCI build trunk_large_dtest #10
> Logs are attached.
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
> File "/home/automaton/cassandra-dtest/dtest.py", line 719, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> Standard Output
> Unexpected error in node3 log, error:
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,460 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,462 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,464 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,464 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-4] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-5] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-7] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-6] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11495) Rename of field in user defined type causes ArrayIndexOutOfBoundsException in select JSON *
[
https://issues.apache.org/jira/browse/CASSANDRA-11495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov resolved CASSANDRA-11495.
-
Resolution: Duplicate
This is a duplicate of [CASSANDRA-11146], validated using the following unit
test:
{code}
@Test
public void userTypeJsonTest() throws Throwable
{
String type = createType("CREATE TYPE %s (a int, b int)");
createTable("CREATE TABLE %s (a int PRIMARY KEY, b frozen<" + KEYSPACE
+ "." + type + ">)");
execute("INSERT INTO %s (a, b) VALUES(1, {a:1, b:1})");
execute("ALTER TYPE " + KEYSPACE + "." + type + " ADD c int");
execute("ALTER TYPE " + KEYSPACE + "." + type + " ADD d int");
assertRowsIgnoringOrder(execute("SELECT JSON * FROM %s"),
row("{\"a\": 1, \"b\": {\"a\": 1, \"b\": 1,
\"c\": null, \"d\": null}}"));
}
{code}
> Rename of field in user defined type causes ArrayIndexOutOfBoundsException in
> select JSON *
> ---
>
> Key: CASSANDRA-11495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11495
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Timothy Wang
>Assignee: Alex Petrov
> Attachments: cassandra_log_output.txt, error.cql
>
>
> Executing the attached error.cql causes
> java.lang.ArrayIndexOutOfBoundsException. This reproduces 100% of the time in
> my environment.
> nodetool -h localhost version
> objc[14803]: Class JavaLaunchHelper is implemented in both
> /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/bin/java and
> /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/libinstrument.dylib.
> One of the two will be used. Which one is undefined.
> ReleaseVersion: 3.3
> cqlsh -f error.cql
> [json]
> -
> {"list_id": "3ae31af7-e72b-11e5-9f1c-77e17196cbff", "card_id":
> "b2f059bb-ecc0-11e5-937c-4760eea11554", "owner_id":
> "8c02459b-dcaf-42dd-874a-41e19ba15075", "book_name": "Hop on Pop (I Can Read
> It All By Myself)", "publication_info": {"author": "Dr. Seuss", "publisher":
> "Beginner Books / Random House", "creative_roles": [], "publication_date":
> "1963-02-12"}}
> (1 rows)
> twangs-MacBook-Pro:migrations twang$ cqlsh -f error.cql
> peer | release_version
> --+-
> (0 rows)
> release_version
> -
> 3.3
> (1 rows)
> [json]
> -
> {"list_id": "3ae31af7-e72b-11e5-9f1c-77e17196cbff", "card_id":
> "b2f059bb-ecc0-11e5-937c-4760eea11554", "owner_id":
> "8c02459b-dcaf-42dd-874a-41e19ba15075", "book_name": "Hop on Pop (I Can Read
> It All By Myself)", "publication_info": {"author": "Dr. Seuss", "publisher":
> "Beginner Books / Random House", "creative_roles": [], "publication_date":
> "1963-02-12"}}
> (1 rows)
> error.cql:49:ServerError: message="java.lang.ArrayIndexOutOfBoundsException: 4">
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12141) dtest failure in consistency_test.TestConsistency.short_read_reversed_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12141:
Assignee: Yuki Morishita
> dtest failure in consistency_test.TestConsistency.short_read_reversed_test
> --
>
> Key: CASSANDRA-12141
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12141
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Yuki Morishita
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/280/testReport/consistency_test/TestConsistency/short_read_reversed_test
> Failed on CassCI build trunk_offheap_dtest #280
> {code}
> Standard Output
> Unexpected error in node2 log, error:
> ERROR [epollEventLoopGroup-2-5] 2016-06-27 19:14:54,412 Slf4JLogger.java:176
> - LEAK: ByteBuf.release() was not called before it's garbage-collected.
> Enable advanced leak reporting to find out where the leak occurred. To enable
> advanced leak reporting, specify the JVM option
> '-Dio.netty.leakDetection.level=advanced' or call
> ResourceLeakDetector.setLevel() See
> http://netty.io/wiki/reference-counted-objects.html for more information.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12142) Add "beta" version native protocol flag
[
https://issues.apache.org/jira/browse/CASSANDRA-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364532#comment-15364532
]
Tyler Hobbs commented on CASSANDRA-12142:
-
[~ifesdjeen] sure, feel free to take this.
bq. As for the flag itself, we probably should add it to each message "flags"
byte (2nd byte of the header) since the protocol version is more associated to
messages than connection in practice.
My thought was that we could avoid wasting the flag bit and simply validate
beta version connections at startup. However, I do agree that the message
header flags are a more natural location, so if you're not concerned about
using a flag bit for that, I'm +1.
> Add "beta" version native protocol flag
> ---
>
> Key: CASSANDRA-12142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12142
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tyler Hobbs
> Labels: protocolv5
>
> As discussed in CASSANDRA-10786, we'd like to add a new flag to the native
> protocol to allow drivers to connect using a "beta" native protocol version.
> This would be used for native protocol versions that are still in development
> and may not have all of the final features. Without the "beta" flag, drivers
> will be prevented from using the protocol version.
> This is primarily useful for driver authors to start work against a new
> protocol version when the work on that spans multiple releases. Users would
> not generally be expected to utilize this flag, although it could potentially
> be used to offer early feedback on new protocol features.
> It seems like the {{STARTUP}} message body is the best place for the new beta
> flag. We may also considering adding protocol information to the
> {{SUPPORTED}} message as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12140) dtest failure in materialized_views_test.TestMaterializedViews.add_write_survey_node_after_mv_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12140:
Assignee: Carl Yeksigian
> dtest failure in
> materialized_views_test.TestMaterializedViews.add_write_survey_node_after_mv_test
> --
>
> Key: CASSANDRA-12140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12140
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Carl Yeksigian
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log,
> node4.log, node4_debug.log, node4_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/409/testReport/materialized_views_test/TestMaterializedViews/add_write_survey_node_after_mv_test
> Failed on CassCI build trunk_novnode_dtest #409
> {code}
> Standard Output
> Unexpected error in node4 log, error:
> ERROR [main] 2016-06-27 15:28:13,139 MigrationManager.java:164 - Migration
> task failed to complete
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12097) dtest failure in materialized_views_test.TestMaterializedViews.view_tombstone_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12097:
Assignee: Carl Yeksigian
> dtest failure in
> materialized_views_test.TestMaterializedViews.view_tombstone_test
> --
>
> Key: CASSANDRA-12097
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12097
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Carl Yeksigian
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/271/testReport/materialized_views_test/TestMaterializedViews/view_tombstone_test
> Failed on CassCI build trunk_offheap_dtest #271
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/materialized_views_test.py", line
> 754, in view_tombstone_test
> [1, 1, 'b', 3.0]
> File "/home/automaton/cassandra-dtest/assertions.py", line 51, in assert_one
> assert list_res == [expected], "Expected %s from %s, but got %s" %
> ([expected], query, list_res)
> "Expected [[1, 1, 'b', 3.0]] from SELECT * FROM t_by_v WHERE v = 1, but got
> [[1, 1, u'b', None]]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12095) [windows] dtest failure in repair_tests.repair_test.TestRepair.no_anticompaction_after_hostspecific_repair_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364531#comment-15364531
]
Joshua McKenzie commented on CASSANDRA-12095:
-
Doubt this is Windows specific. See CASSANDRA-11960.
> [windows] dtest failure in
> repair_tests.repair_test.TestRepair.no_anticompaction_after_hostspecific_repair_test
> ---
>
> Key: CASSANDRA-12095
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12095
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
> Labels: dtest, windows
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log,
> node4.log, node4_debug.log, node4_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest_win32/447/testReport/repair_tests.repair_test/TestRepair/no_anticompaction_after_hostspecific_repair_test
> Failed on CassCI build trunk_dtest_win32 #447
> {code}
> Standard Output
> Unexpected error in node2 log, error:
> ERROR [HintsDispatcher:2] 2016-06-24 16:30:04,790 CassandraDaemon.java:219 -
> Exception in thread Thread[HintsDispatcher:2,1,main]
> java.lang.UnsupportedOperationException: Hints are not seekable.
> at org.apache.cassandra.hints.HintsReader.seek(HintsReader.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.hints.HintsDispatcher.seek(HintsDispatcher.java:79)
> ~[main/:na]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:257)
> ~[main/:na]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
> ~[main/:na]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
> ~[main/:na]
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_51]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12114) Cassandra startup takes an hour because of N*N operation
[
https://issues.apache.org/jira/browse/CASSANDRA-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-12114:
Reviewer: Joel Knighton
> Cassandra startup takes an hour because of N*N operation
>
>
> Key: CASSANDRA-12114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12114
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tom van der Woerdt
>Assignee: Jeff Jirsa
> Fix For: 3.0.x, 3.x
>
>
> (There's a previous version of this ticket, which was very wrong about the
> actual cause. Original is quoted below)
> In java.org.cassandra.db.ColumnFamilyStore, the function scrubDataDirectories
> loops over all sstables and then for each sstable it cleans temporary files
> from its directory.
> Since there are many sstables in a directory, this ends up cleaning the same
> directory many times.
> When using leveledcompactionstrategy on a data set that is ~4TB per node, you
> can easily end up with 200k files.
> Add N and N, and we get a N*N operation (scrubDataDirectories) which ends up
> taking an hour (or more).
> (At this point I should probably point out that no, I am not sure about that.
> At all. But I do know this takes an hour and jstack blames this function)
> As promised, original ticket below :
> {quote}
> A Cassandra cluster of ours has nodes with up to 4TB of data, in a single
> table using leveled compaction having 200k files. While upgrading from 2.2.6
> to 3.0.7 we noticed that it took a while to restart a node. And with "a
> while" I mean we measured it at more than 60 minutes.
> jstack shows something interesting :
> {code}
> "main" #1 prio=5 os_prio=0 tid=0x7f30db0ea400 nid=0xdb22 runnable
> [0x7f30de122000]
>java.lang.Thread.State: RUNNABLE
> at java.io.UnixFileSystem.list(Native Method)
> at java.io.File.list(File.java:1122)
> at java.io.File.listFiles(File.java:1248)
> at
> org.apache.cassandra.io.sstable.Descriptor.getTemporaryFiles(Descriptor.java:172)
> at
> org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:599)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:245)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:557)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:685)
> {code}
> Going by the source of File.listFiles, it puts every file in a directory into
> an array and *then* applies the filter.
> This is actually a known Java issue from 1999:
> http://bugs.java.com/view_bug.do?bug_id=4285834 -- their "solution" was to
> introduce new APIs in JRE7. I guess that makes listFiles deprecated for
> larger directories (like when using LeveledCompactionStrategy).
> tl;dr: because Cassandra uses java.io.File.listFiles, service startup can
> take an hour for larger data sets.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11495) Rename of field in user defined type causes ArrayIndexOutOfBoundsException in select JSON *
[
https://issues.apache.org/jira/browse/CASSANDRA-11495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov reassigned CASSANDRA-11495:
---
Assignee: Alex Petrov
> Rename of field in user defined type causes ArrayIndexOutOfBoundsException in
> select JSON *
> ---
>
> Key: CASSANDRA-11495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11495
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Timothy Wang
>Assignee: Alex Petrov
> Attachments: cassandra_log_output.txt, error.cql
>
>
> Executing the attached error.cql causes
> java.lang.ArrayIndexOutOfBoundsException. This reproduces 100% of the time in
> my environment.
> nodetool -h localhost version
> objc[14803]: Class JavaLaunchHelper is implemented in both
> /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/bin/java and
> /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/libinstrument.dylib.
> One of the two will be used. Which one is undefined.
> ReleaseVersion: 3.3
> cqlsh -f error.cql
> [json]
> -
> {"list_id": "3ae31af7-e72b-11e5-9f1c-77e17196cbff", "card_id":
> "b2f059bb-ecc0-11e5-937c-4760eea11554", "owner_id":
> "8c02459b-dcaf-42dd-874a-41e19ba15075", "book_name": "Hop on Pop (I Can Read
> It All By Myself)", "publication_info": {"author": "Dr. Seuss", "publisher":
> "Beginner Books / Random House", "creative_roles": [], "publication_date":
> "1963-02-12"}}
> (1 rows)
> twangs-MacBook-Pro:migrations twang$ cqlsh -f error.cql
> peer | release_version
> --+-
> (0 rows)
> release_version
> -
> 3.3
> (1 rows)
> [json]
> -
> {"list_id": "3ae31af7-e72b-11e5-9f1c-77e17196cbff", "card_id":
> "b2f059bb-ecc0-11e5-937c-4760eea11554", "owner_id":
> "8c02459b-dcaf-42dd-874a-41e19ba15075", "book_name": "Hop on Pop (I Can Read
> It All By Myself)", "publication_info": {"author": "Dr. Seuss", "publisher":
> "Beginner Books / Random House", "creative_roles": [], "publication_date":
> "1963-02-12"}}
> (1 rows)
> error.cql:49:ServerError: message="java.lang.ArrayIndexOutOfBoundsException: 4">
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator
[ https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364517#comment-15364517 ] Aleksey Yeschenko commented on CASSANDRA-9318: -- I'm not sure that putting anything like this in 3.0.x is reasonable, sorry. It's not technically a bug fix. Whether or not it should be enabled by default in 3.x is up for debate (would have to be an even feature release, too). > Bound the number of in-flight requests at the coordinator > - > > Key: CASSANDRA-9318 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9318 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths, Streaming and Messaging >Reporter: Ariel Weisberg >Assignee: Sergio Bossa > Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png, > limit.btm, no_backpressure.png > > > It's possible to somewhat bound the amount of load accepted into the cluster > by bounding the number of in-flight requests and request bytes. > An implementation might do something like track the number of outstanding > bytes and requests and if it reaches a high watermark disable read on client > connections until it goes back below some low watermark. > Need to make sure that disabling read on the client connection won't > introduce other issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12005) Out of memory error in MessagingService
[
https://issues.apache.org/jira/browse/CASSANDRA-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364484#comment-15364484
]
Chris Powell commented on CASSANDRA-12005:
--
Thanks for the input! I have made some configuration changes to Cassandra and I
am no longer experiencing the problem. (Knock on wood). I specifically have
reduced the number of `concurrent_compactors` and moved the
`memtable_allocation_type` to `offheap_objects`.
> Out of memory error in MessagingService
> ---
>
> Key: CASSANDRA-12005
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12005
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: Ubuntu 14.04.4 LTS 3.13.0-79-generic #123-Ubuntu SMP
> x86_64
> Cassandra ReleaseVersion: 2.2.5
> java version "1.8.0_31"
> Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
>Reporter: Chris Powell
>
> I am periodically loosing nodes due to the below OOM error. The nodes restart
> perfectly fine. It appears intermittent and randomly affects nodes. There are
> no other warnings or errors in the log files.
> I am using the {{GCG1}} with the following options:
> {quote}
> JVM_OPTS="$JVM_OPTS -XX:+UseG1GC"
> JVM_OPTS="$JVM_OPTS -XX:SurvivorRatio=8"
> JVM_OPTS="$JVM_OPTS -XX:MaxTenuringThreshold=1"
> JVM_OPTS="$JVM_OPTS -XX:+UseTLAB"
> JVM_OPTS="$JVM_OPTS -XX:+AlwaysPreTouch"
> JVM_OPTS="$JVM_OPTS -XX:-UseBiasedLocking"
> JVM_OPTS="$JVM_OPTS -XX:+ResizeTLAB"
> JVM_OPTS="$JVM_OPTS -XX:MaxGCPauseMillis=500"
> JVM_OPTS="$JVM_OPTS -XX:G1RSetUpdatingPauseTimePercent=10"
> JVM_OPTS="$JVM_OPTS -XX:InitiatingHeapOccupancyPercent=25"
> {quote}
> ERROR [MessagingService-Incoming-/10.184.11.109] 2016-06-14 13:00:20,237
> CassandraDaemon.java:185 - Exception in thread
> Thread[MessagingService-Incoming-/10.184.11.109,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:361)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:322)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:126)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:109)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:101)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:109)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:322)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:302)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:330)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:272)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:200)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:177)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:91)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> ERROR [MessagingService-Incoming-/10.184.11.109] 2016-06-14 13:00:20,239
> JVMStabilityInspector.java:117 - JVM state determined to be unstable.
> Exiting forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:361)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:322)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:126)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:109)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:101)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.d
[jira] [Commented] (CASSANDRA-11752) histograms/metrics in 2.2 do not appear recency biased
[ https://issues.apache.org/jira/browse/CASSANDRA-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364475#comment-15364475 ] T Jake Luciani commented on CASSANDRA-11752: Thanks for the patch [~eperott] I'm a little concerned about the impact of the lock every 30 minutes. Do you think buffering during the period of locking would be possible to not interrupt the active workload? Where did the 30 minute interval come from? (i.e why not 60) how does the decay look when plotting the quantiles before and after the interval? I'll try running some stress tests and see if there is any meaningful impact as well as try and generate some charts from reporting live data (unless you have some) > histograms/metrics in 2.2 do not appear recency biased > -- > > Key: CASSANDRA-11752 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11752 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Chris Burroughs >Assignee: Per Otterström > Labels: metrics > Fix For: 2.2.8 > > Attachments: 11752-2.2.txt, boost-metrics.png, > c-jconsole-comparison.png, c-metrics.png, default-histogram.png > > > In addition to upgrading to metrics3, CASSANDRA-5657 switched to using a > custom histogram implementation. After upgrading to Cassandra 2.2 > histograms/timer metrics are not suspiciously flat. To be useful for > graphing and alerting metrics need to be biased towards recent events. > I have attached images that I think illustrate this. > * The first two are a comparison between latency observed by a C* 2.2 (us) > cluster shoring very flat lines and a client (using metrics 2.2.0, ms) > showing server performance problems. We can't rule out with total certainty > that something else isn't the cause (that's why we measure from both the > client & server) but they very rarely disagree. > * The 3rd image compares jconsole viewing of metrics on a 2.2 and 2.1 > cluster over several minutes. Not a single digit changed on the 2.2 cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
