[jira] [Assigned] (CASSANDRA-13356) BootstrapMonitor.progress does not store all error messages

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-13356:
--

Assignee: Hao Zhong

> BootstrapMonitor.progress does not store all error messages
> ---
>
> Key: CASSANDRA-13356
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13356
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Hao Zhong
>Assignee: Hao Zhong
> Fix For: 4.x
>
>
> The BootstrapMonitor.progress ignores error messages when for 
> ProgressEventType.ERROR. Indeed, RepairRunner.progress once had a similar 
> bug, but is fixed. The fixed code is:
> {code}
>  public void progress(String tag, ProgressEvent event)
> {
> ProgressEventType type = event.getType();
> String message = String.format("[%s] %s", 
> format.format(System.currentTimeMillis()), event.getMessage());
> if (type == ProgressEventType.PROGRESS)
> {
> message = message + " (progress: " + 
> (int)event.getProgressPercentage() + "%)";
> }
> out.println(message);
> if (type == ProgressEventType.ERROR)
> {
> error = new RuntimeException("Repair job has failed with the 
> error message: " + message);
> }
> if (type == ProgressEventType.COMPLETE)
> {
> condition.signalAll();
> }
> }
> {code}
> Please refer to CASSANDRA-12508 for details.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13357) A possible NPE

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-13357:
--

Assignee: Hao Zhong

> A possible NPE
> --
>
> Key: CASSANDRA-13357
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13357
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Hao Zhong
>Assignee: Hao Zhong
> Fix For: 4.x
>
>
> The GetEndpoints.execute method has the following code:
> {code:title=GetEndpoints.java|borderStyle=solid}
>List endpoints = probe.getEndpoints(ks, table, key);
> for (InetAddress endpoint : endpoints)
> {
> System.out.println(endpoint.getHostAddress());
> }
> {code}
> This code can throw NPE. A similar bug is fixed in CASSANDRA-8950. The buggy 
> code  is 
> {code:title=NodeCmd.java|borderStyle=solid}
>   List endpoints = this.probe.getEndpoints(keySpace, cf, key);
> for (InetAddress anEndpoint : endpoints)
> {
>output.println(anEndpoint.getHostAddress());
> }
> {code}
> The fixed code is:
> {code:title=NodeCmd.java|borderStyle=solid}
> try
> {
> List endpoints = probe.getEndpoints(keySpace, cf, 
> key);
> for (InetAddress anEndpoint : endpoints)
>output.println(anEndpoint.getHostAddress());
> }
> catch (IllegalArgumentException ex)
> {
> output.println(ex.getMessage());
> probe.failed();
> }
> {code}
> The GetEndpoints.execute method shall be modified as CASSANDRA-8950 does.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13356) BootstrapMonitor.progress does not store all error messages

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934113#comment-15934113
 ] 

Jeff Jirsa commented on CASSANDRA-13356:


Hello [~haozhong] - thanks for the bug report. Are you able to follow the 
procedure defined in the 
[documentation|http://cassandra.apache.org/doc/latest/development/patches.html#creating-a-patch]
 for creating a patch? We typically need a patch for each impacted branch, and 
then a reviewer can run unit tests for you and commit.


> BootstrapMonitor.progress does not store all error messages
> ---
>
> Key: CASSANDRA-13356
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13356
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Hao Zhong
> Fix For: 4.x
>
>
> The BootstrapMonitor.progress ignores error messages when for 
> ProgressEventType.ERROR. Indeed, RepairRunner.progress once had a similar 
> bug, but is fixed. The fixed code is:
> {code}
>  public void progress(String tag, ProgressEvent event)
> {
> ProgressEventType type = event.getType();
> String message = String.format("[%s] %s", 
> format.format(System.currentTimeMillis()), event.getMessage());
> if (type == ProgressEventType.PROGRESS)
> {
> message = message + " (progress: " + 
> (int)event.getProgressPercentage() + "%)";
> }
> out.println(message);
> if (type == ProgressEventType.ERROR)
> {
> error = new RuntimeException("Repair job has failed with the 
> error message: " + message);
> }
> if (type == ProgressEventType.COMPLETE)
> {
> condition.signalAll();
> }
> }
> {code}
> Please refer to CASSANDRA-12508 for details.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13357) A possible NPE

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934114#comment-15934114
 ] 

Jeff Jirsa commented on CASSANDRA-13357:


Hello [~haozhong] - thanks for the bug report. Are you able to follow the 
procedure defined in the 
[documentation|http://cassandra.apache.org/doc/latest/development/patches.html#creating-a-patch]
 for creating a patch? We typically need a patch for each impacted branch, and 
then a reviewer can run unit tests for you and commit.


> A possible NPE
> --
>
> Key: CASSANDRA-13357
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13357
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Hao Zhong
> Fix For: 4.x
>
>
> The GetEndpoints.execute method has the following code:
> {code:title=GetEndpoints.java|borderStyle=solid}
>List endpoints = probe.getEndpoints(ks, table, key);
> for (InetAddress endpoint : endpoints)
> {
> System.out.println(endpoint.getHostAddress());
> }
> {code}
> This code can throw NPE. A similar bug is fixed in CASSANDRA-8950. The buggy 
> code  is 
> {code:title=NodeCmd.java|borderStyle=solid}
>   List endpoints = this.probe.getEndpoints(keySpace, cf, key);
> for (InetAddress anEndpoint : endpoints)
> {
>output.println(anEndpoint.getHostAddress());
> }
> {code}
> The fixed code is:
> {code:title=NodeCmd.java|borderStyle=solid}
> try
> {
> List endpoints = probe.getEndpoints(keySpace, cf, 
> key);
> for (InetAddress anEndpoint : endpoints)
>output.println(anEndpoint.getHostAddress());
> }
> catch (IllegalArgumentException ex)
> {
> output.println(ex.getMessage());
> probe.failed();
> }
> {code}
> The GetEndpoints.execute method shall be modified as CASSANDRA-8950 does.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13357) A possible NPE

[Hao Zhong (JIRA)]

Hao Zhong created CASSANDRA-13357:
-

 Summary: A possible NPE
 Key: CASSANDRA-13357
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13357
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Hao Zhong
 Fix For: 4.x


The GetEndpoints.execute method has the following code:

{code:title=GetEndpoints.java|borderStyle=solid}
   List endpoints = probe.getEndpoints(ks, table, key);
for (InetAddress endpoint : endpoints)
{
System.out.println(endpoint.getHostAddress());
}
{code}

This code can throw NPE. A similar bug is fixed in CASSANDRA-8950. The buggy 
code  is 
{code:title=NodeCmd.java|borderStyle=solid}
  List endpoints = this.probe.getEndpoints(keySpace, cf, key);

for (InetAddress anEndpoint : endpoints)
{
   output.println(anEndpoint.getHostAddress());
}
{code}

The fixed code is:

{code:title=NodeCmd.java|borderStyle=solid}
try
{
List endpoints = probe.getEndpoints(keySpace, cf, key);
for (InetAddress anEndpoint : endpoints)
   output.println(anEndpoint.getHostAddress());
}
catch (IllegalArgumentException ex)
{
output.println(ex.getMessage());
probe.failed();
}
{code}

The GetEndpoints.execute method shall be modified as CASSANDRA-8950 does.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933938#comment-15933938
 ] 

Jeff Jirsa commented on CASSANDRA-11530:


Linking related CASSANDRA-11430 for context. 


> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[Cassandra Wiki] Update of "FrontPage" by AnthonyGrasso

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "FrontPage" page has been changed by AnthonyGrasso:
https://wiki.apache.org/cassandra/FrontPage?action=diff=117=118

  = Apache Cassandra Wiki =
  {{{#!wiki red/solid
- The official Cassandra documentation is located 
[[http://cassandra.apache.org/doc/latest/|here]].  Patches to the documentation 
[[https://github.com/apache/cassandra/tree/trunk/doc|can be submitted with 
git]]; this wiki is intended instead to document Cassandra internals and how to 
edit them.
+ Most of the information in this Wiki is being deprecated!
  
- If you would like to contribute to this wiki, please send an email to the 
mailing list dev.at.cassandra.apache-dot-org with your wiki username and we 
will be happy to add you. Contributions welcome!
+ The official Cassandra documentation is located 
[[http://cassandra.apache.org/doc/latest/|here]]. Cassandra is moving away from 
this wiki for user-facing documentation in favor of in-tree docs. Patches to 
the official in-tree documentation are welcome and 
[[https://github.com/apache/cassandra/tree/trunk/doc|can be submitted via a 
GitHub Pull Request]].
+ 
+ This wiki is intended instead to document Cassandra internals and how to edit 
them and thus contains developer-focused information. If you really need access 
to this wiki, please send an email to the mailing list 
d...@cassandra.apache.org with your wiki username and we will add you.
  }}}
  Cassandra is a highly scalable, eventually consistent, distributed, 
structured key-value store. Cassandra brings together the distributed systems 
technologies from 
[[http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf|Dynamo]]
 and the log-structured storage engine from Google's 
[[http://research.google.com/archive/bigtable-osdi06.pdf|BigTable]]. 
  
- Cassandra is moving away from this wiki for user-facing documentation in 
favor of in-tree docs, linked below.  
([[https://github.com/apache/cassandra/tree/trunk/doc|Pull requests welcome]]!)
- 
- Developer-focused information is still hosted on the wiki; see below.
  
  == General Information ==
   * [[http://cassandra.apache.org/|Official Apache Cassandra Website]] 
(download, bug-tracking, mailing-lists, etc)
@@ -18, +17 @@

  
  
  
- == Cassandra developer Documentation ==
+ == Developer Documentation ==
-  * [[HowToBuild|How To Build]]
-  * [[HowToDebug|How to Debug in Eclipse]]
   * [[ArchitectureInternals|Architecture Internals]]
   * [[TopLevelPackages|Top Level Packages]]
   * [[CLI%20Design|CLI Design]]
-  * [[HowToContribute|How To Contribute]]
-  * [[HowToReview|How To Review]]
-  * [[HowToCommit|How To Commit]]
   * [[HowToPublishReleases|How To Release]] (Note: currently a work in 
progress) (Note: only relevant to Cassandra Committers)
   * [[Windows Development|WindowsDevelopment]]
   * [[LoggingGuidelines|Logging Guidelines]]
  
  
+ == Deprecated Developer Documentation ==
+  * [[HowToBuild|How To Build]] (Deprecated by 
[[http://cassandra.apache.org/doc/latest/development/ide.html|Building and IDE 
Integration]])
+  * [[HowToDebug|How to Debug in Eclipse]] (Deprecated by 
[[http://cassandra.apache.org/doc/latest/development/ide.html#setting-up-cassandra-in-eclipse|Eclipse
 IDE Setup]])
+  * [[HowToContribute|How To Contribute]] (Deprecated by 
[[http://cassandra.apache.org/doc/latest/development/patches.html|Contributing 
Code Changes]])
+  * [[HowToReview|How To Review]] (Deprecated by 
[[http://cassandra.apache.org/doc/latest/development/how_to_review.html|Review 
Checklist]])
+  * [[HowToCommit|How To Commit]] (Deprecated by 
[[http://cassandra.apache.org/doc/latest/development/how_to_commit.html|How To 
Commit]])
  
  == Other Languages ==
   * [[%E9%A6%96%E9%A1%B5|SimpleChinese 简体中文]]

[jira] [Commented] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933913#comment-15933913
 ] 

Yuki Morishita commented on CASSANDRA-11530:


Yes, this only removes deprecated methods.

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13356) BootstrapMonitor.progress does not store all error messages

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13356:
---
Description: 
The BootstrapMonitor.progress ignores error messages when for 
ProgressEventType.ERROR. Indeed, RepairRunner.progress once had a similar bug, 
but is fixed. The fixed code is:

{code}
 public void progress(String tag, ProgressEvent event)
{
ProgressEventType type = event.getType();
String message = String.format("[%s] %s", 
format.format(System.currentTimeMillis()), event.getMessage());
if (type == ProgressEventType.PROGRESS)
{
message = message + " (progress: " + 
(int)event.getProgressPercentage() + "%)";
}
out.println(message);
if (type == ProgressEventType.ERROR)
{
error = new RuntimeException("Repair job has failed with the error 
message: " + message);
}
if (type == ProgressEventType.COMPLETE)
{
condition.signalAll();
}
}
{code}

Please refer to CASSANDRA-12508 for details.

  was:
The BootstrapMonitor.progress ignores error messages when for 
ProgressEventType.ERROR. Indeed, RepairRunner.progress once had a similar bug, 
but is fixed. The fixed code is:
 public void progress(String tag, ProgressEvent event)
{
ProgressEventType type = event.getType();
String message = String.format("[%s] %s", 
format.format(System.currentTimeMillis()), event.getMessage());
if (type == ProgressEventType.PROGRESS)
{
message = message + " (progress: " + 
(int)event.getProgressPercentage() + "%)";
}
out.println(message);
if (type == ProgressEventType.ERROR)
{
error = new RuntimeException("Repair job has failed with the error 
message: " + message);
}
if (type == ProgressEventType.COMPLETE)
{
condition.signalAll();
}
}
Please refer to CASSANDRA-12508 for details.


> BootstrapMonitor.progress does not store all error messages
> ---
>
> Key: CASSANDRA-13356
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13356
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Hao Zhong
> Fix For: 4.x
>
>
> The BootstrapMonitor.progress ignores error messages when for 
> ProgressEventType.ERROR. Indeed, RepairRunner.progress once had a similar 
> bug, but is fixed. The fixed code is:
> {code}
>  public void progress(String tag, ProgressEvent event)
> {
> ProgressEventType type = event.getType();
> String message = String.format("[%s] %s", 
> format.format(System.currentTimeMillis()), event.getMessage());
> if (type == ProgressEventType.PROGRESS)
> {
> message = message + " (progress: " + 
> (int)event.getProgressPercentage() + "%)";
> }
> out.println(message);
> if (type == ProgressEventType.ERROR)
> {
> error = new RuntimeException("Repair job has failed with the 
> error message: " + message);
> }
> if (type == ProgressEventType.COMPLETE)
> {
> condition.signalAll();
> }
> }
> {code}
> Please refer to CASSANDRA-12508 for details.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13356) BootstrapMonitor.progress does not store all error messages

[Hao Zhong (JIRA)]

Hao Zhong created CASSANDRA-13356:
-

 Summary: BootstrapMonitor.progress does not store all error 
messages
 Key: CASSANDRA-13356
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13356
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Hao Zhong
 Fix For: 4.x


The BootstrapMonitor.progress ignores error messages when for 
ProgressEventType.ERROR. Indeed, RepairRunner.progress once had a similar bug, 
but is fixed. The fixed code is:
 public void progress(String tag, ProgressEvent event)
{
ProgressEventType type = event.getType();
String message = String.format("[%s] %s", 
format.format(System.currentTimeMillis()), event.getMessage());
if (type == ProgressEventType.PROGRESS)
{
message = message + " (progress: " + 
(int)event.getProgressPercentage() + "%)";
}
out.println(message);
if (type == ProgressEventType.ERROR)
{
error = new RuntimeException("Repair job has failed with the error 
message: " + message);
}
if (type == ProgressEventType.COMPLETE)
{
condition.signalAll();
}
}
Please refer to CASSANDRA-12508 for details.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898641#comment-15898641
 ] 

Ariel Weisberg edited comment on CASSANDRA-13289 at 3/20/17 10:11 PM:
--

||Code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13289?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13289-testall/3/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13289-dtest/3/]|


was (Author: aweisberg):
||Code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13289?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13289-testall/2/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13289-dtest/2/]|

> Make it possible to monitor an ideal consistency level separate from actual 
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter 
> replication and consistency you may want to have more information on from 
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those 
> writes failing to achieve EACH_QUORUM at other data centers. If you failed 
> your application over to one of those data centers roughly how inconsistent 
> might it be given the number of writes that didn't propagate since the last 
> incremental repair?
> You might also want to know roughly what the latency of writes would be if 
> you switched to a different consistency level. For instance you are writing 
> at LOCAL_QUORUM and want to know what would happen if you switched to 
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in 
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is 
> specified no additional tracking is done.
> if an ideal consistency level is specified then the 
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler 
> that tracks whether the ideal consistency level is met before a write times 
> out. It also tracks the latency for achieving the ideal CL  of successful 
> writes.
> These two metrics would be reported on a per keyspace basis.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13355) Trivial typo in JavaDriverClient.java

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13355:
---
External issue URL: https://github.com/apache/cassandra/pull/97
 External issue ID: https://github.com/apache/cassandra/pull/97

> Trivial typo in JavaDriverClient.java
> -
>
> Key: CASSANDRA-13355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13355
> Project: Cassandra
>  Issue Type: Bug
>  Components: Stress
>Reporter: Jeff Jirsa
>Priority: Trivial
> Fix For: 4.0
>
>
> Upstream github PR from Ian Macalinao 
> https://github.com/apache/cassandra/pull/97 - simple typo fix. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13355) Trivial typo in JavaDriverClient.java

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13355:
---
Description: Upstream github PR from Ian Macalinao 
https://github.com/apache/cassandra/pull/97 - simple typo fix. 

> Trivial typo in JavaDriverClient.java
> -
>
> Key: CASSANDRA-13355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13355
> Project: Cassandra
>  Issue Type: Bug
>  Components: Stress
>Reporter: Jeff Jirsa
>Priority: Trivial
> Fix For: 4.0
>
>
> Upstream github PR from Ian Macalinao 
> https://github.com/apache/cassandra/pull/97 - simple typo fix. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13256) add documentation on contributing documentation

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933656#comment-15933656
 ] 

Jeff Jirsa commented on CASSANDRA-13256:


I like the idea, but I think we should - at the ver least - make GH workflow 
less noisy. Sent out [VOTE] thread to dev@ proposing we move GH email to a less 
noisy list.


> add documentation on contributing documentation
> ---
>
> Key: CASSANDRA-13256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13256
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Stefan Podkowinski
> Attachments: 13256.patch
>
>
> The contributing page has nothing on how to contribute documentation, an area 
> the project is lacking. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13256) add documentation on contributing documentation

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13256:
---
Attachment: 13256.patch

> add documentation on contributing documentation
> ---
>
> Key: CASSANDRA-13256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13256
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Stefan Podkowinski
> Attachments: 13256.patch
>
>
> The contributing page has nothing on how to contribute documentation, an area 
> the project is lacking. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13355) Trivial typo in JavaDriverClient.java

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa resolved CASSANDRA-13355.

   Resolution: Fixed
 Reviewer: Jeff Jirsa
Fix Version/s: 4.0

Committed to trunk as {{5b8b1ce26cd073a44ddf7c7a6750da409a343eba}}

> Trivial typo in JavaDriverClient.java
> -
>
> Key: CASSANDRA-13355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13355
> Project: Cassandra
>  Issue Type: Bug
>  Components: Stress
>Reporter: Jeff Jirsa
>Priority: Trivial
> Fix For: 4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13355) Trivial typo in JavaDriverClient.java

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13355:
---
Component/s: Stress

> Trivial typo in JavaDriverClient.java
> -
>
> Key: CASSANDRA-13355
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13355
> Project: Cassandra
>  Issue Type: Bug
>  Components: Stress
>Reporter: Jeff Jirsa
>Priority: Trivial
> Fix For: 4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

cassandra git commit: Fix typo in stress JavaDriverClient.java

[jjirsa]

Repository: cassandra
Updated Branches:
  refs/heads/trunk dd5326646 -> 5b8b1ce26


Fix typo in stress JavaDriverClient.java

Closes #97

Patch by Ian Macalinao; Reviewed by Jeff Jirsa for CASSANDRA-13355


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5b8b1ce2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5b8b1ce2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5b8b1ce2

Branch: refs/heads/trunk
Commit: 5b8b1ce26cd073a44ddf7c7a6750da409a343eba
Parents: dd53266
Author: Ian Macalinao 
Authored: Thu Mar 9 23:35:42 2017 -0600
Committer: Jeff Jirsa 
Committed: Mon Mar 20 14:50:01 2017 -0700

--
 .../src/org/apache/cassandra/stress/util/JavaDriverClient.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5b8b1ce2/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java 
b/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
index e0b4262..3a7c513 100644
--- a/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
+++ b/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
@@ -165,7 +165,7 @@ public class JavaDriverClient
 connectionsPerHost);
 for (Host host : metadata.getAllHosts())
 {
-System.out.printf("Datatacenter: %s; Host: %s; Rack: %s%n",
+System.out.printf("Datacenter: %s; Host: %s; Rack: %s%n",
 host.getDatacenter(), host.getAddress(), host.getRack());
 }
 

[jira] [Created] (CASSANDRA-13355) Trivial typo in JavaDriverClient.java

[Jeff Jirsa (JIRA)]

Jeff Jirsa created CASSANDRA-13355:
--

 Summary: Trivial typo in JavaDriverClient.java
 Key: CASSANDRA-13355
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13355
 Project: Cassandra
  Issue Type: Bug
Reporter: Jeff Jirsa
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13256) add documentation on contributing documentation

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13256:
---
Reviewer: Jeff Jirsa
  Status: Patch Available  (was: In Progress)

Proposed patch for added documentation:

* [GH 
Preview|https://github.com/spodkowinski/cassandra/blob/docs_gettingstarted/doc/source/development/documentation.rst]
* [Branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13256]



> add documentation on contributing documentation
> ---
>
> Key: CASSANDRA-13256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13256
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Stefan Podkowinski
>
> The contributing page has nothing on how to contribute documentation, an area 
> the project is lacking. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933619#comment-15933619
 ] 

Joshua McKenzie commented on CASSANDRA-8844:


Good question for the user or possibly dev mailing list. You'll want to clarify 
what you mean by: "but I am wondering how CDC works with delete,", but as a 
general point: any mutations that are flushed to the CommitLog will end up 
available via CDC.

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Coordination, Local Write-Read Paths
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.8
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order to make 

[jira] [Commented] (CASSANDRA-13289) Make it possible to monitor an ideal consistency level separate from actual consistency level

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933610#comment-15933610
 ] 

Ariel Weisberg commented on CASSANDRA-13289:


bq. maybe only instantiate AbstractWriteResponseHandler#responsesAndExpirations 
in #setIdealCLResponseHandler(), and thus only create the AtomicInteger when 
you know you are actually going to use it.
Sure. There is a part of me that doesn't like having a null field, but then 
again the NPE is asserting something important.

bq. if the ideal CL and the requested CL are the same, should we even bother 
capturing metrics about it? I'm kinda mixed on it...
bq. what happens if the user mixes non-CAS consistency levels with CAS 
consistency levels (or vice versa)? I think the behavior will be correct (we 
won't inadvertantly violate paxos semantics), but the semantic difference 
between CAS and non-CAS requests might not be meaningful. So perhaps ignore the 
idealCl if the CL types are different? wdyt?

So we could try and block people from doing combinations of things that don't 
make sense or aren't useful, but what is the penalty if they do? Their system 
will continue running they just won't get anything useful for this metric. 

When you think about it for the purpose of what this is measuring CAS and 
non-CAS are the same. Only the commit will have a write response handler and 
SERIAL == QUORUM and LOCAL_SERIAL == LOCAL_QURUM.

I am not sure there are invalid values other then ideal == current. The problem 
is that the error occurs at request time not configuration time so I can't 
validate the configuration because I don't know what the CL of subsequent 
requests will be. I could not count, but the operator asked me to do something 
and even if it looks useless maybe they still expect the metrics to be accurate?

I am in favor of providing mechanism and not policy in these cases. I don't 
want to throw an error at the request level. I can't validate the 
configuration. And I don't want to silently not increment the counters. The 
only other viable alternative is maybe a rate limited log warning.

bq. how will timed out message metrics be affected? We create an entry in 
MessagingService#callbacks for each peer contacted for an operation (just 
talking reads/mutations right now), and say the request CL is satisfied, but 
the idealCL doesn't hear back from some nodes. In that case we'll increment the 
timeouts, ConnectionMetrics.totalTimeouts.mark(), even though they weren't 
explicitly part of the user's request. It might be confusing to users or 
operators. I'm not sure how hard it is to code around that, or if it's 
worthwhile. If we feel it's not, perhaps we just document it in the yaml that 
"you may see higher than usual timeout counts". Thoughts?

This doesn't impact how timeouts are counted or callbacks are registered. All 
this does is hook in and maintain a separate set of metrics for ideal CL. It's 
operating within the existing callback that is already being registered and 
timed out. It doesn't register additional callbacks. There should be no impact 
on existing metrics.

> Make it possible to monitor an ideal consistency level separate from actual 
> consistency level
> -
>
> Key: CASSANDRA-13289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13289
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> As an operator there are several issues related to multi-datacenter 
> replication and consistency you may want to have more information on from 
> your production database.
> For instance. If your application writes at LOCAL_QUORUM how often are those 
> writes failing to achieve EACH_QUORUM at other data centers. If you failed 
> your application over to one of those data centers roughly how inconsistent 
> might it be given the number of writes that didn't propagate since the last 
> incremental repair?
> You might also want to know roughly what the latency of writes would be if 
> you switched to a different consistency level. For instance you are writing 
> at LOCAL_QUORUM and want to know what would happen if you switched to 
> EACH_QUORUM.
> The proposed change is to allow an ideal_consistency_level to be specified in 
> cassandra.yaml as well as get/set via JMX. If no ideal consistency level is 
> specified no additional tracking is done.
> if an ideal consistency level is specified then the 
> {{AbstractWriteResponesHandler}} will contain a delegate WriteResponseHandler 
> that tracks whether the ideal consistency level is met before a write times 
> out. It also tracks the latency for achieving the ideal CL  of successful 
> writes.
> These two metrics 

[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)

[Jia Zhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933587#comment-15933587
 ] 

Jia Zhan commented on CASSANDRA-8844:
-

Not sure if I should start a new thread, but I am wondering how CDC works with 
delete, especially whether it supports range delete or partition delete? 

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Coordination, Local Write-Read Paths
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.8
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order to make consuming a change log easy and efficient to do with low 
> latency, the following could supplement the approach 

[jira] [Commented] (CASSANDRA-12961) LCS needlessly checks for L0 STCS candidates multiple times

[Carlos Alonso (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933580#comment-15933580
 ] 

Carlos Alonso commented on CASSANDRA-12961:
---

Hi guys, I'm planning to do this improvement. It's gonna be my first one ever 
so please, bear with me :)

After going through the code I think it makes sense to get the L0 check outside 
the loop as [~jjirsa] suggests as a first approach. Pretty much like we do on 
the first few lines of that method when we check if we're bootstrapping.

On a separate issue we can tackle the improvement [~carlyeks] suggests as it 
will be a different check.

The way I see it is that this function will end up being split into several 
checks. Something along this lines:

# If bootstrapping, check for L0 STCS
# else if L0 is falling behind
#* if there's space in L1. Normal L0 -> L1
#* else compact L0
# else iterate levels descendingly and compact if enough score

This will make for a pretty big function, so splitting in smaller functions 
will also help I think.

> LCS needlessly checks for L0 STCS candidates multiple times
> ---
>
> Key: CASSANDRA-12961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12961
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Priority: Trivial
>  Labels: lhf
>
> It's very likely that the check for L0 STCS candidates (if L0 is falling 
> behind) can be moved outside of the loop, or at very least made so that it's 
> not called on each loop iteration:
> {code}
> for (int i = generations.length - 1; i > 0; i--)
> {
> List sstables = getLevel(i);
> if (sstables.isEmpty())
> continue; // mostly this just avoids polluting the debug log 
> with zero scores
> // we want to calculate score excluding compacting ones
> Set sstablesInLevel = Sets.newHashSet(sstables);
> Set remaining = Sets.difference(sstablesInLevel, 
> cfs.getTracker().getCompacting());
> double score = (double) SSTableReader.getTotalBytes(remaining) / 
> (double)maxBytesForLevel(i, maxSSTableSizeInBytes);
> logger.trace("Compaction score for level {} is {}", i, score);
> if (score > 1.001)
> {
> // before proceeding with a higher level, let's see if L0 is 
> far enough behind to warrant STCS
> CompactionCandidate l0Compaction = 
> getSTCSInL0CompactionCandidate();
> if (l0Compaction != null)
> return l0Compaction;
> ..
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13291) Replace usages of MessageDigest with Guava's Hasher

[Michael Kjellman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933567#comment-15933567
 ] 

Michael Kjellman commented on CASSANDRA-13291:
--

[~snazy] sorry for the delay replying. I never got an email... seems like the 
JIRA watch feature + email is a bit flaky.

I know we don't have that method in trunk anymore, but we'll need something 
similar to it to upgrade to a different hashing algo off of MD5... A new class 
sounds like a reasonable solution. Let me do that now.

Other things comments sound good too. Let me address those now too.

> Replace usages of MessageDigest with Guava's Hasher
> ---
>
> Key: CASSANDRA-13291
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13291
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Attachments: CASSANDRA-13291-trunk.diff
>
>
> During my profiling of C* I frequently see lots of aggregate time across 
> threads being spent inside the MD5 MessageDigest implementation. Given that 
> there are tons of modern alternative hashing functions better than MD5 
> available -- both in terms of providing better collision resistance and 
> actual computational speed -- I wanted to switch out our usage of MD5 for 
> alternatives (like adler128 or murmur3_128) and test for performance 
> improvements.
> Unfortunately, I found given the fact we use MessageDigest everywhere --  
> switching out the hashing function to something like adler128 or murmur3_128 
> (for example) -- which don't ship with the JDK --  wasn't straight forward.
> The goal of this ticket is to propose switching out usages of MessageDigest 
> directly in favor of Hasher from Guava. This means going forward we can 
> change a single line of code to switch the hashing algorithm being used 
> (assuming there is an implementation in Guava).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12104) Handle coalesce efforts for inter-dc traffic discretely from intra-dc traffic

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933484#comment-15933484
 ] 

Ariel Weisberg commented on CASSANDRA-12104:


I had something for this, but it's blocked by 4.0 and Netty now. No sense in 
doing it for code that won't exist in 4.0. Will revisit once that stuff lands.

> Handle coalesce efforts for inter-dc traffic discretely from intra-dc traffic
> -
>
> Key: CASSANDRA-12104
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12104
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Thom Valley
>Assignee: Ariel Weisberg
>Priority: Minor
>
> In relationship to CASSANDRA-8692, we have discovered that pushing coalescing 
> windows to the point where they have a positive impact on inter-dc traffic 
> overhead appears to have causes delays in intra-dc traffic (namely, quorum 
> requests between nodes).  Having the same coalescing strategy apply to all 
> messages (especially intra-dc request/response messages) seems like a bad 
> idea.
> This was in a 5 DC environment with from 30 to 130 ms of latency between the 
> DCs.  Local network was entirely unrestricted 10G ethernet.
> Being able to apply different coalescing rules to those two classifications 
> of traffic would allow much more effective tuning of the coalescing 
> strategies, save inter-dc bandwidth while not having any impact on intra-dc 
> message handling.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928274#comment-15928274
 ] 

Ariel Weisberg edited comment on CASSANDRA-13324 at 3/20/17 8:29 PM:
-

||code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13324-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-testall/5/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-dtest/5/]|


was (Author: aweisberg):
||code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13324-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-testall/4/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-dtest/4/]|

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933481#comment-15933481
 ] 

Ariel Weisberg commented on CASSANDRA-13324:


The tests found some bugs and I made a few small changes. I force pushed 
instead of adding a commit.

It didn't compile because {{MessagingServiceTest}} needed to pass in a 
{{MockBackpressureStrategy}} to construct an {{OutboundTcpConnectionPool}}. 
Also there were issues with {{DatabaseDescriptor.internodeAuthenticator}} being 
null. Rather then add null checks I added made the config not nullable with a 
default of the allow all authenticator.

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928274#comment-15928274
 ] 

Ariel Weisberg edited comment on CASSANDRA-13324 at 3/20/17 8:20 PM:
-

||code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13324-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-testall/4/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-dtest/4/]|


was (Author: aweisberg):
||code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13324-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-testall/3/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-dtest/3/]|

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13153) Reappeared Data when Mixing Incremental and Full Repairs

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13153:
---
   Resolution: Fixed
Fix Version/s: 4.0
   3.11.0
   3.0.13
   2.2.10
Reproduced In: 2.2.8, 2.2.7  (was: 2.2.7, 2.2.8)
   Status: Resolved  (was: Ready to Commit)

Merged as 06316df549c0096bd774893a405d1d32512e97bf

> Reappeared Data when Mixing Incremental and Full Repairs
> 
>
> Key: CASSANDRA-13153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13153
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Tools
> Environment: Apache Cassandra 2.2
>Reporter: Amanda Debrot
>Assignee: Stefan Podkowinski
>  Labels: Cassandra
> Fix For: 2.2.10, 3.0.13, 3.11.0, 4.0
>
> Attachments: log-Reappeared-Data.txt, 
> Step-by-Step-Simulate-Reappeared-Data.txt
>
>
> This happens for both LeveledCompactionStrategy and 
> SizeTieredCompactionStrategy.  I've only tested it on Cassandra version 2.2 
> but it most likely also affects all Cassandra versions after 2.2, if they 
> have anticompaction with full repair.
> When mixing incremental and full repairs, there are a few scenarios where the 
> Data SSTable is marked as unrepaired and the Tombstone SSTable is marked as 
> repaired.  Then if it is past gc_grace, and the tombstone and data has been 
> compacted out on other replicas, the next incremental repair will push the 
> Data to other replicas without the tombstone.
> Simplified scenario:
> 3 node cluster with RF=3
> Intial config:
>   Node 1 has data and tombstone in separate SSTables.
>   Node 2 has data and no tombstone.
>   Node 3 has data and tombstone in separate SSTables.
> Incremental repair (nodetool repair -pr) is run every day so now we have 
> tombstone on each node.
> Some minor compactions have happened since so data and tombstone get merged 
> to 1 SSTable on Nodes 1 and 3.
>   Node 1 had a minor compaction that merged data with tombstone. 1 
> SSTable with tombstone.
>   Node 2 has data and tombstone in separate SSTables.
>   Node 3 had a minor compaction that merged data with tombstone. 1 
> SSTable with tombstone.
> Incremental repairs keep running every day.
> Full repairs run weekly (nodetool repair -full -pr). 
> Now there are 2 scenarios where the Data SSTable will get marked as 
> "Unrepaired" while Tombstone SSTable will get marked as "Repaired".
> Scenario 1:
> Since the Data and Tombstone SSTable have been marked as "Repaired" 
> and anticompacted, they have had minor compactions with other SSTables 
> containing keys from other ranges.  During full repair, if the last node to 
> run it doesn't own this particular key in it's partitioner range, the Data 
> and Tombstone SSTable will get anticompacted and marked as "Unrepaired".  Now 
> in the next incremental repair, if the Data SSTable is involved in a minor 
> compaction during the repair but the Tombstone SSTable is not, the resulting 
> compacted SSTable will be marked "Unrepaired" and Tombstone SSTable is marked 
> "Repaired".
> Scenario 2:
> Only the Data SSTable had minor compaction with other SSTables 
> containing keys from other ranges after being marked as "Repaired".  The 
> Tombstone SSTable was never involved in a minor compaction so therefore all 
> keys in that SSTable belong to 1 particular partitioner range. During full 
> repair, if the last node to run it doesn't own this particular key in it's 
> partitioner range, the Data SSTable will get anticompacted and marked as 
> "Unrepaired".   The Tombstone SSTable stays marked as Repaired.
> Then it’s past gc_grace.  Since Node’s #1 and #3 only have 1 SSTable for that 
> key, the tombstone will get compacted out.
>   Node 1 has nothing.
>   Node 2 has data (in unrepaired SSTable) and tombstone (in repaired 
> SSTable) in separate SSTables.
>   Node 3 has nothing.
> Now when the next incremental repair runs, it will only use the Data SSTable 
> to build the merkle tree since the tombstone SSTable is flagged as repaired 
> and data SSTable is marked as unrepaired.  And the data will get repaired 
> against the other two nodes.
>   Node 1 has data.
>   Node 2 has data and tombstone in separate SSTables.
>   Node 3 has data.
> If a read request hits Node 1 and 3, it will return data.  If it hits 1 and 
> 2, or 2 and 3, however, it would return no data.
> Tested this with single range tokens for simplicity.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[spod]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f4ba9083
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f4ba9083
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f4ba9083

Branch: refs/heads/cassandra-3.0
Commit: f4ba9083e8251144d14cf367b8ef7d23c65ff2da
Parents: 0eebc6e 06316df
Author: Stefan Podkowinski 
Authored: Mon Mar 20 19:44:10 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:46:56 2017 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/compaction/CompactionManager.java  | 12 +++-
 2 files changed, 12 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/CHANGES.txt
--
diff --cc CHANGES.txt
index 10402f3,27dd343..6021315
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,26 -1,8 +1,27 @@@
 -2.2.10
 +3.0.13
 + * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
 + * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
 + * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
 + * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)
 + * Add formatted row output to assertEmpty in CQL Tester (CASSANDRA-13238)
 +Merged from 2.2:
+  * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
   * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
   * Fix queries updating multiple time the same list (CASSANDRA-13130)
   * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 +
 +
 +3.0.12
 + * Prevent data loss on upgrade 2.1 - 3.0 by adding component separator to 
LogRecord absolute path (CASSANDRA-13294)
 + * Improve testing on macOS by eliminating sigar logging (CASSANDRA-13233)
 + * Cqlsh copy-from should error out when csv contains invalid data for 
collections (CASSANDRA-13071)
 + * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
 + * Faster StreamingHistogram (CASSANDRA-13038)
 + * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
 + * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
 + * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 +Merged from 2.2:
   * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
   * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
   * Coalescing strategy sleeps too much (CASSANDRA-13090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 28140e0,d21f1e8..8d470d5
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -22,7 -22,6 +22,8 @@@ import java.io.IOException
  import java.lang.management.ManagementFactory;
  import java.util.*;
  import java.util.concurrent.*;
 +import java.util.function.Predicate;
++import java.util.stream.Collectors;
  import javax.management.MBeanServer;
  import javax.management.ObjectName;
  import javax.management.openmbean.OpenDataException;
@@@ -1222,7 -1212,21 +1223,16 @@@ public class CompactionManager implemen
  logger.info("Performing anticompaction on {} sstables", 
repaired.originals().size());
  
  //Group SSTables
- Collection groupedSSTables = 
cfs.getCompactionStrategyManager().groupSSTablesForAntiCompaction(repaired.originals());
+ Set sstables = repaired.originals();
+ 
+ // Repairs can take place on both unrepaired (incremental + full) and 
repaired (full) data.
+ // Although anti-compaction could work on repaired sstables as well 
and would result in having more accurate
+ // repairedAt values for these, we still avoid anti-compacting 
already repaired sstables, as we currently don't
+ // make use of any actual repairedAt value and splitting up sstables 
just for that is not worth it at this point.
 -Set unrepairedSSTables = 
ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate()
 -{
 -public boolean apply(SSTableReader input)
 -{
 -return !input.isRepaired();
 -}
 -}));
++Set unrepairedSSTables = 

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[spod]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5484bd1a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5484bd1a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5484bd1a

Branch: refs/heads/trunk
Commit: 5484bd1acc95e17a2001297f118fe8472d271924
Parents: 6b8da36 f4ba908
Author: Stefan Podkowinski 
Authored: Mon Mar 20 19:50:48 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:52:31 2017 +0100

--
 CHANGES.txt  |  1 +
 .../cassandra/db/compaction/CompactionManager.java   | 11 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5484bd1a/CHANGES.txt
--
diff --cc CHANGES.txt
index a800f82,6021315..ce8535d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -37,142 -48,6 +37,143 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 +Merged from 2.2:
++ * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
 + * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 + * Use portable stderr for java error in startup (CASSANDRA-13211)
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum 

[01/10] cassandra git commit: Avoid anti-compacting repaired sstables

[spod]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 a69f68855 -> 06316df54
  refs/heads/cassandra-3.0 0eebc6e6b -> f4ba9083e
  refs/heads/cassandra-3.11 6b8da3612 -> 5484bd1ac
  refs/heads/trunk 567f90666 -> dd5326646


Avoid anti-compacting repaired sstables

patch by Stefan Podkowinski; reviewed by Marcus Eriksson for CASSANDRA-13153


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06316df5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06316df5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06316df5

Branch: refs/heads/cassandra-2.2
Commit: 06316df549c0096bd774893a405d1d32512e97bf
Parents: a69f688
Author: Stefan Podkowinski 
Authored: Thu Feb 16 10:32:22 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:36:57 2017 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/compaction/CompactionManager.java  | 16 +++-
 2 files changed, 16 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2a8330e..27dd343 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
  * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
  * Fix queries updating multiple time the same list (CASSANDRA-13130)
  * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 8a3c11e..d21f1e8 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1212,7 +1212,21 @@ public class CompactionManager implements 
CompactionManagerMBean
 logger.info("Performing anticompaction on {} sstables", 
repaired.originals().size());
 
 //Group SSTables
-Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(repaired.originals());
+Set sstables = repaired.originals();
+
+// Repairs can take place on both unrepaired (incremental + full) and 
repaired (full) data.
+// Although anti-compaction could work on repaired sstables as well 
and would result in having more accurate
+// repairedAt values for these, we still avoid anti-compacting already 
repaired sstables, as we currently don't
+// make use of any actual repairedAt value and splitting up sstables 
just for that is not worth it at this point.
+Set unrepairedSSTables = 
ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate()
+{
+public boolean apply(SSTableReader input)
+{
+return !input.isRepaired();
+}
+}));
+
+Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(unrepairedSSTables);
 // iterate over sstables to check if the repaired / unrepaired ranges 
intersect them.
 int antiCompactedSSTableCount = 0;
 for (Collection sstableGroup : groupedSSTables)

[03/10] cassandra git commit: Avoid anti-compacting repaired sstables

[spod]

Avoid anti-compacting repaired sstables

patch by Stefan Podkowinski; reviewed by Marcus Eriksson for CASSANDRA-13153


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06316df5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06316df5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06316df5

Branch: refs/heads/cassandra-3.11
Commit: 06316df549c0096bd774893a405d1d32512e97bf
Parents: a69f688
Author: Stefan Podkowinski 
Authored: Thu Feb 16 10:32:22 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:36:57 2017 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/compaction/CompactionManager.java  | 16 +++-
 2 files changed, 16 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2a8330e..27dd343 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
  * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
  * Fix queries updating multiple time the same list (CASSANDRA-13130)
  * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 8a3c11e..d21f1e8 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1212,7 +1212,21 @@ public class CompactionManager implements 
CompactionManagerMBean
 logger.info("Performing anticompaction on {} sstables", 
repaired.originals().size());
 
 //Group SSTables
-Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(repaired.originals());
+Set sstables = repaired.originals();
+
+// Repairs can take place on both unrepaired (incremental + full) and 
repaired (full) data.
+// Although anti-compaction could work on repaired sstables as well 
and would result in having more accurate
+// repairedAt values for these, we still avoid anti-compacting already 
repaired sstables, as we currently don't
+// make use of any actual repairedAt value and splitting up sstables 
just for that is not worth it at this point.
+Set unrepairedSSTables = 
ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate()
+{
+public boolean apply(SSTableReader input)
+{
+return !input.isRepaired();
+}
+}));
+
+Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(unrepairedSSTables);
 // iterate over sstables to check if the repaired / unrepaired ranges 
intersect them.
 int antiCompactedSSTableCount = 0;
 for (Collection sstableGroup : groupedSSTables)

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[spod]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f4ba9083
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f4ba9083
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f4ba9083

Branch: refs/heads/trunk
Commit: f4ba9083e8251144d14cf367b8ef7d23c65ff2da
Parents: 0eebc6e 06316df
Author: Stefan Podkowinski 
Authored: Mon Mar 20 19:44:10 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:46:56 2017 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/compaction/CompactionManager.java  | 12 +++-
 2 files changed, 12 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/CHANGES.txt
--
diff --cc CHANGES.txt
index 10402f3,27dd343..6021315
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,26 -1,8 +1,27 @@@
 -2.2.10
 +3.0.13
 + * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
 + * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
 + * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
 + * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)
 + * Add formatted row output to assertEmpty in CQL Tester (CASSANDRA-13238)
 +Merged from 2.2:
+  * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
   * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
   * Fix queries updating multiple time the same list (CASSANDRA-13130)
   * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 +
 +
 +3.0.12
 + * Prevent data loss on upgrade 2.1 - 3.0 by adding component separator to 
LogRecord absolute path (CASSANDRA-13294)
 + * Improve testing on macOS by eliminating sigar logging (CASSANDRA-13233)
 + * Cqlsh copy-from should error out when csv contains invalid data for 
collections (CASSANDRA-13071)
 + * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
 + * Faster StreamingHistogram (CASSANDRA-13038)
 + * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
 + * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
 + * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 +Merged from 2.2:
   * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
   * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
   * Coalescing strategy sleeps too much (CASSANDRA-13090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 28140e0,d21f1e8..8d470d5
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -22,7 -22,6 +22,8 @@@ import java.io.IOException
  import java.lang.management.ManagementFactory;
  import java.util.*;
  import java.util.concurrent.*;
 +import java.util.function.Predicate;
++import java.util.stream.Collectors;
  import javax.management.MBeanServer;
  import javax.management.ObjectName;
  import javax.management.openmbean.OpenDataException;
@@@ -1222,7 -1212,21 +1223,16 @@@ public class CompactionManager implemen
  logger.info("Performing anticompaction on {} sstables", 
repaired.originals().size());
  
  //Group SSTables
- Collection groupedSSTables = 
cfs.getCompactionStrategyManager().groupSSTablesForAntiCompaction(repaired.originals());
+ Set sstables = repaired.originals();
+ 
+ // Repairs can take place on both unrepaired (incremental + full) and 
repaired (full) data.
+ // Although anti-compaction could work on repaired sstables as well 
and would result in having more accurate
+ // repairedAt values for these, we still avoid anti-compacting 
already repaired sstables, as we currently don't
+ // make use of any actual repairedAt value and splitting up sstables 
just for that is not worth it at this point.
 -Set unrepairedSSTables = 
ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate()
 -{
 -public boolean apply(SSTableReader input)
 -{
 -return !input.isRepaired();
 -}
 -}));
++Set unrepairedSSTables = sstables.stream().filter((s) 

[04/10] cassandra git commit: Avoid anti-compacting repaired sstables

[spod]

Avoid anti-compacting repaired sstables

patch by Stefan Podkowinski; reviewed by Marcus Eriksson for CASSANDRA-13153


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06316df5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06316df5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06316df5

Branch: refs/heads/trunk
Commit: 06316df549c0096bd774893a405d1d32512e97bf
Parents: a69f688
Author: Stefan Podkowinski 
Authored: Thu Feb 16 10:32:22 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:36:57 2017 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/compaction/CompactionManager.java  | 16 +++-
 2 files changed, 16 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2a8330e..27dd343 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
  * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
  * Fix queries updating multiple time the same list (CASSANDRA-13130)
  * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 8a3c11e..d21f1e8 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1212,7 +1212,21 @@ public class CompactionManager implements 
CompactionManagerMBean
 logger.info("Performing anticompaction on {} sstables", 
repaired.originals().size());
 
 //Group SSTables
-Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(repaired.originals());
+Set sstables = repaired.originals();
+
+// Repairs can take place on both unrepaired (incremental + full) and 
repaired (full) data.
+// Although anti-compaction could work on repaired sstables as well 
and would result in having more accurate
+// repairedAt values for these, we still avoid anti-compacting already 
repaired sstables, as we currently don't
+// make use of any actual repairedAt value and splitting up sstables 
just for that is not worth it at this point.
+Set unrepairedSSTables = 
ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate()
+{
+public boolean apply(SSTableReader input)
+{
+return !input.isRepaired();
+}
+}));
+
+Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(unrepairedSSTables);
 // iterate over sstables to check if the repaired / unrepaired ranges 
intersect them.
 int antiCompactedSSTableCount = 0;
 for (Collection sstableGroup : groupedSSTables)

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[spod]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5484bd1a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5484bd1a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5484bd1a

Branch: refs/heads/cassandra-3.11
Commit: 5484bd1acc95e17a2001297f118fe8472d271924
Parents: 6b8da36 f4ba908
Author: Stefan Podkowinski 
Authored: Mon Mar 20 19:50:48 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:52:31 2017 +0100

--
 CHANGES.txt  |  1 +
 .../cassandra/db/compaction/CompactionManager.java   | 11 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5484bd1a/CHANGES.txt
--
diff --cc CHANGES.txt
index a800f82,6021315..ce8535d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -37,142 -48,6 +37,143 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 +Merged from 2.2:
++ * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
 + * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 + * Use portable stderr for java error in startup (CASSANDRA-13211)
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * 

[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[spod]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd532664
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd532664
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd532664

Branch: refs/heads/trunk
Commit: dd5326646626bdba1739686d1f0e4a024a43186f
Parents: 567f906 5484bd1
Author: Stefan Podkowinski 
Authored: Mon Mar 20 19:54:33 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:54:33 2017 +0100

--
 CHANGES.txt  |  1 +
 .../cassandra/db/compaction/CompactionManager.java   | 11 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd532664/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd532664/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--

[02/10] cassandra git commit: Avoid anti-compacting repaired sstables

[spod]

Avoid anti-compacting repaired sstables

patch by Stefan Podkowinski; reviewed by Marcus Eriksson for CASSANDRA-13153


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06316df5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06316df5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06316df5

Branch: refs/heads/cassandra-3.0
Commit: 06316df549c0096bd774893a405d1d32512e97bf
Parents: a69f688
Author: Stefan Podkowinski 
Authored: Thu Feb 16 10:32:22 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:36:57 2017 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/compaction/CompactionManager.java  | 16 +++-
 2 files changed, 16 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2a8330e..27dd343 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
  * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
  * Fix queries updating multiple time the same list (CASSANDRA-13130)
  * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06316df5/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 8a3c11e..d21f1e8 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1212,7 +1212,21 @@ public class CompactionManager implements 
CompactionManagerMBean
 logger.info("Performing anticompaction on {} sstables", 
repaired.originals().size());
 
 //Group SSTables
-Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(repaired.originals());
+Set sstables = repaired.originals();
+
+// Repairs can take place on both unrepaired (incremental + full) and 
repaired (full) data.
+// Although anti-compaction could work on repaired sstables as well 
and would result in having more accurate
+// repairedAt values for these, we still avoid anti-compacting already 
repaired sstables, as we currently don't
+// make use of any actual repairedAt value and splitting up sstables 
just for that is not worth it at this point.
+Set unrepairedSSTables = 
ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate()
+{
+public boolean apply(SSTableReader input)
+{
+return !input.isRepaired();
+}
+}));
+
+Collection groupedSSTables = 
cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(unrepairedSSTables);
 // iterate over sstables to check if the repaired / unrepaired ranges 
intersect them.
 int antiCompactedSSTableCount = 0;
 for (Collection sstableGroup : groupedSSTables)

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[spod]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f4ba9083
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f4ba9083
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f4ba9083

Branch: refs/heads/cassandra-3.11
Commit: f4ba9083e8251144d14cf367b8ef7d23c65ff2da
Parents: 0eebc6e 06316df
Author: Stefan Podkowinski 
Authored: Mon Mar 20 19:44:10 2017 +0100
Committer: Stefan Podkowinski 
Committed: Mon Mar 20 19:46:56 2017 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/compaction/CompactionManager.java  | 12 +++-
 2 files changed, 12 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/CHANGES.txt
--
diff --cc CHANGES.txt
index 10402f3,27dd343..6021315
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,26 -1,8 +1,27 @@@
 -2.2.10
 +3.0.13
 + * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
 + * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
 + * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
 + * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)
 + * Add formatted row output to assertEmpty in CQL Tester (CASSANDRA-13238)
 +Merged from 2.2:
+  * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
   * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
   * Fix queries updating multiple time the same list (CASSANDRA-13130)
   * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 +
 +
 +3.0.12
 + * Prevent data loss on upgrade 2.1 - 3.0 by adding component separator to 
LogRecord absolute path (CASSANDRA-13294)
 + * Improve testing on macOS by eliminating sigar logging (CASSANDRA-13233)
 + * Cqlsh copy-from should error out when csv contains invalid data for 
collections (CASSANDRA-13071)
 + * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
 + * Faster StreamingHistogram (CASSANDRA-13038)
 + * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
 + * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
 + * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 +Merged from 2.2:
   * Avoid race on receiver by starting streaming sender thread after sending 
init message (CASSANDRA-12886)
   * Fix "multiple versions of ant detected..." when running ant test 
(CASSANDRA-13232)
   * Coalescing strategy sleeps too much (CASSANDRA-13090)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 28140e0,d21f1e8..8d470d5
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -22,7 -22,6 +22,8 @@@ import java.io.IOException
  import java.lang.management.ManagementFactory;
  import java.util.*;
  import java.util.concurrent.*;
 +import java.util.function.Predicate;
++import java.util.stream.Collectors;
  import javax.management.MBeanServer;
  import javax.management.ObjectName;
  import javax.management.openmbean.OpenDataException;
@@@ -1222,7 -1212,21 +1223,16 @@@ public class CompactionManager implemen
  logger.info("Performing anticompaction on {} sstables", 
repaired.originals().size());
  
  //Group SSTables
- Collection groupedSSTables = 
cfs.getCompactionStrategyManager().groupSSTablesForAntiCompaction(repaired.originals());
+ Set sstables = repaired.originals();
+ 
+ // Repairs can take place on both unrepaired (incremental + full) and 
repaired (full) data.
+ // Although anti-compaction could work on repaired sstables as well 
and would result in having more accurate
+ // repairedAt values for these, we still avoid anti-compacting 
already repaired sstables, as we currently don't
+ // make use of any actual repairedAt value and splitting up sstables 
just for that is not worth it at this point.
 -Set unrepairedSSTables = 
ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate()
 -{
 -public boolean apply(SSTableReader input)
 -{
 -return !input.isRepaired();
 -}
 -}));
++Set unrepairedSSTables = 

[jira] [Comment Edited] (CASSANDRA-11935) Add support for arithmetic operators

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933128#comment-15933128
 ] 

T Jake Luciani edited comment on CASSANDRA-11935 at 3/20/17 6:27 PM:
-

There is a issue with this ticket as it changes the serialization format of 
Shorts from variable width to fixed.  This needs to be captured in the 
MessageService, CommitLog, and SSTable versions so we can properly upgrade 
clusters to 4.0


was (Author: tjake):
There is a issue with this ticket as it changes the serialization format of 
Shorts from variable width to fixed.  This needs to be captured in the Message 
service and disk version so we can properly upgrade clusters to 4.0

> Add support for arithmetic operators
> 
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 4.0
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep 
> the scope as small as possible. Dates and string operations will be adressed 
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are 
> performed on different types (e.g. double + int).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (CASSANDRA-11935) Add support for arithmetic operators

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani reopened CASSANDRA-11935:


There is a issue with this ticket as it changes the serialization format of 
Shorts from variable width to fixed.  This needs to be captured in the Message 
service and disk version so we can properly upgrade clusters to 4.0

> Add support for arithmetic operators
> 
>
> Key: CASSANDRA-11935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11935
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 4.0
>
>
> The goal of this ticket is to add support for arithmetic operators:
> * {{-}}: Change the sign of the argument
> * {{+}}: Addition operator
> * {{-}}: Minus operator
> * {{*}}: Multiplication operator
> * {{/}}: Division operator
> * {{%}}: Modulo operator
> This ticket we should focus on adding operator only for numeric types to keep 
> the scope as small as possible. Dates and string operations will be adressed 
> in follow up tickets.
> The operation precedence should be:
> # {{*}}, {{/}}, {{%}}
> # {{+}}, {{-}}
> Some implicit data conversion should be performed when operations are 
> performed on different types (e.g. double + int).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928274#comment-15928274
 ] 

Ariel Weisberg edited comment on CASSANDRA-13324 at 3/20/17 5:11 PM:
-

||code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13324-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-testall/3/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-dtest/3/]|


was (Author: aweisberg):
||code|utests|dtests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:cassandra-13324-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-testall/1/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-13324-trunk-dtest/1/]|

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933059#comment-15933059
 ] 

Alex Petrov edited comment on CASSANDRA-13337 at 3/20/17 5:00 PM:
--

There's another way to reproduce the same issue with slightly different steps:

{code}
CREATE KEYSPACE IF NOT EXISTS "test" WITH REPLICATION = {'class' : 
'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': 1 };

CREATE TABLE IF NOT EXISTS "test"."reproduce" (pk1 int, ck1 int, v1 int, v2 
int, v3 int, v4 int, v5 int, PRIMARY KEY(pk1, ck1));

UPDATE "test"."reproduce" SET v2 = 1, v3 = 3, v4 = 4 WHERE pk1 = 1 AND ck1 = 0;

ALTER TABLE "test"."reproduce" DROP v2;
ALTER TABLE "test"."reproduce" DROP v3;
ALTER TABLE "test"."reproduce" DROP v4;

SELECT * FROM test.reproduce;
{code}

But essentially the problem is that we do return empty rows from local storage. 
For example, when {{UPDATE}} was used to set only a subset of rows, then the 
rows that were used in {{UPDATE}} get dropped. When trying to query, we end up 
with an empty row. This wouldn't happen with {{INSERT}} since for {{INSERT}} we 
have liveness set.

I just see a single small problem: 

{code}
createTable("CREATE TABLE %s(k int PRIMARY KEY, x int, y int)");
execute("UPDATE %s SET x = 1 WHERE k = 0");
flush(doFlush); // (1)
execute("ALTER TABLE %s DROP x");
{code}

If we do flush at point {{1}}, we will end up with a single row {{row(1, 
null)}}. However, if we do not do flush and query directly from memtable, we 
end up with an empty result.


was (Author: ifesdjeen):
There's another way to reproduce the same issue with slightly different steps:

{code}
CREATE KEYSPACE IF NOT EXISTS "test" WITH REPLICATION = {'class' : 
'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': 1 };

CREATE TABLE IF NOT EXISTS "test"."reproduce" (pk1 int, ck1 int, v1 int, v2 
int, v3 int, v4 int, v5 int, PRIMARY KEY(pk1, ck1));

UPDATE "test"."reproduce" SET v1 = 1, v2 = 1, v3 = 3, v4 = 4 WHERE pk1 = 1 AND 
ck1 = 0;

ALTER TABLE "test"."reproduce" DROP v2;
ALTER TABLE "test"."reproduce" DROP v3;
ALTER TABLE "test"."reproduce" DROP v4;

SELECT * FROM test.reproduce;
{code}

But essentially the problem is that we do return empty rows from local storage. 
For example, when {{UPDATE}} was used to set only a subset of rows, then the 
rows that were used in {{UPDATE}} get dropped. When trying to query, we end up 
with an empty row. This wouldn't happen with {{INSERT}} since for {{INSERT}} we 
have liveness set.

I just see a single small problem: 

{code}
createTable("CREATE TABLE %s(k int PRIMARY KEY, x int, y int)");
execute("UPDATE %s SET x = 1 WHERE k = 0");
flush(doFlush); // (1)
execute("ALTER TABLE %s DROP x");
{code}

If we do flush at point {{1}}, we will end up with a single row {{row(1, 
null)}}. However, if we do not do flush and query directly from memtable, we 
end up with an empty result.

> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
>   

[jira] [Comment Edited] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933059#comment-15933059
 ] 

Alex Petrov edited comment on CASSANDRA-13337 at 3/20/17 4:59 PM:
--

There's another way to reproduce the same issue with slightly different steps:

{code}
CREATE KEYSPACE IF NOT EXISTS "test" WITH REPLICATION = {'class' : 
'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': 1 };

CREATE TABLE IF NOT EXISTS "test"."reproduce" (pk1 int, ck1 int, v1 int, v2 
int, v3 int, v4 int, v5 int, PRIMARY KEY(pk1, ck1));

UPDATE "test"."reproduce" SET v1 = 1, v2 = 1, v3 = 3, v4 = 4 WHERE pk1 = 1 AND 
ck1 = 0;

ALTER TABLE "test"."reproduce" DROP v2;
ALTER TABLE "test"."reproduce" DROP v3;
ALTER TABLE "test"."reproduce" DROP v4;

SELECT * FROM test.reproduce;
{code}

But essentially the problem is that we do return empty rows from local storage. 
For example, when {{UPDATE}} was used to set only a subset of rows, then the 
rows that were used in {{UPDATE}} get dropped. When trying to query, we end up 
with an empty row. This wouldn't happen with {{INSERT}} since for {{INSERT}} we 
have liveness set.

I just see a single small problem: 

{code}
createTable("CREATE TABLE %s(k int PRIMARY KEY, x int, y int)");
execute("UPDATE %s SET x = 1 WHERE k = 0");
flush(doFlush); // (1)
execute("ALTER TABLE %s DROP x");
{code}

If we do flush at point {{1}}, we will end up with a single row {{row(1, 
null)}}. However, if we do not do flush and query directly from memtable, we 
end up with an empty result.


was (Author: ifesdjeen):
There's another way to reproduce the same issue with slightly different steps:

{code}
CREATE KEYSPACE IF NOT EXISTS "test" WITH REPLICATION = {'class' : 
'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': 1 };

CREATE TABLE IF NOT EXISTS "test"."reproduce" (pk1 int, ck1 int, v1 int, v2 
int, v3 int, v4 int, v5 int, PRIMARY KEY(pk1, ck1));

UPDATE "test"."reproduce" SET v1 = 1, v2 = 1, v3 = 3, v4 = 4 WHERE pk1 = 1 AND 
ck1 = 0;

ALTER TABLE "test"."reproduce" DROP v2;
ALTER TABLE "test"."reproduce" DROP v3;
ALTER TABLE "test"."reproduce" DROP v4;

SELECT * FROM test.reproduce;
{code}

But essentially the problem is that we do return empty rows from local storage. 
For example, when {{UPDATE}} was used to set only a subset of rows, then the 
rows that were used in {{UPDATE}} get dropped. When trying to query, we end up 
with an empty row. This wouldn't happen with {{INSERT}} since for {{INSERT}} we 
have liveness set.

I just see a single small problem: 

{code}
createTable("CREATE TABLE %s(k int PRIMARY KEY, x int, y int)");
execute("UPDATE %s SET x = 1 WHERE k = 0");
flush(doFlush); // (1)
execute("ALTER TABLE %s DROP x");
{code}

If we do flush at point {{1}}, we will end up with a single row {{row(1, 
null)}}. However, if we do not do flush and query directly from sstable, we end 
up with an empty result.

> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
> 

[jira] [Commented] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933059#comment-15933059
 ] 

Alex Petrov commented on CASSANDRA-13337:
-

There's another way to reproduce the same issue with slightly different steps:

{code}
CREATE KEYSPACE IF NOT EXISTS "test" WITH REPLICATION = {'class' : 
'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': 1 };

CREATE TABLE IF NOT EXISTS "test"."reproduce" (pk1 int, ck1 int, v1 int, v2 
int, v3 int, v4 int, v5 int, PRIMARY KEY(pk1, ck1));

UPDATE "test"."reproduce" SET v1 = 1, v2 = 1, v3 = 3, v4 = 4 WHERE pk1 = 1 AND 
ck1 = 0;

ALTER TABLE "test"."reproduce" DROP v2;
ALTER TABLE "test"."reproduce" DROP v3;
ALTER TABLE "test"."reproduce" DROP v4;

SELECT * FROM test.reproduce;
{code}

But essentially the problem is that we do return empty rows from local storage. 
For example, when {{UPDATE}} was used to set only a subset of rows, then the 
rows that were used in {{UPDATE}} get dropped. When trying to query, we end up 
with an empty row. This wouldn't happen with {{INSERT}} since for {{INSERT}} we 
have liveness set.

I just see a single small problem: 

{code}
createTable("CREATE TABLE %s(k int PRIMARY KEY, x int, y int)");
execute("UPDATE %s SET x = 1 WHERE k = 0");
flush(doFlush); // (1)
execute("ALTER TABLE %s DROP x");
{code}

If we do flush at point {{1}}, we will end up with a single row {{row(1, 
null)}}. However, if we do not do flush and query directly from sstable, we end 
up with an empty result.

> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:509)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:369)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
>   at 
> 

cassandra-builds git commit: Switch all jobs to use latest JDK 1.8 version

[mshuler]

Repository: cassandra-builds
Updated Branches:
  refs/heads/master 3c1a7fc89 -> 4253406d9


Switch all jobs to use latest JDK 1.8 version


Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/4253406d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/4253406d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/4253406d

Branch: refs/heads/master
Commit: 4253406d900246c5df6636f1e5b2bdfb6eee9a93
Parents: 3c1a7fc
Author: Michael Shuler 
Authored: Mon Mar 20 11:54:20 2017 -0500
Committer: Michael Shuler 
Committed: Mon Mar 20 11:54:20 2017 -0500

--
 jenkins-dsl/cassandra_job_dsl_seed.groovy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/4253406d/jenkins-dsl/cassandra_job_dsl_seed.groovy
--
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy 
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 062b4cb..515965d 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -5,7 +5,7 @@
 
 
 def jobDescription = 'Apache Cassandra DSL-generated job - DSL git repo: https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;>cassandra-builds'
-def jdkLabel = 'jdk1.8.0_66-unlimited-security'
+def jdkLabel = 'JDK 1.8 (latest)'
 def slaveLabel = 'cassandra'
 def mainRepo = 'https://git-wip-us.apache.org/repos/asf/cassandra.git'
 def buildsRepo = 'https://git.apache.org/cassandra-builds.git'

[jira] [Commented] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933025#comment-15933025
 ] 

Ariel Weisberg commented on CASSANDRA-13327:


[dtest 
here|https://github.com/riptano/cassandra-dtest/compare/master...aweisberg:cassandra-13327?expand=1]
[~slebresne] can you review this? Since we don't run Jepsen on this it's really 
hard to know if it's going to impact the correctness of LWT.

> Pending endpoints size check for CAS doesn't play nicely with 
> writes-on-replacement
> ---
>
> Key: CASSANDRA-13327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13327
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> Consider this ring:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.4   MR DOWN NORMAL -7148113328562451251
> where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to 
> the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and 
> making no progress.
> Then the down node was replaced so we had:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.5   MR UP JOINING -7148113328562451251
> It’s confusing in the ring - the first JOINING is a genuine bootstrap, the 
> second is a replacement. We now had CAS unavailables (but no non-CAS 
> unvailables). I think it’s because the pending endpoints check thinks that 
> 127.0.0.5 is gaining a range when it’s just replacing.
> The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t 
> unnecessarily fail these requests.
> It also appears like required participants is bumped by 1 during a host 
> replacement so if the replacing host fails you will get unavailables and 
> timeouts.
> This is related to the check added in CASSANDRA-8346



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13337:

Reproduced In: 3.10, 3.0.12  (was: 3.0.12, 3.10)
   Status: Patch Available  (was: Open)

> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:509)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:369)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:58)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:67)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:26)
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:96)
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:227)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:190)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:610)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at 

[jira] [Updated] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13337:

 Reviewer: Alex Petrov
Reproduced In: 3.10, 3.0.12  (was: 3.0.12, 3.10)
   Status: Open  (was: Patch Available)

> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:509)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:369)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:58)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:67)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:26)
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:96)
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:227)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:190)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:610)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> 

[jira] [Commented] (CASSANDRA-12859) Column-level permissions

[Boris Melamed (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932925#comment-15932925
 ] 

Boris Melamed commented on CASSANDRA-12859:
---

Hi [~spo...@gmail.com],
I have suspended my work on this ticket while still awaiting feedback / 
resolution on multiple questions.

h4. Background summary

There was a very helpful initial discussion with [~beobal], based on which I 
made the implementation so far.
In an attempt to do even better, I made GRANTing columns additive (not 
'replacing'), and [~jjordan] has correctly pointed out that:

bq. I think it should be required for grant and revoke to work in a 
complementary fashion in v1 of this. So either grant replaces and revoke 
removes all. Or grant is additive and revoke is subtractive. If this is not how 
it works then it is impossible to remove access to a column without having an 
outage for an application that has stopped using said column.

Seems I'll have to either backtrack and make GRANT behavior replacing, or else 
implement subtractive-ness for REVOKE, after all, even in v1 of this.
Jordan also suggested that columns should be a 3rd level in permission 
hierarchies. However, the initially discussed, specced, and implemented 
approach considers column permissions as restrictions on table permissions. 
This follows what I perceive as the column permission paradigm of RDBMSs such 
as MySQL, DB2, Oracle.

I have just pushed the fixes I mentioned in my previous post to my fork: 
https://github.com/bmel/cassandra/tree/col-perms-12859.

h4. Summary of blocking issues
Before resuming my work, I would like the following questions resolved:

# Agreement on the originally approved approach of column permissions as 
light-weight restrictions on table permissions, as opposed to an additional, 
3rd resource level.

# Is there a spec for what's the expected behavior with permissions and MVs?
There are not many dtests for those, and from reading the code, it seems quite 
different from RDBMS view permissions.
For example, in C*:
#* No permissions are required on an MV for SELECT. Only its base table must be 
granted.
#* For modifying a base table, the user must have permissions on all its MVs. 
This is starkly different from RDBMS, IMHO. I can try and proceed likewise for 
columns, but would like to make sure this is expected.

# The "Remaining questions" (last section) in the uploaded doc Cassandra 
Proposal - Column-level permissions v2.docx

# And lastly... It may make sense for someone to also review the code so far, 
now that the basic feature is 'code-complete' and tested, in:
#* https://github.com/bmel/cassandra/tree/col-perms-12859. 
#* https://github.com/bmel/cassandra-dtest (the last test line fails 
rightfully; it reproduces the current leaking of permissions into re-created 
columns)

Note though that I have not merged from trunk/master since last November.


> Column-level permissions
> 
>
> Key: CASSANDRA-12859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12859
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core, CQL
>Reporter: Boris Melamed
>  Labels: doc-impacting
> Attachments: Cassandra Proposal - Column-level permissions.docx, 
> Cassandra Proposal - Column-level permissions v2.docx
>
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> h4. Here is a draft of: 
> Cassandra Proposal - Column-level permissions.docx (attached)
> h4. Quoting the 'Overview' section:
> The purpose of this proposal is to add column-level (field-level) permissions 
> to Cassandra. It is my intent to soon start implementing this feature in a 
> fork, and to submit a pull request once it’s ready.
> h4. Motivation
> Cassandra already supports permissions on keyspace and table (column family) 
> level. Sources:
> * http://www.datastax.com/dev/blog/role-based-access-control-in-cassandra
> * https://cassandra.apache.org/doc/latest/cql/security.html#data-control
> At IBM, we have use cases in the area of big data analytics where 
> column-level access permissions are also a requirement. All industry RDBMS 
> products are supporting this level of permission control, and regulators are 
> expecting it from all data-based systems.
> h4. Main day-one requirements
> # Extend CQL (Cassandra Query Language) to be able to optionally specify a 
> list of individual columns, in the {{GRANT}} statement. The relevant 
> permission types are: {{MODIFY}} (for {{UPDATE}} and {{INSERT}}) and 
> {{SELECT}}.
> # Persist the optional information in the appropriate system table 
> ‘system_auth.role_permissions’.
> # Enforce the column access restrictions during execution. Details:
> #* Should fit with the existing permission propagation down a role chain.
> #* Proposed message format when a user’s roles give access to the queried 
> table 

[jira] [Commented] (CASSANDRA-13065) Consistent range movements to not require MV updates to go through write paths

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932912#comment-15932912
 ] 

Joshua McKenzie commented on CASSANDRA-13065:
-

It looks like we can always just use {{appendToCommitLog}} for CDC-enabled cfs 
+ the standard streaming code rather than using the write path the way MV's do. 
CDC has no requirement for the data to make it back into a memtable or to go 
through the entire write path; what we really need is to ensure that data makes 
it to the CL and the segments get flagged as containing CDC-enabled cf's which 
this code should do.

> Consistent range movements to not require MV updates to go through write 
> paths 
> ---
>
> Key: CASSANDRA-13065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13065
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 4.0
>
>
> Booting or decommisioning nodes with MVs is unbearably slow as all streams go 
> through the regular write paths. This causes read-before-writes for every 
> mutation and during bootstrap it causes them to be sent to batchlog.
> The makes it virtually impossible to boot a new node in an acceptable amount 
> of time.
> Using the regular streaming behaviour for consistent range movements works 
> much better in this case and does not break the MV local consistency contract.
> Already tested on own cluster.
> Bootstrap case is super easy to handle, decommission case requires 
> CASSANDRA-13064



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Jan Karlsson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932799#comment-15932799
 ] 

Jan Karlsson edited comment on CASSANDRA-13354 at 3/20/17 3:45 PM:
---

Added patch on 4.0 to fix this. Applies cleanly to other versions as 
well(tested 2.2.9).
I have tested this in a cluster and will upload some graphs as well.
Comments and suggestions welcome!


was (Author: jan karlsson):
Added patch on 4.0 to fix this. Should be pretty minimal work to get this to 
apply to other versions as well.

I have tested this in a cluster and will upload some graphs as well.
Comments and suggestions welcome!

> LCS estimated compaction tasks does not take number of files into account
> -
>
> Key: CASSANDRA-13354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.2.9
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Attachments: 13354-trunk.txt
>
>
> In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
> taking the size of a SSTable and multiply it by four. This would give 4*160mb 
> with default settings. This calculation is used to determine whether repaired 
> or repaired data is being compacted.
> Now this works well until you take repair into account. Repair streams over 
> many many sstables which could be smaller than the configured SSTable size 
> depending on your use case. In our case we are talking about many thousands 
> of tiny SSTables. As number of files increases one can run into any number of 
> problems, including GC issues, too many open files or plain increase in read 
> latency.
> With the current algorithm we will choose repaired or unrepaired depending on 
> whichever side has more data in it. Even if the repaired files outnumber the 
> unrepaired files by a large margin.
> Similarily, our algorithm that selects compaction candidates takes up to 32 
> SSTables at a time in L0, however our estimated task calculation does not 
> take this number into account. These two mechanisms should be aligned with 
> each other.
> I propose that we take the number of files in L0 into account when estimating 
> remaining tasks. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Jan Karlsson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Karlsson updated CASSANDRA-13354:
-
Attachment: (was: CASSANDRA-13354)

> LCS estimated compaction tasks does not take number of files into account
> -
>
> Key: CASSANDRA-13354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.2.9
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Attachments: 13354-trunk.txt
>
>
> In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
> taking the size of a SSTable and multiply it by four. This would give 4*160mb 
> with default settings. This calculation is used to determine whether repaired 
> or repaired data is being compacted.
> Now this works well until you take repair into account. Repair streams over 
> many many sstables which could be smaller than the configured SSTable size 
> depending on your use case. In our case we are talking about many thousands 
> of tiny SSTables. As number of files increases one can run into any number of 
> problems, including GC issues, too many open files or plain increase in read 
> latency.
> With the current algorithm we will choose repaired or unrepaired depending on 
> whichever side has more data in it. Even if the repaired files outnumber the 
> unrepaired files by a large margin.
> Similarily, our algorithm that selects compaction candidates takes up to 32 
> SSTables at a time in L0, however our estimated task calculation does not 
> take this number into account. These two mechanisms should be aligned with 
> each other.
> I propose that we take the number of files in L0 into account when estimating 
> remaining tasks. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Jan Karlsson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Karlsson updated CASSANDRA-13354:
-
Attachment: 13354-trunk.txt

> LCS estimated compaction tasks does not take number of files into account
> -
>
> Key: CASSANDRA-13354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.2.9
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Attachments: 13354-trunk.txt
>
>
> In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
> taking the size of a SSTable and multiply it by four. This would give 4*160mb 
> with default settings. This calculation is used to determine whether repaired 
> or repaired data is being compacted.
> Now this works well until you take repair into account. Repair streams over 
> many many sstables which could be smaller than the configured SSTable size 
> depending on your use case. In our case we are talking about many thousands 
> of tiny SSTables. As number of files increases one can run into any number of 
> problems, including GC issues, too many open files or plain increase in read 
> latency.
> With the current algorithm we will choose repaired or unrepaired depending on 
> whichever side has more data in it. Even if the repaired files outnumber the 
> unrepaired files by a large margin.
> Similarily, our algorithm that selects compaction candidates takes up to 32 
> SSTables at a time in L0, however our estimated task calculation does not 
> take this number into account. These two mechanisms should be aligned with 
> each other.
> I propose that we take the number of files in L0 into account when estimating 
> remaining tasks. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11530:

Reviewer: Paulo Motta

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13348) Duplicate tokens after bootstrap

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932827#comment-15932827
 ] 

Paulo Motta commented on CASSANDRA-13348:
-

regardless of the fix here we should detect and prevent nodes from joining the 
ring with conflicting tokens (unless when replacing an existing node) - there's 
a similar check available on CASSANDRA-12485 for another less likely edge case 
that allows a node joining the ring with conflicting tokens

> Duplicate tokens after bootstrap
> 
>
> Key: CASSANDRA-13348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13348
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Priority: Blocker
> Fix For: 3.0.x
>
>
> This one is a bit scary, and probably results in data loss. After a bootstrap 
> of a few new nodes into an existing cluster, two new nodes have chosen some 
> overlapping tokens.
> In fact, of the 256 tokens chosen, 51 tokens were already in use on the other 
> node.
> Node 1 log :
> {noformat}
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,564 
> TokenAllocation.java:61 - Selected tokens [, 2959334889475814712, 
> 3727103702384420083, 7183119311535804926, 6013900799616279548, 
> -1222135324851761575, 1645259890258332163, -1213352346686661387, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:70 - Unexpected growth in standard deviation after 
> allocation.
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:44,150 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:43:14,151 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> Node 2 log:
> {noformat}
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:51,937 
> StorageService.java:971 - Joining ring by operator request
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,630 
> TokenAllocation.java:61 - Selected tokens [.., 2890709530010722764, 
> -2416006722819773829, -5820248611267569511, -5990139574852472056, 
> 1645259890258332163, 9135021011763659240, -5451286144622276797, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,794 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.02 min 0.98 stddev 0.
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,795 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:53,149 
> StorageService.java:1160 - JOINING: sleeping 3 ms for 

[jira] [Updated] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-13337:
-
Fix Version/s: 3.11.x
   3.0.x
Reproduced In: 3.10, 3.0.12  (was: 3.0.12, 3.10)
   Status: Patch Available  (was: Open)

This is definitively wrong, attaching patch for fix (including an unit test to 
reproduce).
| [13337-3.0|https://github.com/pcmanus/cassandra/commits/13337-3.0] | 
[utests|http://cassci.datastax.com/job/pcmanus-13337-3.0-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-13337-3.0-dtest] |
| [13337-3.11|https://github.com/pcmanus/cassandra/commits/13337-3.11] | 
[utests|http://cassci.datastax.com/job/pcmanus-13337-3.11-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-13337-3.11-dtest] |

I'll note that the patch basically disable the error message seen here, instead 
simply ignoring empty rows from disk since they can happen. I suppose it would 
be possible to do a more involved checking to make sure we didn't wrote 
something actually empty, but I'm not sure at all it's worth the trouble (not 
the cost of that check on a pretty hot path) especially given that expecting 
non-empty rows was wrong no only due to dropping, but also because we can 
actually skip columns due to the {{ColumnFilter}} within 
{{SerializationHelper}}. I believe the latter would only potentially impact 
thrift queries, which may be why nobody as yet reported that problem, but it's 
still wrong.


> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:509)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:369)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
>   at 
> 

[jira] [Updated] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Jan Karlsson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Karlsson updated CASSANDRA-13354:
-
Attachment: CASSANDRA-13354

> LCS estimated compaction tasks does not take number of files into account
> -
>
> Key: CASSANDRA-13354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.2.9
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Attachments: CASSANDRA-13354
>
>
> In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
> taking the size of a SSTable and multiply it by four. This would give 4*160mb 
> with default settings. This calculation is used to determine whether repaired 
> or repaired data is being compacted.
> Now this works well until you take repair into account. Repair streams over 
> many many sstables which could be smaller than the configured SSTable size 
> depending on your use case. In our case we are talking about many thousands 
> of tiny SSTables. As number of files increases one can run into any number of 
> problems, including GC issues, too many open files or plain increase in read 
> latency.
> With the current algorithm we will choose repaired or unrepaired depending on 
> whichever side has more data in it. Even if the repaired files outnumber the 
> unrepaired files by a large margin.
> Similarily, our algorithm that selects compaction candidates takes up to 32 
> SSTables at a time in L0, however our estimated task calculation does not 
> take this number into account. These two mechanisms should be aligned with 
> each other.
> I propose that we take the number of files in L0 into account when estimating 
> remaining tasks. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Jan Karlsson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Karlsson updated CASSANDRA-13354:
-
Status: Patch Available  (was: Open)

Added patch on 4.0 to fix this. Should be pretty minimal work to get this to 
apply to other versions as well.

I have tested this in a cluster and will upload some graphs as well.
Comments and suggestions welcome!

> LCS estimated compaction tasks does not take number of files into account
> -
>
> Key: CASSANDRA-13354
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.2.9
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
>
> In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
> taking the size of a SSTable and multiply it by four. This would give 4*160mb 
> with default settings. This calculation is used to determine whether repaired 
> or repaired data is being compacted.
> Now this works well until you take repair into account. Repair streams over 
> many many sstables which could be smaller than the configured SSTable size 
> depending on your use case. In our case we are talking about many thousands 
> of tiny SSTables. As number of files increases one can run into any number of 
> problems, including GC issues, too many open files or plain increase in read 
> latency.
> With the current algorithm we will choose repaired or unrepaired depending on 
> whichever side has more data in it. Even if the repaired files outnumber the 
> unrepaired files by a large margin.
> Similarily, our algorithm that selects compaction candidates takes up to 32 
> SSTables at a time in L0, however our estimated task calculation does not 
> take this number into account. These two mechanisms should be aligned with 
> each other.
> I propose that we take the number of files in L0 into account when estimating 
> remaining tasks. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13354) LCS estimated compaction tasks does not take number of files into account

[Jan Karlsson (JIRA)]

Jan Karlsson created CASSANDRA-13354:


 Summary: LCS estimated compaction tasks does not take number of 
files into account
 Key: CASSANDRA-13354
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13354
 Project: Cassandra
  Issue Type: Bug
  Components: Compaction
 Environment: Cassandra 2.2.9
Reporter: Jan Karlsson
Assignee: Jan Karlsson


In LCS, the way we estimate number of compaction tasks remaining for L0 is by 
taking the size of a SSTable and multiply it by four. This would give 4*160mb 
with default settings. This calculation is used to determine whether repaired 
or repaired data is being compacted.

Now this works well until you take repair into account. Repair streams over 
many many sstables which could be smaller than the configured SSTable size 
depending on your use case. In our case we are talking about many thousands of 
tiny SSTables. As number of files increases one can run into any number of 
problems, including GC issues, too many open files or plain increase in read 
latency.

With the current algorithm we will choose repaired or unrepaired depending on 
whichever side has more data in it. Even if the repaired files outnumber the 
unrepaired files by a large margin.

Similarily, our algorithm that selects compaction candidates takes up to 32 
SSTables at a time in L0, however our estimated task calculation does not take 
this number into account. These two mechanisms should be aligned with each 
other.

I propose that we take the number of files in L0 into account when estimating 
remaining tasks. 




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13332) wrong number of rows in cqlsh

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13332:
---
Reproduced In: 3.10
Fix Version/s: (was: 3.10)

> wrong number of rows in cqlsh
> -
>
> Key: CASSANDRA-13332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13332
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Windows 10
>Reporter: Bogusz Jelinski
>Priority: Trivial
> Attachments: cqlsh.jpg
>
>
> when I send a query via JDBC I receive fewer rows than via CQLSH. What I can 
> see is that at each page/list break the last row is duplicated - replicated 
> to the next page of the list. I have 8 page breaks and the summary of my 
> query gives 8 rows more than my Java client. 
> It might be of no importance - I am querying a table with "WHERE columnA LIKE 
> '%ABCD%'" for which I created an index: 
> CREATE CUSTOM INDEX columnA_idx ON ab.cdef (columnA) using 
> 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = { 'mode': 
> 'CONTAINS' };



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13230) Build RPM packages for release

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932785#comment-15932785
 ] 

Michael Shuler commented on CASSANDRA-13230:


Luke, see CASSANDRA-13252

> Build RPM packages for release
> --
>
> Key: CASSANDRA-13230
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13230
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Packaging
>Reporter: Michael Shuler
>Assignee: Stefan Podkowinski
> Fix For: 2.1.18, 2.2.10, 3.0.12, 3.11.0, 4.0
>
> Attachments: 13230.patch
>
>
> Currently, releases are built locally on Debian/Ubuntu based machines, 
> without native support for building RPM packages. This can be done with a 
> docker image.
> The current cassandra-release scripts are here (please, do not randomly run 
> and push tags..):
> https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=cassandra-release
> A couple incomplete docker run scripts are here:
> https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=docker-wip
> {code}
> git clone https://git-wip-us.apache.org/repos/asf/cassandra-builds.git
> {code}
> Patches for build infra improvements are welcome!
> /cc [~spo...@gmail.com] if you want to assign to yourself, I'd be happy to 
> review :)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12967) Spec for Cassandra RPM Build

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932778#comment-15932778
 ] 

Michael Shuler commented on CASSANDRA-12967:


{{cqlsh}} requires python-2.7. If that can be satisfied by a savvy user prior 
to installation, so the dependency is met, then installation should work. I've 
tried using "supported" third-party python-2.7 packages from SCL on 
CentOS-6.something and it can be excruciatingly painful. My suggestion would be 
to just use RHEL/CentOS-7.

> Spec for Cassandra RPM Build
> 
>
> Key: CASSANDRA-12967
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12967
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Packaging
>Reporter: Bhuvan Rawal
>Assignee: Michael Shuler
> Fix For: 2.1.17, 2.2.9, 3.0.11, 3.10, 4.0
>
>
> Currently there is no RPM packaging for cassandra community. We tried 
> creating a RPM build for cassandra 3.0.9 using cassandra spec found in 
> https://github.com/riptano/cassandra-rpm, but we found some incompatibilities 
> as the spec was meant for 2.X versions. 
> It would be great to have a community RPM build for cassandra, with the 
> recommended cassandra recommendations and tools.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12870) Calculate pending ranges for identical KS settings just once

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932770#comment-15932770
 ] 

Edward Capriolo commented on CASSANDRA-12870:
-

The common case is probably a few tables however some users have 20k tables 
etc. 

> Calculate pending ranges for identical KS settings just once
> 
>
> Key: CASSANDRA-12870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12870
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 4.x
>
> Attachments: 12870-trunk.patch
>
>
> The {{TokenMetadata.calculatePendingRanges()}} operation can be very 
> expensive and already has been subject to micro optimization in 
> CASSANDRA-9258. Instead of further optimizing the calculation itself, I'd 
> like to prevent it from being executed as often by only calling it just once 
> for all keyspaces sharing identical replication settings. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12870) Calculate pending ranges for identical KS settings just once

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-12870:
---
Fix Version/s: (was: 4.0)
   4.x

> Calculate pending ranges for identical KS settings just once
> 
>
> Key: CASSANDRA-12870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12870
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 4.x
>
> Attachments: 12870-trunk.patch
>
>
> The {{TokenMetadata.calculatePendingRanges()}} operation can be very 
> expensive and already has been subject to micro optimization in 
> CASSANDRA-9258. Instead of further optimizing the calculation itself, I'd 
> like to prevent it from being executed as often by only calling it just once 
> for all keyspaces sharing identical replication settings. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12748) Cassandra fails to start up if GREP_COLOR is set

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932768#comment-15932768
 ] 

Michael Shuler commented on CASSANDRA-12748:


Unsetting {{GREP_COLOR}} seems like a reasonable fix. Thanks!

> Cassandra fails to start up if GREP_COLOR is set
> 
>
> Key: CASSANDRA-12748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12748
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Roth
>Priority: Minor
>
> If env var GREP_COLOR is set, CS fails to start as startup command then 
> contains ANSI codes.
> See cassandra-env.sh when JVM_OPTS_FILE is parsed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12748) Cassandra fails to start up if GREP_COLOR is set

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-12748:
---
Summary: Cassandra fails to start up if GREP_COLOR is set  (was: CS fails 
to start up if GREP_COLOR is set)

> Cassandra fails to start up if GREP_COLOR is set
> 
>
> Key: CASSANDRA-12748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12748
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Roth
>Priority: Minor
>
> If env var GREP_COLOR is set, CS fails to start as startup command then 
> contains ANSI codes.
> See cassandra-env.sh when JVM_OPTS_FILE is parsed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12870) Calculate pending ranges for identical KS settings just once

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932756#comment-15932756
 ] 

Stefan Podkowinski commented on CASSANDRA-12870:


Elements from the keyspace list will be removed by 
keyspaces.iterator().remove(). That's why the list is copied before as I don't 
want to remove elements from an "unowned" collection. LinkedList was chosen for 
performance reasons, although totally insignificant at such low cardinality. 
The LHS declaration should be changed to List to make the code style 
coherent, yes. 

> Calculate pending ranges for identical KS settings just once
> 
>
> Key: CASSANDRA-12870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12870
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 4.0
>
> Attachments: 12870-trunk.patch
>
>
> The {{TokenMetadata.calculatePendingRanges()}} operation can be very 
> expensive and already has been subject to micro optimization in 
> CASSANDRA-9258. Instead of further optimizing the calculation itself, I'd 
> like to prevent it from being executed as often by only calling it just once 
> for all keyspaces sharing identical replication settings. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13353) some metrics are missing when using GraphiteReporter with parameter prefix

[Roland O (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roland O updated CASSANDRA-13353:
-
Description: 
we have currently the problem that some metrics are not getting reported 
properly to graphite when using the prefix property in the 
metricsreporterconfigfile

here is our actual config

{noformat}
graphite:
  -
period: 10
timeunit: 'SECONDS'
prefix: 'demo01'
hosts:
 - host: 'our_graphite_host'
   port: 2003
predicate:
  color: "white"
  useQualifiedName: true
  patterns:
- "^org.apache.cassandra.metrics.+"
{noformat} 

we do some queries with cql against that host, but no metrics for

"demo01.org.apache.cassandra.metrics.ClientRequest.Latency.Read.count"

are getting reported.

when omitting the "prefix: 'demo01'"

we geht the proper metrics for

"org.apache.cassandra.metrics.ClientRequest.Latency.Read.count"

  was:
we have currently the problem that some metrics are not getting reported 
properly to graphite when using the prefix property in the 
metricsreporterconfigfile

here is our actual config

{noformat}
graphite:
  -
period: 10
timeunit: 'SECONDS'
prefix: 'demo01'
hosts:
 - host: 'our_graphite_host'
   port: 2003
predicate:
  color: "white"
  useQualifiedName: true
  patterns:
- "^org.apache.cassandra.metrics.+"
{noformat} 

we do some queries with cql against that host, but no metrics for

```demo01.org.apache.cassandra.metrics.ClientRequest.Latency.Read.count```

are getting reported.

when omitting the ```prefix: 'demo01'```

we geht the proper metrics for

```org.apache.cassandra.metrics.ClientRequest.Latency.Read.count```


> some metrics are missing when using GraphiteReporter with parameter prefix
> --
>
> Key: CASSANDRA-13353
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13353
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: cassandra 3.10, metrics-graphite from 
> https://repo1.maven.org/maven2/io/dropwizard/metrics/metrics-graphite/3.1.4/metrics-graphite-3.1.4.jar
>Reporter: Roland O
>
> we have currently the problem that some metrics are not getting reported 
> properly to graphite when using the prefix property in the 
> metricsreporterconfigfile
> here is our actual config
> {noformat}
> graphite:
>   -
> period: 10
> timeunit: 'SECONDS'
> prefix: 'demo01'
> hosts:
>  - host: 'our_graphite_host'
>port: 2003
> predicate:
>   color: "white"
>   useQualifiedName: true
>   patterns:
> - "^org.apache.cassandra.metrics.+"
> {noformat} 
> we do some queries with cql against that host, but no metrics for
> "demo01.org.apache.cassandra.metrics.ClientRequest.Latency.Read.count"
> are getting reported.
> when omitting the "prefix: 'demo01'"
> we geht the proper metrics for
> "org.apache.cassandra.metrics.ClientRequest.Latency.Read.count"



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13353) some metrics are missing when using GraphiteReporter with parameter prefix

[Roland O (JIRA)]

Roland O created CASSANDRA-13353:


 Summary: some metrics are missing when using GraphiteReporter with 
parameter prefix
 Key: CASSANDRA-13353
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13353
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: cassandra 3.10, metrics-graphite from 
https://repo1.maven.org/maven2/io/dropwizard/metrics/metrics-graphite/3.1.4/metrics-graphite-3.1.4.jar
Reporter: Roland O


we have currently the problem that some metrics are not getting reported 
properly to graphite when using the prefix property in the 
metricsreporterconfigfile

here is our actual config

{noformat}
graphite:
  -
period: 10
timeunit: 'SECONDS'
prefix: 'demo01'
hosts:
 - host: 'our_graphite_host'
   port: 2003
predicate:
  color: "white"
  useQualifiedName: true
  patterns:
- "^org.apache.cassandra.metrics.+"
{noformat} 

we do some queries with cql against that host, but no metrics for

```demo01.org.apache.cassandra.metrics.ClientRequest.Latency.Read.count```

are getting reported.

when omitting the ```prefix: 'demo01'```

we geht the proper metrics for

```org.apache.cassandra.metrics.ClientRequest.Latency.Read.count```



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13345) Increasing the per thread stack size to atleast 512k

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp resolved CASSANDRA-13345.
--
Resolution: Incomplete
  Reviewer:   (was: Jason Brown)

Thanks for your understanding, [~amitkumar_ghatwal]!
The approach to do all the development in a fork on github is right.

We'd appreciate to see it working on ppc64le. Looking forward to see progress 
on this!
Resolving this ticket as "incomplete" for now, but feel free to open another 
ticket. #cassandra-dev IRC channel is the right place for questions on 
development and infra questions.

> Increasing the per thread stack size to atleast 512k 
> -
>
> Key: CASSANDRA-13345
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13345
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
> Environment: Set up details
> $ lscpu
> Architecture:  ppc64le
> Byte Order:Little Endian
> CPU(s):160
> On-line CPU(s) list:   0-159
> Thread(s) per core:8
> Core(s) per socket:5
> Socket(s): 4
> NUMA node(s):  4
> Model: 2.1 (pvr 004b 0201)
> Model name:POWER8E (raw), altivec supported
> CPU max MHz:   3690.
> CPU min MHz:   2061.
> L1d cache: 64K
> L1i cache: 32K
> L2 cache:  512K
> L3 cache:  8192K
> NUMA node0 CPU(s): 0-39
> NUMA node1 CPU(s): 40-79
> NUMA node16 CPU(s):80-119
> NUMA node17 CPU(s):120-159
> $ cat /etc/os-release
> NAME="Ubuntu"
> VERSION="16.04.1 LTS (Xenial Xerus)"
> ID=ubuntu
> ID_LIKE=debian
> PRETTY_NAME="Ubuntu 16.04.1 LTS"
> VERSION_ID="16.04"
> HOME_URL="http://www.ubuntu.com/;
> SUPPORT_URL="http://help.ubuntu.com/;
> BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/;
> VERSION_CODENAME=xenial
> UBUNTU_CODENAME=xenial
> $ arch
> ppc64le
> $ java -version
> openjdk version "1.8.0_121"
> OpenJDK Runtime Environment (build 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
>Reporter: Amitkumar Ghatwal
>Priority: Minor
>
> Hi All,
> I followed the below steps 
> ```
> $ git clone https://github.com/apache/cassandra.git
> $ cd cassandra/
> $ ant
> $ bin/cassandra -f
> The stack size specified is too small, Specify at least 328k
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> ```
> After getting this , i had to upgrade the thread stack size to 512kb in 
> 'conf/jvm.options'
> ```
> $ git diff conf/jvm.options
> diff --git a/conf/jvm.options b/conf/jvm.options
> index 49b2196..00c03ce 100644
> --- a/conf/jvm.options
> +++ b/conf/jvm.options
> @@ -99,7 +99,7 @@
>  -XX:+HeapDumpOnOutOfMemoryError
>  # Per-thread stack size.
> --Xss256k
> +-Xss512k
>  # Larger interned string table, for gossip's benefit (CASSANDRA-6410)
>  -XX:StringTableSize=103
> ```
> Thereafter i was able to start the Cassandra server successfully.
> Could you please consider increasing the stack size to '512k' in 
> 'conf/jvm.options.
> Similar to  "https://issues.apache.org/jira/browse/CASSANDRA-13300;. Let me 
> know if we can increase the stack size in the Apache Cassandra trunk.
> Thanks for support provided so far , and let me know
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13246:
---
Component/s: (was: CQL)
 Local Write-Read Paths

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> 

[jira] [Updated] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13246:
---
Component/s: CQL

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[apache-cassandra-3.7.jar:3.7]
>   

[jira] [Comment Edited] (CASSANDRA-13339) java.nio.BufferOverflowException: null

[Chris Richards (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932484#comment-15932484
 ] 

Chris Richards edited comment on CASSANDRA-13339 at 3/20/17 11:51 AM:
--

I've had a look through our logs over the weekend, and we saw several instances 
: all with the same backtrace as the original report.  This was running with 
Netty-4.0.44.

I've looked at one corrupt commit log : it has x1fffd68 bytes of useful stuff 
in it, then the remaining x298 bytes (up to 32MB) is full of zeroes.  I've now 
looked through a further 7 logs; all finish with a small quantity (nothing more 
than 1KB) of zeroes.


was (Author: crichards):
I've had a look through our logs over the weekend, and we saw several instances 
: all with the same backtrace as the original report.  This was running with 
Netty-4.0.44.

I've looked at one corrupt commit log : it has x1fffd68 bytes of useful stuff 
in it, then the remaining x298 bytes (up to 32MB) is full of zeroes.

> java.nio.BufferOverflowException: null
> --
>
> Key: CASSANDRA-13339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13339
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Chris Richards
>
> I'm seeing the following exception running Cassandra 3.9 (with Netty updated 
> to 4.1.8.Final) running on a 2 node cluster.  It would have been processing 
> around 50 queries/second at the time (mixture of 
> inserts/updates/selects/deletes) : there's a collection of tables (some with 
> counters some without) and a single materialized view.
> ERROR [MutationStage-4] 2017-03-15 22:50:33,052 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.serializeForMessaging(SerializationHeader.java:380)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:122)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serialize(PartitionUpdate.java:790)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.serialize(Mutation.java:393)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:279) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:241) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1347)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2539)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_121]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.9.jar:3.9]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> and then again shortly afterwards
> ERROR [MutationStage-3] 2017-03-15 23:27:36,198 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
> 

[jira] [Updated] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13246:
---
   Resolution: Fixed
Fix Version/s: 4.0
   3.11.0
   3.0.13
   Status: Resolved  (was: Ready to Commit)

Committed into 3.0 at 0eebc6e6b7cd7fc801579e57701608e7bf155ee0 and merged into 
3.11 and trunk.

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Fix For: 3.0.13, 3.11.0, 4.0
>
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> 

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[blerer]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/567f9066
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/567f9066
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/567f9066

Branch: refs/heads/trunk
Commit: 567f90666293c05ac0b9b9d74b7cf8e053d6dc4c
Parents: 091e5fb 6b8da36
Author: Benjamin Lerer 
Authored: Mon Mar 20 12:34:33 2017 +0100
Committer: Benjamin Lerer 
Committed: Mon Mar 20 12:34:33 2017 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/filter/RowFilter.java   | 21 +-
 src/java/org/apache/cassandra/db/rows/Row.java  |  2 +-
 .../validation/entities/SecondaryIndexTest.java | 39 ++
 .../cql3/validation/operations/SelectTest.java  | 42 
 5 files changed, 95 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/567f9066/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/567f9066/src/java/org/apache/cassandra/db/filter/RowFilter.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/567f9066/src/java/org/apache/cassandra/db/rows/Row.java
--
diff --cc src/java/org/apache/cassandra/db/rows/Row.java
index 04092a6,2915a46..7449e51
--- a/src/java/org/apache/cassandra/db/rows/Row.java
+++ b/src/java/org/apache/cassandra/db/rows/Row.java
@@@ -132,9 -132,9 +132,9 @@@ public interface Row extends Unfiltered
   * The returned object groups all the cells for the column, as well as 
it's complex deletion (if relevant).
   *
   * @param c the complex column for which to return the complex data.
-  * @return the data for {@code c} or {@code null} is the row has no data 
for this column.
+  * @return the data for {@code c} or {@code null} if the row has no data 
for this column.
   */
 -public ComplexColumnData getComplexColumnData(ColumnDefinition c);
 +public ComplexColumnData getComplexColumnData(ColumnMetadata c);
  
  /**
   * An iterable over the cells of this row.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/567f9066/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/567f9066/test/unit/org/apache/cassandra/cql3/validation/operations/SelectTest.java
--

[5/6] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b8da361
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b8da361
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b8da361

Branch: refs/heads/cassandra-3.11
Commit: 6b8da3612bd01317220067251b13ed2c48fab264
Parents: 1092f5e 0eebc6e
Author: Benjamin Lerer 
Authored: Mon Mar 20 12:25:41 2017 +0100
Committer: Benjamin Lerer 
Committed: Mon Mar 20 12:25:41 2017 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/filter/RowFilter.java   | 21 +-
 src/java/org/apache/cassandra/db/rows/Row.java  |  2 +-
 .../validation/entities/SecondaryIndexTest.java | 39 ++
 .../cql3/validation/operations/SelectTest.java  | 42 
 5 files changed, 95 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/CHANGES.txt
--
diff --cc CHANGES.txt
index 44321ef,10402f3..a800f82
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
 -3.0.13
 +3.11.0
 + * Support unaligned memory access for AArch64 (CASSANDRA-13326)
 + * Improve SASI range iterator efficiency on intersection with an empty range 
(CASSANDRA-12915).
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 + * Address message coalescing regression (CASSANDRA-12676)
 +Merged from 3.0:
+  * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
   * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
   * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
   * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/src/java/org/apache/cassandra/db/filter/RowFilter.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/src/java/org/apache/cassandra/db/rows/Row.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
--
diff --cc 
test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
index c03b0cc,5d43bd2..88c6f17
--- 
a/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
+++ 
b/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
@@@ -420,27 -401,64 +420,66 @@@ public class SecondaryIndexTest extend
  execute("INSERT INTO %s (k, v, l, s, m) VALUES (1, 0, [1, 2, 4], {},  
   {'b' : 1})");
  execute("INSERT INTO %s (k, v, l, s, m) VALUES (1, 1, [4, 5],
{'d'},  {'a' : 1, 'b' : 3})");
  
 -// lists
 -assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 1"), row(1, 
0), row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE k = 0 AND l CONTAINS 
1"), row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 2"), row(1, 
0), row(0, 0));
 -assertEmpty(execute("SELECT k, v FROM %s WHERE l CONTAINS 6"));
 +beforeAndAfterFlush(() -> {
 +// lists
 +assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 1"), 
row(1, 0), row(0, 0), row(0, 2));
 +assertRows(execute("SELECT k, v FROM %s WHERE k = 0 AND l 
CONTAINS 1"), row(0, 0), row(0, 2));
 +assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 2"), 
row(1, 0), row(0, 0));
 +assertEmpty(execute("SELECT k, v FROM %s WHERE l CONTAINS 6"));
  
 -// sets
 -assertRows(execute("SELECT k, v FROM %s WHERE s CONTAINS 'a'"), 
row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE k = 0 AND s CONTAINS 
'a'"), row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE s CONTAINS 'd'"), 
row(1, 1));
 -assertEmpty(execute("SELECT k, v FROM %s  WHERE s CONTAINS 'e'"));
 +// sets
 

[3/6] cassandra git commit: Fix CONTAINS filtering for null collections

[blerer]

Fix CONTAINS filtering for null collections

patch by Mikkel Andersen; reviewed by Benjamin Lerer for CASSANDRA-13246


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0eebc6e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0eebc6e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0eebc6e6

Branch: refs/heads/trunk
Commit: 0eebc6e6b7cd7fc801579e57701608e7bf155ee0
Parents: 1dcb313
Author: Mikkel Andersen 
Authored: Mon Mar 20 12:15:34 2017 +0100
Committer: Benjamin Lerer 
Committed: Mon Mar 20 12:22:40 2017 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/filter/RowFilter.java   | 21 ++
 src/java/org/apache/cassandra/db/rows/Row.java  |  2 +-
 .../validation/entities/SecondaryIndexTest.java | 39 ++
 .../cql3/validation/operations/SelectTest.java  | 43 
 5 files changed, 96 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97d8561..10402f3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
  * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
  * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
  * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/src/java/org/apache/cassandra/db/filter/RowFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/RowFilter.java 
b/src/java/org/apache/cassandra/db/filter/RowFilter.java
index 11cfb87..5ffe2ab 100644
--- a/src/java/org/apache/cassandra/db/filter/RowFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/RowFilter.java
@@ -670,17 +670,20 @@ public abstract class RowFilter implements 
Iterable
 if (column.isComplex())
 {
 ComplexColumnData complexData = 
row.getComplexColumnData(column);
-for (Cell cell : complexData)
+if (complexData != null)
 {
-if (type.kind == CollectionType.Kind.SET)
+for (Cell cell : complexData)
 {
-if 
(type.nameComparator().compare(cell.path().get(0), value) == 0)
-return true;
-}
-else
-{
-if 
(type.valueComparator().compare(cell.value(), value) == 0)
-return true;
+if (type.kind == CollectionType.Kind.SET)
+{
+if 
(type.nameComparator().compare(cell.path().get(0), value) == 0)
+return true;
+}
+else
+{
+if 
(type.valueComparator().compare(cell.value(), value) == 0)
+return true;
+}
 }
 }
 return false;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/src/java/org/apache/cassandra/db/rows/Row.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/Row.java 
b/src/java/org/apache/cassandra/db/rows/Row.java
index 74d8664..a61f365 100644
--- a/src/java/org/apache/cassandra/db/rows/Row.java
+++ b/src/java/org/apache/cassandra/db/rows/Row.java
@@ -129,7 +129,7 @@ public interface Row extends Unfiltered, 
Collection
  * The returned object groups all the cells for the column, as well as 
it's complex deletion (if relevant).
  *
  * @param c the complex column for which to return the complex data.
- * @return the data for {@code c} or {@code null} is the row has no data 
for this column.
+ * @return the data for {@code c} or {@code null} if the row has no data 
for this column.
  */
 public ComplexColumnData getComplexColumnData(ColumnDefinition c);
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java

[2/6] cassandra git commit: Fix CONTAINS filtering for null collections

[blerer]

Fix CONTAINS filtering for null collections

patch by Mikkel Andersen; reviewed by Benjamin Lerer for CASSANDRA-13246


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0eebc6e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0eebc6e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0eebc6e6

Branch: refs/heads/cassandra-3.11
Commit: 0eebc6e6b7cd7fc801579e57701608e7bf155ee0
Parents: 1dcb313
Author: Mikkel Andersen 
Authored: Mon Mar 20 12:15:34 2017 +0100
Committer: Benjamin Lerer 
Committed: Mon Mar 20 12:22:40 2017 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/filter/RowFilter.java   | 21 ++
 src/java/org/apache/cassandra/db/rows/Row.java  |  2 +-
 .../validation/entities/SecondaryIndexTest.java | 39 ++
 .../cql3/validation/operations/SelectTest.java  | 43 
 5 files changed, 96 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97d8561..10402f3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
  * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
  * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
  * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/src/java/org/apache/cassandra/db/filter/RowFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/RowFilter.java 
b/src/java/org/apache/cassandra/db/filter/RowFilter.java
index 11cfb87..5ffe2ab 100644
--- a/src/java/org/apache/cassandra/db/filter/RowFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/RowFilter.java
@@ -670,17 +670,20 @@ public abstract class RowFilter implements 
Iterable
 if (column.isComplex())
 {
 ComplexColumnData complexData = 
row.getComplexColumnData(column);
-for (Cell cell : complexData)
+if (complexData != null)
 {
-if (type.kind == CollectionType.Kind.SET)
+for (Cell cell : complexData)
 {
-if 
(type.nameComparator().compare(cell.path().get(0), value) == 0)
-return true;
-}
-else
-{
-if 
(type.valueComparator().compare(cell.value(), value) == 0)
-return true;
+if (type.kind == CollectionType.Kind.SET)
+{
+if 
(type.nameComparator().compare(cell.path().get(0), value) == 0)
+return true;
+}
+else
+{
+if 
(type.valueComparator().compare(cell.value(), value) == 0)
+return true;
+}
 }
 }
 return false;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/src/java/org/apache/cassandra/db/rows/Row.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/Row.java 
b/src/java/org/apache/cassandra/db/rows/Row.java
index 74d8664..a61f365 100644
--- a/src/java/org/apache/cassandra/db/rows/Row.java
+++ b/src/java/org/apache/cassandra/db/rows/Row.java
@@ -129,7 +129,7 @@ public interface Row extends Unfiltered, 
Collection
  * The returned object groups all the cells for the column, as well as 
it's complex deletion (if relevant).
  *
  * @param c the complex column for which to return the complex data.
- * @return the data for {@code c} or {@code null} is the row has no data 
for this column.
+ * @return the data for {@code c} or {@code null} if the row has no data 
for this column.
  */
 public ComplexColumnData getComplexColumnData(ColumnDefinition c);
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java

[4/6] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11

[blerer]

Merge branch cassandra-3.0 into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b8da361
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b8da361
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b8da361

Branch: refs/heads/trunk
Commit: 6b8da3612bd01317220067251b13ed2c48fab264
Parents: 1092f5e 0eebc6e
Author: Benjamin Lerer 
Authored: Mon Mar 20 12:25:41 2017 +0100
Committer: Benjamin Lerer 
Committed: Mon Mar 20 12:25:41 2017 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/filter/RowFilter.java   | 21 +-
 src/java/org/apache/cassandra/db/rows/Row.java  |  2 +-
 .../validation/entities/SecondaryIndexTest.java | 39 ++
 .../cql3/validation/operations/SelectTest.java  | 42 
 5 files changed, 95 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/CHANGES.txt
--
diff --cc CHANGES.txt
index 44321ef,10402f3..a800f82
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
 -3.0.13
 +3.11.0
 + * Support unaligned memory access for AArch64 (CASSANDRA-13326)
 + * Improve SASI range iterator efficiency on intersection with an empty range 
(CASSANDRA-12915).
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 + * Address message coalescing regression (CASSANDRA-12676)
 +Merged from 3.0:
+  * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
   * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
   * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
   * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/src/java/org/apache/cassandra/db/filter/RowFilter.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/src/java/org/apache/cassandra/db/rows/Row.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b8da361/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
--
diff --cc 
test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
index c03b0cc,5d43bd2..88c6f17
--- 
a/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
+++ 
b/test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
@@@ -420,27 -401,64 +420,66 @@@ public class SecondaryIndexTest extend
  execute("INSERT INTO %s (k, v, l, s, m) VALUES (1, 0, [1, 2, 4], {},  
   {'b' : 1})");
  execute("INSERT INTO %s (k, v, l, s, m) VALUES (1, 1, [4, 5],
{'d'},  {'a' : 1, 'b' : 3})");
  
 -// lists
 -assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 1"), row(1, 
0), row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE k = 0 AND l CONTAINS 
1"), row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 2"), row(1, 
0), row(0, 0));
 -assertEmpty(execute("SELECT k, v FROM %s WHERE l CONTAINS 6"));
 +beforeAndAfterFlush(() -> {
 +// lists
 +assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 1"), 
row(1, 0), row(0, 0), row(0, 2));
 +assertRows(execute("SELECT k, v FROM %s WHERE k = 0 AND l 
CONTAINS 1"), row(0, 0), row(0, 2));
 +assertRows(execute("SELECT k, v FROM %s WHERE l CONTAINS 2"), 
row(1, 0), row(0, 0));
 +assertEmpty(execute("SELECT k, v FROM %s WHERE l CONTAINS 6"));
  
 -// sets
 -assertRows(execute("SELECT k, v FROM %s WHERE s CONTAINS 'a'"), 
row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE k = 0 AND s CONTAINS 
'a'"), row(0, 0), row(0, 2));
 -assertRows(execute("SELECT k, v FROM %s WHERE s CONTAINS 'd'"), 
row(1, 1));
 -assertEmpty(execute("SELECT k, v FROM %s  WHERE s CONTAINS 'e'"));
 +// sets
 +

[1/6] cassandra git commit: Fix CONTAINS filtering for null collections

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 1dcb3131a -> 0eebc6e6b
  refs/heads/cassandra-3.11 1092f5e71 -> 6b8da3612
  refs/heads/trunk 091e5fbe4 -> 567f90666


Fix CONTAINS filtering for null collections

patch by Mikkel Andersen; reviewed by Benjamin Lerer for CASSANDRA-13246


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0eebc6e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0eebc6e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0eebc6e6

Branch: refs/heads/cassandra-3.0
Commit: 0eebc6e6b7cd7fc801579e57701608e7bf155ee0
Parents: 1dcb313
Author: Mikkel Andersen 
Authored: Mon Mar 20 12:15:34 2017 +0100
Committer: Benjamin Lerer 
Committed: Mon Mar 20 12:22:40 2017 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/filter/RowFilter.java   | 21 ++
 src/java/org/apache/cassandra/db/rows/Row.java  |  2 +-
 .../validation/entities/SecondaryIndexTest.java | 39 ++
 .../cql3/validation/operations/SelectTest.java  | 43 
 5 files changed, 96 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 97d8561..10402f3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
  * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)
  * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320)
  * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/src/java/org/apache/cassandra/db/filter/RowFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/RowFilter.java 
b/src/java/org/apache/cassandra/db/filter/RowFilter.java
index 11cfb87..5ffe2ab 100644
--- a/src/java/org/apache/cassandra/db/filter/RowFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/RowFilter.java
@@ -670,17 +670,20 @@ public abstract class RowFilter implements 
Iterable
 if (column.isComplex())
 {
 ComplexColumnData complexData = 
row.getComplexColumnData(column);
-for (Cell cell : complexData)
+if (complexData != null)
 {
-if (type.kind == CollectionType.Kind.SET)
+for (Cell cell : complexData)
 {
-if 
(type.nameComparator().compare(cell.path().get(0), value) == 0)
-return true;
-}
-else
-{
-if 
(type.valueComparator().compare(cell.value(), value) == 0)
-return true;
+if (type.kind == CollectionType.Kind.SET)
+{
+if 
(type.nameComparator().compare(cell.path().get(0), value) == 0)
+return true;
+}
+else
+{
+if 
(type.valueComparator().compare(cell.value(), value) == 0)
+return true;
+}
 }
 }
 return false;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0eebc6e6/src/java/org/apache/cassandra/db/rows/Row.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/Row.java 
b/src/java/org/apache/cassandra/db/rows/Row.java
index 74d8664..a61f365 100644
--- a/src/java/org/apache/cassandra/db/rows/Row.java
+++ b/src/java/org/apache/cassandra/db/rows/Row.java
@@ -129,7 +129,7 @@ public interface Row extends Unfiltered, 
Collection
  * The returned object groups all the cells for the column, as well as 
it's complex deletion (if relevant).
  *
  * @param c the complex column for which to return the complex data.
- * @return the data for {@code c} or {@code null} is the row has no data 
for this column.
+ * @return the data for {@code c} or {@code null} if the row has no data 
for this column.
  */
 public ComplexColumnData 

[jira] [Commented] (CASSANDRA-13339) java.nio.BufferOverflowException: null

[Chris Richards (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932484#comment-15932484
 ] 

Chris Richards commented on CASSANDRA-13339:


I've had a look through our logs over the weekend, and we saw several instances 
: all with the same backtrace as the original report.  This was running with 
Netty-4.0.44.

I've looked at one corrupt commit log : it has x1fffd68 bytes of useful stuff 
in it, then the remaining x298 bytes (up to 32MB) is full of zeroes.

> java.nio.BufferOverflowException: null
> --
>
> Key: CASSANDRA-13339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13339
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Chris Richards
>
> I'm seeing the following exception running Cassandra 3.9 (with Netty updated 
> to 4.1.8.Final) running on a 2 node cluster.  It would have been processing 
> around 50 queries/second at the time (mixture of 
> inserts/updates/selects/deletes) : there's a collection of tables (some with 
> counters some without) and a single materialized view.
> ERROR [MutationStage-4] 2017-03-15 22:50:33,052 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.serializeForMessaging(SerializationHeader.java:380)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:122)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serialize(PartitionUpdate.java:790)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.serialize(Mutation.java:393)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:279) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:241) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1347)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2539)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_121]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.9.jar:3.9]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> and then again shortly afterwards
> ERROR [MutationStage-3] 2017-03-15 23:27:36,198 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> 

[jira] [Assigned] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne reassigned CASSANDRA-13337:


Assignee: Sylvain Lebresne

> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:509)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:369)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:58)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:67)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:26)
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:96)
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:227)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:190)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:610)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> 

[jira] [Updated] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13246:
---
Status: Ready to Commit  (was: Patch Available)

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> 

[jira] [Commented] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932405#comment-15932405
 ] 

Benjamin Lerer commented on CASSANDRA-13246:


CI results look good.
|| 3.0 | [utests| 
http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13246-3.0-testall/] 
| [dtests| 
http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13246-3.0-dtest/]|
|| 3.11 | [utests| 
http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13246-3.11-testall/] 
| [dtests| 
http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13246-3.11-dtest/]|
|| trunk | [utests| 
http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13246-trunk-testall/]
 | [dtests| 
http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13246-trunk-dtest/]|

I just added an extra unit test to test filtering without secondary index.

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  

[jira] [Commented] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932375#comment-15932375
 ] 

Benjamin Lerer commented on CASSANDRA-13246:


Do not worry, I am just mentionning it for you to know (for your next patches 
;-) )
{{Ready to commit}} should normally be set by the reviewer to tell that the 
review is complete. If the reviewer is also a committer, he might simply commit 
the patch and skip that state.
So the workflow is simply:  

{noformat}
  +-+ 
+--+
+--- ok > | READY TO COMMIT | --> | 
RESOLVED | or something like that
+--++--+| +-+ 
+--+
| OPEN | -> | PATCH AVAILABLE  | ---+
+--++--+|  +--+ 
+-- not ok --> | OPEN |
   +--+ 
{noformat}
  
 

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> 

[jira] [Updated] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Mikkel Andersen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikkel Andersen updated CASSANDRA-13246:


Sorry Benjamin - did not mean to cause problems... could you send me the
link to where the workflow is described?

On Mon, Mar 20, 2017 at 9:40 AM, Benjamin Lerer (JIRA) 




-- 
Venlig Hilsen
Mikkel T. Andersen
Skjoldborgvej 8
7100 Vejle
Mobil: +45 40 26 79 26


> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  

[jira] [Comment Edited] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932326#comment-15932326
 ] 

Benjamin Lerer edited comment on CASSANDRA-13246 at 3/20/17 8:40 AM:
-

[~mikkel.t.ander...@gmail.com] Be carefull, the patch should be marked as 
{{Ready To Commit}} only when the reviewer give his green light. Which I did 
not do yet as I was waiting for the CI results as mentioned in my comment.


was (Author: blerer):
[~mikkel.t.ander...@gmail.com] Be carefull, the patch should be marked as 
{{Ready To Commit}} only when the reviewer give his green light. Which I did 
not do yet as I was waiting for the CI result as mentioned in my comment.

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> 

[jira] [Commented] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932326#comment-15932326
 ] 

Benjamin Lerer commented on CASSANDRA-13246:


[~mikkel.t.ander...@gmail.com] Be carefull, the patch should be marked as 
{{Ready To Commit}} only when the reviewer give his green light. Which I did 
not do yet as I was waiting for the CI result as mentioned in my comment.

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  

[jira] [Updated] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13246:
---
Status: Open  (was: Ready to Commit)

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> 

[jira] [Updated] (CASSANDRA-13246) Querying by secondary index on collection column returns NullPointerException sometimes

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13246:
---
Status: Patch Available  (was: Open)

> Querying by secondary index on collection column returns NullPointerException 
> sometimes
> ---
>
> Key: CASSANDRA-13246
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13246
> Project: Cassandra
>  Issue Type: Bug
> Environment: [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native 
> protocol v4] 
> One cassandra node up, with consistency ONE
>Reporter: hochung
>  Labels: easyfix
> Attachments: cassandra-13246.diff
>
>
> Not sure if this is the absolute minimal case that produces the bug, but here 
> are the steps for reproducing.
> 1. Create table
> {code}
> CREATE TABLE test (
> id text,
> ck1 text,
> ck2 text,
> static_value text static,
> set_value set,
> primary key (id, ck1, ck2)
> );
> {code}
> 2. Create secondary indices on the clustering columns, static column, and 
> collection column
> {code}
> create index on test (set_value);
> create index on test (static_value);
> create index on test (ck1);
> create index on test (ck2);
> {code}
> 3. Insert a null value into the `set_value` column
> {code}
> insert into test (id, ck1, ck2, static_value, set_value) values ('id', 
> 'key1', 'key2', 'static', {'one', 'two'} );
> {code}
> Sanity check: 
> {code}
> select * from test;
>  id | ck1  | ck2  | static_value | set_value
> +--+--+--+
>  id | key1 | key2 |   static | {'one', 'two'}
> {code}
> 4. Set the set_value to be empty
> {code}
> update test set set_value = {} where id = 'id' and ck1 = 'key1' and ck2 = 
> 'key2';
> {code}
> 5. Make a select query that uses `CONTAINS` in the `set_value` column
> {code}
> select * from test where ck2 = 'key2' and static_value = 'static' and 
> set_value contains 'one' allow filtering;
> {code}
> Here we get a ReadFailure:
> {code}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] 
> message="Operation failed - received 0 responses and 1 failures" 
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 
> 'consistency': 'ONE'}
> {code}
> Logs show a NullPointerException
> {code}
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.7.jar:3.7]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:720)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:303)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:120) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:293)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:281)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> 

[jira] [Resolved] (CASSANDRA-13350) getByPk() utility methods in Datastax session object

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-13350.
--
Resolution: Invalid

I'm pretty sure this is a request for the [DataStax Java 
driver|https://github.com/datastax/java-driver] which is a separate. Please 
inquire about this with said project.

> getByPk() utility methods in Datastax session object
> 
>
> Key: CASSANDRA-13350
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13350
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sachin Goyal
>
> Data modeling in Cassandra is the key to querying.
> Best way to query is to have tables where you always query by primary-key or 
> by partition-key.
> And yet there is no method in the datastax's session object that simplifies 
> this process.
> It would be great to have methods like:
> # session.getByPrimaryKey (String tableName, Object []primaryKeys)
> # session.getByPartitionKey (String tableName, Object []partitionKeys)
> # session.getByPartitionKeys (String tableName, Object [][]partitionKeys) // 
> Like an in-query
> # session.getByPrimaryKeys (String tableName, Object [][]primaryKeys)
> The last is an unsupported feature yet in Cassandra but it would be really 
> awesome to have the same. It would be like a read equivalent of the 
> batch-statements in write.
> Advantages:
> # Ease-of-use: User does not have to create a string query or a prepared 
> query.
> # User does not have to worry about [using prepared statements with select * 
> queries|https://docs.datastax.com/en/developer/java-driver/3.1/manual/statements/prepared/#avoid-preparing-select-queries].
>  I am not yet sure how the driver would handle it but if it can, wow!
> # If murmur-3 hashing in the client is same as the cluster, clients can query 
> just the right node (Better token-aware?)
> Such methods are present in several other software. Examples:
> # Hibernate: 
> [session.get()|https://www.mkyong.com/hibernate/different-between-session-get-and-session-load/]
>   and
> # JPA: [find()|http://www.java2s.com/Code/Java/JPA/GetEntitybyID.htm].
> # Solr: 
> [getById()|https://lucene.apache.org/solr/6_4_1/solr-solrj/org/apache/solr/client/solrj/SolrClient.html#getById-java.lang.String-java.util.Collection-org.apache.solr.common.params.SolrParams-]
>  and several flavors of the same.
> (Please note that these links are just an example, not meant to provide 
> implementation details or the behavior).
> As a feature, *session.getByPrimaryKeys (String tableName, Object 
> [][]primaryKeys)* should  help get a performance boost to the users because 
> it allows running queries for different partitions in parallel and also 
> allows getting results from the same partition in one query. We can put this 
> in a separate JIRA task if it is seen as a useful feature by all.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13351) CASSANDRA-11345 why not fix on 2.1.x?

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-13351.
--
Resolution: Won't Fix

> CASSANDRA-11345 why not fix on 2.1.x?
> -
>
> Key: CASSANDRA-13351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13351
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: cassandra 2.1.15
> jdk 1.8
>Reporter: zhaoyan
>
> I found same problem on cassandra 2.1.15
> the problem was reproduced In 2.1.13
> I don't know why not fix it on 2.1.x ?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13352) Cassandra does not respond back in 12000ms

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-13352.
--
Resolution: Invalid

There is nothing indicative of a bug here, so please use the user mailing list 
for such inquiries.

But to elaborate quickly, your query will have read the entirely of the table 
(the need to add {{ALLOW FILTERING}} is C* telling you that the performance of 
this query is likely to very very bad) so it's not particularly surprising that 
it can't even answer in 12 seconds.

> Cassandra does not respond back in 12000ms
> --
>
> Key: CASSANDRA-13352
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13352
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, CQL
>Reporter: Chirag
> Fix For: 3.10
>
>
> I have my system with a table design, 
> Create Table lamscope_dashboard.events (
> ToolId text,
> Date timestamp,
> End_Time timestamp,
> DeviceId text,
> Logtype text,
> EventId text,
> MaterialId text,
> CfgId text,
> MaterialType text,
> Status text,
> SlotNo text,
> LotId text,
> RecipeId text,
> StepNum int ,
> Fromdevice text,
> Fromslot text,
> ToDevice text,
> ToSlot int ,
> FlowRecipeId text,
> Flowinfo text,
> CarrierId text,
> JobId text,
> Data text,
> PRIMARY KEY( ToolId, Date, MaterialId, DeviceId, EventId))
> WITH CLUSTERING ORDER BY (Date ASC,MaterialId ASC, DeviceId ASC)
> AND COMPRESSION = { 'sstable_compression' : 'SnappyCompressor'} ;
> Query 
> select * from events where eventid='x' allow filtering;
> It is a single instance cluster.
> System is not responding back in 12000 ms. 
> Query goes timeout.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13196) test failure in snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13196:

Reviewer:   (was: Alex Petrov)

> test failure in 
> snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address
> -
>
> Key: CASSANDRA-13196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13196
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Aleksandr Sorokoumov
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1487/testReport/snitch_test/TestGossipingPropertyFileSnitch/test_prefer_local_reconnect_on_listen_address
> {code}
> {novnode}
> Error Message
> Error from server: code=2200 [Invalid query] message="keyspace keyspace1 does 
> not exist"
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-k6b0iF
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> cassandra.policies: INFO: Using datacenter 'dc1' for DCAwareRoundRobinPolicy 
> (via host '127.0.0.1'); if incorrect, please specify a local_dc to the 
> constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  discovered
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/snitch_test.py", line 87, in 
> test_prefer_local_reconnect_on_listen_address
> new_rows = list(session.execute("SELECT * FROM {}".format(stress_table)))
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 1998, in execute
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile, paging_state).result()
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 3784, in result
> raise self._final_exception
> 'Error from server: code=2200 [Invalid query] message="keyspace keyspace1 
> does not exist"\n >> begin captured logging << 
> \ndtest: DEBUG: cluster ccm directory: 
> /tmp/dtest-k6b0iF\ndtest: DEBUG: Done setting configuration options:\n{   
> \'initial_token\': None,\n\'num_tokens\': \'32\',\n
> \'phi_convict_threshold\': 5,\n\'range_request_timeout_in_ms\': 1,\n  
>   \'read_request_timeout_in_ms\': 1,\n\'request_timeout_in_ms\': 
> 1,\n\'truncate_request_timeout_in_ms\': 1,\n
> \'write_request_timeout_in_ms\': 1}\ncassandra.policies: INFO: Using 
> datacenter \'dc1\' for DCAwareRoundRobinPolicy (via host \'127.0.0.1\'); if 
> incorrect, please specify a local_dc to the constructor, or limit contact 
> points to local cluster nodes\ncassandra.cluster: INFO: New Cassandra host 
>  discovered\n- >> end captured 
> logging << -'
> {novnode}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-7826) support non-frozen, nested collections

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-7826:
--

Assignee: (was: Alex Petrov)

> support non-frozen, nested collections
> --
>
> Key: CASSANDRA-7826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7826
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Tupshin Harper
>  Labels: ponies
> Fix For: 3.11.x
>
>
> The inability to nest collections is one of the bigger data modelling 
> limitations we have right now.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13350) getByPk() utility methods in Datastax session object

[Sachin Goyal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sachin Goyal updated CASSANDRA-13350:
-
Summary: getByPk() utility methods in Datastax session object  (was: Having 
utility methods in session object)

> getByPk() utility methods in Datastax session object
> 
>
> Key: CASSANDRA-13350
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13350
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sachin Goyal
>
> Data modeling in Cassandra is the key to querying.
> Best way to query is to have tables where you always query by primary-key or 
> by partition-key.
> And yet there is no method in the datastax's session object that simplifies 
> this process.
> It would be great to have methods like:
> # session.getByPrimaryKey (String tableName, Object []primaryKeys)
> # session.getByPartitionKey (String tableName, Object []partitionKeys)
> # session.getByPartitionKeys (String tableName, Object [][]partitionKeys) // 
> Like an in-query
> # session.getByPrimaryKeys (String tableName, Object [][]primaryKeys)
> The last is an unsupported feature yet in Cassandra but it would be really 
> awesome to have the same. It would be like a read equivalent of the 
> batch-statements in write.
> Advantages:
> # Ease-of-use: User does not have to create a string query or a prepared 
> query.
> # User does not have to worry about [using prepared statements with select * 
> queries|https://docs.datastax.com/en/developer/java-driver/3.1/manual/statements/prepared/#avoid-preparing-select-queries].
>  I am not yet sure how the driver would handle it but if it can, wow!
> # If murmur-3 hashing in the client is same as the cluster, clients can query 
> just the right node (Better token-aware?)
> Such methods are present in several other software. Examples:
> # Hibernate: 
> [session.get()|https://www.mkyong.com/hibernate/different-between-session-get-and-session-load/]
>   and
> # JPA: [find()|http://www.java2s.com/Code/Java/JPA/GetEntitybyID.htm].
> # Solr: 
> [getById()|https://lucene.apache.org/solr/6_4_1/solr-solrj/org/apache/solr/client/solrj/SolrClient.html#getById-java.lang.String-java.util.Collection-org.apache.solr.common.params.SolrParams-]
>  and several flavors of the same.
> (Please note that these links are just an example, not meant to provide 
> implementation details or the behavior).
> As a feature, *session.getByPrimaryKeys (String tableName, Object 
> [][]primaryKeys)* should  help get a performance boost to the users because 
> it allows running queries for different partitions in parallel and also 
> allows getting results from the same partition in one query. We can put this 
> in a separate JIRA task if it is seen as a useful feature by all.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932203#comment-15932203
 ] 

sankalp kohli commented on CASSANDRA-11530:
---

This does not remove any repair options but only deprecated method correct? 

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13350) Having utility methods in session object

[Sachin Goyal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sachin Goyal updated CASSANDRA-13350:
-
Description: 
Data modeling in Cassandra is the key to querying.
Best way to query is to have tables where you always query by primary-key or by 
partition-key.

And yet there is no method in the datastax's session object that simplifies 
this process.
It would be great to have methods like:
# session.getByPrimaryKey (String tableName, Object []primaryKeys)
# session.getByPartitionKey (String tableName, Object []partitionKeys)
# session.getByPartitionKeys (String tableName, Object [][]partitionKeys) // 
Like an in-query
# session.getByPrimaryKeys (String tableName, Object [][]primaryKeys)

The last is an unsupported feature yet in Cassandra but it would be really 
awesome to have the same. It would be like a read equivalent of the 
batch-statements in write.

Advantages:
# Ease-of-use: User does not have to create a string query or a prepared query.
# User does not have to worry about [using prepared statements with select * 
queries|https://docs.datastax.com/en/developer/java-driver/3.1/manual/statements/prepared/#avoid-preparing-select-queries].
 I am not yet sure how the driver would handle it but if it can, wow!
# If murmur-3 hashing in the client is same as the cluster, clients can query 
just the right node (Better token-aware?)

Such methods are present in several other software. Examples:
# Hibernate: 
[session.get()|https://www.mkyong.com/hibernate/different-between-session-get-and-session-load/]
  and
# JPA: [find()|http://www.java2s.com/Code/Java/JPA/GetEntitybyID.htm].
# Solr: 
[getById()|https://lucene.apache.org/solr/6_4_1/solr-solrj/org/apache/solr/client/solrj/SolrClient.html#getById-java.lang.String-java.util.Collection-org.apache.solr.common.params.SolrParams-]
 and several flavors of the same.

(Please note that these links are just an example, not meant to provide 
implementation details or the behavior).

As a feature, *session.getByPrimaryKeys (String tableName, Object 
[][]primaryKeys)* should  help get a performance boost to the users because it 
allows running queries for different partitions in parallel and also allows 
getting results from the same partition in one query. We can put this in a 
separate JIRA task if it is seen as a useful feature by all.

  was:
Data modeling in Cassandra is the key to querying.
Best way to query is to have tables where you always query by primary-key or by 
partition-key.

And yet there is no method in the datastax's session object that simplifies 
this process.
It would be great to have methods like:
# session.getByPrimaryKey (String tableName, Object []primaryKeys)
# session.getByPartitionKey (String tableName, Object []partitionKeys)
# session.getByPartitionKeys (String tableName, Object [][]partitionKeys) // 
Like an in-query
# session.getByPrimaryKeys (String tableName, Object [][]primaryKeys)

The last is an unsupported feature yet in Cassandra but it would be really 
awesome to have the same. It would be like a read equivalent of the 
batch-statements in write.

Advantages:
# Ease-of-use: User does not have to create a string query or a prepared query.
# User does not have to worry about [using prepared statements with select * 
queries|https://docs.datastax.com/en/developer/java-driver/3.1/manual/statements/prepared/#avoid-preparing-select-queries].
 I am not yet sure how the driver would handle it but if it can, wow!
# If murmur-3 hashing in the client is same as the cluster, clients can query 
just the right node (Better token-aware?)

Tools like Hibernate provide such a feature. Examples: 
# 
[session.get()|https://www.mkyong.com/hibernate/different-between-session-get-and-session-load/]
  and
# [JPA.find|http://www.java2s.com/Code/Java/JPA/GetEntitybyID.htm].

(Please note that these links are just an example, not meant to provide 
implementation details or the behavior).

As a feature, *session.getByPrimaryKeys (String tableName, Object 
[][]primaryKeys)* should  help get a performance boost to the users because it 
allows running queries for different partitions in parallel and also allows 
getting results from the same partition in one query. We can put this in a 
separate JIRA task if it is seen as a useful feature by all.


> Having utility methods in session object
> 
>
> Key: CASSANDRA-13350
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13350
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sachin Goyal
>
> Data modeling in Cassandra is the key to querying.
> Best way to query is to have tables where you always query by primary-key or 
> by partition-key.
> And yet there is no method in the datastax's session object that simplifies 
> this process.
> It would be great to have methods like:
> #