[jira] [Updated] (CASSANDRA-11949) GC log directory should be created in startup scripts

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-11949:
--
Summary: GC log directory should be created in startup scripts  (was: GC 
log directory should be present/created in fresh clone)

> GC log directory should be created in startup scripts
> -
>
> Key: CASSANDRA-11949
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11949
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Joel Knighton
>Priority: Minor
>
> In [CASSANDRA-10140], we enabled GC logging by default, since the overhead 
> was low and asking people providing diagnostics to restart can often make it 
> more difficult to diagnose problems.
> The default GC log path is set to {{$CASSANDRA_HOME/logs/gc.log}} in 
> {{cassandra-env.sh}}, a directory that is not present in a fresh 
> clone/install. Even if logback creates this directory later in startup, it is 
> not present when the JVM initiates GC logging, so GC logging will silently 
> fail for this first Cassandra run
> I haven't tested this in Windows but suspect the same problem may occur. 
> Since lots of tooling around Cassandra won't create this directory, we should 
> instead consider attempting to create it in our startup scripts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11949) GC log directory should be present/created in fresh clone

[Joel Knighton (JIRA)]

Joel Knighton created CASSANDRA-11949:
-

 Summary: GC log directory should be present/created in fresh clone
 Key: CASSANDRA-11949
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11949
 Project: Cassandra
  Issue Type: Bug
Reporter: Joel Knighton
Priority: Minor


In [CASSANDRA-10140], we enabled GC logging by default, since the overhead was 
low and asking people providing diagnostics to restart can often make it more 
difficult to diagnose problems.

The default GC log path is set to {{$CASSANDRA_HOME/logs/gc.log}} in 
{{cassandra-env.sh}}, a directory that is not present in a fresh clone/install. 
Even if logback creates this directory later in startup, it is not present when 
the JVM initiates GC logging, so GC logging will silently fail for this first 
Cassandra run

I haven't tested this in Windows but suspect the same problem may occur. Since 
lots of tooling around Cassandra won't create this directory, we should instead 
consider attempting to create it in our startup scripts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11933) Improve Repair performance

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313449#comment-15313449
 ] 

Paulo Motta commented on CASSANDRA-11933:
-

Don't worry, they seem to be unrelated (flakey tests addressed elsewhere).

> Improve Repair performance
> --
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
>
> During  a full repair on a ~ 60 nodes cluster, I've been able to see that 
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
>  calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it 
> takes more than 99% of the time. This call takes 600ms when there is no load 
> on the cluster and more if there is. So for 10k ranges, you can imagine that 
> it takes at least 1.5 hours just to compute ranges. 
> Underneath it calls 
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
>  which can get pretty inefficient ([~jbellis]'s 
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend 
> hours on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11933) Improve Repair performance

[Mahdi Mohammadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313430#comment-15313430
 ] 

Mahdi Mohammadi commented on CASSANDRA-11933:
-

CI Reports 5 test failures in dtest and testall together. Does that mean 
something is wrong with my change?

> Improve Repair performance
> --
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
>
> During  a full repair on a ~ 60 nodes cluster, I've been able to see that 
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
>  calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it 
> takes more than 99% of the time. This call takes 600ms when there is no load 
> on the cluster and more if there is. So for 10k ranges, you can imagine that 
> it takes at least 1.5 hours just to compute ranges. 
> Underneath it calls 
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
>  which can get pretty inefficient ([~jbellis]'s 
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend 
> hours on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11933) Improve Repair performance

[Mahdi Mohammadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313426#comment-15313426
 ] 

Mahdi Mohammadi commented on CASSANDRA-11933:
-

You are right. Will rename that and check for other versions too.

> Improve Repair performance
> --
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
>
> During  a full repair on a ~ 60 nodes cluster, I've been able to see that 
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
>  calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it 
> takes more than 99% of the time. This call takes 600ms when there is no load 
> on the cluster and more if there is. So for 10k ranges, you can imagine that 
> it takes at least 1.5 hours just to compute ranges. 
> Underneath it calls 
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
>  which can get pretty inefficient ([~jbellis]'s 
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend 
> hours on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313417#comment-15313417
 ] 

Paulo Motta commented on CASSANDRA-8523:


I had a closer look at CASSANDRA-11559 and it should be possible to support 
that on trunk/3.x series without breaking compatibility, thus making this 
viable before 4.0 in a robust way. I will provide an initial patch for that 
soon.

We now need to decide whether we want to have a partial/fragile solution to 
this ticket before the ideal fix on 3.x leveraging #11559.

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8928) Add downgradesstables

[Kaide Mu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kaide Mu reassigned CASSANDRA-8928:
---

Assignee: Kaide Mu

> Add downgradesstables
> -
>
> Key: CASSANDRA-8928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8928
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Kaide Mu
>Priority: Minor
>  Labels: gsoc2016, mentor
>
> As mentioned in other places such as CASSANDRA-8047 and in the wild, 
> sometimes you need to go back.  A downgrade sstables utility would be nice 
> for a lot of reasons and I don't know that supporting going back to the 
> previous major version format would be too much code since we already support 
> reading the previous version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11875) Create sstableconvert tool with support to ma format

[Kaide Mu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313350#comment-15313350
 ] 

Kaide Mu commented on CASSANDRA-11875:
--

Hello Paulo, new 
[patch|https://github.com/apache/cassandra/compare/trunk...kdmu:trunk-11875?expand=1]
 available. Changed isCompatibleForWriting(String version) -> 
isCompatibleForWriting() since we are dealing directly with versions so it 
would be redundant to pass it.

bq.We should probably add a Descriptor.isCompatibleForWriting so we can check 
that directly from the descriptor, and use that to throw an exception on 
SSTableConverter if the version is incompatible

Added Descriptor.isCompatibleWriting(), but in case of {{SSTableConverter}} we 
don't have specified version {{Descriptor}} so we have to check it from 
{{Version}} 
{{DatabaseDescriptor.getSSTableFormat().info.getVersion(options.version).isCompatibleForWriting()}},
 or maybe there's some way to obtain it? 

bq.The idea of your WIP is correct, but instead of calling the main method of 
StandaloneUpgrader you could create a common method that receives an Options 
object, which in the case of the upgrader it would have a special flag 
indicating upgrade to latest version. The argument parsing and Options creation 
could be removed from the Options object to the main class so it's different 
between the converter and the upgrader. You can do different printing based on 
the options flag.

I modified Options access level to protected and parseArgs now varies on 
boolean isLatestVersion, so now {{StandaloneUpgrader}} only contains the 
execution code which is identical than {{StandaloneConverter}}, do you think we 
should abstract it into a common method SSTableConverter.execute(Option 
options)?

Thanks.

> Create sstableconvert tool with support to ma format
> 
>
> Key: CASSANDRA-11875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11875
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Kaide Mu
>Priority: Minor
> Attachments: trunk-11875-WIP-V1.patch
>
>
> Currently {{Upgrader}} receives an sstable in any readable format, and writes 
> into {{BigFormat.getLatestVersion()}}. We should generalize it by making it 
> receive a {{target}} version and probably also rename it to 
> {{SSTableConverter}}. 
> Based on this we can create an {{StandaloneDowngrader}} tool which will 
> perform downgrade of specified sstables to a target version. To start with, 
> we should support only downgrading to {{ma}} format (from current format 
> {{mb}}), downgrade to any other version should be forbidden. Since we already 
> support serializing to "ma" we will not need to do any data conversion.
> We should also create a test suite that creates an sstable with data in the 
> current format, perform the downgrade, and verify data in the new format is 
> correct. This will be the base tests suite for more advanced conversions in 
> the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11948) Wrong ByteBuffer comparisons in TimeTypeTest

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11948:
---
Assignee: Rei Odaira
  Status: Patch Available  (was: Open)

> Wrong ByteBuffer comparisons in TimeTypeTest
> 
>
> Key: CASSANDRA-11948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11948
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 11948-2.2.txt
>
>
> The last three tests in org.apache.cassandra.db.marshal.TimeTypeTest compare 
> the returned values of AbstractType.compare() with -1 or 1. However,  
> AbstractType.compare() is supposed to return a negative integer, zero, or a 
> positive integer, and it does not necessarily return -1 or 1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11948) Wrong ByteBuffer comparisons in TimeTypeTest

[Rei Odaira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rei Odaira updated CASSANDRA-11948:
---
  Flags: Patch
 Attachment: 11948-2.2.txt
Description: The last three tests in 
org.apache.cassandra.db.marshal.TimeTypeTest compare the returned values of 
AbstractType.compare() with -1 or 1. However,  AbstractType.compare() is 
supposed to return a negative integer, zero, or a positive integer, and it does 
not necessarily return -1 or 1.  (was: The last three tests in 
org.apache.cassandra.db.marshal.TimeTypeTest compare the returned values of 
AbstractType.compare() with -1 or 1. However,  AbstractType.compare() is 
supposed to return a negative integer, zero, or a positive integer, and it does 
not necessarily return -1 or 1. I will submit a patch shortly.)

> Wrong ByteBuffer comparisons in TimeTypeTest
> 
>
> Key: CASSANDRA-11948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11948
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Rei Odaira
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 11948-2.2.txt
>
>
> The last three tests in org.apache.cassandra.db.marshal.TimeTypeTest compare 
> the returned values of AbstractType.compare() with -1 or 1. However,  
> AbstractType.compare() is supposed to return a negative integer, zero, or a 
> positive integer, and it does not necessarily return -1 or 1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11944) sstablesInBounds might not actually give all sstables within the bounds due to having start positions moved in sstables

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11944:

Reviewer: Benedict

> sstablesInBounds might not actually give all sstables within the bounds due 
> to having start positions moved in sstables
> ---
>
> Key: CASSANDRA-11944
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11944
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> Same problem as with CASSANDRA-11886 - if we try to fetch sstablesInBounds 
> for CANONICAL_SSTABLES, we can miss some actually overlapping sstables. In 
> 3.0+ we state which SSTableSet we want when calling the method.
> Looks like the only issue this could cause is that we include a few too many 
> sstables in compactions that we think contain only droppable tombstones



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10862:

Assignee: Chen Shen
Reviewer: Paulo Motta

> LCS repair: compact tables before making available in L0
> 
>
> Key: CASSANDRA-10862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction, Streaming and Messaging
>Reporter: Jeff Ferland
>Assignee: Chen Shen
>
> When doing repair on a system with lots of mismatched ranges, the number of 
> tables in L0 goes up dramatically, as correspondingly goes the number of 
> tables referenced for a query. Latency increases dramatically in tandem.
> Eventually all the copied tables are compacted down in L0, then copied into 
> L1 (which may be a very large copy), finally reducing the number of SSTables 
> per query into the manageable range.
> It seems to me that the cleanest answer is to compact after streaming, then 
> mark tables available rather than marking available when the file itself is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313225#comment-15313225
 ] 

Paulo Motta commented on CASSANDRA-10862:
-

bq. Also, l'm willing to jump on this if it's not your top priority 

Definitely, go for it! I'm happy to review it.

You cannot yet be assigned to this issue but I will assign you when some admin 
add you to the assignable list.

> LCS repair: compact tables before making available in L0
> 
>
> Key: CASSANDRA-10862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction, Streaming and Messaging
>Reporter: Jeff Ferland
>
> When doing repair on a system with lots of mismatched ranges, the number of 
> tables in L0 goes up dramatically, as correspondingly goes the number of 
> tables referenced for a query. Latency increases dramatically in tandem.
> Eventually all the copied tables are compacted down in L0, then copied into 
> L1 (which may be a very large copy), finally reducing the number of SSTables 
> per query into the manageable range.
> It seems to me that the cleanest answer is to compact after streaming, then 
> mark tables available rather than marking available when the file itself is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11933) Improve Repair performance

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313220#comment-15313220
 ] 

Paulo Motta commented on CASSANDRA-11933:
-

This looks good, thanks! Just a minor nitpick: can you rename the variable from 
{{keyspaceLocalRange}} to {{keyspaceLocalRange*s*}}, or maybe just 
{{localRanges}} (it's implicit that is for a given keyspace)?

Also, could you check if this patch merges to cassandra-2.2 all the way up to 
trunk (via cassandra-3.7), and if not, provide patch for conflicting versions?

Submitted CI unit and dtests for 2.1:
||2.1||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.1...pauloricardomg:11933-2.1]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-11933-2.1-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-11933-2.1-dtest/lastCompletedBuild/testReport/]|



> Improve Repair performance
> --
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
>
> During  a full repair on a ~ 60 nodes cluster, I've been able to see that 
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
>  calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it 
> takes more than 99% of the time. This call takes 600ms when there is no load 
> on the cluster and more if there is. So for 10k ranges, you can imagine that 
> it takes at least 1.5 hours just to compute ranges. 
> Underneath it calls 
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
>  which can get pretty inefficient ([~jbellis]'s 
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend 
> hours on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Cassandra Wiki] Update of "ArticlesAndPresentations" by MahdiMohammadi

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "ArticlesAndPresentations" page has been changed by MahdiMohammadi:
https://wiki.apache.org/cassandra/ArticlesAndPresentations?action=diff=138=139

Comment:
Added a link to a new article about 'How Apache Cassandra executes CQL queries?'

   * 
[[http://posulliv.github.com/2009/09/07/building-a-small-cassandra-cluster-for-testing-and-development.html|Building
 a Small Cassandra Cluster for Testing]], September 2009
   * 
[[http://www.slideshare.net/hjort/persistncia-nas-nuvens-com-no-sql-hjort|Persistência
 nas Nuvens com NoSQL]], Brazilian Portuguese, June 2011
   * [[http://www.emtg.net78.net/2011/10/21/cassandra_hector.html|Cassandra y 
Hector]], Spanish, October 2011
+  * 
[[http://www.mahdix.com/blog/2016/06/01/how-apache-cassandra-executes-cql-queries/|How
 Apache Cassandra executes CQL queries?]], English, June 2016
  
  = Presentations =
   * [[http://www.roviyo.com/documents/NoSQL-Apache_Cassandra.pdf|Bases de 
Datos NoSQL - Apache Cassandra (in spanish)]] - Roberto Amor, April 2012

[jira] [Comment Edited] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Chen Shen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313089#comment-15313089
 ] 

Chen Shen edited comment on CASSANDRA-10862 at 6/2/16 10:36 PM:


Sorry for the late reply due to vacation. 
[~pauloricardomg] This is happening during streaming stage and we are on 2.2.5 
by running nodetool repair -pr. So it seems CASSANDRA-6851 should have been 
included in this version and we should perform STCS on received sstables right? 

Also, l'm willing to jump on this if it's not your top priority :)


was (Author: scv...@gmail.com):
Sorry for the late reply due to vacation. 
[~pauloricardomg] This is happening during streaming stage and we are on 2.2.5. 
So it seems CASSANDRA-6851 should have been included in this version and we 
should perform STCS on received sstables right? 

Also, l'm willing to jump on this if it's not your top priority :)

> LCS repair: compact tables before making available in L0
> 
>
> Key: CASSANDRA-10862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction, Streaming and Messaging
>Reporter: Jeff Ferland
>
> When doing repair on a system with lots of mismatched ranges, the number of 
> tables in L0 goes up dramatically, as correspondingly goes the number of 
> tables referenced for a query. Latency increases dramatically in tandem.
> Eventually all the copied tables are compacted down in L0, then copied into 
> L1 (which may be a very large copy), finally reducing the number of SSTables 
> per query into the manageable range.
> It seems to me that the cleanest answer is to compact after streaming, then 
> mark tables available rather than marking available when the file itself is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11933) Improve Repair performance

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11933:

Reviewer: Paulo Motta

> Improve Repair performance
> --
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
>
> During  a full repair on a ~ 60 nodes cluster, I've been able to see that 
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
>  calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it 
> takes more than 99% of the time. This call takes 600ms when there is no load 
> on the cluster and more if there is. So for 10k ranges, you can imagine that 
> it takes at least 1.5 hours just to compute ranges. 
> Underneath it calls 
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
>  which can get pretty inefficient ([~jbellis]'s 
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend 
> hours on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11933) Improve Repair performance

[Mahdi Mohammadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313183#comment-15313183
 ] 

Mahdi Mohammadi commented on CASSANDRA-11933:
-

How can I run my branch in ci (for testall and dtests)?

> Improve Repair performance
> --
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
>
> During  a full repair on a ~ 60 nodes cluster, I've been able to see that 
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
>  calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it 
> takes more than 99% of the time. This call takes 600ms when there is no load 
> on the cluster and more if there is. So for 10k ranges, you can imagine that 
> it takes at least 1.5 hours just to compute ranges. 
> Underneath it calls 
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
>  which can get pretty inefficient ([~jbellis]'s 
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend 
> hours on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11933) Improve Repair performance

[Mahdi Mohammadi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313181#comment-15313181
 ] 

Mahdi Mohammadi commented on CASSANDRA-11933:
-

[Branch for 2.1|https://github.com/mm-binary/cassandra/tree/11933-2.1]

> Improve Repair performance
> --
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
>
> During  a full repair on a ~ 60 nodes cluster, I've been able to see that 
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
>  calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it 
> takes more than 99% of the time. This call takes 600ms when there is no load 
> on the cluster and more if there is. So for 10k ranges, you can imagine that 
> it takes at least 1.5 hours just to compute ranges. 
> Underneath it calls 
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
>  which can get pretty inefficient ([~jbellis]'s 
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend 
> hours on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Chen Shen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313089#comment-15313089
 ] 

Chen Shen edited comment on CASSANDRA-10862 at 6/2/16 9:37 PM:
---

Sorry for the late reply due to vacation. 
[~pauloricardomg] This is happening during streaming stage and we are on 2.2.5. 
So it seems CASSANDRA-6851 should have been included in this version and we 
should perform STCS on received sstables right? 

Also, l'm willing to jump on this if it's not your top priority :)


was (Author: scv...@gmail.com):
[~pauloricardomg] This is happening during streaming stage and we are on 2.2.5. 
So it seems CASSANDRA-6851 should have been included in this version and we 
should perform STCS on received sstables right? 

> LCS repair: compact tables before making available in L0
> 
>
> Key: CASSANDRA-10862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction, Streaming and Messaging
>Reporter: Jeff Ferland
>
> When doing repair on a system with lots of mismatched ranges, the number of 
> tables in L0 goes up dramatically, as correspondingly goes the number of 
> tables referenced for a query. Latency increases dramatically in tandem.
> Eventually all the copied tables are compacted down in L0, then copied into 
> L1 (which may be a very large copy), finally reducing the number of SSTables 
> per query into the manageable range.
> It seems to me that the cleanest answer is to compact after streaming, then 
> mark tables available rather than marking available when the file itself is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Chen Shen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313089#comment-15313089
 ] 

Chen Shen commented on CASSANDRA-10862:
---

[~pauloricardomg] This is happening during streaming stage and we are on 2.2.5. 
So it seems CASSANDRA-6851 should have been included in this version and we 
should perform STCS on received sstables right? 

> LCS repair: compact tables before making available in L0
> 
>
> Key: CASSANDRA-10862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction, Streaming and Messaging
>Reporter: Jeff Ferland
>
> When doing repair on a system with lots of mismatched ranges, the number of 
> tables in L0 goes up dramatically, as correspondingly goes the number of 
> tables referenced for a query. Latency increases dramatically in tandem.
> Eventually all the copied tables are compacted down in L0, then copied into 
> L1 (which may be a very large copy), finally reducing the number of SSTables 
> per query into the manageable range.
> It seems to me that the cleanest answer is to compact after streaming, then 
> mark tables available rather than marking available when the file itself is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11948) Wrong ByteBuffer comparisons in TimeTypeTest

[Rei Odaira (JIRA)]

Rei Odaira created CASSANDRA-11948:
--

 Summary: Wrong ByteBuffer comparisons in TimeTypeTest
 Key: CASSANDRA-11948
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11948
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Rei Odaira
Priority: Minor
 Fix For: 2.2.x, 3.0.x, 3.x


The last three tests in org.apache.cassandra.db.marshal.TimeTypeTest compare 
the returned values of AbstractType.compare() with -1 or 1. However,  
AbstractType.compare() is supposed to return a negative integer, zero, or a 
positive integer, and it does not necessarily return -1 or 1. I will submit a 
patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11930) Range tombstone serialisation between 2.1 and 3.0 nodes (in 3.0 -> 2.1 direction) is broken for some Thrift deletions

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313032#comment-15313032
 ] 

Tyler Hobbs commented on CASSANDRA-11930:
-

Patch and pending CI runs:

||branch||testall||dtest||
|[CASANDRA-11930-3.0|https://github.com/thobbs/cassandra/tree/CASANDRA-11930-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASANDRA-11930-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASANDRA-11930-3.0-dtest]|
|[CASANDRA-11930-3.7|https://github.com/thobbs/cassandra/tree/CASANDRA-11930-3.7]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASANDRA-11930-3.7-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASANDRA-11930-3.7-dtest]|
|[CASANDRA-11930-trunk|https://github.com/thobbs/cassandra/tree/CASANDRA-11930-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASANDRA-11930-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASANDRA-11930-trunk-dtest]|

Dtest PR that reproduces the issue (and can reproduce the ordering problem 
described above, but ignores it for now): 
https://github.com/riptano/cassandra-dtest/pull/1016

> Range tombstone serialisation between 2.1 and 3.0 nodes (in 3.0 -> 2.1 
> direction) is broken for some Thrift deletions
> -
>
> Key: CASSANDRA-11930
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11930
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Tyler Hobbs
> Fix For: 3.0.x, 3.x
>
>
> {{LegacyLayout.LegacyRangeTombstoneList::serialize()}} has a broken 
> assumption that a range tombstone implies {{CompositeType}}. This is 
> incorrect, as you can have non-{{CompositeType}} range tombstones created via 
> Thrift in 2.1, and as such wrapping {{clusteringComparator.subtypes()}} in a 
> {{CompositeType}} is incorrect. On 2.1/2.2 side, when decoding the range 
> tombstone list, {{RangeTombstoneList::deserialize()}} will use the raw type 
> to decode start and end bounds, which will end up being confused by the extra 
> 2 bytes in the beginning (short length) plus end of component header in the 
> end.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11930) Range tombstone serialisation between 2.1 and 3.0 nodes (in 3.0 -> 2.1 direction) is broken for some Thrift deletions

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-11930:

Reviewer: Aleksey Yeschenko
  Status: Patch Available  (was: In Progress)

> Range tombstone serialisation between 2.1 and 3.0 nodes (in 3.0 -> 2.1 
> direction) is broken for some Thrift deletions
> -
>
> Key: CASSANDRA-11930
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11930
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Tyler Hobbs
> Fix For: 3.0.x, 3.x
>
>
> {{LegacyLayout.LegacyRangeTombstoneList::serialize()}} has a broken 
> assumption that a range tombstone implies {{CompositeType}}. This is 
> incorrect, as you can have non-{{CompositeType}} range tombstones created via 
> Thrift in 2.1, and as such wrapping {{clusteringComparator.subtypes()}} in a 
> {{CompositeType}} is incorrect. On 2.1/2.2 side, when decoding the range 
> tombstone list, {{RangeTombstoneList::deserialize()}} will use the raw type 
> to decode start and end bounds, which will end up being confused by the extra 
> 2 bytes in the beginning (short length) plus end of component header in the 
> end.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11844) Create compaction-stress

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313017#comment-15313017
 ] 

T Jake Luciani commented on CASSANDRA-11844:


Slight changes added to maintain support for CASSANDRA-8671 



> Create compaction-stress
> 
>
> Key: CASSANDRA-11844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11844
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Compaction, Tools
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.x
>
>
> A tool like cassandra-stress that works with stress yaml but:
>   * writes directly to a specified dir using CQLSSTableWriter. 
>   * lets you run just compaction on that directory and generates a report on 
> compaction throughput.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11582) Slow table creation

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312972#comment-15312972
 ] 

Chris Lohfink commented on CASSANDRA-11582:
---

possibly related to https://datastax-oss.atlassian.net/browse/PYTHON-531

> Slow table creation
> ---
>
> Key: CASSANDRA-11582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11582
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: C* 3.5
> OpenSUSE 42.1
> JDK 1.8u66
>Reporter: Jaroslav Kamenik
>
> In last versions of Cassandra we experienced much slower creation of tables. 
> It happens even at single PC, where there is no need to do broadcast schema 
> change etc. It works well, but it is little annoying to wait when you have to 
> recreate lots of tables... 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11947) Ability to create a Materialized View without primary key column that has '=' relation

[Nimi Wariboko Jr. (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nimi Wariboko Jr. updated CASSANDRA-11947:
--
Description: 
I have a use case where I'd like to create a materialized view where one of the 
columns is filtered on a '=' relation and the primary key for that materialized 
view doesn't include that column.

Example:
{code}
CREATE TABLE analytics (
  resourceId text,
  country text,
  date timestamp,
  clicks int,
  PRIMARY KEY(resourceId, date, country)
);

CREATE MATERIALIZED VIEW analytics_ww AS
   SELECT * FROM analytics
   WHERE country = 'WW'
   PRIMARY KEY (resourceId, date);
{code}

This way I can have a table filtered on a high cardinality column, and I don't 
also have to specify that column, even though I already know thats the only 
value in the view.

  was:
I have a use case where I'd like to create a materialized view where one of the 
columns is filtered on a '=' relation and the primary key for that materialized 
view doesn't include that column.

Example:
{code}
CREATE TABLE analytics (
  resourceId text,
  country text,
  date timestamp,
  clicks int,
  PRIMARY KEY(resourceId, date, country)
);

CREATE MATERIALIZED VIEW analytics_ww AS
   SELECT * FROM analytics
   WHERE country = 'WW'
   PRIMARY KEY (resourceId, date);
{/code}

This way I can have a table filtered on a high cardinality column, and I don't 
also have to specify that column, even though I already know thats the only 
value in the view.


> Ability to create a Materialized View without primary key column that has '=' 
> relation
> --
>
> Key: CASSANDRA-11947
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11947
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Nimi Wariboko Jr.
>
> I have a use case where I'd like to create a materialized view where one of 
> the columns is filtered on a '=' relation and the primary key for that 
> materialized view doesn't include that column.
> Example:
> {code}
> CREATE TABLE analytics (
>   resourceId text,
>   country text,
>   date timestamp,
>   clicks int,
>   PRIMARY KEY(resourceId, date, country)
> );
> CREATE MATERIALIZED VIEW analytics_ww AS
>SELECT * FROM analytics
>WHERE country = 'WW'
>PRIMARY KEY (resourceId, date);
> {code}
> This way I can have a table filtered on a high cardinality column, and I 
> don't also have to specify that column, even though I already know thats the 
> only value in the view.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11947) Ability to create a Materialized View without primary key column that has '=' relation

[Nimi Wariboko Jr. (JIRA)]

Nimi Wariboko Jr. created CASSANDRA-11947:
-

 Summary: Ability to create a Materialized View without primary key 
column that has '=' relation
 Key: CASSANDRA-11947
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11947
 Project: Cassandra
  Issue Type: Improvement
Reporter: Nimi Wariboko Jr.


I have a use case where I'd like to create a materialized view where one of the 
columns is filtered on a '=' relation and the primary key for that materialized 
view doesn't include that column.

Example:
{code}
CREATE TABLE analytics (
  resourceId text,
  country text,
  date timestamp,
  clicks int,
  PRIMARY KEY(resourceId, date, country)
);

CREATE MATERIALIZED VIEW analytics_ww AS
   SELECT * FROM analytics
   WHERE country = 'WW'
   PRIMARY KEY (resourceId, date);
{/code}

This way I can have a table filtered on a high cardinality column, and I don't 
also have to specify that column, even though I already know thats the only 
value in the view.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11875) Create sstableconvert tool with support to ma format

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312874#comment-15312874
 ] 

Paulo Motta commented on CASSANDRA-11875:
-

Thanks for the update. Overall the patch is looking much better, just a few 
minor nits:
* CONVERT_SSTABLES("Converte sstables") -> CONVERT_SSTABLES("Convert sstables"),
* {{compatibleForWriting}} set on BigFormat is a constant, so should probably 
be named {{supported_write_formats}} and move next to {{current_version}} and 
{{earliest_supported_version}} to be consistent with other constants in 
{{BigVersion}}
* We should probably add a {{Descriptor.isCompatibleForWriting}} so we can 
check that directly from the descriptor, and use that to throw an exception on 
{{SSTableConverter}} if the version is incompatible
** You may have a look on a similar situation on 
{{ColumnFamilyStore.loadNewSSTables}}

On the tests, it would be nice if you could read the data before and after 
converting (SELECT) to verify it's consistent. Similarly, you should check that 
commit log lower bound is different from {{ReplayPosition.NONE}} before the 
conversion. On {{testUnsupportedVersionShouldFail}} the test does not guarantee 
the program failed with an error, so it's probably better to test that an 
exception is being thrown when you try to convert with {{SSTableConverter}} 
with an unsupported version.

bq. I'm not sure if we have to make StandaloneUpgrader subclass of 
StandaloneConverter. I did a WIP for that, since we have to validate passed 
args (validate and add latestVersion to it), we should maintain original 
parseArgs of StandaloneUpgrader, also since printUsage is different from 
StandaloneConverter we should also maintain it. 

The idea of your WIP is correct, but instead of calling the main method of 
{{StandaloneUpgrader}} you could create a common method that receives an 
{{Options}} object, which in the case of the upgrader it would have a special 
flag indicating upgrade to latest version. The argument parsing and Options 
creation could be removed from the Options object to the main class so it's 
different between the converter and the upgrader. You can do different printing 
based on the options flag.

> Create sstableconvert tool with support to ma format
> 
>
> Key: CASSANDRA-11875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11875
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Kaide Mu
>Priority: Minor
> Attachments: trunk-11875-WIP-V1.patch
>
>
> Currently {{Upgrader}} receives an sstable in any readable format, and writes 
> into {{BigFormat.getLatestVersion()}}. We should generalize it by making it 
> receive a {{target}} version and probably also rename it to 
> {{SSTableConverter}}. 
> Based on this we can create an {{StandaloneDowngrader}} tool which will 
> perform downgrade of specified sstables to a target version. To start with, 
> we should support only downgrading to {{ma}} format (from current format 
> {{mb}}), downgrade to any other version should be forbidden. Since we already 
> support serializing to "ma" we will not need to do any data conversion.
> We should also create a test suite that creates an sstable with data in the 
> current format, perform the downgrade, and verify data in the new format is 
> correct. This will be the base tests suite for more advanced conversions in 
> the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11067) Improve SASI syntax

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-11067:

Labels: client-impacting sasi  (was: client-impacting)

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Pavel Yaskevich
>  Labels: client-impacting, sasi
> Fix For: 3.4
>
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11752) histograms/metrics in 2.2 do not appear recency biased

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312796#comment-15312796
 ] 

Per Otterström commented on CASSANDRA-11752:


By using sliding window only, all collected metrics from the last 5 minutes 
will be equally significant.

On the other hand, by using decay only, the user may very well see values on 
the percentiles even thought the node has not processed any requests for the 
last five minutes.

The reason for having both a sliding window and decay would be to make the 
metrics from the last minute more significant than the metrics collected 5 
minutes ago. And an idling node would show zero-values on the percentiles after 
5 minutes. But I may be over engineering this a bit. A pure decay approach is 
probably good enough.

> histograms/metrics in 2.2 do not appear recency biased
> --
>
> Key: CASSANDRA-11752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11752
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Chris Burroughs
>  Labels: metrics
> Attachments: boost-metrics.png, c-jconsole-comparison.png, 
> c-metrics.png, default-histogram.png
>
>
> In addition to upgrading to metrics3, CASSANDRA-5657 switched to using  a 
> custom histogram implementation.  After upgrading to Cassandra 2.2 
> histograms/timer metrics are not suspiciously flat.  To be useful for 
> graphing and alerting metrics need to be biased towards recent events.
> I have attached images that I think illustrate this.
>  * The first two are a comparison between latency observed by a C* 2.2 (us) 
> cluster shoring very flat lines and a client (using metrics 2.2.0, ms) 
> showing server performance problems.  We can't rule out with total certainty 
> that something else isn't the cause (that's why we measure from both the 
> client & server) but they very rarely disagree.
>  * The 3rd image compares jconsole viewing of metrics on a 2.2 and 2.1 
> cluster over several minutes.  Not a single digit changed on the 2.2 cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11456) support for PreparedStatement with LIKE

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-11456:

Labels: sasi  (was: )

> support for PreparedStatement with LIKE
> ---
>
> Key: CASSANDRA-11456
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11456
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Pavel Yaskevich
>Assignee: Sam Tunnicliffe
>Priority: Minor
>  Labels: sasi
> Fix For: 3.6
>
>
> Using the Java driver for example:
> {code}
> PreparedStatement pst = session.prepare("select * from test.users where 
> first_name LIKE ?");
> BoundStatement bs = pst.bind("Jon%");
> {code}
> The first line fails with {{SyntaxError: line 1:47 mismatched input '?' 
> expecting STRING_LITERAL}} (which makes sense since it's how it's declared in 
> the grammar). Other operators declare the right-hand side value as a 
> {{Term.Raw}}, which can also be a bind marker.
> I think users will expect to be able to bind the argument this way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11324) Make SSTABLESCRUB and related jobs multi-threaded / parallelize somehow

[Thom Valley (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312686#comment-15312686
 ] 

Thom Valley commented on CASSANDRA-11324:
-

I disagree.

SSTABLESCRUB is run on a keyspace or keyspace/table.

A single table can have hundreds of SSTABLEs, which _could_ be run in parallel.

Expecting a user to write a script to efficiently do basic maintenance is a 
serious usability problem.

> Make SSTABLESCRUB and related jobs multi-threaded  / parallelize somehow
> 
>
> Key: CASSANDRA-11324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11324
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Thom Valley
>Priority: Minor
>
> SSTABLE scrub right now appears to be single threaded in offline mode (unsure 
> about online).  When scrubbing a node, especially a well provisioned one, the 
> node sits mostly idle for potentially hours while scrubbing.
> Making better use of available resources through parallelization / 
> multi-threading of the scrub process seems like a good idea.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11324) Make SSTABLESCRUB and related jobs multi-threaded / parallelize somehow

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-11324.
---
Resolution: Not A Problem

> Make SSTABLESCRUB and related jobs multi-threaded  / parallelize somehow
> 
>
> Key: CASSANDRA-11324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11324
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Thom Valley
>Priority: Minor
>
> SSTABLE scrub right now appears to be single threaded in offline mode (unsure 
> about online).  When scrubbing a node, especially a well provisioned one, the 
> node sits mostly idle for potentially hours while scrubbing.
> Making better use of available resources through parallelization / 
> multi-threading of the scrub process seems like a good idea.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11324) Make SSTABLESCRUB and related jobs multi-threaded / parallelize somehow

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312675#comment-15312675
 ] 

Aleksey Yeschenko commented on CASSANDRA-11324:
---

You can initiate multiple scrub processes to run in parallel, processing one 
sstable each. It's not possible to parallelise scrub on a single sstable, 
however, as you really need to go byte by byte, without skipping. As such, 
closing as Not A Problem.

> Make SSTABLESCRUB and related jobs multi-threaded  / parallelize somehow
> 
>
> Key: CASSANDRA-11324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11324
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Thom Valley
>Priority: Minor
>
> SSTABLE scrub right now appears to be single threaded in offline mode (unsure 
> about online).  When scrubbing a node, especially a well provisioned one, the 
> node sits mostly idle for potentially hours while scrubbing.
> Making better use of available resources through parallelization / 
> multi-threading of the scrub process seems like a good idea.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11582) Slow table creation

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312666#comment-15312666
 ] 

Aleksey Yeschenko commented on CASSANDRA-11582:
---

This is most likely a driver-related issue, as nothing really changed wrt 
schema propagation in 3.0 or later. Try java-driver/python-driver JIRAs if you 
can still reproduce at all.

> Slow table creation
> ---
>
> Key: CASSANDRA-11582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11582
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: C* 3.5
> OpenSUSE 42.1
> JDK 1.8u66
>Reporter: Jaroslav Kamenik
>
> In last versions of Cassandra we experienced much slower creation of tables. 
> It happens even at single PC, where there is no need to do broadcast schema 
> change etc. It works well, but it is little annoying to wait when you have to 
> recreate lots of tables... 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11582) Slow table creation

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-11582.
---
Resolution: Not A Problem

> Slow table creation
> ---
>
> Key: CASSANDRA-11582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11582
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: C* 3.5
> OpenSUSE 42.1
> JDK 1.8u66
>Reporter: Jaroslav Kamenik
>
> In last versions of Cassandra we experienced much slower creation of tables. 
> It happens even at single PC, where there is no need to do broadcast schema 
> change etc. It works well, but it is little annoying to wait when you have to 
> recreate lots of tables... 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11752) histograms/metrics in 2.2 do not appear recency biased

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312659#comment-15312659
 ] 

Chris Lohfink commented on CASSANDRA-11752:
---

I am confused on why you have a sliding window and decaying function. can kinda 
just do one or the other. 

Sliding Window: if keeping 5 copies of the EH why not just have the same as EH 
but keep a buffer of long[5][offsetsize] that every minute takes the current 
values of the bucket and stores them. then on read compare oldest in the buffer 
to the current, and the delta is the last 5 minutes. can treat the long[5] as a 
cyclic buffer. Can still provide the current "getValues" as the current total 
value but the percentiles can be computed on the delta.

Decayed: I was thinking more like a 
http://www.source-code.biz/snippets/java/11.htm per bucket (a forward decaying 
i think would be best). Why keep the 5 sets of them?

Decaying the snapshots sounds like it has the overhead of a sliding window and 
the complexity/performance/accuracy impact of a decaying algorithm. although I 
might have to see the implementation and may be missing something though.

> histograms/metrics in 2.2 do not appear recency biased
> --
>
> Key: CASSANDRA-11752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11752
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Chris Burroughs
>  Labels: metrics
> Attachments: boost-metrics.png, c-jconsole-comparison.png, 
> c-metrics.png, default-histogram.png
>
>
> In addition to upgrading to metrics3, CASSANDRA-5657 switched to using  a 
> custom histogram implementation.  After upgrading to Cassandra 2.2 
> histograms/timer metrics are not suspiciously flat.  To be useful for 
> graphing and alerting metrics need to be biased towards recent events.
> I have attached images that I think illustrate this.
>  * The first two are a comparison between latency observed by a C* 2.2 (us) 
> cluster shoring very flat lines and a client (using metrics 2.2.0, ms) 
> showing server performance problems.  We can't rule out with total certainty 
> that something else isn't the cause (that's why we measure from both the 
> client & server) but they very rarely disagree.
>  * The 3rd image compares jconsole viewing of metrics on a 2.2 and 2.1 
> cluster over several minutes.  Not a single digit changed on the 2.2 cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11847) Cassandra dies on a specific node in a multi-DC environment

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-11847.
---
Resolution: Not A Problem

> Cassandra dies on a specific node in a multi-DC environment
> ---
>
> Key: CASSANDRA-11847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11847
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Core
> Environment: Cassandra 2.0.11, JDK build 1.7.0_79-b15
>Reporter: Rajesh Babu
> Attachments: java_error19030.log, java_error2912.log, 
> java_error4571.log, java_error7539.log, java_error9552.log
>
>
> We've a customer who runs a 16 node 2 DC (8 nodes each) environment where 
> Cassandra pid dies randomly but on a specific node.
> Whenever Cassandra dies, admin has to manually restart Cassandra only on that 
> node.
> I tried upgrading their environment from java 1.7 (patch 60) to java 1.7 
> (patch 79) but it still seems to be an issue. 
> Is this a known hardware related bug or should is this issue fixed in later 
> Cassandra versions? 
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x7f4542d5a27f, pid=19030, tid=139933154096896
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_79-b15) (build 
> 1.7.0_79-b15)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode 
> linux-amd64 compressed oops)
> # Problematic frame:
> # C  [libjava.so+0xe027f]  _fini+0xbd5f7
> #
> # Core dump written. Default location: /tmp/core or core.19030
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
> ---  T H R E A D  ---
> Current thread (0x7f453c89f000):  JavaThread "COMMIT-LOG-WRITER" 
> [_thread_in_vm, id=19115, stack(0x7f44b9ed3000,0x7f44b9f14000)]
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=2 (SEGV_ACCERR), 
> si_addr=0x7f4542d5a27f
> Registers:
> RAX=0x, RBX=0x7f453c564ad0, RCX=0x0001, 
> RDX=0x0020
> RSP=0x7f44b9f125a0, RBP=0x7f44b9f125b0, RSI=0x, 
> RDI=0x0001
> R8 =0x7f453c564ad8, R9 =0x4aab, R10=0x7f453917a52c, 
> R11=0x0006fae57068
> R12=0x7f453c564ad8, R13=0x7f44b9f125d0, R14=0x, 
> R15=0x7f453c89f000
> RIP=0x7f4542d5a27f, EFLAGS=0x00010246, CSGSFS=0x0033, 
> ERR=0x0014
>   TRAPNO=0x000e
> -
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x7f28e08787a4, pid=2912, tid=139798767699712
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_79-b15) (build 
> 1.7.0_79-b15)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode 
> linux-amd64 compressed oops)
> # Problematic frame:
> # C  0x7f28e08787a4
> #
> # Core dump written. Default location: /tmp/core or core.2912
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
> ---  T H R E A D  ---
> Current thread (0x7f2640008000):  JavaThread "ValidationExecutor:15" 
> daemon [_thread_in_Java, id=7393, 
> stack(0x7f256fdf8000,0x7f256fe39000)]
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=2 (SEGV_ACCERR), 
> si_addr=0x7f28e08787a4
> Registers:
> RAX=0x, RBX=0x3f8bb878, RCX=0xc77040d6, 
> RDX=0xc770409a
> RSP=0x7f256fe37430, RBP=0x00063b820710, RSI=0x00063b820530, 
> RDI=0x
> R8 =0x3f8bb888, R9 =0x, R10=0x3f8bb888, 
> R11=0x3f8bb878
> R12=0x, R13=0x00063b820530, R14=0x000b, 
> R15=0x7f2640008000
> RIP=0x7f28e08787a4, EFLAGS=0x00010246, CSGSFS=0x0033, 
> ERR=0x0015
>   TRAPNO=0x000e



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9811) Tie counter shards' logical clock value to timestamps

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-9811:
---
Status: Ready to Commit  (was: Patch Available)

> Tie counter shards' logical clock value to timestamps
> -
>
> Key: CASSANDRA-9811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9811
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.x
>
>
> CASSANDRA-6506 will get rid of counter shards, turning them into elements of 
> a map. And we'll be using the logical clock values as map element timestamps.
> To avoid unpleasant initial performance (being unable to perform certain 
> sstable optimisations on the read path), we should switch the shards to use 
> {{max(timestamp, clock + 1)}} now, in advance of CASSANDRA-6506 switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9811) Tie counter shards' logical clock value to timestamps

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312660#comment-15312660
 ] 

Sam Tunnicliffe commented on CASSANDRA-9811:


Pending CI, LGTM.

> Tie counter shards' logical clock value to timestamps
> -
>
> Key: CASSANDRA-9811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9811
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.x
>
>
> CASSANDRA-6506 will get rid of counter shards, turning them into elements of 
> a map. And we'll be using the logical clock values as map element timestamps.
> To avoid unpleasant initial performance (being unable to perform certain 
> sstable optimisations on the read path), we should switch the shards to use 
> {{max(timestamp, clock + 1)}} now, in advance of CASSANDRA-6506 switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11187) DESC table on a table with UDT's should also print it's Types

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-11187.
---
Resolution: Not A Problem

I can see how this is potentially useless, but would argue that {{DESC TABLE}} 
should only describe the table itself. If needed, you can stitch schema for 
types manually, which is only slightly inconvenient.

> DESC table on a table with UDT's should also print it's Types
> -
>
> Key: CASSANDRA-11187
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11187
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sebastian Estevez
>Priority: Minor
>
> Lot's of folks use desc table to capture table definitions. When you describe 
> a table with UDT's today it doesn't also spit out it's CREATE TYPE statements 
> which makes it tricky and inconvenient to share tabe definitions with UDT's.
> Current functionality:
> {code}
> > desc TABLE payments.payments ;
> CREATE TABLE payments.payments (
> branch text,
> timebucket text,
> create_ts timestamp,
> eventid text,
> applicable_manufacturer_or_applicable_gpo_making_payment_country text,
> applicable_manufacturer_or_applicable_gpo_making_payment_id text,
> applicable_manufacturer_or_applicable_gpo_making_payment_name text,
> applicable_manufacturer_or_applicable_gpo_making_payment_state text,
> charity_indicator text,
> city_of_travel text,
> contextual_information text,
> country_of_travel text,
> covered_recipient_type text,
> date_of_payment timestamp,
> delay_in_publication_indicator text,
> dispute_status_for_publication text,
> form_of_payment_or_transfer_of_value text,
> name_of_associated_covered_device_or_medical_supply1 text,
> name_of_associated_covered_device_or_medical_supply2 text,
> name_of_associated_covered_device_or_medical_supply3 text,
> name_of_associated_covered_device_or_medical_supply4 text,
> name_of_associated_covered_device_or_medical_supply5 text,
> name_of_associated_covered_drug_or_biological1 text,
> name_of_associated_covered_drug_or_biological2 text,
> name_of_associated_covered_drug_or_biological3 text,
> name_of_associated_covered_drug_or_biological4 text,
> name_of_associated_covered_drug_or_biological5 text,
> name_of_third_party_entity_receiving_payment_or_transfer_of_value text,
> nature_of_payment_or_transfer_of_value text,
> ndc_of_associated_covered_drug_or_biological1 text,
> ndc_of_associated_covered_drug_or_biological2 text,
> ndc_of_associated_covered_drug_or_biological3 text,
> ndc_of_associated_covered_drug_or_biological4 text,
> ndc_of_associated_covered_drug_or_biological5 text,
> number_of_payments_included_in_total_amount double,
> payment_publication_date timestamp,
> physicians set,
> product_indicator text,
> program_year text,
> record_id text,
> solr_query text,
> state_of_travel text,
> submitting_applicable_manufacturer_or_applicable_gpo_name text,
> teaching_hospital_id text,
> teaching_hospital_name text,
> third_party_equals_covered_recipient_indicator text,
> third_party_payment_recipient_indicator text,
> total_amount_of_payment_usdollars double,
> PRIMARY KEY ((branch, timebucket), create_ts, eventid)
> )WITH CLUSTERING ORDER BY (create_ts ASC, eventid ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Desired functionality:
> {code}
> CREATE TYPE physician(
> physician_first_name text,
> physician_last_name text,
> physician_license_state_code1 text,
> physician_license_state_code2 text,
> physician_license_state_code3 text,
> physician_license_state_code4 text,
> physician_license_state_code5 text,
> physician_middle_name text,
> physician_name_suffix text,
> physician_ownership_indicator text,
> physician_primary_type text,
> physician_profile_id text,
> physician_specialty text
> );
> CREATE TYPE recipient(
> recipient_city text,
> recipient_country text,
> recipient_postal_code text,
> recipient_primary_business_street_address_line1 text,
>

[jira] [Resolved] (CASSANDRA-7666) Range-segmented sstables

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7666.
--
   Resolution: Later
Fix Version/s: (was: 3.x)

bq. I think that it's sufficient to let this be dormant until or unless it is 
needed to support other features. DTCS covers most of the immediate benefit.

+1

> Range-segmented sstables
> 
>
> Key: CASSANDRA-7666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7666
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
>  Labels: dense-storage
>
> It would be useful to segment sstables by data range (not just token range as 
> envisioned by CASSANDRA-6696).
> The primary use case is to allow deleting those data ranges for "free" by 
> dropping the sstables involved.  We should also (possibly as a separate 
> ticket) be able to leverage this information in query planning to avoid 
> unnecessary sstable reads.
> Relational databases typically call this "partitioning" the table, but 
> obviously we use that term already for something else: 
> http://www.postgresql.org/docs/9.1/static/ddl-partitioning.html
> Tokutek's take for mongodb: 
> http://docs.tokutek.com/tokumx/tokumx-partitioned-collections.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11875) Create sstableconvert tool with support to ma format

[Kaide Mu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312634#comment-15312634
 ] 

Kaide Mu commented on CASSANDRA-11875:
--

New [patch|https://github.com/kdmu/cassandra/tree/trunk-11875] available. 
Working test suit included.

I'm not sure if we have to make {{StandaloneUpgrader}} subclass of 
{{StandaloneConverter}}. I did a 
[WIP|https://gist.github.com/anonymous/29b850c4abed44702c7490606c09fbdc] for 
that, since we have to validate passed args (validate and add latestVersion to 
it), we should maintain original parseArgs of {{StandaloneUpgrader}}, also 
since printUsage is different from {{StandaloneConverter}} we should also 
maintain it. 

Looking forward for your feedback,

Thanks!

> Create sstableconvert tool with support to ma format
> 
>
> Key: CASSANDRA-11875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11875
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Kaide Mu
>Priority: Minor
> Attachments: trunk-11875-WIP-V1.patch
>
>
> Currently {{Upgrader}} receives an sstable in any readable format, and writes 
> into {{BigFormat.getLatestVersion()}}. We should generalize it by making it 
> receive a {{target}} version and probably also rename it to 
> {{SSTableConverter}}. 
> Based on this we can create an {{StandaloneDowngrader}} tool which will 
> perform downgrade of specified sstables to a target version. To start with, 
> we should support only downgrading to {{ma}} format (from current format 
> {{mb}}), downgrade to any other version should be forbidden. Since we already 
> support serializing to "ma" we will not need to do any data conversion.
> We should also create a test suite that creates an sstable with data in the 
> current format, perform the downgrade, and verify data in the new format is 
> correct. This will be the base tests suite for more advanced conversions in 
> the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10235) Support for queries with extra parentheses

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-10235.
---
Resolution: Won't Fix

This has come up before, and every time the resolution is the same: CQL is not 
SQL, and we are not going to please a random SQL ORM by adding syntax to CQL 
that doesn't make sense.

> Support for queries with extra parentheses
> --
>
> Key: CASSANDRA-10235
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10235
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Tom van der Woerdt
>
> Currently, Cassandra does not support queries that have top-level parentheses 
> around them :
> {noformat}
> select * from table where (a = 1 and b = 2);
> {noformat}
> One could argue that these are pointless here, but some automated ORM code 
> may output these assuming that the database will handle them properly.
> Since Cassandra does not support OR, it could just ignore these parentheses 
> and have compatibility with those ORMs again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9811) Tie counter shards' logical clock value to timestamps

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-9811:
---
Reviewer: Sam Tunnicliffe

> Tie counter shards' logical clock value to timestamps
> -
>
> Key: CASSANDRA-9811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9811
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.x
>
>
> CASSANDRA-6506 will get rid of counter shards, turning them into elements of 
> a map. And we'll be using the logical clock values as map element timestamps.
> To avoid unpleasant initial performance (being unable to perform certain 
> sstable optimisations on the read path), we should switch the shards to use 
> {{max(timestamp, clock + 1)}} now, in advance of CASSANDRA-6506 switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9110) Bounded/RingBuffer CQL Collections

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-9110.
--
   Resolution: Won't Fix
Fix Version/s: (was: 3.x)

Unfortunately this is not efficiently implementable in Cassandra without a 
distributed read-before-write, which is a known anti-pattern.

> Bounded/RingBuffer CQL Collections
> --
>
> Key: CASSANDRA-9110
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9110
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jim Plush
>Priority: Minor
>
> Feature Request:
> I've had frequent use cases for bounded and RingBuffer based collections. 
> For example: 
> I want to store the first 100 times I've see this thing.
> I want to store the last 100 times I've seen this thing.
> Currently that means having to do application level READ/WRITE operations and 
> we like to keep some of our high scale apps to write only where possible. 
> While probably expensive for exactly N items an approximation should be good 
> enough for most applications. Where N in our example could be 100 or 102, or 
> even make that tunable on the type or table. 
> For the RingBuffer example, consider I only want to store the last N login 
> attempts for a user. Once N+1 comes in it issues a delete for the oldest one 
> in the collection, or waits until compaction to drop the overflow data as 
> long as the CQL returns the right bounds.
> A potential implementation idea, given the rowkey would live on a single node 
> would be to have an LRU based counter cache (tunable in the yaml settings in 
> MB) that keeps a current count of how many items are already in the 
> collection for that rowkey. If > than bound, toss. 
> something akin to:
> CREATE TABLE users (
>   user_id text PRIMARY KEY,
>   first_name text,
>   first_logins set
>   last_logins set
> );



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11566) read time out when do count(*)

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-11566.
---
   Resolution: Won't Fix
Fix Version/s: (was: 3.3)

> read time out when do count(*)
> --
>
> Key: CASSANDRA-11566
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11566
> Project: Cassandra
>  Issue Type: Bug
> Environment: staging
>Reporter: nizar
>
> Hello I using Cassandra Datastax 3.3, I keep getting read time out even if I 
> set the limit to 1, it would make sense if the limit is high number .. 
> However only limit 1 and still timing out sounds odd?
> [cqlsh 5.0.1 | Cassandra 3.3 | CQL spec 3.4.0 | Native protocol v4]
> cqlsh:test> select count(*) from test.my_view where s_id=? and flag=false 
> limit 1;
> OperationTimedOut: errors={}, last_host=
> my key look like this :
> CREATE MATERIALIZED VIEW test.my_view AS
>   SELECT *
>   FROM table_name
>   WHERE id IS NOT NULL AND processed IS NOT NULL AND time IS  NOT NULL AND id 
> IS NOT NULL
>   PRIMARY KEY ( ( s_id, flag ), time, id )
>   WITH CLUSTERING ORDER BY ( time ASC );
>  I have 5 nodes with replica 3
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'dc': '3'}  AND durable_writes = true;
> Below was the result for nodetoolcfstats
> Keyspace: test
> Read Count: 128770
> Read Latency: 1.42208769123243 ms.
> Write Count: 0
> Write Latency: NaN ms.
> Pending Flushes: 0
> Table: tableName
> SSTable count: 3
> Space used (live): 280777032
> Space used (total): 280777032
> Space used by snapshots (total): 0
> Off heap memory used (total): 2850227
> SSTable Compression Ratio: 0.24706731995327527
> Number of keys (estimate): 1277211
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 0
> Local read count: 3
> Local read latency: 0.396 ms
> Local write count: 0
> Local write latency: NaN ms
> Pending flushes: 0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 1589848
> Bloom filter off heap memory used: 1589824
> Index summary off heap memory used: 1195691
> Compression metadata off heap memory used: 64712
> Compacted partition minimum bytes: 311
> Compacted partition maximum bytes: 535
> Compacted partition mean bytes: 458
> Average live cells per slice (last five minutes): 102.92671205446536
> Maximum live cells per slice (last five minutes): 103
> Average tombstones per slice (last five minutes): 1.0
> Maximum tombstones per slice (last five minutes): 1
> Table: my_view
> SSTable count: 4
> Space used (live): 126114270
> Space used (total): 126114270
> Space used by snapshots (total): 0
> Off heap memory used (total): 91588
> SSTable Compression Ratio: 0.1652453778228639
> Number of keys (estimate): 8
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 0
> Local read count: 128767
> Local read latency: 1.590 ms
> Local write count: 0
> Local write latency: NaN ms
> Pending flushes: 0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 96
> Bloom filter off heap memory used: 64
> Index summary off heap memory used: 140
> Compression metadata off heap memory used: 91384
> Compacted partition minimum bytes: 3974
> Compacted partition maximum bytes: 386857368
> Compacted partition mean bytes: 26034715
> Average live cells per slice (last five minutes): 102.99462595230145
> Maximum live cells per slice (last five minutes): 103
> Average tombstones per slice (last five minutes): 1.0
> Maximum tombstones per slice (last five minutes): 1
> Thank you.
> Nizar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11827) Node is not getting back to UN

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312629#comment-15312629
 ] 

Aleksey Yeschenko commented on CASSANDRA-11827:
---

This is unfortunately insufficient to reproduce the issue. Can you provide 
steps to reproduce at least semi-reliably?

> Node is not getting back to UN
> --
>
> Key: CASSANDRA-11827
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11827
> Project: Cassandra
>  Issue Type: Task
>  Components: Configuration, Observability
> Environment: Ubuntu 12.04.5 LTS
>Reporter: Gaurav Sharma
> Fix For: 2.1.x
>
>
> Hi , 
> One of node went down due to high CPU/Memory utilization , now when we are 
> trying to start it again , it is not joining cluster . On other nodes in 
> system.log we can see below error ::
> INFO  [HANDSHAKE-/172.16.1.71] 2016-05-18 15:21:27,011 
> OutboundTcpConnection.java:485 - Handshaking version with /172.16.1.71
> DEBUG [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,011 
> OutboundTcpConnection.java:412 - Target max version is -2147483648; no 
> version information yet, will retry
> INFO  [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,011 
> YamlConfigurationLoader.java:92 - Loading settings from 
> file:/etc/cassandra/cassandra.yaml
> INFO  [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,013 
> YamlConfigurationLoader.java:135 - Node 
> configuration:[authenticator=AllowAllAuthenticator; authorizer=AllowAl
> lAuthorizer; auto_snapshot=false; batch_size_warn_threshold_in_kb=5; 
> batchlog_replay_throttle_in_kb=1024; broadcast_rpc_address=172.16.1.73; 
> cas_contention_timeout_in_ms=5000; clien
> t_encryption_options=; cluster_name=wynkframework; 
> column_index_size_in_kb=64; commit_failure_policy=stop; 
> commitlog_directory=/mnt/cassandra/commitlog; commitlog_segment_
> size_in_mb=32; commitlog_sync=periodic; commitlog_sync_period_in_ms=1; 
> compaction_large_partition_warning_threshold_mb=100; 
> compaction_throughput_mb_per_sec=32; concurrent_compa
> ctors=4; concurrent_counter_writes=48; concurrent_reads=48; 
> concurrent_writes=32; counter_cache_save_period=7200; 
> counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=
> 1; cross_node_timeout=true; 
> data_file_directories=[/var/lib/cassandra/data, /data/cassandra, 
> /data1/cassandra]; disk_failure_policy=stop; 
> dynamic_snitch_badness_threshold=0.1; d
> ynamic_snitch_reset_interval_in_ms=60; 
> dynamic_snitch_update_interval_in_ms=100; endpoint_snitch=SimpleSnitch; 
> hinted_handoff_enabled=false; hinted_handoff_throttle_in_kb=1024;
> incremental_backups=false; index_summary_capacity_in_mb=null; 
> index_summary_resize_interval_in_minutes=60; inter_dc_tcp_nodelay=false; 
> internode_compression=all; key_cache_save_peri
> od=14400; key_cache_size_in_mb=1024; listen_address=172.16.1.73; 
> max_hint_window_in_ms=1080; max_hints_delivery_threads=4; 
> memory_allocator=JEMallocAllocator; memtable_allocatio
> n_type=offheap_objects; memtable_cleanup_threshold=0.11; 
> memtable_flush_writers=4; native_transport_max_concurrent_connections=100; 
> native_transport_port=9042; num_tokens=256; parti
> tioner=org.apache.cassandra.dht.Murmur3Partitioner; 
> permissions_validity_in_ms=2000; range_request_timeout_in_ms=3; 
> read_request_timeout_in_ms=2; request_scheduler=org.apach
> e.cassandra.scheduler.NoScheduler; request_timeout_in_ms=3; 
> row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=172.16.1.73; 
> rpc_keepalive=true; rpc_max_threads=2048; r
> pc_min_threads=16; rpc_port=9160; rpc_server_type=sync; 
> saved_caches_directory=/mnt/cassandra/saved_caches; 
> seed_provider=[{class_name=org.apache.cassandra.locator.SimpleSeedProvide
> r, parameters=[{seeds=172.16.1.103,172.16.1.73}]}]; 
> server_encryption_options=; snapshot_before_compaction=false; 
> ssl_storage_port=7001; sstable_preemptive_open_interval_i
> n_mb=50; start_native_transport=true; start_rpc=true; storage_port=7000; 
> thrift_framed_transport_size_in_mb=15; tombstone_failure_threshold=10; 
> tombstone_warn_threshold=1000; tr
> ickle_fsync=false; trickle_fsync_interval_in_kb=10240; 
> truncate_request_timeout_in_ms=6; write_request_timeout_in_ms=3]
> TRACE [HANDSHAKE-/172.16.1.71] 2016-05-18 15:21:32,015 
> OutboundTcpConnection.java:492 - Cannot handshake version with /172.16.1.71
> java.nio.channels.AsynchronousCloseException: null



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11827) Node is not getting back to UN

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11827:
--
Status: Awaiting Feedback  (was: Open)

> Node is not getting back to UN
> --
>
> Key: CASSANDRA-11827
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11827
> Project: Cassandra
>  Issue Type: Task
>  Components: Configuration, Observability
> Environment: Ubuntu 12.04.5 LTS
>Reporter: Gaurav Sharma
> Fix For: 2.1.x
>
>
> Hi , 
> One of node went down due to high CPU/Memory utilization , now when we are 
> trying to start it again , it is not joining cluster . On other nodes in 
> system.log we can see below error ::
> INFO  [HANDSHAKE-/172.16.1.71] 2016-05-18 15:21:27,011 
> OutboundTcpConnection.java:485 - Handshaking version with /172.16.1.71
> DEBUG [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,011 
> OutboundTcpConnection.java:412 - Target max version is -2147483648; no 
> version information yet, will retry
> INFO  [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,011 
> YamlConfigurationLoader.java:92 - Loading settings from 
> file:/etc/cassandra/cassandra.yaml
> INFO  [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,013 
> YamlConfigurationLoader.java:135 - Node 
> configuration:[authenticator=AllowAllAuthenticator; authorizer=AllowAl
> lAuthorizer; auto_snapshot=false; batch_size_warn_threshold_in_kb=5; 
> batchlog_replay_throttle_in_kb=1024; broadcast_rpc_address=172.16.1.73; 
> cas_contention_timeout_in_ms=5000; clien
> t_encryption_options=; cluster_name=wynkframework; 
> column_index_size_in_kb=64; commit_failure_policy=stop; 
> commitlog_directory=/mnt/cassandra/commitlog; commitlog_segment_
> size_in_mb=32; commitlog_sync=periodic; commitlog_sync_period_in_ms=1; 
> compaction_large_partition_warning_threshold_mb=100; 
> compaction_throughput_mb_per_sec=32; concurrent_compa
> ctors=4; concurrent_counter_writes=48; concurrent_reads=48; 
> concurrent_writes=32; counter_cache_save_period=7200; 
> counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=
> 1; cross_node_timeout=true; 
> data_file_directories=[/var/lib/cassandra/data, /data/cassandra, 
> /data1/cassandra]; disk_failure_policy=stop; 
> dynamic_snitch_badness_threshold=0.1; d
> ynamic_snitch_reset_interval_in_ms=60; 
> dynamic_snitch_update_interval_in_ms=100; endpoint_snitch=SimpleSnitch; 
> hinted_handoff_enabled=false; hinted_handoff_throttle_in_kb=1024;
> incremental_backups=false; index_summary_capacity_in_mb=null; 
> index_summary_resize_interval_in_minutes=60; inter_dc_tcp_nodelay=false; 
> internode_compression=all; key_cache_save_peri
> od=14400; key_cache_size_in_mb=1024; listen_address=172.16.1.73; 
> max_hint_window_in_ms=1080; max_hints_delivery_threads=4; 
> memory_allocator=JEMallocAllocator; memtable_allocatio
> n_type=offheap_objects; memtable_cleanup_threshold=0.11; 
> memtable_flush_writers=4; native_transport_max_concurrent_connections=100; 
> native_transport_port=9042; num_tokens=256; parti
> tioner=org.apache.cassandra.dht.Murmur3Partitioner; 
> permissions_validity_in_ms=2000; range_request_timeout_in_ms=3; 
> read_request_timeout_in_ms=2; request_scheduler=org.apach
> e.cassandra.scheduler.NoScheduler; request_timeout_in_ms=3; 
> row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=172.16.1.73; 
> rpc_keepalive=true; rpc_max_threads=2048; r
> pc_min_threads=16; rpc_port=9160; rpc_server_type=sync; 
> saved_caches_directory=/mnt/cassandra/saved_caches; 
> seed_provider=[{class_name=org.apache.cassandra.locator.SimpleSeedProvide
> r, parameters=[{seeds=172.16.1.103,172.16.1.73}]}]; 
> server_encryption_options=; snapshot_before_compaction=false; 
> ssl_storage_port=7001; sstable_preemptive_open_interval_i
> n_mb=50; start_native_transport=true; start_rpc=true; storage_port=7000; 
> thrift_framed_transport_size_in_mb=15; tombstone_failure_threshold=10; 
> tombstone_warn_threshold=1000; tr
> ickle_fsync=false; trickle_fsync_interval_in_kb=10240; 
> truncate_request_timeout_in_ms=6; write_request_timeout_in_ms=3]
> TRACE [HANDSHAKE-/172.16.1.71] 2016-05-18 15:21:32,015 
> OutboundTcpConnection.java:492 - Cannot handshake version with /172.16.1.71
> java.nio.channels.AsynchronousCloseException: null



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9811) Tie counter shards' logical clock value to timestamps

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-9811:
-
Assignee: Aleksey Yeschenko
  Status: Patch Available  (was: Open)

> Tie counter shards' logical clock value to timestamps
> -
>
> Key: CASSANDRA-9811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9811
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.x
>
>
> CASSANDRA-6506 will get rid of counter shards, turning them into elements of 
> a map. And we'll be using the logical clock values as map element timestamps.
> To avoid unpleasant initial performance (being unable to perform certain 
> sstable optimisations on the read path), we should switch the shards to use 
> {{max(timestamp, clock + 1)}} now, in advance of CASSANDRA-6506 switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9811) Tie counter shards' logical clock value to timestamps

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312607#comment-15312607
 ] 

Aleksey Yeschenko commented on CASSANDRA-9811:
--

Trivial (one-line) trunk-based change set:

||branch||testall||dtest||
|[9811|https://github.com/iamaleksey/cassandra/tree/9811]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-9811-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-9811-dtest]|

> Tie counter shards' logical clock value to timestamps
> -
>
> Key: CASSANDRA-9811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9811
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.x
>
>
> CASSANDRA-6506 will get rid of counter shards, turning them into elements of 
> a map. And we'll be using the logical clock values as map element timestamps.
> To avoid unpleasant initial performance (being unable to perform certain 
> sstable optimisations on the read path), we should switch the shards to use 
> {{max(timestamp, clock + 1)}} now, in advance of CASSANDRA-6506 switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-8523:
---
Status: Awaiting Feedback  (was: In Progress)

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312567#comment-15312567
 ] 

Paulo Motta commented on CASSANDRA-8523:


Solving this when the replacing node has a different ip address is quite simple:
* Add a new non-dead gossip state for replace BOOT_REPLACE
* When receiving BOOT_REPLACE, other node adds the replacing node as 
bootstrapping AND replacing endpoint
* Pending ranges are calculated, and writes are sent to the replacing node 
during replace
* When replacing node changes state to NORMAL, the old node is removed and the 
new node becomes a natural endpoint on {{TokenMetadata}}

I have a WIP patch with this approach 
[here|https://github.com/pauloricardomg/cassandra/commit/a4be7f7a0f298b1d52821f67ff2e927a1d1f1c5e]
 that is functional. The tricky part is when the replacing node has the same IP 
as the old node, because if this node ever joins gossip with a non-dead state, 
{{FailureDetector.isAlive(InetAddress)}} return true and reads are sent to the 
replacing node, since the original node is a natural endpoint.

My initial idea of removing the old node as a natural endpoint from 
{{TokenMetadata}} obviously does not work because this changes range 
assignments making reads go the following node in the ring, which has no data 
for that range. I also considered replacing the node IP in TokenMetadata with a 
special marker, but this will also not play along well with snitches and 
topologies.

A clean fix to this is to enhance the node representation on {{TokenMetadata}}, 
which is currently a plain {{InetAddresss}}, to include the information if an 
endpoint is available for reads (CASSANDRA-11559), but this cannot be done 
before cassandra 4.0 because it will change {{TokenMetadata}} and 
{{AbstractReplicationStrategy}} public interfaces and other related classes, so 
it's a major refactoring in the codebase. One step further is to change the 
{{FailureDetector}} interface to query nodes by {{UUIDs}} instead of 
{{InetAddress}}, so a replacing endpoint will not be confused with the original 
node since they will have different ids.

One workaround before that would be to send a read to a normal endpoint if both 
{{FailureDetector.isAlive(endpoint)}} is true and the endpoint is on {{NORMAL}} 
state, so we avoid sending reads to a replacing endpoint with the same IP 
address. A big downside is that this check should also be peformed for other 
operations such as bootstrap, repairs, hints, batches, etc. I experimented with 
that route [in this 
commit|https://github.com/pauloricardomg/cassandra/commit/a439edcbd8d4186ea2e3301681b7cc0b5012a265]
 by creating a method {{StorageService.isAvailable(InetAddress endpoint, 
boolean isPending) = \{FailureDetector.instance.isAlive(endpoint) && 
(isPendingEndpoint || Gossiper.instance.isNormalStatus(endpoint))\}}} that 
should be used instead of {{FailureDetector.isAlive(endpoint)}}, but I'm not 
very comfortable with this approach for the following reasons:
* It's pretty fragile, because that check must be used instead of 
{{FailureDetector.isAlive()}} in every place throughout the code
* Other developers/drivers/tools wil also need to use the same approach or a 
similar logic
* We will need to modify the write path to segregate writes between natural and 
pending endpoints (see an example on 
[doPaxosCommit|https://github.com/pauloricardomg/cassandra/commit/a439edcbd8d4186ea2e3301681b7cc0b5012a265#diff-71f06c193f5b5e270cf8ac695164f43aR498])

So unless someone comes up with a magic idea, I think we're left with two 
options to fix this before CASSANDRA-11559:
1. Go ahead with {{StorageService.isAvailable}} approach even with its 
downsides and fragility
2. Fix this only for replacing endpoints with a different IP address, falling 
back to the previous behavior when the replacing node has the same IP as the 
old node, which should already fix this in most cloud deployments, where 
replacement nodes usually have different IPs

I'm leaning more towards 2 since it's much simpler/safer and work on a proper 
general solution for 4.0 after CASSANDRA-11559. WDYT?

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent 

[jira] [Updated] (CASSANDRA-9811) Tie counter shards' logical clock value to timestamps

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-9811:
-
Priority: Minor  (was: Major)

> Tie counter shards' logical clock value to timestamps
> -
>
> Key: CASSANDRA-9811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9811
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.x
>
>
> CASSANDRA-6506 will get rid of counter shards, turning them into elements of 
> a map. And we'll be using the logical clock values as map element timestamps.
> To avoid unpleasant initial performance (being unable to perform certain 
> sstable optimisations on the read path), we should switch the shards to use 
> {{max(timestamp, clock + 1)}} now, in advance of CASSANDRA-6506 switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11752) histograms/metrics in 2.2 do not appear recency biased

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312467#comment-15312467
 ] 

Per Otterström commented on CASSANDRA-11752:


I'm also missing the simplicity of just graphing the percentiles directly. 
Still, I like the accuracy you get from getValues(), wasn't aware of it until 
now.

I'd be willing to create a patch for this. I'm thinking of something like this:
- Keep the existing EH implementation as it is. I think we wan't to keep the 
getValues() implementation for external tools to use. Also, the EH class seem 
to have some complexity due to the fact that is is used for SSTable metadata 
which don't match well with decay functionality.
- Instead, create a new DecayingEstimatedHistogram implementation which keeps 5 
arrays of buckets and switch out the oldest one every minute. Also, every 
minute, perform backward decay on values in the old arrays with a factor of two 
in order to make the recent minute more significant in the percentiles. It 
should be sufficient to use int rather than long in the buckets in order to 
save some memory.
- Every time a new metric value is registered, it is added to both an EH and a 
DecayingEH.
- When reading the bean a call to getValues() is directed to the EH, while 
calls to min/max/mean and percentiles are directed to DecayingEH.

Some code would be duplicated in the EH and DecayingEH class, but I think I 
would prefer this approach over adding decay complexity to the existing EH. One 
option would be to do the decay operation on read. Another option would be to 
use forward decay.

WDYT?

> histograms/metrics in 2.2 do not appear recency biased
> --
>
> Key: CASSANDRA-11752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11752
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Chris Burroughs
>  Labels: metrics
> Attachments: boost-metrics.png, c-jconsole-comparison.png, 
> c-metrics.png, default-histogram.png
>
>
> In addition to upgrading to metrics3, CASSANDRA-5657 switched to using  a 
> custom histogram implementation.  After upgrading to Cassandra 2.2 
> histograms/timer metrics are not suspiciously flat.  To be useful for 
> graphing and alerting metrics need to be biased towards recent events.
> I have attached images that I think illustrate this.
>  * The first two are a comparison between latency observed by a C* 2.2 (us) 
> cluster shoring very flat lines and a client (using metrics 2.2.0, ms) 
> showing server performance problems.  We can't rule out with total certainty 
> that something else isn't the cause (that's why we measure from both the 
> client & server) but they very rarely disagree.
>  * The 3rd image compares jconsole viewing of metrics on a 2.2 and 2.1 
> cluster over several minutes.  Not a single digit changed on the 2.2 cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11765) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312383#comment-15312383
 ] 

Stefania commented on CASSANDRA-11765:
--

No worries, thanks for clarifying! :)

> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11765
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Stefania
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> Example failure:
> http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/12/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD/rolling_upgrade_test_2/
> The test is seeing a corrupt hint sstable after upgrade from 2.2.5 to 3.6. 
> Relevant stack trace is
> {code}
> ERROR [main] 2016-05-11 16:22:25,180 CassandraDaemon.java:727 - Exception 
> encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:119)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.compactLegacyHints(LegacyHintsMigrator.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.migrate(LegacyHintsMigrator.java:92)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:321) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:710) 
> [main/:na]
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
> ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:115)
>  ~[main/:na]
>   ... 5 common frames omitted
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: 
> Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:351)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:245)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:463)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$10.runMayThrow(CompactionManager.java:805)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> 

[jira] [Resolved] (CASSANDRA-11945) CQLSH COPY FROM attempts to parse null timestamp value representatives

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania resolved CASSANDRA-11945.
--
Resolution: Duplicate

Should be fine in 3.0.6/3.6.

> CQLSH COPY FROM attempts to parse null timestamp value representatives
> --
>
> Key: CASSANDRA-11945
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11945
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: OSX / [cqlsh 5.0.1 | Cassandra 3.5 | CQL spec 3.4.0 | 
> Native protocol v4]
>Reporter: craig mcmillan
>Assignee: Stefania
>
> CQLSH COPY FROM is broken when the table includes timestamp columns with null 
> values
> create table foo (id uuid primary key, t timestamp);
> insert into foo (now(), null);
> copy foo to 'foo.csv';
> copy foo from 'foo.csv';
> ... borks with 
> Failed to import 1 rows: ParseError - can't interpret '' as a date with 
> format %Y-%m-%d %H:%M:%S.%f%z or as int,  given up without retries
> Failed to process 1 rows; failed rows written to import_erdev_foo.err



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11765) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312369#comment-15312369
 ] 

Robert Stupp commented on CASSANDRA-11765:
--

bq. why would it have been a problem given that JVMStabilityInspector.inspect() 
recursively calls itself if getClause() is not null
Oh! You're right! No problem then. Sorry for the confusion!

> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11765
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Stefania
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> Example failure:
> http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/12/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD/rolling_upgrade_test_2/
> The test is seeing a corrupt hint sstable after upgrade from 2.2.5 to 3.6. 
> Relevant stack trace is
> {code}
> ERROR [main] 2016-05-11 16:22:25,180 CassandraDaemon.java:727 - Exception 
> encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:119)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.compactLegacyHints(LegacyHintsMigrator.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.migrate(LegacyHintsMigrator.java:92)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:321) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:710) 
> [main/:na]
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
> ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:115)
>  ~[main/:na]
>   ... 5 common frames omitted
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: 
> Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:351)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:245)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:463)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$10.runMayThrow(CompactionManager.java:805)
>  ~[main/:na]
>   at 
> 

[jira] [Commented] (CASSANDRA-11944) sstablesInBounds might not actually give all sstables within the bounds due to having start positions moved in sstables

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312371#comment-15312371
 ] 

Benedict commented on CASSANDRA-11944:
--

Habits imply the future, at which date my life will hopefully not be so hectic. 
 Currently mid-demolition and rebuild of my home, which is eating into my free 
and non-free time alike.

> sstablesInBounds might not actually give all sstables within the bounds due 
> to having start positions moved in sstables
> ---
>
> Key: CASSANDRA-11944
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11944
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> Same problem as with CASSANDRA-11886 - if we try to fetch sstablesInBounds 
> for CANONICAL_SSTABLES, we can miss some actually overlapping sstables. In 
> 3.0+ we state which SSTableSet we want when calling the method.
> Looks like the only issue this could cause is that we include a few too many 
> sstables in compactions that we think contain only droppable tombstones



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-11945) CQLSH COPY FROM attempts to parse null timestamp value representatives

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania reassigned CASSANDRA-11945:


Assignee: Stefania

> CQLSH COPY FROM attempts to parse null timestamp value representatives
> --
>
> Key: CASSANDRA-11945
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11945
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: OSX / [cqlsh 5.0.1 | Cassandra 3.5 | CQL spec 3.4.0 | 
> Native protocol v4]
>Reporter: craig mcmillan
>Assignee: Stefania
>
> CQLSH COPY FROM is broken when the table includes timestamp columns with null 
> values
> create table foo (id uuid primary key, t timestamp);
> insert into foo (now(), null);
> copy foo to 'foo.csv';
> copy foo from 'foo.csv';
> ... borks with 
> Failed to import 1 rows: ParseError - can't interpret '' as a date with 
> format %Y-%m-%d %H:%M:%S.%f%z or as int,  given up without retries
> Failed to process 1 rows; failed rows written to import_erdev_foo.err



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11765) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312352#comment-15312352
 ] 

Stefania commented on CASSANDRA-11765:
--

Thank you for the review.

bq. We could probably just run the tasks one after the other instead?

Done. Previously they were running in the compaction executor and it would have 
been a problem with a single threaded executor.

bq. And a code comment - in CompactionStrategyManager;

Comment added and typo fixed, thanks for spotting this.

bq. The catch (InterruptedException | ExecutionException e) { 
JVMStabilityInspector.inspectThrowable(e); ... doesn't work as expected - you'd 
need to pass ((ExecutionException)e).getCause() to JVMStabilityInspector. 
However - I'd prefer to add support for ExecutionException to 
JVMStabilityInspector since we have similar usages elsewhere.

It's no longer relevant but, why would it have been a problem given that 
JVMStabilityInspector.inspect() recursively calls itself if getClause() is not 
null?

I've rebased on 3.7 and cherry-picked to trunk:

||3.7||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11765-3.7]|[patch|https://github.com/stef1927/cassandra/commits/11765]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11765-3.7-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11765-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11765-3.7-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11765-dtest/]|

The tests are currently running on 3.7, if they are OK I will launch them on 
trunk as well. 

The new dtest is 
[here|https://github.com/stef1927/cassandra-dtest/commits/11765] and it is 
included in the CI jobs currently running.


> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11765
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Stefania
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> Example failure:
> http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/12/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD/rolling_upgrade_test_2/
> The test is seeing a corrupt hint sstable after upgrade from 2.2.5 to 3.6. 
> Relevant stack trace is
> {code}
> ERROR [main] 2016-05-11 16:22:25,180 CassandraDaemon.java:727 - Exception 
> encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:119)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.compactLegacyHints(LegacyHintsMigrator.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.migrate(LegacyHintsMigrator.java:92)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:321) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:710) 
> [main/:na]
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
> ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:115)
>  ~[main/:na]
>   ... 5 common frames omitted
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: 
> Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:351)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265)
>  ~[main/:na]
>   at 
> 

[jira] [Created] (CASSANDRA-11946) Use the return type when resolving function on ambiguous calls

[Sylvain Lebresne (JIRA)]

Sylvain Lebresne created CASSANDRA-11946:


 Summary: Use the return type when resolving function on ambiguous 
calls
 Key: CASSANDRA-11946
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11946
 Project: Cassandra
  Issue Type: Improvement
  Components: CQL
Reporter: Sylvain Lebresne
 Fix For: 3.x


Currently, when we have multiple overloads of a function we only use the 
argument to try to resolve the function. When that resolution is ambiguous, we 
currently throw an error, but in many case (in the {{WHERE}} clause at least) 
we know which type the result is supposed to be so we could use that 
information to try to disambiguate.

The main use case I'm thinking of is the {{now()}} function. Currently, we have 
it only for {{timeuuid}}. But we should likely provide the equivalent for other 
time-base types ({{timestamp}}, {{date}} and {{time}}). Except that currently 
we'd have to use other names that {{now}} and that would probably be a bit 
ugly. If we implement what's above, we'll be able to have overloads of 
{{now()}} for all date types and in many case it'll work how users want out of 
the bose (that is, {{WHERE t = now()}} will work whatever date-based type {{t}} 
is). And in the cases where you can't disambiguate, having to do 
{{(time)now()}} is not really worth than if we had a {{timeNow()}} function 
specific to the {{time}} type.

Also, in principle the change is just a few lines of code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11944) sstablesInBounds might not actually give all sstables within the bounds due to having start positions moved in sstables

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312291#comment-15312291
 ] 

Joshua McKenzie commented on CASSANDRA-11944:
-

No doubt - my hope was that the context of this would be similar enough to 
11886 that the delta would be pretty small to add it on top. Not looking to 
make a habit of it. :)

> sstablesInBounds might not actually give all sstables within the bounds due 
> to having start positions moved in sstables
> ---
>
> Key: CASSANDRA-11944
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11944
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> Same problem as with CASSANDRA-11886 - if we try to fetch sstablesInBounds 
> for CANONICAL_SSTABLES, we can miss some actually overlapping sstables. In 
> 3.0+ we state which SSTableSet we want when calling the method.
> Looks like the only issue this could cause is that we include a few too many 
> sstables in compactions that we think contain only droppable tombstones



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-10786:

Status: Open  (was: Patch Available)

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9935) Repair fails with RuntimeException

[Ahmed ELJAMI (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312239#comment-15312239
 ] 

Ahmed ELJAMI commented on CASSANDRA-9935:
-

[~mlowicki] When you had a problem with repair, streamthroughput had the 
default value (200Mb/s) ? We lowered this value (50Mb/s) and since the repair 
fails with the same error like you!

> Repair fails with RuntimeException
> --
>
> Key: CASSANDRA-9935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9935
> Project: Cassandra
>  Issue Type: Bug
> Environment: C* 2.1.8, Debian Wheezy
>Reporter: mlowicki
>Assignee: Paulo Motta
> Fix For: 2.1.15, 3.6, 3.0.6, 2.2.7
>
> Attachments: 9935.patch, db1.sync.lati.osa.cassandra.log, 
> db5.sync.lati.osa.cassandra.log, system.log.10.210.3.117, 
> system.log.10.210.3.221, system.log.10.210.3.230
>
>
> We had problems with slow repair in 2.1.7 (CASSANDRA-9702) but after upgrade 
> to 2.1.8 it started to work faster but now it fails with:
> {code}
> ...
> [2015-07-29 20:44:03,956] Repair session 23a811b0-3632-11e5-a93e-4963524a8bde 
> for range (-5474076923322749342,-5468600594078911162] finished
> [2015-07-29 20:44:03,957] Repair session 336f8740-3632-11e5-a93e-4963524a8bde 
> for range (-8631877858109464676,-8624040066373718932] finished
> [2015-07-29 20:44:03,957] Repair session 4ccd8430-3632-11e5-a93e-4963524a8bde 
> for range (-5372806541854279315,-5369354119480076785] finished
> [2015-07-29 20:44:03,957] Repair session 59f129f0-3632-11e5-a93e-4963524a8bde 
> for range (8166489034383821955,8168408930184216281] finished
> [2015-07-29 20:44:03,957] Repair session 6ae7a9a0-3632-11e5-a93e-4963524a8bde 
> for range (6084602890817326921,6088328703025510057] finished
> [2015-07-29 20:44:03,957] Repair session 8938e4a0-3632-11e5-a93e-4963524a8bde 
> for range (-781874602493000830,-781745173070807746] finished
> [2015-07-29 20:44:03,957] Repair command #4 finished
> error: nodetool failed, check server logs
> -- StackTrace --
> java.lang.RuntimeException: nodetool failed, check server logs
> at 
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:290)
> at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:202)
> {code}
> After running:
> {code}
> nodetool repair --partitioner-range --parallel --in-local-dc sync
> {code}
> Last records in logs regarding repair are:
> {code}
> INFO  [Thread-173887] 2015-07-29 20:44:03,956 StorageService.java:2952 - 
> Repair session 09ff9e40-3632-11e5-a93e-4963524a8bde for range 
> (-7695808664784761779,-7693529816291585568] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,956 StorageService.java:2952 - 
> Repair session 17d8d860-3632-11e5-a93e-4963524a8bde for range 
> (806371695398849,8065203836608925992] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,956 StorageService.java:2952 - 
> Repair session 23a811b0-3632-11e5-a93e-4963524a8bde for range 
> (-5474076923322749342,-5468600594078911162] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,956 StorageService.java:2952 - 
> Repair session 336f8740-3632-11e5-a93e-4963524a8bde for range 
> (-8631877858109464676,-8624040066373718932] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,957 StorageService.java:2952 - 
> Repair session 4ccd8430-3632-11e5-a93e-4963524a8bde for range 
> (-5372806541854279315,-5369354119480076785] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,957 StorageService.java:2952 - 
> Repair session 59f129f0-3632-11e5-a93e-4963524a8bde for range 
> (8166489034383821955,8168408930184216281] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,957 StorageService.java:2952 - 
> Repair session 6ae7a9a0-3632-11e5-a93e-4963524a8bde for range 
> (6084602890817326921,6088328703025510057] finished
> INFO  [Thread-173887] 2015-07-29 20:44:03,957 StorageService.java:2952 - 
> Repair session 8938e4a0-3632-11e5-a93e-4963524a8bde for range 
> (-781874602493000830,-781745173070807746] finished
> {code}
> but a bit above I see (at least two times in attached log):
> {code}
> ERROR [Thread-173887] 2015-07-29 20:44:03,853 StorageService.java:2959 - 
> Repair session 1b07ea50-3608-11e5-a93e-4963524a8bde for range 
> (5765414319217852786,5781018794516851576] failed with error 
> org.apache.cassandra.exceptions.RepairException: [repair 
> #1b07ea50-3608-11e5-a93e-4963524a8bde on sync/entity_by_id2, 
> (5765414319217852786,5781018794516851576]] Validation failed in /10.195.15.162
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
> org.apache.cassandra.exceptions.RepairException: [repair 
> #1b07ea50-3608-11e5-a93e-4963524a8bde on sync/entity_by_id2, 
> (5765414319217852786,5781018794516851576]] Validation failed in /10.195.15.162
> at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> [na:1.7.0_80]
> 

[jira] [Created] (CASSANDRA-11945) CQLSH COPY FROM attempts to parse null timestamp value representatives

[craig mcmillan (JIRA)]

craig mcmillan created CASSANDRA-11945:
--

 Summary: CQLSH COPY FROM attempts to parse null timestamp value 
representatives
 Key: CASSANDRA-11945
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11945
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: OSX / [cqlsh 5.0.1 | Cassandra 3.5 | CQL spec 3.4.0 | 
Native protocol v4]
Reporter: craig mcmillan


CQLSH COPY FROM is broken when the table includes timestamp columns with null 
values

create table foo (id uuid primary key, t timestamp);
insert into foo (now(), null);
copy foo to 'foo.csv';
copy foo from 'foo.csv';

... borks with 

Failed to import 1 rows: ParseError - can't interpret '' as a date with format 
%Y-%m-%d %H:%M:%S.%f%z or as int,  given up without retries
Failed to process 1 rows; failed rows written to import_erdev_foo.err



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11944) sstablesInBounds might not actually give all sstables within the bounds due to having start positions moved in sstables

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312213#comment-15312213
 ] 

Benedict commented on CASSANDRA-11944:
--

Life is very busy right now, but sure...

> sstablesInBounds might not actually give all sstables within the bounds due 
> to having start positions moved in sstables
> ---
>
> Key: CASSANDRA-11944
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11944
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> Same problem as with CASSANDRA-11886 - if we try to fetch sstablesInBounds 
> for CANONICAL_SSTABLES, we can miss some actually overlapping sstables. In 
> 3.0+ we state which SSTableSet we want when calling the method.
> Looks like the only issue this could cause is that we include a few too many 
> sstables in compactions that we think contain only droppable tombstones



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11055:
-
   Resolution: Fixed
Fix Version/s: (was: 2.1.x)
   2.1.15
   Status: Resolved  (was: Patch Available)

> C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no 
> attribute 'replace'
> ---
>
> Key: CASSANDRA-11055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11055
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Simon Ashley
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.15
>
> Attachments: 11055-driver-2.7.2.patch, data.tar.gz
>
>
> C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns:
> {code}
>  'NoneType' object has no attribute 'replace' 
> {code}
> for thrift CF's originally created in C* 1.2.
> Repro:
> 1. Create cf in cassandra-cli on C* 1.2.x  (1.2.9 was used here)
> {code}
> [default@ks1] CREATE COLUMN FAMILY t1
> ...   WITH column_type='Standard'
> ...   AND 
> comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> ...   AND default_validation_class='UTF8Type'
> ...   AND key_validation_class='UTF8Type'
> ...   AND read_repair_chance=0.1
> ...   AND dclocal_read_repair_chance=0.0
> ...   AND gc_grace=864000
> ...   AND min_compaction_threshold=4
> ...   AND max_compaction_threshold=32
> ...   AND replicate_on_write=true
> ...   AND compaction_strategy='LeveledCompactionStrategy' AND 
> compaction_strategy_options={sstable_size_in_mb: 32}
> ...   AND caching='KEYS_ONLY'
> ...   AND compression_options={sstable_compression:SnappyCompressor, 
> chunk_length_kb:64};
> qlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {
>   'class': 'NetworkTopologyStrategy',
>   'datacenter1': '1'
> };
> USE ks1;
> CREATE TABLE t1 (
>   key text,
>   column1 text,
>   column2 text,
>   value text,
>   PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE AND
>   bloom_filter_fp_chance=0.10 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.00 AND
>   gc_grace_seconds=864000 AND
>   read_repair_chance=0.10 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '32', 'class': 
> 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '64', 'sstable_compression': 
> 'SnappyCompressor'};
> cqlsh> select keyspace_name, columnfamily_name,column_aliases,key_aliases 
> from system.schema_columnfamilies where keyspace_name= 'ks1';
>  keyspace_name | columnfamily_name | column_aliases | key_aliases
> ---+---++-
>ks1 |t1 | [] |  []
> 2/ Upgrade -> C* 2.0.9 -> nodetool upgradesstables -a
> At this stage , DESCRIBE in cqlsh is working
> 3/ Upgrade -> C* 2.1.12 -> nodetool upgradesstables -a
> DESCRIBE now fails:
> cqlsh> describe table ks1.t1;
> 'NoneType' object has no attribute 'replace'
> cqlsh> describe keyspace ks1;
> 'NoneType' object has no attribute 'replace'
> {code}
> You can workaround by manually updating {{system.schema_columnfamilies}}
> {code}
>  UPDATE system.schema_columnfamilies SET column_aliases 
> ='["column1","column2"]' WHERE keyspace_name = 'ks1' AND columnfamily_name = 
> 't1';
> {code}
> Once you exit and restart cqlsh, {{DESCRIBE}} is not working as per C* 1.2
> {code}
> cqlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1'}  AND durable_writes = true;
> CREATE TABLE ks1.t1 (
> key text,
> column1 text,
> column2 text,
> value text,
> PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE
> AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'sstable_size_in_mb': '32', 'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'chunk_length_kb': '64', 'sstable_compression': 
> 'org.apache.cassandra.io.compress.SnappyCompressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.1
> AND speculative_retry = '99.0PERCENTILE';
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312205#comment-15312205
 ] 

Stefania commented on CASSANDRA-11055:
--

CI is fine, committed to 2.1 as 06bb6b9d095205869f22cfe4fbd8877dff006f9b and 
merged upwards with {{-s ours}} since the problem does not occur in 2.2+ 
(driver version 3.0.0).

> C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no 
> attribute 'replace'
> ---
>
> Key: CASSANDRA-11055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11055
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Simon Ashley
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.15
>
> Attachments: 11055-driver-2.7.2.patch, data.tar.gz
>
>
> C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns:
> {code}
>  'NoneType' object has no attribute 'replace' 
> {code}
> for thrift CF's originally created in C* 1.2.
> Repro:
> 1. Create cf in cassandra-cli on C* 1.2.x  (1.2.9 was used here)
> {code}
> [default@ks1] CREATE COLUMN FAMILY t1
> ...   WITH column_type='Standard'
> ...   AND 
> comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> ...   AND default_validation_class='UTF8Type'
> ...   AND key_validation_class='UTF8Type'
> ...   AND read_repair_chance=0.1
> ...   AND dclocal_read_repair_chance=0.0
> ...   AND gc_grace=864000
> ...   AND min_compaction_threshold=4
> ...   AND max_compaction_threshold=32
> ...   AND replicate_on_write=true
> ...   AND compaction_strategy='LeveledCompactionStrategy' AND 
> compaction_strategy_options={sstable_size_in_mb: 32}
> ...   AND caching='KEYS_ONLY'
> ...   AND compression_options={sstable_compression:SnappyCompressor, 
> chunk_length_kb:64};
> qlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {
>   'class': 'NetworkTopologyStrategy',
>   'datacenter1': '1'
> };
> USE ks1;
> CREATE TABLE t1 (
>   key text,
>   column1 text,
>   column2 text,
>   value text,
>   PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE AND
>   bloom_filter_fp_chance=0.10 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.00 AND
>   gc_grace_seconds=864000 AND
>   read_repair_chance=0.10 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '32', 'class': 
> 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '64', 'sstable_compression': 
> 'SnappyCompressor'};
> cqlsh> select keyspace_name, columnfamily_name,column_aliases,key_aliases 
> from system.schema_columnfamilies where keyspace_name= 'ks1';
>  keyspace_name | columnfamily_name | column_aliases | key_aliases
> ---+---++-
>ks1 |t1 | [] |  []
> 2/ Upgrade -> C* 2.0.9 -> nodetool upgradesstables -a
> At this stage , DESCRIBE in cqlsh is working
> 3/ Upgrade -> C* 2.1.12 -> nodetool upgradesstables -a
> DESCRIBE now fails:
> cqlsh> describe table ks1.t1;
> 'NoneType' object has no attribute 'replace'
> cqlsh> describe keyspace ks1;
> 'NoneType' object has no attribute 'replace'
> {code}
> You can workaround by manually updating {{system.schema_columnfamilies}}
> {code}
>  UPDATE system.schema_columnfamilies SET column_aliases 
> ='["column1","column2"]' WHERE keyspace_name = 'ks1' AND columnfamily_name = 
> 't1';
> {code}
> Once you exit and restart cqlsh, {{DESCRIBE}} is not working as per C* 1.2
> {code}
> cqlsh> describe keyspace ks1;
> CREATE KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1'}  AND durable_writes = true;
> CREATE TABLE ks1.t1 (
> key text,
> column1 text,
> column2 text,
> value text,
> PRIMARY KEY (key, column1, column2)
> ) WITH COMPACT STORAGE
> AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'sstable_size_in_mb': '32', 'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'chunk_length_kb': '64', 'sstable_compression': 
> 'org.apache.cassandra.io.compress.SnappyCompressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.1
> AND speculative_retry = '99.0PERCENTILE';
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[10/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bedc7425
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bedc7425
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bedc7425

Branch: refs/heads/trunk
Commit: bedc7425796afe311e148669a3e7b5255e5150a2
Parents: 1e82695 bb88b8c
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:36 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:36 2016 +0200

--

--

[06/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[stefania]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bb88b8cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bb88b8cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bb88b8cd

Branch: refs/heads/trunk
Commit: bb88b8cdc98a4290c4db6409719cc4e7f3440f7f
Parents: 6c445d6 06bb6b9
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:14 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:14 2016 +0200

--

--

[08/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[stefania]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bb88b8cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bb88b8cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bb88b8cd

Branch: refs/heads/cassandra-3.0
Commit: bb88b8cdc98a4290c4db6409719cc4e7f3440f7f
Parents: 6c445d6 06bb6b9
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:14 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:14 2016 +0200

--

--

[11/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bedc7425
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bedc7425
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bedc7425

Branch: refs/heads/cassandra-3.7
Commit: bedc7425796afe311e148669a3e7b5255e5150a2
Parents: 1e82695 bb88b8c
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:36 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:36 2016 +0200

--

--

[02/15] cassandra git commit: Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

[stefania]

Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

patch by Stefania Alborghetti; reviewed by Adam Holmberg for CASSANDRA-11055


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06bb6b9d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06bb6b9d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06bb6b9d

Branch: refs/heads/cassandra-2.2
Commit: 06bb6b9d095205869f22cfe4fbd8877dff006f9b
Parents: 4008e9b
Author: Stefania Alborghetti 
Authored: Thu May 19 10:00:35 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:06:33 2016 +0200

--
 CHANGES.txt   |   1 +
 ...ssandra-driver-internal-only-2.7.2-5d33cb4.zip | Bin 0 -> 226546 bytes
 lib/cassandra-driver-internal-only-2.7.2.zip  | Bin 229600 -> 0 bytes
 3 files changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a437322..619dc61 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables 
(CASSANDRA-11055)
  * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
  * Backport CASSANDRA-11578 (CASSANDRA-11750)
  * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
new file mode 100644
index 000..c59208b
Binary files /dev/null and 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2.zip 
b/lib/cassandra-driver-internal-only-2.7.2.zip
deleted file mode 100644
index f2e75f1..000
Binary files a/lib/cassandra-driver-internal-only-2.7.2.zip and /dev/null differ

[13/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e08d197f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e08d197f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e08d197f

Branch: refs/heads/cassandra-3.7
Commit: e08d197f8509b8a01263629bc79cc6c24942508b
Parents: dc6ffc2 bedc742
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:59 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:59 2016 +0200

--

--

[01/15] cassandra git commit: Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

[stefania]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 4008e9bd3 -> 06bb6b9d0
  refs/heads/cassandra-2.2 6c445d6b7 -> bb88b8cdc
  refs/heads/cassandra-3.0 1e8269511 -> bedc74257
  refs/heads/cassandra-3.7 dc6ffc25a -> e08d197f8
  refs/heads/trunk adfbf518e -> f5ec3699e


Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

patch by Stefania Alborghetti; reviewed by Adam Holmberg for CASSANDRA-11055


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06bb6b9d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06bb6b9d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06bb6b9d

Branch: refs/heads/cassandra-2.1
Commit: 06bb6b9d095205869f22cfe4fbd8877dff006f9b
Parents: 4008e9b
Author: Stefania Alborghetti 
Authored: Thu May 19 10:00:35 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:06:33 2016 +0200

--
 CHANGES.txt   |   1 +
 ...ssandra-driver-internal-only-2.7.2-5d33cb4.zip | Bin 0 -> 226546 bytes
 lib/cassandra-driver-internal-only-2.7.2.zip  | Bin 229600 -> 0 bytes
 3 files changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a437322..619dc61 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables 
(CASSANDRA-11055)
  * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
  * Backport CASSANDRA-11578 (CASSANDRA-11750)
  * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
new file mode 100644
index 000..c59208b
Binary files /dev/null and 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2.zip 
b/lib/cassandra-driver-internal-only-2.7.2.zip
deleted file mode 100644
index f2e75f1..000
Binary files a/lib/cassandra-driver-internal-only-2.7.2.zip and /dev/null differ

[03/15] cassandra git commit: Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

[stefania]

Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

patch by Stefania Alborghetti; reviewed by Adam Holmberg for CASSANDRA-11055


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06bb6b9d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06bb6b9d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06bb6b9d

Branch: refs/heads/trunk
Commit: 06bb6b9d095205869f22cfe4fbd8877dff006f9b
Parents: 4008e9b
Author: Stefania Alborghetti 
Authored: Thu May 19 10:00:35 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:06:33 2016 +0200

--
 CHANGES.txt   |   1 +
 ...ssandra-driver-internal-only-2.7.2-5d33cb4.zip | Bin 0 -> 226546 bytes
 lib/cassandra-driver-internal-only-2.7.2.zip  | Bin 229600 -> 0 bytes
 3 files changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a437322..619dc61 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables 
(CASSANDRA-11055)
  * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
  * Backport CASSANDRA-11578 (CASSANDRA-11750)
  * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
new file mode 100644
index 000..c59208b
Binary files /dev/null and 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2.zip 
b/lib/cassandra-driver-internal-only-2.7.2.zip
deleted file mode 100644
index f2e75f1..000
Binary files a/lib/cassandra-driver-internal-only-2.7.2.zip and /dev/null differ

[07/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[stefania]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bb88b8cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bb88b8cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bb88b8cd

Branch: refs/heads/cassandra-2.2
Commit: bb88b8cdc98a4290c4db6409719cc4e7f3440f7f
Parents: 6c445d6 06bb6b9
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:14 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:14 2016 +0200

--

--

[12/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bedc7425
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bedc7425
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bedc7425

Branch: refs/heads/cassandra-3.0
Commit: bedc7425796afe311e148669a3e7b5255e5150a2
Parents: 1e82695 bb88b8c
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:36 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:36 2016 +0200

--

--

[04/15] cassandra git commit: Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

[stefania]

Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables

patch by Stefania Alborghetti; reviewed by Adam Holmberg for CASSANDRA-11055


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06bb6b9d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06bb6b9d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06bb6b9d

Branch: refs/heads/cassandra-3.0
Commit: 06bb6b9d095205869f22cfe4fbd8877dff006f9b
Parents: 4008e9b
Author: Stefania Alborghetti 
Authored: Thu May 19 10:00:35 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:06:33 2016 +0200

--
 CHANGES.txt   |   1 +
 ...ssandra-driver-internal-only-2.7.2-5d33cb4.zip | Bin 0 -> 226546 bytes
 lib/cassandra-driver-internal-only-2.7.2.zip  | Bin 229600 -> 0 bytes
 3 files changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a437322..619dc61 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables 
(CASSANDRA-11055)
  * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
  * Backport CASSANDRA-11578 (CASSANDRA-11750)
  * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip
new file mode 100644
index 000..c59208b
Binary files /dev/null and 
b/lib/cassandra-driver-internal-only-2.7.2-5d33cb4.zip differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06bb6b9d/lib/cassandra-driver-internal-only-2.7.2.zip
--
diff --git a/lib/cassandra-driver-internal-only-2.7.2.zip 
b/lib/cassandra-driver-internal-only-2.7.2.zip
deleted file mode 100644
index f2e75f1..000
Binary files a/lib/cassandra-driver-internal-only-2.7.2.zip and /dev/null differ

[15/15] cassandra git commit: Merge branch 'cassandra-3.7' into trunk

[stefania]

Merge branch 'cassandra-3.7' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f5ec3699
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f5ec3699
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f5ec3699

Branch: refs/heads/trunk
Commit: f5ec3699e64c183b2d625b7d46e949dabe093293
Parents: adfbf51 e08d197
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:08:19 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:08:19 2016 +0200

--

--

[14/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e08d197f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e08d197f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e08d197f

Branch: refs/heads/trunk
Commit: e08d197f8509b8a01263629bc79cc6c24942508b
Parents: dc6ffc2 bedc742
Author: Stefania Alborghetti 
Authored: Thu Jun 2 14:07:59 2016 +0200
Committer: Stefania Alborghetti 
Committed: Thu Jun 2 14:07:59 2016 +0200

--

--

[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312195#comment-15312195
 ] 

Joshua McKenzie commented on CASSANDRA-8844:


Yep - CommitLogReader, sort logs in data/commitlog by timestamp, and exploit 
the fact that we only ever actively append to one file in that directory. 
Requires more book-keeping to do the live tailing but it shouldn't be a 
super-difficult thing to implement.

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Coordination, Local Write-Read Paths
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.x
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order 

[jira] [Updated] (CASSANDRA-7190) Add schema to snapshot manifest

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-7190:
---
Status: Patch Available  (was: Open)

Thank you for the review. 

I've found a couple of "edge cases" in generation that have affected the 
implementation: create a column with a UDT, drop column and then drop UDT, type 
referenced in the {{DroppedColumn}} is now just a tuple type. That was also the 
reason to not include types of dropped columns together with the other {{CREATE 
TYPE}} statements. Also, re-adding static columns. During {{ALTER TABLE ADD}}, 
it's important to use the non-dropped definition. 

Things like dropping type that's present in dropped column is also covered with 
tests. 

I've added a file called {{schema.cql}} that contains all re-create schema 
information, all consolidated in a single file. It starts with creating types, 
followed by the table creation, then dropped columns (with re-creation, where 
column first created in {{CREATE TABLE}} statement, then dropped with a 
timestamp and then re-added back), and finally followed by indexes, as they 
require a full schema. 

Also, the test with drop/re-add static column is added to make sure behaviour 
is as desired.

|[trunk|https://github.com/ifesdjeen/cassandra/tree/7190-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-7190-trunk-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-7190-trunk-dtest/]|
|[3.0|https://github.com/ifesdjeen/cassandra/tree/7190-3.0]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-7190-3.0-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-7190-3.0-dtest/]|
|[3.7|https://github.com/ifesdjeen/cassandra/tree/7190-3.7]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-7190-3.7-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-7190-3.7-dtest/]|

> Add schema to snapshot manifest
> ---
>
> Key: CASSANDRA-7190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7190
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, lhf
> Fix For: 3.0.x, 3.x
>
>
> followup from CASSANDRA-6326



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9039) CommitLog compressed configuration not run in several unit tests

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9039:
--
   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 2.2.x)
   (was: 3.x)
   3.0.7
   3.7
   2.2.7
   Status: Resolved  (was: Patch Available)

Committed in 2.2 at 6c445d6b7f3c8933a0bfd599ba8455b7254a323d and merged into 
3.0, 3.7 and trunk. 

> CommitLog compressed configuration not run in several unit tests
> 
>
> Key: CASSANDRA-9039
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9039
> Project: Cassandra
>  Issue Type: Test
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Benjamin Lerer
> Fix For: 2.2.7, 3.7, 3.0.7
>
> Attachments: 9039-2.2.txt, 9039-3.0.txt, 9039-3.7.txt
>
>
> CommitLogTest, RecoveryManagerTest, RecoveryManagerTruncateTest, 
> RecoveryManager2Test, RecoveryManager3Test
> Some or all of these should be run with a commit log configured to use 
> compression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9039) CommitLog compressed configuration not run in several unit tests

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9039:
--
Attachment: 9039-3.7.txt
9039-3.0.txt
9039-2.2.txt

Latest patches.

> CommitLog compressed configuration not run in several unit tests
> 
>
> Key: CASSANDRA-9039
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9039
> Project: Cassandra
>  Issue Type: Test
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Benjamin Lerer
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 9039-2.2.txt, 9039-3.0.txt, 9039-3.7.txt
>
>
> CommitLogTest, RecoveryManagerTest, RecoveryManagerTruncateTest, 
> RecoveryManager2Test, RecoveryManager3Test
> Some or all of these should be run with a commit log configured to use 
> compression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9039) CommitLog compressed configuration not run in several unit tests

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312136#comment-15312136
 ] 

Benjamin Lerer commented on CASSANDRA-9039:
---

Thanks for the review.

> CommitLog compressed configuration not run in several unit tests
> 
>
> Key: CASSANDRA-9039
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9039
> Project: Cassandra
>  Issue Type: Test
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Benjamin Lerer
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 9039-2.2.txt, 9039-3.0.txt, 9039-3.7.txt
>
>
> CommitLogTest, RecoveryManagerTest, RecoveryManagerTruncateTest, 
> RecoveryManager2Test, RecoveryManager3Test
> Some or all of these should be run with a commit log configured to use 
> compression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/4] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e826951
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e826951
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e826951

Branch: refs/heads/trunk
Commit: 1e82695115c7f191d4de60a92f7b9fd078ebbc68
Parents: 7eb4647 6c445d6
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:36:14 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:44:45 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   |  71 +-
 .../db/commitlog/CommitLogSegment.java  |   7 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  37 +++---
 .../db/commitlog/CompressedSegment.java |   6 +-
 .../db/commitlog/CommitLogStressTest.java   |   2 +-
 .../db/RecoveryManagerFlushedTest.java  |  38 +-
 .../db/RecoveryManagerMissingHeaderTest.java|  36 -
 .../cassandra/db/RecoveryManagerTest.java   |  47 +--
 .../db/RecoveryManagerTruncateTest.java |  42 +-
 .../db/commitlog/CommitLogDescriptorTest.java   | 102 ++
 .../cassandra/db/commitlog/CommitLogTest.java   | 132 +++
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   2 +-
 13 files changed, 389 insertions(+), 134 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e826951/CHANGES.txt
--
diff --cc CHANGES.txt
index 0cafa83,9752d16..70da4ad
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,5 +1,19 @@@
 -2.2.7
 +3.0.7
 + * Fix Directories instantiations where CFS.initialDirectories should be used 
(CASSANDRA-11849)
 + * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912)
 + * Fix sstables not being protected from removal during index build 
(CASSANDRA-11905)
 + * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032)
 + * Remove unneeded code to repair index summaries that have
 +   been improperly down-sampled (CASSANDRA-11127)
 + * Avoid WriteTimeoutExceptions during commit log replay due to materialized
 +   view lock contention (CASSANDRA-11891)
 + * Prevent OOM failures on SSTable corruption, improve tests for corruption 
detection (CASSANDRA-9530)
 + * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705)
 + * Allow compaction strategies to disable early open (CASSANDRA-11754)
 + * Refactor Materialized View code (CASSANDRA-11475)
 + * Update Java Driver (CASSANDRA-11615)
 +Merged from 2.2:
+  * Run CommitLog tests with different compression settings (CASSANDRA-9039)
   * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
   * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
   * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e826951/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --cc src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index c0e12c5,460ecfe..dcdd855
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@@ -70,11 -70,10 +70,10 @@@ public class CommitLog implements Commi
  final CommitLogMetrics metrics;
  final AbstractCommitLogService executor;
  
- final ICompressor compressor;
- public ParameterizedClass compressorClass;
+ volatile Configuration configuration;
  final public String location;
  
 -static private CommitLog construct()
 +private static CommitLog construct()
  {
  CommitLog log = new 
CommitLog(DatabaseDescriptor.getCommitLogLocation(), 
CommitLogArchiver.construct());
  
@@@ -433,6 -432,14 +431,14 @@@
  }
  
  /**
+  * FOR TESTING PURPOSES.
+  */
+ public void resetConfiguration()
+ {
 -this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
++configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
+ }
+ 
+ /**
   * FOR TESTING PURPOSES.  See CommitLogAllocator
   */
  public int restartUnsafe() throws IOException
@@@ -487,4 -494,59 +493,59 @@@
  throw new 
AssertionError(DatabaseDescriptor.getCommitFailurePolicy());
  }
  }
+ 
+ public static final class Configuration
+ {
+ /**
+  * The compressor class.
+  */
+ private final ParameterizedClass compressorClass;
+ 
+ /**
+  * The compressor used to compress the segments.
+  */
+ private final ICompressor 

[3/4] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.7

[blerer]

Merge branch cassandra-3.0 into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dc6ffc25
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dc6ffc25
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dc6ffc25

Branch: refs/heads/trunk
Commit: dc6ffc25a8d00659385a1219d0189bd068ef110d
Parents: dbf0310 1e82695
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:47:03 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:50:19 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   | 102 +-
 .../db/commitlog/CommitLogSegment.java  |  15 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  17 +-
 .../db/commitlog/CompressedSegment.java |   4 +-
 .../db/commitlog/EncryptedSegment.java  |   4 +-
 .../db/commitlog/CommitLogStressTest.java   |  12 +-
 .../db/RecoveryManagerFlushedTest.java  |  40 
 .../db/RecoveryManagerMissingHeaderTest.java|  38 +++-
 .../cassandra/db/RecoveryManagerTest.java   | 167 ++---
 .../db/RecoveryManagerTruncateTest.java |  38 
 .../db/commitlog/CommitLogDescriptorTest.java   |   3 +-
 .../cassandra/db/commitlog/CommitLogTest.java   | 187 ++-
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   4 +-
 14 files changed, 407 insertions(+), 225 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc6ffc25/CHANGES.txt
--
diff --cc CHANGES.txt
index a54f4fd,70da4ad..2a66eb4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -22,11 -20,10 +22,12 @@@ Merged from 2.2
   * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
   * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
   * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
   * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
  Merged from 2.1:
++ * Run CommitLog tests with different compression settings (CASSANDRA-9039)
   * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
 - * Backport CASSANDRA-11578 (CASSANDRA-11750)
   * Clear out parent repair session if repair coordinator dies 
(CASSANDRA-11824)
   * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840)
   * Do not consider local node a valid source during replace (CASSANDRA-11848)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc6ffc25/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --cc src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 10bc91a,dcdd855..4a660ca
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@@ -96,13 -92,10 +94,11 @@@ public class CommitLog implements Commi
  @VisibleForTesting
  CommitLog(String location, CommitLogArchiver archiver)
  {
- compressorClass = DatabaseDescriptor.getCommitLogCompression();
  this.location = location;
- ICompressor compressor = compressorClass != null ? 
CompressionParams.createCompressor(compressorClass) : null;
 -this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
++this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression(),
++   
DatabaseDescriptor.getEncryptionContext());
  DatabaseDescriptor.createAllDirectories();
- encryptionContext = DatabaseDescriptor.getEncryptionContext();
  
- this.compressor = compressor;
  this.archiver = archiver;
  metrics = new CommitLogMetrics();
  
@@@ -146,7 -139,7 +142,8 @@@
  };
  
  // submit all existing files in the commit log dir for archiving 
prior to recovery - CASSANDRA-6904
--for (File file : new 
File(DatabaseDescriptor.getCommitLogLocation()).listFiles(unmanagedFilesFilter))
++File[] listFiles = new 
File(DatabaseDescriptor.getCommitLogLocation()).listFiles(unmanagedFilesFilter);
++for (File file : listFiles)
  {
  archiver.maybeArchive(file.getPath(), file.getName());
  archiver.maybeWaitForArchiving(file.getName());
@@@ -420,6 -413,6 +418,15 @@@
  }
  
  /**
++ * FOR TESTING PURPOSES.
++ */
++public void resetConfiguration()
++{
++configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression(),
++

[1/4] cassandra git commit: Run CommitLog tests with different compression settings

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk eb5a59a31 -> adfbf518e


Run CommitLog tests with different compression settings

patch by Benjamin Lerer; reviewed by Branimir Lambov for CASSANDRA-9039


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c445d6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c445d6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c445d6b

Branch: refs/heads/trunk
Commit: 6c445d6b7f3c8933a0bfd599ba8455b7254a323d
Parents: b8f5c1f
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:31:31 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:31:31 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   |  71 ++-
 .../db/commitlog/CommitLogSegment.java  |   5 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  15 ++-
 .../db/commitlog/CompressedSegment.java |   6 +-
 .../db/commitlog/CommitLogStressTest.java   |   2 +-
 .../cassandra/db/RecoveryManager2Test.java  |  36 ++
 .../cassandra/db/RecoveryManager3Test.java  |  33 +
 .../cassandra/db/RecoveryManagerTest.java   |  42 +--
 .../db/RecoveryManagerTruncateTest.java |  35 ++
 .../db/commitlog/CommitLogDescriptorTest.java   | 103 
 .../cassandra/db/commitlog/CommitLogTest.java   | 121 ++-
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   2 +-
 13 files changed, 358 insertions(+), 114 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c97293d..9752d16 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.7
+ * Run CommitLog tests with different compression settings (CASSANDRA-9039)
  * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
  * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
  * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 9a6ba34..460ecfe 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -70,8 +70,7 @@ public class CommitLog implements CommitLogMBean
 final CommitLogMetrics metrics;
 final AbstractCommitLogService executor;
 
-final ICompressor compressor;
-public ParameterizedClass compressorClass;
+volatile Configuration configuration;
 final public String location;
 
 static private CommitLog construct()
@@ -93,12 +92,10 @@ public class CommitLog implements CommitLogMBean
 @VisibleForTesting
 CommitLog(String location, CommitLogArchiver archiver)
 {
-compressorClass = DatabaseDescriptor.getCommitLogCompression();
 this.location = location;
-ICompressor compressor = compressorClass != null ? 
CompressionParameters.createCompressor(compressorClass) : null;
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
 DatabaseDescriptor.createAllDirectories();
 
-this.compressor = compressor;
 this.archiver = archiver;
 metrics = new CommitLogMetrics();
 
@@ -412,6 +409,7 @@ public class CommitLog implements CommitLogMBean
 public int resetUnsafe(boolean deleteSegments) throws IOException
 {
 stopUnsafe(deleteSegments);
+resetConfiguration();
 return restartUnsafe();
 }
 
@@ -434,6 +432,14 @@ public class CommitLog implements CommitLogMBean
 }
 
 /**
+ * FOR TESTING PURPOSES.
+ */
+public void resetConfiguration()
+{
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
+}
+
+/**
  * FOR TESTING PURPOSES.  See CommitLogAllocator
  */
 public int restartUnsafe() throws IOException
@@ -488,4 +494,59 @@ public class CommitLog implements CommitLogMBean
 throw new 
AssertionError(DatabaseDescriptor.getCommitFailurePolicy());
 }
 }
+
+public static final class Configuration
+{
+/**
+ * The compressor class.
+ */
+private final ParameterizedClass compressorClass;
+
+/**
+ * The compressor used to compress the segments.
+ */
+private final ICompressor compressor;
+
+public 

[4/4] cassandra git commit: Merge branch cassandra-3.7 into trunk

[blerer]

Merge branch cassandra-3.7 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/adfbf518
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/adfbf518
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/adfbf518

Branch: refs/heads/trunk
Commit: adfbf518e041595a481cd902a033e6b081f50f82
Parents: eb5a59a dc6ffc2
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:51:58 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:52:10 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   | 102 +-
 .../db/commitlog/CommitLogSegment.java  |  15 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  17 +-
 .../db/commitlog/CompressedSegment.java |   4 +-
 .../db/commitlog/EncryptedSegment.java  |   4 +-
 .../db/commitlog/CommitLogStressTest.java   |  12 +-
 .../db/RecoveryManagerFlushedTest.java  |  40 
 .../db/RecoveryManagerMissingHeaderTest.java|  38 +++-
 .../cassandra/db/RecoveryManagerTest.java   | 167 ++---
 .../db/RecoveryManagerTruncateTest.java |  38 
 .../db/commitlog/CommitLogDescriptorTest.java   |   3 +-
 .../cassandra/db/commitlog/CommitLogTest.java   | 187 ++-
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   4 +-
 14 files changed, 407 insertions(+), 225 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/adfbf518/CHANGES.txt
--

[3/3] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.7

[blerer]

Merge branch cassandra-3.0 into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dc6ffc25
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dc6ffc25
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dc6ffc25

Branch: refs/heads/cassandra-3.7
Commit: dc6ffc25a8d00659385a1219d0189bd068ef110d
Parents: dbf0310 1e82695
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:47:03 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:50:19 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   | 102 +-
 .../db/commitlog/CommitLogSegment.java  |  15 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  17 +-
 .../db/commitlog/CompressedSegment.java |   4 +-
 .../db/commitlog/EncryptedSegment.java  |   4 +-
 .../db/commitlog/CommitLogStressTest.java   |  12 +-
 .../db/RecoveryManagerFlushedTest.java  |  40 
 .../db/RecoveryManagerMissingHeaderTest.java|  38 +++-
 .../cassandra/db/RecoveryManagerTest.java   | 167 ++---
 .../db/RecoveryManagerTruncateTest.java |  38 
 .../db/commitlog/CommitLogDescriptorTest.java   |   3 +-
 .../cassandra/db/commitlog/CommitLogTest.java   | 187 ++-
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   4 +-
 14 files changed, 407 insertions(+), 225 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc6ffc25/CHANGES.txt
--
diff --cc CHANGES.txt
index a54f4fd,70da4ad..2a66eb4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -22,11 -20,10 +22,12 @@@ Merged from 2.2
   * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
   * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
   * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
   * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
  Merged from 2.1:
++ * Run CommitLog tests with different compression settings (CASSANDRA-9039)
   * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
 - * Backport CASSANDRA-11578 (CASSANDRA-11750)
   * Clear out parent repair session if repair coordinator dies 
(CASSANDRA-11824)
   * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840)
   * Do not consider local node a valid source during replace (CASSANDRA-11848)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc6ffc25/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --cc src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 10bc91a,dcdd855..4a660ca
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@@ -96,13 -92,10 +94,11 @@@ public class CommitLog implements Commi
  @VisibleForTesting
  CommitLog(String location, CommitLogArchiver archiver)
  {
- compressorClass = DatabaseDescriptor.getCommitLogCompression();
  this.location = location;
- ICompressor compressor = compressorClass != null ? 
CompressionParams.createCompressor(compressorClass) : null;
 -this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
++this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression(),
++   
DatabaseDescriptor.getEncryptionContext());
  DatabaseDescriptor.createAllDirectories();
- encryptionContext = DatabaseDescriptor.getEncryptionContext();
  
- this.compressor = compressor;
  this.archiver = archiver;
  metrics = new CommitLogMetrics();
  
@@@ -146,7 -139,7 +142,8 @@@
  };
  
  // submit all existing files in the commit log dir for archiving 
prior to recovery - CASSANDRA-6904
--for (File file : new 
File(DatabaseDescriptor.getCommitLogLocation()).listFiles(unmanagedFilesFilter))
++File[] listFiles = new 
File(DatabaseDescriptor.getCommitLogLocation()).listFiles(unmanagedFilesFilter);
++for (File file : listFiles)
  {
  archiver.maybeArchive(file.getPath(), file.getName());
  archiver.maybeWaitForArchiving(file.getName());
@@@ -420,6 -413,6 +418,15 @@@
  }
  
  /**
++ * FOR TESTING PURPOSES.
++ */
++public void resetConfiguration()
++{
++configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression(),
++

[2/3] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e826951
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e826951
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e826951

Branch: refs/heads/cassandra-3.7
Commit: 1e82695115c7f191d4de60a92f7b9fd078ebbc68
Parents: 7eb4647 6c445d6
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:36:14 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:44:45 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   |  71 +-
 .../db/commitlog/CommitLogSegment.java  |   7 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  37 +++---
 .../db/commitlog/CompressedSegment.java |   6 +-
 .../db/commitlog/CommitLogStressTest.java   |   2 +-
 .../db/RecoveryManagerFlushedTest.java  |  38 +-
 .../db/RecoveryManagerMissingHeaderTest.java|  36 -
 .../cassandra/db/RecoveryManagerTest.java   |  47 +--
 .../db/RecoveryManagerTruncateTest.java |  42 +-
 .../db/commitlog/CommitLogDescriptorTest.java   | 102 ++
 .../cassandra/db/commitlog/CommitLogTest.java   | 132 +++
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   2 +-
 13 files changed, 389 insertions(+), 134 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e826951/CHANGES.txt
--
diff --cc CHANGES.txt
index 0cafa83,9752d16..70da4ad
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,5 +1,19 @@@
 -2.2.7
 +3.0.7
 + * Fix Directories instantiations where CFS.initialDirectories should be used 
(CASSANDRA-11849)
 + * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912)
 + * Fix sstables not being protected from removal during index build 
(CASSANDRA-11905)
 + * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032)
 + * Remove unneeded code to repair index summaries that have
 +   been improperly down-sampled (CASSANDRA-11127)
 + * Avoid WriteTimeoutExceptions during commit log replay due to materialized
 +   view lock contention (CASSANDRA-11891)
 + * Prevent OOM failures on SSTable corruption, improve tests for corruption 
detection (CASSANDRA-9530)
 + * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705)
 + * Allow compaction strategies to disable early open (CASSANDRA-11754)
 + * Refactor Materialized View code (CASSANDRA-11475)
 + * Update Java Driver (CASSANDRA-11615)
 +Merged from 2.2:
+  * Run CommitLog tests with different compression settings (CASSANDRA-9039)
   * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
   * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
   * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e826951/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --cc src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index c0e12c5,460ecfe..dcdd855
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@@ -70,11 -70,10 +70,10 @@@ public class CommitLog implements Commi
  final CommitLogMetrics metrics;
  final AbstractCommitLogService executor;
  
- final ICompressor compressor;
- public ParameterizedClass compressorClass;
+ volatile Configuration configuration;
  final public String location;
  
 -static private CommitLog construct()
 +private static CommitLog construct()
  {
  CommitLog log = new 
CommitLog(DatabaseDescriptor.getCommitLogLocation(), 
CommitLogArchiver.construct());
  
@@@ -433,6 -432,14 +431,14 @@@
  }
  
  /**
+  * FOR TESTING PURPOSES.
+  */
+ public void resetConfiguration()
+ {
 -this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
++configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
+ }
+ 
+ /**
   * FOR TESTING PURPOSES.  See CommitLogAllocator
   */
  public int restartUnsafe() throws IOException
@@@ -487,4 -494,59 +493,59 @@@
  throw new 
AssertionError(DatabaseDescriptor.getCommitFailurePolicy());
  }
  }
+ 
+ public static final class Configuration
+ {
+ /**
+  * The compressor class.
+  */
+ private final ParameterizedClass compressorClass;
+ 
+ /**
+  * The compressor used to compress the segments.
+  */
+ private final 

[1/3] cassandra git commit: Run CommitLog tests with different compression settings

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.7 dbf03103c -> dc6ffc25a


Run CommitLog tests with different compression settings

patch by Benjamin Lerer; reviewed by Branimir Lambov for CASSANDRA-9039


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c445d6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c445d6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c445d6b

Branch: refs/heads/cassandra-3.7
Commit: 6c445d6b7f3c8933a0bfd599ba8455b7254a323d
Parents: b8f5c1f
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:31:31 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:31:31 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   |  71 ++-
 .../db/commitlog/CommitLogSegment.java  |   5 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  15 ++-
 .../db/commitlog/CompressedSegment.java |   6 +-
 .../db/commitlog/CommitLogStressTest.java   |   2 +-
 .../cassandra/db/RecoveryManager2Test.java  |  36 ++
 .../cassandra/db/RecoveryManager3Test.java  |  33 +
 .../cassandra/db/RecoveryManagerTest.java   |  42 +--
 .../db/RecoveryManagerTruncateTest.java |  35 ++
 .../db/commitlog/CommitLogDescriptorTest.java   | 103 
 .../cassandra/db/commitlog/CommitLogTest.java   | 121 ++-
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   2 +-
 13 files changed, 358 insertions(+), 114 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c97293d..9752d16 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.7
+ * Run CommitLog tests with different compression settings (CASSANDRA-9039)
  * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
  * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
  * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 9a6ba34..460ecfe 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -70,8 +70,7 @@ public class CommitLog implements CommitLogMBean
 final CommitLogMetrics metrics;
 final AbstractCommitLogService executor;
 
-final ICompressor compressor;
-public ParameterizedClass compressorClass;
+volatile Configuration configuration;
 final public String location;
 
 static private CommitLog construct()
@@ -93,12 +92,10 @@ public class CommitLog implements CommitLogMBean
 @VisibleForTesting
 CommitLog(String location, CommitLogArchiver archiver)
 {
-compressorClass = DatabaseDescriptor.getCommitLogCompression();
 this.location = location;
-ICompressor compressor = compressorClass != null ? 
CompressionParameters.createCompressor(compressorClass) : null;
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
 DatabaseDescriptor.createAllDirectories();
 
-this.compressor = compressor;
 this.archiver = archiver;
 metrics = new CommitLogMetrics();
 
@@ -412,6 +409,7 @@ public class CommitLog implements CommitLogMBean
 public int resetUnsafe(boolean deleteSegments) throws IOException
 {
 stopUnsafe(deleteSegments);
+resetConfiguration();
 return restartUnsafe();
 }
 
@@ -434,6 +432,14 @@ public class CommitLog implements CommitLogMBean
 }
 
 /**
+ * FOR TESTING PURPOSES.
+ */
+public void resetConfiguration()
+{
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
+}
+
+/**
  * FOR TESTING PURPOSES.  See CommitLogAllocator
  */
 public int restartUnsafe() throws IOException
@@ -488,4 +494,59 @@ public class CommitLog implements CommitLogMBean
 throw new 
AssertionError(DatabaseDescriptor.getCommitFailurePolicy());
 }
 }
+
+public static final class Configuration
+{
+/**
+ * The compressor class.
+ */
+private final ParameterizedClass compressorClass;
+
+/**
+ * The compressor used to compress the segments.
+ */
+private final ICompressor compressor;
+
+  

[1/2] cassandra git commit: Run CommitLog tests with different compression settings

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 7eb464734 -> 1e8269511


Run CommitLog tests with different compression settings

patch by Benjamin Lerer; reviewed by Branimir Lambov for CASSANDRA-9039


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c445d6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c445d6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c445d6b

Branch: refs/heads/cassandra-3.0
Commit: 6c445d6b7f3c8933a0bfd599ba8455b7254a323d
Parents: b8f5c1f
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:31:31 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:31:31 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   |  71 ++-
 .../db/commitlog/CommitLogSegment.java  |   5 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  15 ++-
 .../db/commitlog/CompressedSegment.java |   6 +-
 .../db/commitlog/CommitLogStressTest.java   |   2 +-
 .../cassandra/db/RecoveryManager2Test.java  |  36 ++
 .../cassandra/db/RecoveryManager3Test.java  |  33 +
 .../cassandra/db/RecoveryManagerTest.java   |  42 +--
 .../db/RecoveryManagerTruncateTest.java |  35 ++
 .../db/commitlog/CommitLogDescriptorTest.java   | 103 
 .../cassandra/db/commitlog/CommitLogTest.java   | 121 ++-
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   2 +-
 13 files changed, 358 insertions(+), 114 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c97293d..9752d16 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.7
+ * Run CommitLog tests with different compression settings (CASSANDRA-9039)
  * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
  * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
  * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 9a6ba34..460ecfe 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -70,8 +70,7 @@ public class CommitLog implements CommitLogMBean
 final CommitLogMetrics metrics;
 final AbstractCommitLogService executor;
 
-final ICompressor compressor;
-public ParameterizedClass compressorClass;
+volatile Configuration configuration;
 final public String location;
 
 static private CommitLog construct()
@@ -93,12 +92,10 @@ public class CommitLog implements CommitLogMBean
 @VisibleForTesting
 CommitLog(String location, CommitLogArchiver archiver)
 {
-compressorClass = DatabaseDescriptor.getCommitLogCompression();
 this.location = location;
-ICompressor compressor = compressorClass != null ? 
CompressionParameters.createCompressor(compressorClass) : null;
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
 DatabaseDescriptor.createAllDirectories();
 
-this.compressor = compressor;
 this.archiver = archiver;
 metrics = new CommitLogMetrics();
 
@@ -412,6 +409,7 @@ public class CommitLog implements CommitLogMBean
 public int resetUnsafe(boolean deleteSegments) throws IOException
 {
 stopUnsafe(deleteSegments);
+resetConfiguration();
 return restartUnsafe();
 }
 
@@ -434,6 +432,14 @@ public class CommitLog implements CommitLogMBean
 }
 
 /**
+ * FOR TESTING PURPOSES.
+ */
+public void resetConfiguration()
+{
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
+}
+
+/**
  * FOR TESTING PURPOSES.  See CommitLogAllocator
  */
 public int restartUnsafe() throws IOException
@@ -488,4 +494,59 @@ public class CommitLog implements CommitLogMBean
 throw new 
AssertionError(DatabaseDescriptor.getCommitFailurePolicy());
 }
 }
+
+public static final class Configuration
+{
+/**
+ * The compressor class.
+ */
+private final ParameterizedClass compressorClass;
+
+/**
+ * The compressor used to compress the segments.
+ */
+private final ICompressor compressor;
+
+  

[2/2] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e826951
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e826951
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e826951

Branch: refs/heads/cassandra-3.0
Commit: 1e82695115c7f191d4de60a92f7b9fd078ebbc68
Parents: 7eb4647 6c445d6
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:36:14 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:44:45 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   |  71 +-
 .../db/commitlog/CommitLogSegment.java  |   7 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  37 +++---
 .../db/commitlog/CompressedSegment.java |   6 +-
 .../db/commitlog/CommitLogStressTest.java   |   2 +-
 .../db/RecoveryManagerFlushedTest.java  |  38 +-
 .../db/RecoveryManagerMissingHeaderTest.java|  36 -
 .../cassandra/db/RecoveryManagerTest.java   |  47 +--
 .../db/RecoveryManagerTruncateTest.java |  42 +-
 .../db/commitlog/CommitLogDescriptorTest.java   | 102 ++
 .../cassandra/db/commitlog/CommitLogTest.java   | 132 +++
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   2 +-
 13 files changed, 389 insertions(+), 134 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e826951/CHANGES.txt
--
diff --cc CHANGES.txt
index 0cafa83,9752d16..70da4ad
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,5 +1,19 @@@
 -2.2.7
 +3.0.7
 + * Fix Directories instantiations where CFS.initialDirectories should be used 
(CASSANDRA-11849)
 + * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912)
 + * Fix sstables not being protected from removal during index build 
(CASSANDRA-11905)
 + * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032)
 + * Remove unneeded code to repair index summaries that have
 +   been improperly down-sampled (CASSANDRA-11127)
 + * Avoid WriteTimeoutExceptions during commit log replay due to materialized
 +   view lock contention (CASSANDRA-11891)
 + * Prevent OOM failures on SSTable corruption, improve tests for corruption 
detection (CASSANDRA-9530)
 + * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705)
 + * Allow compaction strategies to disable early open (CASSANDRA-11754)
 + * Refactor Materialized View code (CASSANDRA-11475)
 + * Update Java Driver (CASSANDRA-11615)
 +Merged from 2.2:
+  * Run CommitLog tests with different compression settings (CASSANDRA-9039)
   * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
   * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
   * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e826951/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --cc src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index c0e12c5,460ecfe..dcdd855
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@@ -70,11 -70,10 +70,10 @@@ public class CommitLog implements Commi
  final CommitLogMetrics metrics;
  final AbstractCommitLogService executor;
  
- final ICompressor compressor;
- public ParameterizedClass compressorClass;
+ volatile Configuration configuration;
  final public String location;
  
 -static private CommitLog construct()
 +private static CommitLog construct()
  {
  CommitLog log = new 
CommitLog(DatabaseDescriptor.getCommitLogLocation(), 
CommitLogArchiver.construct());
  
@@@ -433,6 -432,14 +431,14 @@@
  }
  
  /**
+  * FOR TESTING PURPOSES.
+  */
+ public void resetConfiguration()
+ {
 -this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
++configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
+ }
+ 
+ /**
   * FOR TESTING PURPOSES.  See CommitLogAllocator
   */
  public int restartUnsafe() throws IOException
@@@ -487,4 -494,59 +493,59 @@@
  throw new 
AssertionError(DatabaseDescriptor.getCommitFailurePolicy());
  }
  }
+ 
+ public static final class Configuration
+ {
+ /**
+  * The compressor class.
+  */
+ private final ParameterizedClass compressorClass;
+ 
+ /**
+  * The compressor used to compress the segments.
+  */
+ private final 

cassandra git commit: Run CommitLog tests with different compression settings

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 b8f5c1fce -> 6c445d6b7


Run CommitLog tests with different compression settings

patch by Benjamin Lerer; reviewed by Branimir Lambov for CASSANDRA-9039


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c445d6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c445d6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c445d6b

Branch: refs/heads/cassandra-2.2
Commit: 6c445d6b7f3c8933a0bfd599ba8455b7254a323d
Parents: b8f5c1f
Author: Benjamin Lerer 
Authored: Thu Jun 2 12:31:31 2016 +0200
Committer: Benjamin Lerer 
Committed: Thu Jun 2 12:31:31 2016 +0200

--
 CHANGES.txt |   1 +
 .../cassandra/db/commitlog/CommitLog.java   |  71 ++-
 .../db/commitlog/CommitLogSegment.java  |   5 +-
 .../db/commitlog/CommitLogSegmentManager.java   |  15 ++-
 .../db/commitlog/CompressedSegment.java |   6 +-
 .../db/commitlog/CommitLogStressTest.java   |   2 +-
 .../cassandra/db/RecoveryManager2Test.java  |  36 ++
 .../cassandra/db/RecoveryManager3Test.java  |  33 +
 .../cassandra/db/RecoveryManagerTest.java   |  42 +--
 .../db/RecoveryManagerTruncateTest.java |  35 ++
 .../db/commitlog/CommitLogDescriptorTest.java   | 103 
 .../cassandra/db/commitlog/CommitLogTest.java   | 121 ++-
 .../db/commitlog/CommitLogUpgradeTestMaker.java |   2 +-
 13 files changed, 358 insertions(+), 114 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c97293d..9752d16 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.7
+ * Run CommitLog tests with different compression settings (CASSANDRA-9039)
  * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
  * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
  * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c445d6b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 9a6ba34..460ecfe 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -70,8 +70,7 @@ public class CommitLog implements CommitLogMBean
 final CommitLogMetrics metrics;
 final AbstractCommitLogService executor;
 
-final ICompressor compressor;
-public ParameterizedClass compressorClass;
+volatile Configuration configuration;
 final public String location;
 
 static private CommitLog construct()
@@ -93,12 +92,10 @@ public class CommitLog implements CommitLogMBean
 @VisibleForTesting
 CommitLog(String location, CommitLogArchiver archiver)
 {
-compressorClass = DatabaseDescriptor.getCommitLogCompression();
 this.location = location;
-ICompressor compressor = compressorClass != null ? 
CompressionParameters.createCompressor(compressorClass) : null;
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
 DatabaseDescriptor.createAllDirectories();
 
-this.compressor = compressor;
 this.archiver = archiver;
 metrics = new CommitLogMetrics();
 
@@ -412,6 +409,7 @@ public class CommitLog implements CommitLogMBean
 public int resetUnsafe(boolean deleteSegments) throws IOException
 {
 stopUnsafe(deleteSegments);
+resetConfiguration();
 return restartUnsafe();
 }
 
@@ -434,6 +432,14 @@ public class CommitLog implements CommitLogMBean
 }
 
 /**
+ * FOR TESTING PURPOSES.
+ */
+public void resetConfiguration()
+{
+this.configuration = new 
Configuration(DatabaseDescriptor.getCommitLogCompression());
+}
+
+/**
  * FOR TESTING PURPOSES.  See CommitLogAllocator
  */
 public int restartUnsafe() throws IOException
@@ -488,4 +494,59 @@ public class CommitLog implements CommitLogMBean
 throw new 
AssertionError(DatabaseDescriptor.getCommitFailurePolicy());
 }
 }
+
+public static final class Configuration
+{
+/**
+ * The compressor class.
+ */
+private final ParameterizedClass compressorClass;
+
+/**
+ * The compressor used to compress the segments.
+ */
+private final ICompressor compressor;
+
+  

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312088#comment-15312088
 ] 

Alex Petrov commented on CASSANDRA-10786:
-

I've started implementing the changes on the driver side, and re-read your 
suggestion and there's a bit more room for improvement. It looks cleaner and 
should be closer to what you described. Also, requires less changes both client 
and server side. You can find it 
[here|https://github.com/ifesdjeen/cassandra/commit/98d46116fba1f1568416ca525c930f0e79595713]
 (diff against prev revision of changes). In short, I reuse {{Rows}} and add a 
flag to indicate that we "force" the metadata. At least in Java driver that 
makes the implementation significantly simpler, as we can check flag, check 
presence of two fields and update metadata or look it up otherwise, all in one 
block without having to change too much.

I might be possible to omit {{statementId}} from the rows response. I've 
included it for now, but we can change it any time, it's a small and isolated 
change, hopefully.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11349) MerkleTree mismatch when multiple range tombstones exists for the same partition and interval

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312087#comment-15312087
 ] 

Stefan Podkowinski commented on CASSANDRA-11349:


I've now attached a patch for the last mentioned implementation as 
{{11349-2.1-v4.patch}} and {{11349-2.2-v4.patch}} to the ticket.

Test results are as follows (reported failures cannot be reproduced locally and 
seem to be unrelated to me):

||2.1||2.2||
|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-11349-2.1]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-11349-2.2]|
|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.2-dtest/]|
|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.2-testall/]|

Anyone willing to take another look and actually commit a patch for this issue? 
I've pushed my WIP branch 
[here|https://github.com/spodkowinski/cassandra/commits/WIP2-11349] with 
individual commits that might help during the review.


> MerkleTree mismatch when multiple range tombstones exists for the same 
> partition and interval
> -
>
> Key: CASSANDRA-11349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11349
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Stefan Podkowinski
>  Labels: repair
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 11349-2.1-v2.patch, 11349-2.1-v3.patch, 
> 11349-2.1-v4.patch, 11349-2.1.patch, 11349-2.2-v4.patch
>
>
> We observed that repair, for some of our clusters, streamed a lot of data and 
> many partitions were "out of sync".
> Moreover, the read repair mismatch ratio is around 3% on those clusters, 
> which is really high.
> After investigation, it appears that, if two range tombstones exists for a 
> partition for the same range/interval, they're both included in the merkle 
> tree computation.
> But, if for some reason, on another node, the two range tombstones were 
> already compacted into a single range tombstone, this will result in a merkle 
> tree difference.
> Currently, this is clearly bad because MerkleTree differences are dependent 
> on compactions (and if a partition is deleted and created multiple times, the 
> only way to ensure that repair "works correctly"/"don't overstream data" is 
> to major compact before each repair... which is not really feasible).
> Below is a list of steps allowing to easily reproduce this case:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 float,
> c4 float,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 2);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush 
> ccm node1 cqlsh
> USE test_rt;
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 3);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair
> # now grep the log and observe that there was some inconstencies detected 
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> Consequences of this are a costly repair, accumulating many small SSTables 
> (up to thousands for a rather short period of time when using VNodes, the 
> time for compaction to absorb those small files), but also an increased size 
> on disk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)